QA Reef vs testRigor
Plain English across far more surfaces than we support, against Playwright code, an open core, and a verdict that can refuse.
Facts about testRigor were read from testrigor.com on 30 August 2026 and are described as their published material states them — check the current version before you buy. QA Reef is not affiliated with testRigor; the name is used here only to identify the product being compared.
The short version
testRigor in one line
Plain-English test automation. Their site describes free-flowing English commands like click "Submit", explicitly aimed at teams without automation engineers.
QA Reef in one line
An open-core QA agent you deploy like a Vercel project: record or describe a flow, get a deterministic Playwright spec, run it on your hardware or ours, and get PASS, FAIL, or UNMEASURED back.
testRigor covers more surfaces than we do — web, mobile, desktop, API, even mainframe — and does it in English. QA Reef is narrower and lower-level: one surface, an open core, and a Playwright file at the end that an engineer can read and diff.
Side by side
| QA Reef | testRigor | |
|---|---|---|
| Open source | Open core. The qareef CLI is Apache-2.0. The automation core under packages/core is MIT; extraction to a public repo is in progress. The generated tests are plain Playwright you already own. |
Not open source. Their public GitHub holds auxiliary pieces — a Playwright plugin, a Selenium extension, an MCP connector — not the platform. |
| Self-host | Yes. Node 22 + Playwright + a model key on your own machines, or hosted by us. Per-workspace data roots either way. | Yes. Their FAQ states an on-premise solution is available. |
| Pricing model | Not published — pre-launch. Self-hosted you pay only for your own model calls, each logged with its model id and cost. A local-model backend makes that $0. | Partly published. Their sign-up page lists a free-forever tier for public tests and a Private tier “from $300/month” with a 14-day trial; the higher Private and Enterprise tiers are not published. Their material frames billing around parallel infrastructure rather than test count. |
| Who writes the tests | You do — by recording in a hosted browser, or by giving the agent a goal. Deterministic codegen writes the Playwright spec; the model only proposes a title and assertions, stamped decided_by. |
You do, in English. Their docs show commands like enter "Peter" into "First Name" and check that page contains "Welcome". |
| How flakiness is handled | A heal ladder: wait-and-retry → recorded alternates → DOM heuristic → model proposal → OCR → coordinates. Every heal must clear a same-control gate. Model heals are quarantined for human review, never auto-promoted. A coordinate click, or a run that healed more than 20% of its steps, returns UNMEASURED rather than a pass. | Their site describes extracting unique issues across tests and grouping affected cases for batch fixing, and states their AI does not depend on XPath. |
| Legacy / canvas UIs | Yes. The operator reads the screen with Apple Vision OCR or a vision model, so canvas and legacy UIs with no stable locators are still driveable. | Not described for canvas, but the surface list is unusually broad — desktop, mainframe, email, SMS and 2FA are all named. |
| CI integration | npx qareef CLI with real exit codes, a GitHub Action, and a Vercel Deployment Check that blocks a promote. |
Named: Jenkins, CircleCI, Azure DevOps, GitLab, GitHub Actions, Spinnaker, plus Jira, TestRail and Zephyr. |
| Data ownership | Self-hosted: flows, screenshots, traces and the model-call ledger never leave your disk. Hosted: per-workspace data root, API tokens hashed at rest. | Hosted, with a stated on-premise option. |
Choose testRigor if…
- Your testers should write English, not code, and never see a repo.
- You need mainframe, desktop, SMS or 2FA coverage — genuinely rare, and we have none of it.
- You want an on-prem option from a shipping product today.
- You want a free tier to try immediately and a published entry price.
Choose QA Reef if…
- You want an open core rather than a closed engine with an on-prem installer.
- You want the test artifact to be Playwright code under version control.
- You need OCR-driven operation of canvas and map UIs.
- You want a run to return UNMEASURED rather than a guess.
Where QA Reef falls short
testRigor covers far more platforms than we do and has a real on-prem product shipping now. On breadth this is not close.
Sitewide caveats, on every page: QA Reef is pre-launch. No customers, no case studies, no published price, no SOC 2 report. Mobile (native app) testing is not supported. There is no managed human-QA service.
Questions
What does testRigor cost?
Their sign-up page publishes a free-forever tier for public tests and a Private tier from $300/month with a 14-day trial. Higher tiers and Enterprise are not published.
Is plain English better than code?
It is better for authoring and worse for reviewing. An English step is easy to write and hard to diff; a Playwright spec is the reverse. QA Reef picks the reviewable side on purpose.
Can both run on-prem?
Their FAQ states an on-premise solution is available. QA Reef self-hosts as its default mode, including recording and evidence storage.
Does QA Reef do mainframe or desktop?
No. Web via Chromium only.
Sources
Read 30 August 2026. Every statement about testRigor above comes from one of these; nothing is inferred, estimated, or taken from a third-party comparison.
Comparing more than one?
Every rival we have written up, in one table — QA Wolf, the codeless platforms, the monitoring tools, the agencies, and rolling your own Playwright.
See the full comparison index →