Run it as a test
qareef deploy snapshots the tree, fans the flows out on the hosted runner under a concurrency cap, rolls the results up, and files a bug with evidence when one fails. Exit code 1 fails your CI job.
🪸 The QA engine for the AI age
One recorded flow. Four ways to run it.
AI writes far more code, far faster, so the bottleneck moves to verification, and verification has to be driven by agents too, with evidence a person can audit. Paste a URL, drive it once in a live browser, and QA Reef keeps what you did as a flow. Then run that flow as a test, as a bulk data-entry job, as a long-running scraper, or as an API.
The unit
The flow is the thing you own
Every click, keystroke, scroll and navigation becomes a step with a durable locator resolved in the page at event time: test id → role+name → label → placeholder → stable #id → exact text → short CSS path, with a live uniqueness count, every alternate kept, and a screenshot. Locators descend into iframes, pierce shadow DOM and reject generated ids; a step is never stored as a bare pair of coordinates.
Flows nest. Every node carries a time-ordered UUID v7 and a stable path, and keeps its id across versions, so one file is the same flow at v1 and at v37.
wf/checkout
└ suite/payments
└ flow/apple-pay
├ step/1 navigate /cart
├ step/2 click role=button name="Pay"
└ step/3 expect text "Order confirmed"
Four faces
Same flow. Four run modes.
QA is the entry point, not the ceiling. Nothing is re-recorded to change mode.
Run it once per row
Parameterize what you typed and the flow becomes a bulk data-entry job: a form filed, a record migrated, a field updated, once for every row of input, each with its own verdict and screenshot.
Run it on a schedule
Point the same flow at a target list or an interval and it becomes a durable scraper: accumulating rows, each one carrying the screenshot and trace of the page it was read from.
Run it as an API
GET /api/flows/:id/schema declares the inputs and outputs; GET|POST /api/flows/:id/call runs the flow and answers JSON. A site with no API gets one.
Primary interface
The CLI drives it
Zero dependencies, Node 20+, pure HTTP. It never launches a browser on your machine. Recording, running and the agent all happen in the hosted browser, or in your own with --url.
npx qareef login # store a workspace token (0600)
npx qareef init # qareef.json + flows/default/smoke.flow.json
npx qareef record https://app.example # hosted browser opens; you drive it
npx qareef record --save checkout # pull the recording into flows/
npx qareef deploy --watch # flows/** → a workflow version → dpl_… (live log)
npx qareef run wf/checkout/suite/payments/flow/apple-pay
npx qareef logs dpl_3be4537f5cd3 -f # stream the deployment log
npx qareef inspect dpl_3be4537f5cd3 # per-flow status, errors, run ids
npx qareef promote dpl_3be4537f5cd3 # or: npx qareef rollback
npx qareef agent "a shopper can reach checkout" --site https://app.example --budget 0.25Also ls, whoami, env pull|add|ls, open. Global flags --url, --token, --json. Exit codes: 0 ok · 1 a flow failed · 2 args · 3 auth · 4 network · 5 config/unmeasured · 6 timeout. Full table in the CLI readme.
Scale it up
Record once, then talk to the agent
A recording is the start of a conversation, not a finished artifact. qareef agent explores the target, proposes a suite grounded strictly in what it saw, generates the steps, deploys them, and heals what broke, inside a budget you set. You can also hand it a saved flow with instructions to extend it.
The model never writes step logic. It proposes a plan and assertions; deterministic codegen writes the Playwright.
Roadmap, plainly: fanning one recording out across a spreadsheet and generating error-case variants from a conversation is what we are building next. The agent loop, the budget and the provenance stamping run today.
Bounded by default
--budget: $0.50 default, charged against the real per-call cost in the ledger--max-steps: 40 page visits, model calls and deployments- 20 minutes of wall clock
- One heal round; a heal makes a new workflow version, leaving v1 and its deployment intact
Every call lands in artifacts/llm-calls.jsonl with its provider, model, purpose and cost.
A step further
It reads the screen and moves the mouse
Most tools resolve a DOM selector and dispatch a synthetic click. QA Reef can also do it the way a person does: screenshot → tiled OCR → the model picks a visible label → the cursor travels a curved, jittered path → click.
That is what makes canvas apps, embedded PDF viewers, cross-origin iframes and frame-era portals automatable at all: there is no selector to resolve, but there is a screen.
When the operator is unsure, it asks you mid-run. The run pauses, your answer resumes it, and the answer is kept in the workspace, so the same question is never asked twice.
Region steps go further. Drag a rectangle on the page and attach an instruction: capture that region's text as a value, assert it shows something, or branch on it.
Measured, not claimed. Accurate-mode OCR word recall is 86–100% (mean 94%) across 7 pages, scored against DOM visible text as ground truth.
Standing rule: numbers always come from the DOM, never from OCR. Screen reading is for clicking, for churning SPAs, and for surfaces with no DOM worth reading. ocr-click and ocr-type are first-class step types: the agent designs flows anchored to on-screen text where the DOM has no selector.
CLI and app, one workspace
Start in the terminal, watch it in the browser
The CLI and the web app talk to the same API against the same workspace. qareef record <url> opens the hosted browser and prints the link you drive it in; the steps you record, the runs, their screenshots and traces, the filed bugs and the schedules all show up in the app. Each workspace drives its own isolated browser; nothing is shared between tenants.
Where it is honest: workflows and deployments are CLI and API only today: ls, inspect and logs -f are the surface. A UI page for them is on the roadmap.
Auditable by design
Verdicts and provenance
An agent that verifies other agents' work has to show its own.
PASS / FAIL / UNMEASURED
A third verdict for a run that never got a clean look. UNMEASURED is never a pass, and any unmeasured flow blocks the pass on roll-up.
decided_by on every decision
When the agent (not you) chooses a click, an assertion or a heal, the exact model is recorded with it, along with what the call cost. Nothing it decides is anonymous.
Heals are quarantined
A model-proposed heal is used for that run, parked for review, and never promoted to the canonical locator without a person. More than max(2, 20%) of steps healed and the run is UNMEASURED: the app changed materially, re-record.
Self-hosting
Point the CLI at your own server with --url, or set QAREEF_URL. You need Node 22, Playwright, and either a Mac for on-device Apple Vision OCR or a 27B-class vision model reachable over the network, plus a model key for the cheap text model that decides what to click.
Open-source core
The automation core is being built in the open under packages/core, MIT-licensed: the flow runtime, the browser-engine adapter, the OCR-driven operator, OCR and pixel measurement, and the LLM adapter with cost logging.
The repository is not public yet. The link goes up at launch. What is in the core →
FAQ
Questions about the platform
What is a flow in QA Reef?
What a person did in a browser, captured as ordered steps. Each step keeps a durable locator resolved in the page at the moment of the event: test id, then role plus accessible name, label, placeholder, stable id, exact text, short CSS path, with a live uniqueness count, every alternate kept, and a screenshot. Locators descend into iframes, pierce shadow DOM and reject generated ids. Flows nest inside suites inside a workflow, and every node has a time-ordered UUID v7 that survives re-deploys.
What are the four run modes?
The same recorded flow can run as a test (deployed and fanned out on the hosted runner, with a bug filed on failure), as a bulk data-entry job (once per row of input), as a long-running scrape (on a schedule or over a target list), or as an HTTP API (a declared input schema and a call endpoint returning JSON). One artifact with four faces, not four products.
Do I have to use the CLI?
No, but it is the primary interface. It is zero-dependency, pure HTTP, and never launches a browser on your machine. Recording, running and the agent all happen in the hosted browser, or in your own if you self-host. Everything the CLI does goes through the same API as the web app and against the same workspace, so recordings, runs, evidence and bugs show up in the app.
What does UNMEASURED mean?
A third verdict alongside PASS and FAIL, for a run that never got a clean look: the page never loaded, the runner could not execute the flow, the agent was challenged. An UNMEASURED flow never counts as a pass and blocks the pass on roll-up, so a bad measurement is never reported as a real regression or a real success.
Can I self-host QA Reef?
Yes, the CLI takes a --url flag pointing at your own server. The automation core is being built in the open under packages/core, MIT-licensed: the flow runtime, the browser adapter, the OCR-driven operator, OCR and pixel measurement, and the LLM adapter with cost logging. The repository is not public yet; the link goes up at launch.
Does QA Reef solve CAPTCHAs?
On an app you own or are authorized to test, yes: QA Reef handles the CAPTCHA with vendor test keys, a bypass token, session reuse, or a solver service you configure, all gated on your attestation that you control or are authorized to test the target. On a site you do not control it does not solve; the run stops, records the screen as evidence, and hands it to a person.
Bring us a workflow.
Tell us the thing your team does by hand in a browser. We will record it once and show you all four faces of it.
Talk to the team