Run it as a test
qareef deploy snapshots the tree, fans the flows out on the hosted runner under a concurrency cap, rolls the results up, and files a bug with evidence when one fails. Exit code 1 fails your CI job.
🪸 The QA engine for the AI age
One recorded flow. Four ways to run it.
AI writes far more code, far faster — so the bottleneck moves to verification, and verification has to be driven by agents too, with evidence a person can audit. Paste a URL, drive it once in a live browser, and QA Reef keeps what you did as a flow. Then run that flow as a test, as a bulk data-entry job, as a long-running scraper, or as an API.
The unit
The flow is the thing you own
Every click, keystroke, scroll and navigation becomes a step with a durable locator resolved in the page at event time — test id → role+name → label → placeholder → stable #id → exact text → short CSS path — with a real match count, every alternate kept, and a screenshot.
Flows nest. Every node carries a time-ordered UUID v7 and a stable path, and keeps its id across versions — so one file is the same flow at v1 and at v37.
wf/checkout
└ suite/payments
└ flow/apple-pay
├ step/1 navigate /cart
├ step/2 click role=button name="Pay"
└ step/3 expect text "Order confirmed"
Four faces
Same flow. Four run modes.
QA is the entry point, not the ceiling. Nothing is re-recorded to change mode.
Run it once per row
Parameterise what you typed and the flow becomes a bulk data-entry job: a form filed, a record migrated, a field updated — once for every row of input, each with its own verdict and screenshot.
Run it on a schedule
Point the same flow at a target list or an interval and it becomes a durable scraper — accumulating rows, each one carrying the screenshot and trace of the page it was read from.
Run it as an API
GET /api/flows/:id/schema declares the inputs and outputs; GET|POST /api/flows/:id/call runs the flow and answers JSON. A site with no API gets one.
Primary interface
The CLI drives it
Zero dependencies, Node 20+, pure HTTP. It never launches a browser on your machine — recording, running and the agent all happen in the hosted browser, or in your own with --url.
npx qareef login # store a workspace token (0600)
npx qareef init # qareef.json + flows/default/smoke.flow.json
npx qareef record https://app.example # hosted browser opens; you drive it
npx qareef record --save checkout # pull the recording into flows/
npx qareef deploy --watch # flows/** → a workflow version → dpl_… (live log)
npx qareef run wf/checkout/suite/payments/flow/apple-pay
npx qareef logs dpl_3be4537f5cd3 -f # stream the deployment log
npx qareef inspect dpl_3be4537f5cd3 # per-flow status, errors, run ids
npx qareef promote dpl_3be4537f5cd3 # or: npx qareef rollback
npx qareef agent "a shopper can reach checkout" --site https://app.example --budget 0.25Also ls, whoami, env pull|add|ls, open. Global flags --url, --token, --json. Exit codes: 0 ok · 1 a flow failed · 2 args · 3 auth · 4 network · 5 config/unmeasured · 6 timeout. Full table in the CLI readme.
Scale it up
Record once, then talk to the agent
A recording is the start of a conversation, not a finished artifact. qareef agent explores the target, proposes a suite grounded strictly in what it saw, generates the steps, deploys them, and heals what broke — inside a budget you set.
The model never writes step logic. It proposes a plan and assertions; deterministic codegen writes the Playwright.
Roadmap, plainly: fanning one recording out across a spreadsheet and generating error-case variants from a conversation is what we are building next. The agent loop, the budget and the provenance stamping run today.
Bounded by default
--budget— $0.50 default, charged against the real per-call cost in the ledger--max-steps— 40 page visits, model calls and deployments- 20 minutes of wall clock
- One heal round; a heal makes a new workflow version, leaving v1 and its deployment intact
Every call lands in artifacts/llm-calls.jsonl with its provider, model, purpose and cost.
A step further
It reads the screen and moves the mouse
Most tools resolve a DOM selector and dispatch a synthetic click. QA Reef can also do it the way a person does: screenshot → tiled OCR → the model picks a visible label → the cursor travels a curved, jittered path → click.
That is what makes canvas apps, embedded PDF viewers, cross-origin iframes and frame-era portals automatable at all — there is no selector to resolve, but there is a screen.
Measured, not claimed. Accurate-mode OCR word recall is 86–100% (mean 94%) across 7 pages, scored against DOM visible text as ground truth.
Standing rule: numbers always come from the DOM, never from OCR. Screen reading is for clicking, for churning SPAs, and for surfaces with no DOM worth reading. The operator (cv/operator.js, cv/see.js, cv/vlm.js) is in the repo today and is being wired in as a first-class step type.
CLI and app, one workspace
Start in the terminal, watch it in the browser
The CLI and the web app talk to the same API against the same workspace. qareef record <url> opens the hosted browser and prints the link you drive it in; the steps you record, the runs, their screenshots and traces, the filed bugs and the schedules all show up in the app.
Where it is honest: workflows and deployments are CLI and API only today — ls, inspect and logs -f are the surface. A UI page for them is on the roadmap.
Auditable by design
Verdicts and provenance
An agent that verifies other agents' work has to show its own.
PASS / FAIL / UNMEASURED
A third verdict for a run that never got a clean look. UNMEASURED is never a pass, and any unmeasured flow blocks the pass on roll-up.
decided_by on every decision
When the agent — not you — chooses a click, an assertion or a heal, the exact model is recorded with it, along with what the call cost. Nothing it decides is anonymous.
Heals are quarantined
A model-proposed heal is used for that run, parked for review, and never promoted to the canonical locator without a person. More than max(2, 20%) of steps healed and the run is UNMEASURED — the app changed materially, re-record.
Self-hosting
Point the CLI at your own server with --url, or set QAREEF_URL. You need Node 22, Playwright, and either a Mac for on-device Apple Vision OCR or a 27B-class vision model reachable over the network, plus a model key for the cheap text model that decides what to click.
Open-source core
The automation core is being built in the open under packages/core, MIT-licensed: the flow runtime, the browser-engine adapter, the OCR-driven operator, OCR and pixel measurement, and the LLM adapter with cost logging.
The repository is not public yet — the link goes up at launch. What is in the core →
FAQ
Questions about the platform
What is a flow in QA Reef?
What a person did in a browser, captured as ordered steps. Each step keeps a durable locator resolved in the page at the moment of the event — test id, then role plus accessible name, label, placeholder, stable id, exact text, short CSS path — with a real match count, every alternate kept, and a screenshot. Flows nest inside suites inside a workflow, and every node has a time-ordered UUID v7 that survives re-deploys.
What are the four run modes?
The same recorded flow can run as a test (deployed and fanned out on the hosted runner, with a bug filed on failure), as a bulk data-entry job (once per row of input), as a long-running scrape (on a schedule or over a target list), or as an HTTP API (a declared input schema and a call endpoint returning JSON). One artifact with four faces, not four products.
Do I have to use the CLI?
No, but it is the primary interface. It is zero-dependency, pure HTTP, and never launches a browser on your machine — recording, running and the agent all happen in the hosted browser, or in your own if you self-host. Everything the CLI does goes through the same API as the web app and against the same workspace, so recordings, runs, evidence and bugs show up in the app.
What does UNMEASURED mean?
A third verdict alongside PASS and FAIL, for a run that never got a clean look — the page never loaded, the runner could not execute the flow, the agent was challenged. An UNMEASURED flow never counts as a pass and blocks the pass on roll-up, so a bad measurement is never reported as a real regression or a real success.
Can I self-host QA Reef?
Yes — the CLI takes a --url flag pointing at your own server. The automation core is being built in the open under packages/core, MIT-licensed: the flow runtime, the browser adapter, the OCR-driven operator, OCR and pixel measurement, and the LLM adapter with cost logging. The repository is not public yet; the link goes up at launch.
Does QA Reef solve CAPTCHAs?
No, never. If a site challenges the agent the run stops, records the screen as evidence, and hands it to a person. QA Reef automates work a person is already authorised to do in a browser; it does not defeat access controls.
Bring us a workflow.
Tell us the thing your team does by hand in a browser. We will record it once and show you all four faces of it.
Talk to the team