Agentic QA glossary: flows, mapping sessions, smoke suites, self-healing tests
"Agentic QA" is young enough that the same word means different things depending on whose docs you're reading. These are the terms as we use them, with notes where QA Wolf, the best-known platform in the category, means something else. Sourced where it matters, opinionated where it doesn't.
Flow
A named, ordered recording of browser steps: navigate, click, type, drag. The unit everything else builds on. QA Reef stores flows as JSON step records that generate Playwright code; QA Wolf stores them as .flow.ts modules exporting flow(name, target, callback). Different formats, same idea: the flow is the thing you replay or compile, not the test code itself.
Mapping session
The live browser session used to author a flow. Either a person drives the app and every action is recorded, or an agent drives it from a plain-English goal, reading the screen and logging what it did. QA Wolf's "Mapping AI" is the equivalent capability, operating against the DOM. Ours can also design the flow around OCR anchors, ocr-click and ocr-type steps that target visible text, when there's no DOM worth mapping.
Automate job / Automation AI
The step that turns a mapped flow into test code. Our rule: the generator is deterministic, and the model is only asked for a title and up to three assertions, never step logic, with every output stamped with the model id that proposed it. QA Wolf's Automation AI has a wider remit and can write Playwright code from a prompt (their blog on multi-agent design). Wider is not automatically better: code a model wrote is code someone now maintains.
Smoke suite
The handful of flows that mean "the app is fundamentally up": log in, reach the dashboard, complete the core action. Run on every deploy. A green smoke suite means "safe to keep going," not "nothing is broken." Teams that forget the difference ship the second kind of surprise.
Self-healing test
Automation that adapts when the page changes instead of failing on a stale locator. QA Wolf's writeups describe DOM-based healing: DOM diffs, network traces, console errors (qawolf.com/blog). Ours is a ladder with refusals built in. First wait and retry, because most "locator failed" is a race, and a heal recorded for a race poisons the flow. Then structural heuristics. Then a model. Then OCR coordinates. Every heal must prove it found the same control: same ARIA family, then same test id or name, else a name-similarity score of at least 0.8, which is exactly why "Delete" can never heal to "Delete all" (they score 0.71). Model-tier heals are quarantined: used for that run, parked for human review, never silently promoted to the canonical locator. A heal is a maintenance signal wearing a fix's clothes; treat it like one.
Dark control
A gate tested on its negative path: proving the paywall, permission check, or feature flag shows nothing to the person it must refuse, and that the nothing came from a refusal rather than a crash. Our term, not an industry standard. Full argument: dark controls.
Evidence
What a run leaves behind besides a verdict: a screenshot, a Playwright trace, and for agent steps the OCR text acted on plus the model and cost of each decision. The test of good evidence is whether a failure can be diagnosed without re-running anything.
PASS / FAIL / UNMEASURED
Three verdicts, not two. UNMEASURED means the check never got a valid observation: the page never settled, the viewport drifted, the check's own self-test failed. It never collapses into FAIL and never counts as a zero in an aggregate. Full argument: why a verdict needs a third state.
Runner
Overloaded, so watch the context. Usually: the Playwright process executing a test. At QA Wolf specifically, also a coordinate-based screenshot-and-act tool their CLI exposes for external agents to drive. Their own docs are explicit that it screenshots on demand, once per call. Nobody in this category runs a continuous vision model over every frame, whatever the marketing sounds like.
Open-core
Part of the codebase open, the rest closed. The useful question is where the boundary sits. QA Wolf's sits low: Playwright underneath, plus an open flow-runtime CLI, with Mapping AI, Automation AI, the grid, and the human service closed. We're drawing ours higher: the operator, the LLM adapter, and evidence storage go into an MIT-licensed core, not just the transport underneath it.
Related: Why we read the screen with OCR instead of trusting selectors · QA Wolf vs QA Reef, in depth · Docs