Case study: a whole app

Most tool demos pick one component. This is what happened when flawless-react was pointed at every component of a real Next.js 16 app and told to produce the fullest suite it could, with the rule that nothing may be hand-written. Where a spec was wrong, the generator was fixed rather than the spec.

The target

The 181 matters more than the 97. A shadcn-style ui/ layer puts many components in one file — card.tsx exports seven, dropdown-menu.tsx fifteen — so a file count understates the surface by roughly a factor of two. --roots exported is what makes each of them an entry in its own right.

flawless-react "src/**/*.tsx" \
  --nextjs . --roots exported --colocate --api openapi.json \
  --tasks perf-harness,unit-browser,perf-static,a11y,react-rules,e2e,load --force

What it produced

172 colocated spec files, 512 tests. Plus a browser profiling host covering 94 components, 172 static performance reports, accessibility and React-correctness findings, 31 route e2e specs and 33 load pipes.

Every one of those files came out of the command above. None was edited afterwards.

The number that actually mattered

The first full sweep looked healthy — 427 passing tests — and was not.

68 of the 172 spec files contained no expect() at all. They rendered a component and asserted nothing. The flagship case was ui/button.tsx: 65 lines, six tests, and two void statements present only to stop the linter complaining about unused variables.

test("Button renders at defaults without crashing", () => {
  renderTarget();
  void screen;
});

describe("Button variants", () => {
  for (const v of VARIANTS) {
    test(`renders with ${JSON.stringify(v)}`, () => {
      renderTarget(v);
    });
  }
});

The cva extraction had worked perfectly — VARIANTS held the real variants read out of buttonVariants. There was simply nothing the generator knew how to assert about a component that renders no literal text of its own, which is the entire ui/ layer.

A green suite that proves nothing is worse than a red one, because it is believed.

What it took to fix

The shortest description of the fix is that the tool had been describing the wrong render.

extractRootTag, extractStaticText and extractRenderMode all read the last return in a component’s body. For a component that opens

const [open, setOpen] = useState(false);
if (!open) return <button onClick={() => setOpen(true)}>Record payment</button>;

that is the expanded branch — so eight getByText assertions were written against copy behind a collapsed disclosure, and a querySelector("#notes") could only ever return null. Twelve failing tests in two files, all one cause.

A new module answers two questions against the props a spec will actually pass: which JSX renders first, and whether a given node inside it is reached. It is a small interpreter over a closed set of expressions — identifiers, literals, !, ===, numeric comparison, &&/||/??, .length — returning “unknown” for everything else, and unknown propagates.

Its treatment of an undecidable guard is deliberately asymmetric:

• decidable — follow it exactly; that is new evidence.
• undecidable, guard returns nothing — fall through to the tail return, which is what the code did before, and flag the result.
• undecidable, guard returns content — no answer at all, because no single render describes the component.

Bailing in the middle case would have deleted working assertions from every component guarding on a prop whose type cannot be resolved. Those specs pass.

With the right render identified, three signals became assertable: the root tag, children passthrough, and className passthrough — each only when the value provably reaches the DOM, not merely when the prop is accepted.

What it still refuses to assert

The className rule only fires on a host element. <SeparatorPrimitive className={cn(…)} /> hands the class to somebody else’s component, and whether it reaches a DOM node is that component’s business.

The cva rule is the sharper example. The obvious assertion is that variant="outline" renders border-input. That is wrong often enough to matter: cn() is usually tailwind-merge, which drops base classes a later variant conflicts with. Rendering the real button at size="sm" produces markup with no rounded-lg on it, so a token assertion would fail against correct code. The tool also cannot know whether a given project’s cn merges or merely concatenates.

What the cva table does support is that two options whose class strings differ must render differently:

test("Button: every `variant` option renders a distinct class list", () => {
  const seen = ["default", "outline", "secondary", "ghost", "destructive", "link"]
    .map((option) => {
      const { container, unmount } = renderTarget({ variant: option });
      const el = container.querySelector("[class]");
      expect(el, `variant=${option} rendered no element carrying a class`).not.toBeNull();
      const cls = el?.getAttribute("class") ?? "";
      unmount();
      return cls;
    });
  expect(new Set(seen).size, "two options rendered identical markup").toBe(seen.length);
});

That proves the variant prop reaches the DOM and that every option is distinguishable, with no Tailwind knowledge at all. It bails when two options share a class string, because then they legitimately render the same.

Where it ended up

The 11 skips are the honest residue, and each states its reason in the file: a disclosure trigger with no selectable handle, a component whose early return may render nothing until a prop it cannot synthesise is supplied, a data fixture whose shape could not be inferred.

The second suite told the same story again

The unit specs went green first. The Playwright browser specs — the same 181 components driven in a real Chromium against the profiling host — then ran for the first time and reported 40 failed, 157 passed. Not one was an application bug.

They were the same nine mistakes, made by a scaffolder that had not learned what the unit one had. The two emitters had drifted, and every place they disagreed was a place the browser one was still guessing.

• The submit button was matched by a list of verbs — /submit|save|send|invite|create|add|confirm/i. Against copy like “Update password” it matched nothing and Playwright waited the full 30 seconds: 18 failures, and about nine minutes of every run. What submits a form is written down in the markup. A host <button> submits unless its type says otherwise; a component tag counts only with an explicit type="submit", because shadcn and base-ui both render type="button" by default, so absence is not evidence.
• getByText matches a case-insensitive substring. getByText("Try again") also matched “We couldn’t load this page. Please try again.” A text label is the element’s whole literal copy, so it now asks for exactly that.
• A bare tag is not a name. page.locator("button").first() resolved to a disabled “Cast vote” while the handler belonged to a vote option — and clicking a disabled control means waiting for it to become enabled until the test times out. The tag only names the element when it is the only one of its kind, counting <Button> towards button.
• Fills ignored the field. .fill() against sixteen checkboxes sharing one name, against a checkbox, against a hidden input.
• A <form onChange> was driven as a text box. React’s change events bubble; that is delegation, not an input.
• Functions cannot cross page.evaluate. The props a browser spec passes go through a structured clone, which drops them silently, so onClick={reset} called undefined. The host now holds those names and supplies no-ops on its own side of the boundary — and reset matches no naming convention, so being bound to an event attribute is the evidence.
• Three error boundaries were failed for doing their job. Each logs what it caught; the spec asserted no console errors. A component that calls console.error, or .error() on an imported binding, cannot satisfy that — so the spec drops the one assertion it cannot justify, keeps the rest, and says which call it saw.
• Seven table primitives were failed for a warning the harness caused. <TableBody> rendered into a <div> makes React log “In HTML, tbody cannot be a child of div”. The required-ancestor chain is the HTML specification, not an inference, so the host now nests the mount through it and keeps those components covered instead of excluding them.
• One component’s DEFAULTS were the wrong ones. The browser emitter passed the raw props shape where the profiler passes synthesised stand-ins, so ProfileForm was green in one spec and dead in the other.

One more, which was not a failure but cost the most time: the host reported “did not render within 5000ms” whatever the cause. It now carries the error React swallowed.

Most of the nine minutes that disappeared were 30-second waits — a locator looking for a button whose copy did not match a list of verbs, or an actionability check on a control that was disabled from the moment it mounted. Each is now a decision made at generation time, which is both faster and truthful. The 13 skips name what stopped them: more than one <button> renders here and nothing distinguishes this one, its submit control is disabled at these props, <form> holds no value to change.

The things that were not the analyser

Roughly half the fixes were not inference at all. They are worth listing because they are the failure modes any generator of this kind will hit.

• A silently-passing selector. The last-resort locator was container.querySelector("<TagName>"). For a component tag that matches no element, so userEvent.click(null) was a no-op and the test passed having touched nothing. Four specs were green on that basis. The emitter now returns nothing and the spec skips, naming what the element would need to be testable.
• A guessed ARIA role. Any capitalised tag was assumed to expose role="button". <DropdownMenuItem> is role="menuitem"; <SelectTrigger> is combobox. Every such query failed confidently and wrongly.
• A duplicate import identifier. The profiling host keyed its registry on the bare component name, and the app has two components called CommentForm. Two identical identifiers in one module do not compile — and because the host is a single bundle, that is not two broken specs, it is all 500.
• 84 components that vanished. The same host deduplicated on file path alone, so a file exporting seven contributed one. Nothing said so.
• A test directory collision. The profile specs were emitted into flawless/tests/. Bun treats a positional argument to bun test as a path substring, so the app’s own bun test tests collected 94 Playwright specs and turned a 207-pass suite into 207 pass / 94 fail. They now live in flawless/profile/.
• Two Reacts. react-dom/server was the one react-dom subpath the generated vitest config did not pin, so the document-root specs rendered through a different React instance than their components imported — every hook call failing with Cannot read properties of null.

Reading it yourself

Every run writes a REPORT.md that accounts for all of it:

181 component(s) across 97 file(s).

| Render mode   | Mountable in the browser host | Not mountable |
| client        |                            94 |             7 |
| compound-part |                             0 |            46 |
| document-root |                             0 |             2 |
| rsc           |                             0 |            32 |
| total         |                            94 |            87 |

94 + 87 = 181 — every component the analyser found is in exactly one of those
columns. A further 2 file(s) yielded no component at all; they are listed below.

The arithmetic is printed rather than implied, because “181 components covered” is a file count wearing a suit unless every one of them is either in a spec or named with a reason it has none.

Update: dogfood round 4 (August 2026)

The same app, revisited after twelve tool commits, with four iterations that exercised the paths the first run could not — upgrade-in-place, real e2e and load against a seeded database and production build, hand-hardening through the extension points, and the --changed/--force flows.

Measured line coverage of the generated unit layer alone, before the browser, e2e and load layers: 64.7% of all component-file lines (66.2% lines, 88.9% branches inside components/).

A no-force regeneration against the newer tool changed exactly one file that mattered — the write-once set and all 172 colocated specs held. A --force regeneration’s entire clobber surface was the three specs that had been hand-hardened, restored from git in one command. The round also sent four fixes upstream (.mts style adoption, the two new component detections, and suppression noise in the sweeps), which is the point of the exercise: where a spec was wrong, the generator was fixed rather than the spec.

Update: dogfood round 5 (late August 2026)

The same app once more, this time against the CI-published packages — the first artifacts the tool’s own pipeline ever shipped — after the waves that closed the review ledger to 33/34. The round-4 pain points, retested:

The clusters that did not burn are the bail policies working, not gaps: this app’s server actions mix literal returns with toFormState(err) calls (mixed shapes refuse to guess), and every Promise<T> annotation imports its T from a workspace package the resolver deliberately doesn’t chase — one precise wall, now a named backlog item, rather than scattered failures.

Two fixes went upstream from live pain, in the round’s own spirit: nullable unions (X | null members, the dominant DB-shaped annotation) now resolve to the non-null arm — a member of the declared union, never a guess — and a hand-customized flawless:* script survives regeneration, recorded durably in REPORT.md. That second one bit mid-round: a refresh clobbered this app’s customized flawless:load, and the crust setup module failed loudly instead of measuring a 404 handler — both tools doing exactly what their honesty rules promise.

Update: dogfood round 6 (late August 2026)

The round that closed the ledger (34/34 — the --ci process contract was the last open item) and shipped the named wall from round 5: workspace package type resolution. Bare specifiers like @mybodycorp-ai/shared now resolve through the repo’s workspace map into the package’s source tree, and barrel re-exports — star, renamed, local clauses — are chased to the declaring file, with an adversarial review pass that caught and fixed a private-declaration shadow, star-vs-explicit precedence, and a breadth-starved walk before merge.

Then the honest part. The refresh on this app ran clean (exit 0, version stamps only; the five hand-hardened specs kept; the customized flawless:load command kept and reported), all five layers green again — unit 508 · browser 191 · profile 254 · e2e 39/39 · load 33/33 — and the TODO count moved from 118 to 118. Zero. The round-5 diagnosis of the 42-marker return-shapes wall was wrong, and this round measured the real composition per callee: 21 are an unannotated getDb client factory (no fixture is derivable in principle), a handful are dotted local calls, two are a React cache()-wrapped export the function mapper cannot unwrap, and exactly one is the shared-typed dep the wall was named after. That one was probed live: the new resolution chain lands on the right interface in the right file — and then bails, because z.infer enum aliases sit as members of the interface and the bail-on-any-hole policy refuses the whole value rather than guess a hole.

So the deliverable of round 6 is a corrected map, not a burned-down count: three named frontiers (zod-schema inference, cache() unwrapping, and a browser-host stubbing edge for dist-built workspace packages) replaced one misdiagnosed wall. The resolution machinery works — the execution test proves it end to end by mutating the shared type at its source and watching the fixture re-derive — and the count stayed flat because the truth was further downstream than the diagnosis. Counting that honestly is the whole point of the exercise.

Update: dogfood round 7 (late August 2026)

Round 6 ended with a corrected map rather than a burned-down count: of the 42 “return shapes could not be inferred” markers, exactly one was the shared-typed dep the wall had been named after. The rest were an unannotated database-client factory (21 of them), dotted local calls, and a cache()-wrapped export. Round 7 built the thing that one dep needed — zod schema inference, reading z.infer aliases as syntax — and shipped it.

The marker count moved 118 → 117. That is the honest headline, and it is the number a burn-down chart would show.

The useful movement is underneath it. listMeetingParticipants now carries a complete fixture with all three of its enum members resolved. More importantly, enum-typed props started producing variant matrices that did not exist before: RsvpButtons gained a four-row matrix over its RsvpStatus prop, JobControls likewise. Generated unit tests went 508 → 517, all nine of them executing green on the first run against components whose specs had previously passed undefined and asserted nothing about those states. Static perf findings dropped 245 → 241 as unresolved-prop warnings retired themselves.

The same round pointed nine adversarial finders at the tool itself and found four paths where it could report a pass nobody earned: an unknown task id filtered away (exit 0, zero tasks run), a --changed ref git could not resolve reported as “nothing affected”, a file the linter could not parse reported clean, and an emitted assertion — document.body.innerHTML.length > 0 — that cannot fail because the test renderer always appends a container. All four were fixed inside the round, each with a control proving the new guard can fail. Fixing the fourth introduced a scoping bug in the emitted spec, which the execution tests caught immediately: that is what “verified by executing the artefact, never by parsing it” buys.

Update: six dogfood rounds, on two platforms (late August 2026)

The same web app, plus the monorepo’s Expo mobile app for the first time — 31 expo-router screens that had zero component tests, because the project’s own testing notes said screens were not testable.

Six full cycles: regenerate, run under the real runners, take coverage, read what the numbers said, fix the tool, repeat.

Over component files only — .tsx under src/ for web, .tsx under the app for mobile — which is the same basis the round-4 figure above uses. Counting every instrumented file, mobile is 63.1% / 44.8% / 53.2%: the difference is lib/api.ts and its siblings, which the generated unit layer mocks rather than exercises. Both numbers are true; they answer different questions, and a coverage figure with no stated denominator answers neither.

174 web suites green (514 pass / 15 skip). 31 mobile suites green (138 pass / 24 skip, up from 96 / 64).

Two of those web skips are the RSC fixture-missing escape, which fires at run time rather than emit time: it is what a spec does when a data mock’s shape could not be inferred and the component then crashes on it.

That is a cause, not noise. An earlier version of this page said the split “can move by one without anything changing” — three observations with no mechanism behind them, which is a guess about flakiness. It tracks how many specs take that escape, which tracks how many mock shapes resolve, and the fix below moved it by exactly one on purpose.

The finding all six rounds converged on

Every plateau was a fixture the component’s own code rejects — never a missing test.

Web coverage sat at 67.0% lines and 50.4% branches, identical to the digit, across five consecutive rounds. That should have been read as a signal much earlier than it was. The cause: 22 of 32 page specs were passing having rendered nothing.

A page looks up the viewer’s membership for the building in the URL:

const { id } = await params;
const memberships = await listMembershipsForUser(db, user.id);
const mine = memberships.find((m) => m.building_id === id && m.status === "active");
if (!mine) notFound();

The synthesised row carried status: "flawless-status" and the params fixture was { id: "" }. So find returned undefined, the page called notFound(), the generated spec recognised that as a declared guard path — which it correctly is — and reported a pass. On one page, 117 of its 158 lines never ran.

Every individual step was right. The composition was a green test over an empty render. It is the same class of defect this tool exists to refuse, living on the fixture side rather than the assertion side, where no probe reached.

What closed it

Two rules, both reading the component’s own source rather than inventing anything:

• A field compared with === to a literal takes that literal. m.status === "active" yields status: "active". !== is not used — it says what a value must not be, which does not pick one out.
• A field compared to one of the component’s props takes that prop’s value, read from the same fixture the spec’s DEFAULTS carries, so the two agree by construction rather than by both happening to spell flawless-id.

And a Next page’s params never carries an empty segment. An empty string is a perfectly good default for a caption; Next does not match a dynamic route against one, so { params: { id: "" } } is a request that cannot happen.

Guard exits went from 22 of 32 to 8 of 32. The notFound() path keeps its own test, which mocks the lookup to return nothing — the coverage is additive, not traded.

Two things the project had written down that were wrong

Both had been recorded as deliberate decisions, with reasons, and both survived because nothing had executed them.

“A partial fixture is worse than none, so heritage bails.” The reasoning is right. The bail was wider than it: a base that resolves is not partial. Merging bases (own members last, and still bailing whole if any base fails) took the mobile app from seven uninferred API shapes to zero — and one of those seven was poisoning an entire screen, because const [{ poll: p }] = await Promise.all([getPoll(…), …]) binds p to undefined and the next member read throws into the app’s own catch, so the error state rendered while the test looked for a row.

“Those screens hold their rows in an external store the harness does not reach.” That was the comment on a test that shipped permanently skipped. It was a hypothesis written as a finding. Enabling all 21 recipes live produced 11 failures with three distinct causes, every one of them visible in the AST: a mutation called from a submit handler, a call behind if (isTrustee), and a collection read to decide a permission boolean. Fourteen screens qualify under the corrected rules and all fourteen pass.

The accessibility half

The mobile app’s unaddressable controls and its accessibility gaps turned out to be the same list. A <Pressable> with no testID, no accessibilityLabel and no text under it is unfindable to a test for exactly the reason it is silent to a screen reader.

48 accessibilityLabel and 28 accessibilityRole props went into the app — 20 text inputs whose visible label was never attached to the control, and 28 icon-only back buttons. The generated suite went from 96 to 124 passing tests and 64 to 36 skips.

The coverage is the consequence. The controls were unusable with VoiceOver and TalkBack either way, and no test run was needed to make that true.

What was deliberately not fixed

Web branch coverage is 56.2% against 74.9% lines, and 8 of 32 page specs still exit through a guard. The remaining gap is that a fixture makes a predicate true, so userIsTrustee ? A : B only ever renders A — mocking to {} and mocking to the annotation’s true take the same branch.

Covering the other side needs a second mount with the predicate inverted, plus a rule for which predicates are safe to invert. That is a design decision rather than a fixture fix, and it is on the backlog rather than half-done.

After the rounds: two defects wearing one symptom

The rounds closed with a ranked list of what was left. The smallest item on it — one function, five consumers — turned out to be hiding a second defect, and the pair is a fair illustration of how this codebase gets things wrong.

exportedFunctions skipped any variable declaration whose initializer is a call, so the standard Next.js data-access idiom was invisible to every consumer:

export const getCurrentUser = cache(async (): Promise<User> => …)

The dogfood app’s lib/dal.ts uses it three times. Downstream that is a {} mock and a “could not be inferred” marker on a function whose return type is written two characters away.

Fixing it made those three exports visible and still unresolvable. User reaches them through import type { User } …; export type { User };, and the re-export chase looked only for a local declaration of that name, found none, and stopped. Two independent defects, one symptom — separated with a three-file probe before either was touched, so each could be shown to move its own case and nothing else.

RSC fixture-missing skips  2 → 1        tests  514/15 → 515/14
lines  74.9% → 75.0%   branches  56.2% → 56.4%   functions  79.4% → 79.8%

Small, and the mechanism is the point: an enriched dependency drops the skip escape, so a page’s whole render test stops being skipped and starts running. The marker count — 104 to 101 — is the least interesting number here.

The wrapper list stays closed: cache and unstable_cache, which hand back the same call signature. Not “any call with one function argument”, because the original skip’s reasoning is real for most wrappers — withPagination(async (): Promise<Row[]> => …) may well return Promise<Page<Row>>, and unwrapping it hands the generator a shape the function never produces.

None of it moved the mobile app, whose API client uses plain export async function and direct type imports. A fix that helps one target and not the other is worth saying so about.