flawless-react

Route tests, load, native and the API

The flawless-e2e, flawless-load, flawless-expo and flawless-api skills — the global-setup login guard, params.json ids and landings, LOAD_BASE and LOAD_AUTH, the anti-vacuous probes and real baselines, what --expo refuses, and the runFlawless, testRun and --ci contracts.

Four of the fourteen skills in .agents/skills/ cover what happens outside the generator: the suites that hit your real app, the native plane that needs a real device, and the surface you drive the tool from when nobody reads a terminal. Three of them guard the same failure — a number that looks like an answer and is about nothing.

flawless-react "src/**/*.tsx" --nextjs . --tasks e2e,load --no-interactive

Both route suites need a route table — --nextjs, or --expo with expo-router — and are emitted together so they share params.json. Which half of this page is about which one: flawless-e2e owns the specs, the login and the ids; flawless-load owns the crust pipes and their baselines. They share a directory and nothing else.

flawless-e2e

Loads when you are about to run flawless:e2e, when e2e fails at global setup, or when a route spec passes on suspiciously little.

Knows the seeded-DB prerequisites, the global-setup login and its guard, params.json ids and landings, and the probes that decide whether a pass means a render. Its premise: this suite runs against the real app, because MSW cannot intercept a server component’s direct database access and mocking it would be a lie. That realism is where every vacuous pass hides.

bun run db:up && bun run db:migrate && bun run db:seed   # Postgres + seed users
bun run dev:web                                          # app on :3001
bun run flawless:e2e

The login guard is the test, not an obstacle

flawless/e2e/global-setup.ts is where you sign in as the seeded user and persist storageState to e2e/.auth.json. The tool emits a stub writing an empty state — so the config’s storageState always resolves — plus a worked example. It is never overwritten, --force included: yours once you touch it. Your version should throw if login lands back on /login. That throw is an anti-vacuous guard, not a bug; when it fires the seed drifted — wrong user, wrong password, unseeded DB — so you fix the seed.

Every emitted route spec then asserts it landed on the route it asked for, in both the render test and the interaction test, so an unauthenticated suite fails naming its escapes instead of quietly testing the login screen. Until the hook is filled REPORT.md’s hand-work table says so, by byte-identity against the stub. The skill bounds the claim too: the assertion catches redirects that change the URL, not a middleware rewrite serving the login page at the requested path, a 401 body, or a late redirect.

A route that redirects on purpose — a default sub-tab, a locale landing — would otherwise fail forever. Record the destination in flawless/e2e/params.json under landing, keyed by route template ("/dash" to "/dash/overview"), not in the spec: --force regenerates specs and leaves params.json alone. Dynamic segments come from that same write-once file, and a missing id fixture never fabricates a URL: the route’s specs emit a test.skip naming the file to fill (no id fixture for /posts/[id] — fill flawless/e2e/params.json), so the gap is a visible skip rather than a 404 measured as a pass.

The resolver tries four things in order — a FLAWLESS_E2E_<PARAM> environment override, the per-route entry for a segment set that has to be consistent, defaults, and the legacy FLAWLESS_E2E_ID escape hatch — and a params.json you have broken by hand fails the run with is not valid JSON rather than being quietly replaced, because its contents are yours.

flawless-load

Loads when you are about to run flawless:load, or when pipes report suspiciously fast p95s.

Knows routes.pipes, LOAD_BASE and LOAD_AUTH, the setup module that seeds the ids, the baselines that live here and nowhere else in the tool, and the probes that decide whether the numbers mean anything. Its premise is the same realism with a sharper edge: measuring anything other than a production build is fiction.

A production build and proven auth

bun run build:web && PORT=3001 bunx next start &         # PROD build on :3001
SESSION=$(jq -r '.cookies[] | select(.name=="session").value' flawless/e2e/.auth.json)
export LOAD_BASE=http://localhost:3001
export LOAD_AUTH="cookie: session=$SESSION"
bun run flawless:load    # routes.setup.ts seeds the ids and proves the auth

next dev compiles routes on demand, so the first request to each route measures the compiler — check the banner says next start, and if the app’s start script has no port flag :3000 may collide, so PORT=3001 wins. LOAD_AUTH is a header line: cookie: session= plus the value from .auth.json, not a Bearer token.

You no longer source env.sh. flawless/load/routes.setup.ts runs first (crust awaits a pipes file’s sibling setup module), seeds every LOAD_<PARAM> from params.json, and throws if it seeded none — env.sh needed jq and without it exported nothing silently while two thirds of the pipes measured the 404 handler.

That module also probes every static route and throws if one lands elsewhere; the check is enforced, not advised. An expired cookie redirects to a login page answering 200, so expect 200 passes and the file reports healthy percentiles for a login form — measured: 20/20 status 200, p50 19.7 ms, exit 0, every request served /login. It is write-once, so a check that knows your app survives.

A green pipe run means nothing until you have probed it

A load pipe has no landing assertion of its own, and crust will report a fast, green p95 for 500 redirects to a login page. Run these before the pipes:

echo "$LOAD_ID"                                   # must be non-empty
curl -s -H "$LOAD_AUTH" "$LOAD_BASE/dashboard/buildings/$LOAD_ID" | grep -ci "login"

The curl must return real page HTML — tens of KB — and the grep no login form. Afterwards the trap wears a second face: a p95 that is suspiciously fast is measuring the 404 or the redirect. Real SSR routes on a dev box sit at 10–15 ms p95, not 2 ms. Each pipe gates p95 against baseline-<name>.json; a missing baseline passes by design because the first run arms it, and you arm one by copying a good last-<name>.json. Refresh only when you can explain the shift in the commit body, never a batch to turn a red run green.

flawless-expo

Loads when generating or fixing native specs, when a native suite is red, or when deciding whether a task refusal is a bug or the design. It is the largest skill in the corpus because a React Native plane disagrees with the web assumptions everywhere the web plane took them for granted.

What it knows. What --expo detects and what each reading changes: sdkMajor, because jest-expo’s major must match the project’s own; routerRoot, app/ or src/app/ and only when expo-router is actually a dependency; reactCompiler, which turns three perf-static rules into advice about work the compiler already did — on the dogfood app 136 of 168 findings were exactly those three; nativewind; and webPort, which is Metro’s 8081 and not 3000. Three spec shapes are chosen from the file, the navigation context is always the object form, fireEvent in route specs and userEvent in component specs, and no emitted spec touches the network.

What it stops you getting wrong. That a skip is not a gap: a screen the selector ladder cannot reach emits a skip naming what it needed, and the ladder — testID, then accessibility label, then text — is what you extend by giving the element an identifier, not by deleting the assertion. And that perf-harness refusing is measured-and-declined, not deferred: an expo-router app’s population is route screens, which a browser host cannot mount, so the measured ceiling was about zero of 32 components. The refusal names the evidence and the condition that would reopen it. Route e2e does run here, against expo start --web, with a third assertion the Next flavour does not need: a copy witness proving the screen’s own longest proven literal rendered, because an auth wall rendered in place passes a landing check and fails that one.

flawless-api

Loads when you are scripting suite generation (CI, sandboxes, agents), building a custom report over the analysis, or embedding the tool in another workflow.

Knows runFlawless/runCore options and results, the exit-code and error contracts, the analyse()/ComponentTree surface, the tasks registry, and the --ci JSON contract.

import { runFlawless } from "flawless-react";

const { exitCode, outDir, results, trees } = await runFlawless({
  entry: "src/components/Widget.tsx",   // path or glob; optional in Next.js mode
  nextjs: ".",                          // enables Next.js mode, "." = cwd
  tasks: ["unit-browser", "perf-static"],
  colocate: true,
});

runFlawless is the friendly wrapper (entry/tasks/out); runCore from core.ts is the same engine with CLI-shaped names (pattern/taskIds/outDir) plus changed and heaps. runTests, force, report, log and quiet are on both.

1. Exit 0 does not mean the suites passed

Read testRun, under one invariant: mode: "none" if and only if zero suites executed, so green never appears on a phase that ran nothing.

  • { mode: "generated" | "affected", green } — suites really ran; green is what folded into exitCode.
  • { mode: "none", reason } — nothing ran, and reason names which of three causes: no selected task generates a suite --run-tests executes (only perf-harness, unit-browser and e2e do — load emits pipes you run with bun run flawless:load), nothing was affected, or the tasks plan no run for the work that was.
  • absent — the run never reached the test-run phase. Not “no run was asked for”: a request blocked by a failed task is also absent, exits 1, and is explained by results.

2. stdout is the contract; stderr is the account

--ci is --no-interactive --json: stdout is exactly one JSON blob, { exitCode, outDir, results, keptScripts, testRun, errors? }, and nothing else. Parse the whole of stdout, never a line of it, and never scrape the human console output instead.

stderr still carries diagnostics — skipped files, an unparseable spec, script collisions, a failed Next.js emit — the only account of work the run quietly dropped. Two flags decide what speaks: log (may this run write to stdout? off by default programmatically, gating the progress narration and the per-file findings listing) and quiet (suppress runCore’s own stderr diagnostics? off by default, so warnings speak even when log is off). One flag for both is how --ci became the mode that lost its warnings.

3. Exit 1 has two causes, and neither is a crash

The API never calls process.exit. Bad input throws UsageError, exported from the package and from types.ts; everything else is a plain error. Exit 2 is a usage error and 0 is clean; 1 means either a task failed — results[].error names it — or a --run-tests/--changed suite run went red, in which case every results[] entry is ok: true and the outcome lives in exitCode and CoreResult.testRun. errors is the third shape: failures belonging to no task, such as a Next.js emit that threw and set exitCode: 1 with every task ok: true. Omitted on a clean run, so branch on its presence.

results is TaskResult[]: { id, ok, outputs, findings, summary, error?, status? }, and status — "generated", "refreshed", "kept", perf-harness only for now — places the run in the suite lifecycle. A re-run over an existing suite is not a failure: ok: true plus "refreshed", exit 0. A --changed run keeps the harness ("kept", outputs empty).

Cwd matters too: nextjs: "." resolves against the process cwd and the run merges flawless:* scripts into that root’s package.json, so pin it with process.chdir or absolute paths. The merge never clobbers a user-customised flawless:* value; kept keys surface in CoreResult.keptScripts, the --ci blob and REPORT.md.

The analysis surface

const tree = await analyse({ entry: "src/Nav.tsx", depth: 3, includeChildren: true });
// ComponentTree — entry: ComponentInfo (propsShape, variants, hooks, interactions,
// dataDeps, unresolvedProps/unresolvedTypes), children: ComponentTree[]

analyse() is pure static analysis — no files written — and the right base for custom reporting. Preserve its honesty contract: an extraction that fails is recorded (unresolvedProps, unresolvedTypes, uncertain on data deps), never silently empty, so an empty shape with empty unresolved records is a bug to report, not a component with no props. Tasks are a registry — findTask(id).run(ctx) returning a TaskResult — so a custom task is one more entry in src/tasks/index.ts.

Installing them

flawless-react skills install --select flawless-e2e,flawless-load
flawless-react skills install --select flawless-expo,flawless-api
flawless-react skills list            # current / edited / missing, in both scopes

Each skill is written once, to .agents/skills/<name>/SKILL.md, with its own link at .claude/skills/<name>, and project scope wins when both a project and a user copy exist.

A stale copy is worse than none. The repository copy is the source of truth and moves when the tool moves: this pair of skills is the split of what used to be one file, and a copy predating routes.setup.ts would still send an agent to source env.sh — a file that exports nothing. A stale skill loads with the same confidence as a current one, which is why skills list reports a differing file as edited and install refuses to overwrite it without --force. Re-run the installer after upgrading.