Wire it into CI

Goal: run flawless-react unattended, capture a result a script can branch on, and never let a green step mean “nothing ran”.

The smallest CI-safe run

bunx flawless-react "src/**/*.tsx" --nextjs . --tasks perf-static,a11y,react-rules --ci

--ci is exactly --no-interactive --json, and both halves earn their place. --no-interactive turns a missing argument into an error instead of a prompt: with no positional and no --tasks, a TTY run opens a checkbox menu, and a runner that allocates a pty sits on it until the job’s timeout kills it. --json makes stdout exactly one JSON object — the summary table is suppressed and the progress narration goes away.

What it does not do is select every task. A non-interactive run with no --tasks runs perf-harness alone — not all seven. Name the ids every time: perf-harness, unit-browser, perf-static, a11y, react-rules, e2e, load.

What lands on stdout

{
  "exitCode": 0,
  "outDir": "/repo/flawless",
  "results": [
    { "id": "perf-harness", "ok": true, "status": "generated", "findings": [],
      "outputs": ["/repo/flawless/playwright.config.ts", "…"],
      "summary": "scaffolded perf suite for 3 of 3 component(s): Card, Dashboard, Hello" },
    { "id": "perf-static", "ok": true,
      "outputs": ["/repo/flawless/reports/Card.perf-static.json"],
      "findings": [
        { "rule": "perf/hook-extraneous-dependency", "severity": "warn", "source": "perf-static",
          "file": "/repo/src/Card.tsx", "line": 11, "column": 3,
          "message": "Card: useEffect lists \"open\" in its deps but never reads it — the hook re-fires whenever \"open\" changes, for no benefit." }
      ],
      "summary": "static perf analysis: 2 finding(s) across 3 root(s)" }
  ],
  "keptScripts": []
}

Parse the whole of stdout, not a line of it — the blob is pretty-printed.

Two streams, one rule

Suppressing the table is not a vow of silence. stdout is the JSON; diagnostics stay on stderr — and --ci still emits them:

warning: skipping /repo/src/util.tsx: parseFile: could not locate a React component in /repo/src/util.tsx. Expected an exported function/arrow component or a `const Name: React.FC<...> = ...` declaration.
warning: "test": "bun test" has no path filter, so it will also collect the colocated *.flawless.test.tsx specs and run them under the wrong runner. Scope it (e.g. "bun test tests") or run flawless without --colocate.
nextjs: could not parse ./openapi.json as OpenAPI JSON — ignoring

Every one of those runs exits 0, with every task ok: true and a short findings list — short because a file was skipped and the spec that would have filled the mock fixtures was unreadable. That is the failure mode: a CI step that captures only stdout throws away the only account of the work the run quietly dropped.

Capture both. The clean way is to redirect the script, not the command:

#!/usr/bin/env bash
# scripts/flawless-ci.sh — stdout is the artefact, stderr stays in the job log
set -uo pipefail
exec 3>&1 1>flawless-run.json

bunx flawless-react "src/**/*.tsx" --nextjs . --tasks perf-harness,unit-browser,e2e --ci --run-tests
code=$?

exec 1>&3 3>&-
exit "$code"

flawless-run.json is the machine-readable result; the warnings still land in the job log.

Per-task status — generated, refreshed, kept

perf-harness carries a status word saying where the run sat in the suite lifecycle, read off the version banner on its playwright.config.ts. None of the three is a failure — a re-run is never refused:

Gating on a clean git status after generation works: emission is deterministic and REPORT.md has no timestamp, so a same-version refresh usually diffs nothing. A tool upgrade moves the derived files and that banner.

--run-tests: what actually executes

Only three tasks produce a suite --run-tests executes: unit-browser runs flawless:unit, perf-harness runs flawless:perf, e2e runs flawless:e2e, and the first two together also run flawless:browser — browser specs mount through the perf host, so that script exists only when both were generated. Standalone suites use test:* and have no e2e script. Each invocation is announced on stderr:

› bun run flawless:unit

› bun run flawless:perf

failed suites: perf

Before any Playwright suite the runner attempts bunx playwright install chromium — no --with-deps, which sudo-prompts and would hang an unattended run — under a five-minute kill timer. Set FLAWLESS_SKIP_BROWSER_INSTALL=1 if your image bakes the browser in.

load is the deliberate exception: it emits flawless/load/routes.pipes and declares flawless:load in the same breath, but --run-tests does not execute it — a default CI run should not fire a paced load at whatever environment is reachable. Run it as its own step.

Exit 0 does not mean the suites passed

Ask for a test run with a task selection that generates no runnable suite, and the run exits 0:

flawless-react "src/**/*.tsx" --nextjs . --tasks perf-static --ci --run-tests

{
  "exitCode": 0,
  "outDir": "/repo/flawless",
  "keptScripts": [],
  "testRun": {
    "mode": "none",
    "reason": "--run-tests ran no suite: none of the selected task(s) (perf-static) generates one that --run-tests executes. Select at least one of perf-harness, unit-browser, e2e."
  }
}

Nothing failed, and the static analysis really did run. But zero specs executed, and a gate reading exitCode alone has just reported a passing test suite that does not exist. testRun is what makes the exit code explicable rather than merely correct:

The invariant to rely on: mode: "none" if and only if nothing executed, so green never appears on a phase that ran nothing.

Absent is not “no run was asked for”. Add a11y to that command on a machine with no biome and the run asks for a test run it never reaches:

{ "id": "a11y", "ok": false, "outputs": [], "findings": [], "summary": "a11y scan failed",
  "error": "biome executable not found. Looked in this package's node_modules, in /repo/node_modules/.bin/biome and every ancestor, and on PATH." }

exitCode is 1, there is no testRun key at all, and results names the cause — the standalone binary cannot carry biome, so a11y and react-rules need one in the target project. A task failed, so the suite run never happened.

The gate that reads all of this is three lines of jq over the captured blob:

jq -e '.exitCode == 0'              flawless-run.json
jq -e '.testRun.mode != "none"'     flawless-run.json   # something actually ran
jq -e '(.errors // []) | length==0' flawless-run.json   # nothing failed outside a task

The middle line does double duty: it also fails when testRun is absent — the phase was never reached, which is not a pass either.

Narrowing the run — --changed [REF]

A full suite is the wrong unit of work when three components changed. --changed narrows only the run — analysis and generation stay complete:

flawless-react "src/**/*.tsx" --nextjs . --tasks perf-harness,unit-browser,e2e --ci --changed origin/main

With no value it compares the worktree against HEAD. It selects:

• unit specs through vitest’s module graph (vitest related), so it is transitively correct for free;
• profile and browser specs by component name, with an anchored --grep — anchored because a bare Card would also select CardHeader and CardTitle;
• e2e routes whose page tree renders a changed component — the relationship a path filter cannot see, since editing a leaf StatusPill changes what every page rendering it produces without touching those pages’ files.

Generation is deliberately not narrowed: the profiling host is a single bundle of every component, so a partial regeneration would leave a host that no longer matches its specs. It is not where the time goes anyway — analysing 99 files took 463 ms median, while running every spec they produce took seventeen minutes.

Over an existing suite the harness is kept, which is a success, so any task selection reaches the affected tests. On stderr:

› bun run flawless:perf -- --grep \b(Hello): 

› bun run flawless:e2e -- --grep \bGET (/items/\[id\]) 

The narration that names the narrowing — changed: 1 source file(s) → 1 component spec(s), 0 route(s) — is stdout, so --ci suppresses it with the table. Those › lines are what you have instead: one more reason to capture stream 2.

Nothing affected is not a pass:

{ "exitCode": 0, "testRun": { "mode": "none", "reason": "changed: nothing affected — no specs to run" } }

Pass --changed and --run-tests together and stderr says it is taking the narrow set, rather than resolving the conflict in silence. --changed --force regenerates instead of keeping: an explicit force outranks the keep.

Flags

A CI job you can lift

flawless:
  runs-on: ubuntu-latest
  env:
    FLAWLESS_SKIP_BROWSER_INSTALL: "1"   # the runner image already has chromium
  steps:
    - uses: actions/checkout@v4
      with: { fetch-depth: 0 }           # --changed needs the base ref
    - uses: oven-sh/setup-bun@v2
    - run: bun install
    - name: generate + run affected specs
      run: ./scripts/flawless-ci.sh
    - name: gate on the result
      run: |
        jq -e '.exitCode == 0' flawless-run.json
        jq -e '.testRun.mode != "none"' flawless-run.json
        jq -e '(.errors // []) | length == 0' flawless-run.json

    - name: keep the evidence
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: flawless
        path: |
          flawless-run.json
          flawless/REPORT.md
          flawless/reports/

fetch-depth: 0 is load-bearing: a shallow clone has no origin/main, and a ref git cannot resolve is a hard exit 2 naming the cause, not a comparison that quietly finds nothing.

git cannot resolve the ref "origin/main" in /repo. Pass a ref that exists here (a shallow CI clone may not have it).

And if: always() on the upload — the evidence you most want is from the red run.

Three artefacts are worth uploading. flawless-run.json is the only place exitCode, testRun and errors appear together. flawless/REPORT.md accounts for every component found — how many mount in the browser host, how many cannot and why, which files yielded none, kept scripts, the remaining TODO(flawless) count, and a byte-stable repro line. flawless/reports/ holds the per-task JSON plus a summary.json carrying a ts field, so it churns every run — do not gate git status on it.

Exit codes

Limits

Honest about what doesn’t work yet:

• Only perf-harness carries a status word. The other six have no lifecycle field, so “was this a refresh?” is answerable for the harness alone.
• load is emit-only. No flag runs the pipes as part of a flawless invocation; bun run flawless:load is a separate step with its own environment contract.
• A clean findings list means “nothing this analysis can prove”, not “nothing wrong”. There is no type checker behind it — every extraction that fails is recorded rather than silently dropped.

Driving the tool from TypeScript rather than a shell — runFlawless, analyse(), UsageError — is in the CLI reference and the agent skills.

Related

• Regenerate without losing work — read before a CI job starts passing --force: what a re-run rewrites, what stays yours.
• Triage the findings — when the blob comes back full of perf-static, a11y and react-rules findings: fix versus suppress.
• Route e2e against a seeded database — the environment --tasks e2e needs before its specs mean anything.
• Load pipes with crust — the step --run-tests will not run for you, and the setup module that stops it measuring a login page.