Load pipes with crust

Goal: get a load run whose numbers are about your routes — not about a login page, not about a 404 handler — and a gate that fails when they get worse.

The load task is an emitter. It writes crust pipelines for the routes your app serves and the GET endpoints your components fetch, plus one setup module that refuses to let a run start when the measurement would be meaningless. flawless-react measures nothing itself.

What crust is

crust is a pipeline-first test and devops toolkit — one binary, no runtime, no config file. Its vocabulary is a shell pipeline whose every stage is a real function: load 10s 50/s paces requests against a wall clock, parallel 16 bounds concurrency, GET/POST speak HTTP, stats reduces a stream of responses to a summary object, and assert (s => …) runs an ordinary JavaScript predicate over it. Falsy fails the pipeline with exit 1 and prints the object it judged. test-pipes, test-fixture, mock-server, gen-fixtures and verify-web-links are builtins of the same binary.

Two properties are why the load task targets crust rather than emitting a bespoke runner:

• A .pipes file is one pipeline per line of plain text. No AST, no imports, no framework. That makes it something flawless-react can merge into rather than overwrite, and something you can read in a diff.
• stats.rps is the measured rate, never the requested one. When the target cannot keep up, crust skips stale slots rather than bursting to catch up, and reports the shortfall on stderr. A generated gate saying s.rps > 40 cannot be satisfied by a run that only achieved 25/s — which is what makes the emitted thresholds worth emitting.

Emit the pipes

The task needs --nextjs: its targets come from the router’s page files and the fetch calls in the component trees under them. Without it the task fails with load task requires --nextjs mode.

bunx flawless-react "src/**/*.tsx" --nextjs . --tasks e2e,load --no-interactive

Run e2e alongside load when you can: both read flawless/e2e/params.json, and the e2e task is what normally creates it. --tasks load alone creates it too — with a blank slot per id its pipes reference — because otherwise every instruction that run emits would point at a file that did not exist.

flawless/load/
├── routes.pipes        # the pipelines — yours after generation, merged not replaced
├── routes.setup.ts     # WRITE-ONCE: seeds the ids, proves the auth
├── env.sh              # legacy shell exports; nothing needs to source it
├── README.md           # regenerated by --force
├── last-<name>.json    # written by every run — gitignore these
└── baseline-<name>.json # you promote these by hand

And one script merged into your project’s package.json:

{ "flawless:load": "crust -c 'test-pipes --target flawless/load/routes.pipes --timeout 60000'" }

What a pipe looks like

One line per target. Here is a real one, wrapped for reading — on disk it is a single line:

load 10s 50/s | parallel 16 | GET $LOAD_BASE/dashboard/buildings/$LOAD_ID -H "$LOAD_AUTH" --timeout 2s | expect 200 \
  | stats --out flawless/load/last-dashboard-buildings-id.json \
  | assert (async s => { const b = Bun.file("flawless/load/baseline-dashboard-buildings-id.json"); return !(await b.exists()) || s.p95 < 2 * (await b.json()).summary.p95 })

Stage by stage:

• load 10s 50/s — 500 requests, paced to absolute wall-clock slots.
• parallel 16 — at most sixteen in flight. If the app cannot keep up, slots are dropped and reported, not queued into a flattering latency figure.
• GET $LOAD_BASE/… -H "$LOAD_AUTH" — URL and header line both from the environment; one $LOAD_<PARAM> per dynamic segment.
• --timeout 2s — a per-request bound. A hung route yields {status: 0, timedOut: true} records instead of pinning the pool until CI gives up.
• expect 200 — drains and fails the line on any other status.
• stats --out …/last-<name>.json — the summary object, and a versioned artifact written before the summary reaches the gate, so a failing run still leaves its numbers behind.
• assert (async s => …) — the baseline comparison. Missing baseline passes, by design: the first run is what arms it.

One assert per threshold is crust’s CI-gate shape, and the emitter follows it. A chained failure names the exact predicate that broke and prints the whole summary; a single p95 < X && rps > Y tells you only “false”. Add links to the chain by hand — the merge never touches a body you edited:

… | stats --out … | assert (s => s.status["200"] === s.count) | assert (s => !s.status["0"]) \
  | assert (s => s.rps > 40) | assert (async s => { … baseline comparison, unchanged … })

Targets are page routes and GET operations only. Mutations are excluded deliberately — a default load run should not write five hundred rows of junk into whatever environment somebody pointed it at. Add POST flows by hand, once you have a target you are willing to dirty.

Running them

bun run build:web && PORT=3001 bunx next start &          # a PRODUCTION build
SESSION=$(jq -r '.cookies[] | select(.name=="session").value' flawless/e2e/.auth.json)
export LOAD_BASE=http://localhost:3001
export LOAD_AUTH="cookie: session=$SESSION"
bun run flawless:load

What you’ll see

  PASS  routes.pipes:23  load 10s 50/s | parallel 16 | GET $LOAD_BASE/login -H "$LOAD_AUTH" --timeout 2s | expect 200 |  (9987.1ms)
  PASS  routes.pipes:29  load 10s 50/s | parallel 16 | GET $LOAD_BASE/api/me/notifications -H "$LOAD_AUTH" --timeout 2s  (9986.2ms)
  PASS  routes.pipes:59  load 10s 50/s | parallel 16 | GET $LOAD_BASE/dashboard/buildings/$LOAD_ID -H "$LOAD_AUTH" --ti  (9991.7ms)
  ...

1 file(s): 33 pass, 0 fail

Ten seconds per pipe, so thirty-three of them is about five and a half minutes. That is the first knob to turn: load 10s 50/s is a starting rate the emitter cannot derive from your source, and the file’s header says so.

LOAD_BASE must be a production build

next dev compiles routes on demand. The first request to each route measures the compiler and every later one measures a warm cache, so the p95 you get is a mixture of two things that are both not your app. Check the banner says next start.

If the app’s start script carries no port flag, the default :3000 may collide with something already on the box — PORT=3001 wins, and the urls field in last-<name>.json records which port actually answered.

LOAD_AUTH is a header line, not a token

The value is split at the first colon into a header name and value:

export LOAD_AUTH="cookie: session=e30dc1…"          # a cookie session
export LOAD_AUTH="authorization: Bearer eyJhbGc…"   # a token session
export LOAD_AUTH="accept: */*"                      # a route that needs no auth

A value with no colon is rejected before anything runs, naming what it wanted. Most Next.js apps are the first case, and the cookie is sitting in flawless/e2e/.auth.json from your last e2e run.

The setup module — routes.setup.ts

crust imports the sibling <name>.setup.ts of a .pipes file and awaits its default export before the first line runs. flawless/load/routes.setup.ts uses that window for two jobs.

One: it seeds every LOAD_<PARAM> from params.json — defaults first, then per-route overrides, with blanks never clobbering a default that is set. The load pipes and the e2e specs cannot drift apart over which records they target, because there is one file of ids.

And it throws if it seeded none:

routes.setup: no ids in flawless/e2e/params.json — every dynamic route below would request a URL with an empty segment and measure the 404 handler

That throw exists because of the thing it replaced. flawless/load/env.sh is a bash/zsh portability shim that reads the same ids with jq. Two failure modes, both silent:

• No jq on the box. The shell said jq: command not found and carried on. Nothing said what that cost, so the pipes ran against URLs with empty path segments and reported the 404 handler’s numbers as the application’s — 22 of 33 pipes, in the run that found this.
• Sourced from zsh. The shebang does not apply to a sourced script, so ${BASH_SOURCE[0]} was empty, dirname returned ., the lookup missed, and every LOAD_<PARAM> silently went unset. Same outcome.

env.sh is still emitted for shells that want the exports; nothing needs to source it any more. The module has one failure mode and it is loud.

It is write-once. No regeneration overwrites it, --force included — a check that knows your app beats the generated one, and an authenticated /api/me answering 401 is a sharper signal than a redirect. It is written when absent, so a suite generated before the guard existed picks it up on its next run.

The auth probe, and the run that bought it

The module’s second job is to probe every static route and throw if one lands somewhere else.

Here is the failure it exists for, measured on a real app: 20/20 status 200, p50 19.7ms, exit 0 — and every single request was served /login. The session cookie had expired. crust follows redirects, so the protected route 302’d to a login page answering 200 with a clean body. expect 200 passed, stats reported healthy percentiles, and the file went green having load-tested somebody else’s form.

Nothing downstream can catch that. In load mode crust’s per-request record is {status, ms, url} and that url is the requested one — byte-identical whether the session is live or dead. Only a plain fetch sees where a request landed, which is why the check lives in the setup module and not in the pipeline.

routes.setup: /dashboard landed on /login (status 200). Either LOAD_AUTH no longer authenticates, or this route redirects by design — record the destination under "landing" in flawless/e2e/params.json. Every measurement below would be of /login, not /dashboard.
EXIT=2

Nothing measured, nothing written, and both explanations named rather than one picked.

Every static route is probed, not one. An earlier cut checked a single static route, drew /forgot-password on the dogfood app, and would have passed with a dead cookie — a guard that structurally could not fail. Dynamic routes are excluded: an unfilled id lands on the 404 handler and reads as an auth failure, which is a different bug with a different fix.

A route that redirects on purpose — a default sub-tab, a locale landing — goes under landing in params.json, keyed by route template:

{ "landing": { "/dash": "/dash/overview" } }

The e2e specs read that same map, so a by-design redirect is recorded once instead of in two places that drift. Put it there and not in the pipe: --force regenerates around params.json and never edits it.

Seeing it yourself

The probe is the automated form of two commands worth running by hand the first time:

echo "$LOAD_ID"                                                     # must be non-empty
curl -s -H "$LOAD_AUTH" "$LOAD_BASE/dashboard/buildings/$LOAD_ID" | grep -ci "login"

Real page HTML is tens of kilobytes and contains no login form. The trap wears a second face after a run: a pipe whose p95 is suspiciously fast is measuring the redirect or the 404, not the route. Real SSR routes on a dev box sit around 10–15 ms p95, not 2 ms.

Ids that are not filled

A pipe whose parameters are blank does not measure its route — it measures the app’s 404 handler. GET /invite/ with an empty token answered 500 on every one of 500 requests, and the run reported the route as failing when the truth was that nothing seeds an invitation.

So a pipe with an unfilled id is emitted commented out, above a line naming exactly which key to fill:

# TODO(flawless): no value for LOAD_TOKEN. Fill "token" in flawless/e2e/params.json
# load 10s 50/s | parallel 16 | GET $LOAD_BASE/invite/$LOAD_TOKEN -H "$LOAD_AUTH" --timeout 2s | expect 200 | stats --out … | assert (…)

The route stays visible, so the file itself says what is not being measured. Fill the id, then turn the pipe back on:

bunx flawless-react "src/**/*.tsx" --nextjs . --tasks e2e,load --force --no-interactive

The key is named in the casing your source used — "token", not TOKEN — because that is how params.json is keyed, and $LOAD_BUILDINGID cannot be turned back into buildingId. Shouting is lossy.

The task’s summary then counts what the file measures, not how many routes were found:

load pipes for 33 target(s) — 31 live, 2 commented out pending ids (buildingId, token) — run with bun run flawless:load

That string is what lands in REPORT.md. Counting a gated pipe as a covered target would be the same mistake REPORT.md exists to prevent.

Baselines

Every run writes last-<name>.json beside the pipe that produced it:

{
  "crustStats": 1, "startedAt": "2026-08-26T22:02:04.772Z",
  "urls": ["http://localhost:3001/dashboard/buildings/22222222-2222-4222-8222-000000000001"],
  "summary": { "count": 500, "wallMs": 9991, "rps": 50, "status": { "200": 500 },
               "p50": 10.1, "p95": 14.3, "p99": 16.4, "meanMs": 10.2 }
}

last-*.json is gitignored. To arm the gate, promote one by hand:

cp flawless/load/last-dashboard-buildings-id.json flawless/load/baseline-dashboard-buildings-id.json
git add flawless/load/baseline-dashboard-buildings-id.json

Promotion is manual on purpose: a tool that refreshed baselines automatically would make every regression permanent one run after it landed.

The emitted gate is 2x the baseline p95 — relative, so it tightens as the app gets faster, and loose enough that a noisy CI runner does not red the build over jitter. Against a baseline of p95: 13.3, a run at 14.3 passes; a run at 30.2 does not:

  FAIL  routes.pipes:59  load 10s 50/s | parallel 16 | GET $LOAD_BASE/dashboard/buildings/$LOAD_ID -H "$LOAD_AUTH" --ti
        assert: item 1 failed (async s => { const b = Bun.file("flawless/load/baseline-dashboard-buildings-id.json"); return !(await b.exists()) || s.p95 < 2 * (await b.json()).summary.p95 }) — got {"count":500,"wallMs":9994,"rps":50,"status":{"200":500},"p50":21.8,"p95":30.2,"p99":34.1,"meanMs":22.0}

1 file(s): 32 pass, 1 fail

The failure prints the whole summary, so the red log already carries the count, the status histogram and every percentile — no re-run needed to find out what happened, and last-dashboard-buildings-id.json is on disk to diff against the baseline.

Refresh a baseline only when you can explain the shift in the commit body. Never refresh a batch of them to make a red run green; that is the one move that turns this whole file into decoration.

--force merges routes.pipes

--force folds this run’s routes into the file on disk rather than replacing it. Only a pipe’s comment state changes — the body is never rewritten, so a rate you tuned survives being gated and un-gated. Pipes are matched by the <name> in their stats --out .../last-<name>.json, not by line number.

Nothing is pruned. A stale generated pipe and a POST flow you added by hand look identical to a line-oriented parser, and losing the second to tidy up the first is not a trade worth making.

The header block is the exception: generated prose, refreshed by a run that still recognises its own first line. Keep your notes below it, where they are preserved verbatim and in place.

Flags

When crust is not installed

The task succeeds, and says so:

load pipes for 33 target(s) — 33 live — crust not found on PATH

Emission does not need crust; only running the pipes does. Failing a whole generation run over a missing optional tool would mean a developer without crust could not scaffold the rest of the suite either. The check looks on PATH and for node_modules/.bin/crust in any ancestor directory, since bun run resolves that too.

--run-tests does not execute the load pipes either — a load run wants a production build, a live session and several minutes, none of which a generation run can assume. The summary says so rather than pretending: load pipes are a suite, but not one --run-tests executes.

Exit codes

Two processes, two contracts. The generation run exits 0 whenever the pipes were emitted — crust present or not — and reports the task failed when --nextjs is missing. The pipes run is the one CI gates on:

The 2 is the useful one: it means nothing was measured, separating “your app got slower” from “this run never had a chance”. A gate that measured nothing is not a pass anywhere in crust — stats over an empty stream is tagged empty: true and assert refuses it, because {count: 0, p95: 0} satisfies every threshold anyone would write.

Crust beyond the load task

The same binary does other jobs here, and one of them is this website. Its deploy pipeline publishes the built site, then crawls the live URL:

/srv/apps/tools/crust/latest/crust -c "verify-web-links --base-url $SITE_URL --concurrency 8 --timeout 15000"

A dead link, a redirect chain or a missing #fragment target on the deployed site fails the build. It runs against what the web server is actually serving, not against dist/ — the same principle as pointing LOAD_BASE at a production build: assert on the artifact people will hit.

Limits

Honest about what doesn’t work yet:

• --nextjs or --expo only — there is no route table to derive targets from otherwise.
• GET only. Mutating flows are yours to add, deliberately.
• One rate for every pipe. load 10s 50/s | parallel 16 is a starting point the analysis cannot improve on — a dashboard and a static marketing page get the same numbers until you tune them.
• The probe covers static routes only. A dynamic route with a stale-but-non-empty id still measures whatever that id resolves to.
• The gate is p95 against a baseline. No status-histogram assert, no rps floor, no windowed gating for soaks. All three are one chained assert away; none are emitted.
• Nothing is pruned, and baselines are promoted by hand — with no tooling to tell you one has gone stale.

Related

• Route e2e against a seeded database — the params.json ids these pipes read, and the login hook that produces the cookie LOAD_AUTH carries.
• Wire it into CI — the --ci JSON contract and the stdout/stderr split; the pipes run as their own step, not through --run-tests.
• Regenerate without losing work — the ownership tiers behind write-once routes.setup.ts and the merged routes.pipes.
• Adopt it in a Next.js app — start here if --nextjs has never been run against this project.