crust

Log mining

Forensic queries over log trees, live streams from any command via stdin, and the interactive logs session that holds a stream while you iterate on filters.

Goal: answer “which logs contain X, how many times, and what’s around them” using crust’s mixed shell + TS pipeline. Use grep when grep is enough; reach for a filter stage when the predicate is structural, and a TS lambda when you need JSON parsing or per-line shaping.

Count error lines across a tree

lines **/*.log | grep ERROR | wc -l

Filter with a TS predicate

When you want a structural filter — say, only lines from a specific user — a filter stage is friendlier than a grep regex.

# grep narrows files to ERROR lines; filter applies the TS predicate per line
lines **/*.log | grep ERROR | filter (l => l.includes('user=42'))

filter (line => …) keeps items whose predicate is truthy and drops the rest (plain JS truthiness — 0, "", and null all drop). Async predicates are awaited; thrown errors fail the pipeline. A plain (line => …) lambda is different: it maps — a boolean-returning lambda emits true/false lines instead of dropping anything, and an empty filter result passes silently (use assert when nothing-matched should fail).

Structured logs: JSON per line

If your logs are JSON-per-line, parsing and projecting is one lambda each.

# Read each .log file, split into lines, parse as JSON, keep
# severity=error, project the message field
lines **/*.json.log
  | grep .
  | (line => JSON.parse(line))
  | filter (j => j.severity === 'error')
  | (j => j.msg)

Shell stages linearize: whatever they print becomes one item per output line, which is why grep . turns whole-file items into per-line items (and drops blanks). A mid-pipeline lambda does not flatten arrays — (text => text.split('\n')) emits the whole array as a single item. Array flattening only happens for function-as-source return values.

Group + count via reduce

# Extract error codes; filter drops the lines that had none
lines **/*.log
  | grep ERROR
  | (l => (l.match(/ERR_[A-Z0-9_]+/) ?? [null])[0])
  | filter (c => c)
# → stream the codes; pipe into your favorite counter

Tail-and-watch

Crust ships a native tail source — last-N lines, tail -F-style follow, multi-file merging, and rotation detection — so log streams stay inside the pipeline model without shelling out.

# Last 100 lines of a log, filter ERRORs through grep
tail -n 100 application.log | grep ERROR

# Follow forever, alert on errors
tail -F application.log | grep ERROR | POST :9000/alerts

# Follow only — skip the initial cut, only stream new lines
tail -F -n 0 application.log | (l => JSON.parse(l))

# Multi-file: glob or explicit list. Lines from all files merge into one stream.
tail logs/*.log | grep ERROR > combined.log
tail -F services/*/access.log | (l => l.toUpperCase()) | tee combined.log
tail -F api.log worker.log scheduler.log

tail mirrors POSIX flags — -F/-f for follow, -n N or --lines=N for the initial cut. The TS API takes tail(paths, { lines, follow, pollMs }), with defaults lines: 10, follow: false, pollMs: 200. paths is a string (path or glob) or an array.

Multi-file semantics: each file gets its own inode/size loop, rotation handling, and initial-N-lines cut. Lines yield as they arrive — non-deterministic across files, deterministic within one file. The same tail -f a.log b.log behavior you’d get from POSIX.

Rotation: the follow loop tracks each file by inode and size. A rotate-and-recreate (logrotate, mv app.log app.log.1 && touch app.log) is detected via inode change, and the stream switches to the new file automatically. A truncate that shrinks the file below the current offset is also detected and resets the stream. A truncate-and-immediate-overwrite to a size ≥ the prior offset is indistinguishable from an append via stat alone, so it’s treated as an append — matches GNU tail -F behavior.

Falling back to system tail: unrecognized flags (-c, --pid, …) fall through to the system binary via the shell, so tail -c 200 app.log and tail --help keep working.

Bounded initial read: the -n N cut scans backward from EOF in 64KB blocks and reads only the last-N-lines window — tail -n 3 on a 500k-line log answers in ~20ms without ever holding the file in memory, and tail -n 0 -F reads nothing before following.

Live streams: two traps crust removes

tail -F app.log | grep ERROR is the classic follow pipeline — and with system grep it silently stalls, because GNU grep block-buffers ~4KB when writing into a pipe. Crust runs the safe grep subset (-i/-v/-F, one pattern) as a native line-buffered stage, so every match streams the moment it arrives; anything fancier (combined flags, $VARs, POSIX classes, file grep) still gets exact system-grep semantics via sh.

The second trap: getting any command’s output into the pipeline. The stdin source (alias -) bridges the invoking pipe:

# Live error feed from a container, pino JSON → filtered → projected
docker logs -f my-app | crust -c 'stdin | (l => { try { return JSON.parse(l) } catch { return null } }) | filter (e => e && e.level >= 40) | (e => e.msg)'

# Rolling latency/status windows over live traffic
docker logs -f my-app | crust -c 'stdin | (l => { try { return JSON.parse(l) } catch { return null } }) | filter (e => e && e.status) | (e => ({status: e.status, ms: e.duration_ms})) | stats --every 3'

# journald, priority err and above
journalctl -f -o json | crust -c 'stdin | (l => JSON.parse(l)) | filter (e => e.PRIORITY <= 3) | (e => e.MESSAGE)'

Real capture of the second line against a container under load:

{"window":1,"count":121,"wallMs":3002,"rps":40,"status":{"200":121},"p50":1,"p95":1,"p99":2,"meanMs":0.9}
{"window":2,"count":120,"wallMs":3002,"rps":40,"status":{"200":120},"p50":1,"p95":1,"p99":2,"meanMs":0.8}

Wrap JSON.parse in a try/catch on mixed streams — real container logs carry startup banners between the JSON lines, and a bare parse dies on the first one. Note the bare-pipe rule: cmd | crust treats piped stdin as a script, so data pipes always go through crust -c 'stdin | …'.

Interactive: hold the stream, iterate on filters

Re-running a follow pipeline restarts the tail and loses the recent past. The logs builtin holds one live source open, buffers the last 10k items (--buffer N to change), and every line you type is an ordinary pipeline fragment run over that buffer first, then over the live stream:

logs docker logs -f --tail 300 my-app     # any shell command's stdout
logs tail -n 0 -F app.log                 # or a file follow
logs procs({web: "bun run dev"})          # or a whole process group

A real session against a live container:

logs: docker logs -f --tail 300 my-app — buffering last 10000 items
logs> grep api_request
{"level":30,"msg":"api_request","path":"/api/health","status":200,"duration_ms":1}
…                       ← matches from the buffer (the recent past)
-- live --
…                       ← new matches as they arrive
^C
logs> json on
json: on — string items parsing to JSON objects/arrays reach queries parsed
logs[json]> filter (e => e.status) | (e => e.path + " " + e.duration_ms + "ms")
-- live --
/api/health 1ms
/api/health 0ms
^C
logs[json]> search timeout
search: 2 matching item(s) of 551 buffered
logs[json]> buffer 50000
buffer: resized 10000 → 50000 (kept 551 item(s))
logs[json]> exit

json on is the session’s honest answer to NDJSON streams: parsing happens at query time (the buffer stays raw, json off reverts instantly), lines that aren’t JSON objects flow through unchanged and are counted out loud — so a mixed stream can neither crash a query (the old (l => JSON.parse(l)) lambda threw on the first plain line) nor silently lose data. search <text> answers instantly from the buffer with a match count (zero included) and highlighting; buffer N widens the window without restarting the held source.

Because every query is ordinary pipeline grammar, rendering is just another stage — filter on the raw JSON first, then hand the matches to pino-pretty:

logs> grep api_request | pino-pretty --colorize --singleLine
[11:46:18.313] INFO: api_request {"request_id":"…","path":"/api/health","status":200,"duration_ms":4}
-- live --
…colorized lines keep streaming until ^C…

No node_modules/.bin/ prefix needed: shell stages get npm-run-style PATH — every ancestor node_modules/.bin is prepended, so locally-installed tools work bare. --colorize forces ANSI even though pino-pretty runs behind a pipe. Keep the grep/filter before the pretty stage — after it, you’d be matching against ANSI-decorated text.

The parts worth knowing: Ctrl-C once ends the live view gracefully — the stream finishes, so a query ending in bare stats prints its summary exactly there; Ctrl-C twice hard-cancels a stuck query; the session and the held source survive both. A fragment runs twice (buffer, then live), so append with >>, never >. exit/Ctrl-D tears the held source down — procs groups get the SIGTERM→SIGKILL escalation.

Tips

  • Mix grep and lambdas — grep is unbeatable for plain substring/regex filters; reserve lambdas for things grep can’t see (JSON shape, async lookups, multi-line state).
  • lines vs read vs Bun.file().text() — the shell stage read <glob> yields whole-file items (one per file), lines <glob> yields one item per line, and the TypeScript read(path) streams lines like the latter. They print identically on a terminal, so the difference only shows when a downstream filter/lambda sees one giant string instead of lines. grep masks it by splitting internally — which is why read **/*.log | grep ERROR | filter (…) works and swapping those last two stages silently does not.
  • Pair with time "label" to know how long a forensic query takes — the timer fires even if a downstream stage throws.
  • Intro — refresher on sources, transforms, sinks.
  • Quickstart — every source/transform/sink in one tour.