Velocity Measurement

A performance claim is worth exactly as much as the measurement behind it. This is the part of Velocity that makes the rest arguable with evidence rather than intuition.

Live surfaces: /showcases/observability, /admin/perf

Latency tracing

Problem. A slow response is a single number. Nobody can act on it until it decomposes.

A response should be explainable:

request                   620ms

middleware                 14ms
authentication             11ms
domain                      8ms
Redis                       6ms
PostgreSQL                163ms
Neo4j                     121ms
external API               92ms
SSR                        31ms
unattributed              174ms

Architecture. A v10r-owned API with an adapter beneath it, so domain code never couples to a telemetry vendor:

$lib/server/http/request-timing.ts    ← what domain code records into
        ↓
Server-Timing header  (today)   |   an OpenTelemetry exporter  (if ever needed)

event.locals.timing is stamped by the requestTiming handler, which sits first in sequence() — a handler can only measure what it wraps, and the chain itself is worth measuring: the gap between total and the spans downstream layers recorded is the middleware cost. securityHeaders stays second and still stamps locals.clientIp before any limiter or auth handler reads it, so the invariant its position exists to hold is untouched. handle-chain.gate.test.ts asserts both facts.

Two properties are load-bearing.

Cheap. A span is one performance.now() pair and one array push. MAX_SPANS (64) caps the array so instrumenting inside a loop degrades to silence rather than to unbounded memory. Instrumentation that costs what it measures is a lie.

Truthful about what it does not know. unattributed is the wall-clock time no span claimed. It is a floor, not a figure: concurrent spans overlap, so their durations can sum past the wall clock, and the value clamps at zero rather than going negative. A zero means "spans covered it", never "nothing was missed".

Disclosure. Server-Timing is readable by any client, so the span breakdown maps internal architecture and hands out a timing oracle. Full detail goes only to admins and debug-paired devices; everyone else gets total, which they could measure anyway. Durations round to 0.1 ms — finer resolution is noise at this scale and a sharper clock is a sharper side channel.

Coverage, stated rather than implied. Prerendered routes never reach the handler on Vercel: they are served as static files by the CDN. Tracing covers dynamic responses.

Invariants.

• Instrumentation does not materially damage performance.
• Traces preserve application boundaries — a span is named for the layer, not the call.
• Sensitive data never enters telemetry; span names are sanitized to Server-Timing tokens, which also stops a raw name injecting a delimiter and corrupting the header.
• Performance data distinguishes real-user, lab and development contexts (below).

Implementation. src/lib/server/http/request-timing.ts, src/hooks.server.ts, tests in request-timing.test.ts and handle-chain.gate.test.ts.

Field, lab and development — never merged

Two measurement contexts, and scoring one against the other's budget is the specific mistake budgets.json splits kind: "lab" from kind: "field" to prevent.

Field data cannot catch a bundle regression: weight does not appear in RUM until it has already shipped. Lab data cannot tell you the site is slow for users in Australia. ttfb_ms (600 ms, warm local preview) and field_ttfb_ms (1800 ms, p75 across the public internet plus a cold start plus TLS) are separate budgets for the same reason.

The third context is development, and it is the one that lies quietly. Dev samples come from localhost — no network, no TLS, no cold start — so they pull every percentile down. Over 30 days, 72% of samples were dev rows and TTFB p75 read 1051 ms against an honest 1431 ms. Contamination made the site look faster, so nothing about the number invited suspicion. TelemetryOrigin (db/analytics/telemetry-origin.ts) is the discriminator and Velocity introduces no second one.

Full detail: ../../stack/quality/performance.md.

Performance budget and ratchet

Problem. Performance erodes a kilobyte at a time, and a gate wired to an aspiration gets muted the afternoon it is added.

Two thresholds, two jobs, and keeping them distinct is the whole pattern.

Targets (budgets in budgets.json) are where we want to be. Scored on /admin/perf, never build-failing. Several are red today and saying so is the point — the heaviest route is ~609 KB gzipped against a 250 KB target.

Ratchets (ceilings) are the measured value at the moment it was accepted. Asserted by snapshot.gate.test.ts. Moved down to bank an improvement; up only as an explicit, reviewed decision once feature work has used the headroom — never to absorb a dependency.

A gate wired to the target would have failed on the commit that introduced the number and been disabled the same day. Ratchets stop the number growing while the target stays visible as the thing still to fix.

Ceilings carry ~3% headroom over the measurement on purpose: set exactly at the measured value, adding one showcase page tripped the gate, and a gate that fails on ordinary work is a gate that gets muted. The headroom absorbs a feature; it does not absorb a heavy dependency, which is what it is for.

Bundle weight suits a gate unusually well — invisible in review (a one-line import can add 100 KB), deterministic (same source, same number), and only catchable before shipping.

Regenerating the snapshot must use a production build. A dev-mode build compiles both halves differently and inflates client JS ~9%; the snapshot records its NODE_ENV and the gate refuses to score anything that does not say production. compose.yaml sets NODE_ENV=development, so the -e override is required locally.

podman exec -e NODE_ENV=production -e GIT_SHA=$(git rev-parse --short HEAD) v10r bun run perf:snapshot

GIT_SHA is required, not decorative: the container has no git, and the script refuses to write a snapshot it cannot attribute to a revision. vr passes it on every container run (-dirty when the tree differs from HEAD).

Invariants.

• Targets and regression gates stay distinct.
• Ratchets move down, not up, except through explicit architectural review.
• Development measurements are never mixed with production or lab measurements.
• A ratchet is set only on a metric stable enough to be deterministic — which is why cold-start time is reported and not gated.

Implementation. src/lib/server/perf/budgets.json, budgets.ts, snapshot.ts, snapshot.gate.test.ts; probes in scripts/perf/.

Performance scenario harness

Problem. A system is not proven fast because localhost is fast. Every number the observatory reports was taken with a warm cache, an answering dependency and nothing else running — which is exactly the condition under which the interesting questions cannot be asked.

Design. One scenario makes one condition adverse, drives the real mechanism against a simulated dependency, and reports two kinds of column:

• Deterministic — origin calls and an outcome string. Same input, same value, every run, on any machine. scenarios.gate.test.ts asserts these against the committed results, so a behaviour change arrives as a reviewed diff.
• Reported — latency. It moves with the host, so gating it would gate the machine's mood. It is committed anyway because the shape is the finding.

The committed run:

scenario                        origin  outcome
cold-cache                           1  origin-miss
warm-cache                           0  local-hit
expiry-burst                         1  coalesced-50-to-1
slow-dependency                      1  deadline-exceeded          400ms dependency, 60ms wait
failing-dependency                   3  open-after-3-of-12
concurrent-load                      8  admitted-8-refused-12
retry-after-transient-failure        3  succeeded-on-attempt-3
refusal-not-retried                  1  attempted-1-of-5
shed-under-pressure                  0  critical:admitted deferred:budget background:load

Three of those rows are the only proof their pattern has. refusal-not-retried is the retry policy's most consequential decision — a refusal does not become a success by being asked again, and retrying one is how a busy system becomes an overloaded one. shed-under-pressure is load shedding's only measurement anywhere: it has no showcase panel because its two arms do not produce a comparison a stopwatch can make honestly, and a scenario row is what an honest measurement of it actually looks like.

The harness runs with no Redis, on purpose. A lab measurement that depends on whether the developer happens to have Upstash credentials in their shell is two different measurements sharing a name — and the in-process path is worth exercising anyway, since it is what a preview deployment runs.

Why it runs inside vitest. The scenarios drive the real cache and resilience modules, and those reach $env/dynamic/private, a specifier that only exists inside Vite. Bare Bun cannot resolve it, so scripts/perf/scenarios.ts spawns the gate in write mode rather than importing the harness the way scripts/perf/snapshot.ts can. The alternative — a harness restricted to the modules that happen to avoid $env — would measure a convenient subset of the system and present it as the system.

podman exec v10r bun run perf:scenarios     # re-record src/lib/server/perf/scenarios.json

What is not covered, stated rather than implied. Cold compute needs a real deployment, slow network needs a real network, large dataset needs a real corpus, and optimistic-mutation failure needs a browser. Those rows of the design's grid are empty and this file does not colour them in.

Invariants.

• Scenario results distinguish lab from field, exactly as budgets.json already does.
• A scenario that cannot be measured reproducibly is reported, not gated — the same stance that keeps cold-start time off the ratchet list.
• The recorded numbers and the asserted numbers come from one code path, not two.

Implementation. src/lib/server/perf/scenarios.ts, scenarios.json, scenarios.gate.test.ts, scripts/perf/scenarios.ts.

Reading the showcase's numbers honestly

/showcases/velocity executes both arms of each comparison on the request that asked for it. What is simulated is the dependency: simulatedOrigin sleeps instead of querying Postgres, because a public page must not be able to load the database (Neon's free tier is 100 CU-h for the whole project, and a showcase has taken it down before), a fixed cost makes the arms comparable, and a real query's variance would drown the effect.

originCalls is usually the more honest of the two figures. Concurrency can make a hundred rebuilds finish in about the wall-clock time of one while still costing a hundred round trips upstream; the count is what the dependency actually felt.

Implementation. src/lib/server/showcases/velocity/measurements.ts.