Turn trace

Status: BUILT 2026-09-12 (Phases 1, 3 and 5 of the AI context + trace refactor), validate-green; the turn graph and the recorded ancestry landed 2026-09-13; the ai.conversation_step → ai.model_call rename DDL has been applied. Phase 2 (profiles, profiles.md) sits under it; Phase 4 (the pointer-layer retirement) landed the same day; Phase 5 gave the deskbot page the same inspector.

What this is

Every chatbot and deskbot turn now leaves one recorded account of itself: what was available to it, what each grounding source considered, what entered each model call's request, which tools ran with what result, how the provider attempts went, and what the answer cited. It is recorded from the actual execution — never a second run — streamed to the client while the turn runs, written once when it finishes, and read back by the turn's owner.

Before this, the chatbot streamed pipeline:* events as message-metadata and dropped them; conversation_step held tokens/provider/model/duration per SDK step and nothing else; chatbot tool calls were never persisted; a reloaded thread had no citations; the full prompt text was gated to dev builds and admins; the showcase's "Context orchestration" was a separate run.

The contract

$lib/types/turn-trace.ts — framework-free, imported by the server recorder, the chat components and the showcase's turn inspector.

TurnTrace
├─ messageId · conversationId · surface · requestId · profileVersion · outcome · errorKind
├─ timings        preStreamMs · embedMs · generateMs · firstTokenMs[] · finalize{catalog,persist,budget}
├─ awareness      locale · authCeiling · page (chatbot) · scopes · workspace · layout · panels (deskbot)
├─ activations    { id: CapabilityId, active, reason }   every capability's verdict for the turn
├─ blocks         { id, capability, section, text, chars, stable }   the system prompt, block by block, WITH text
├─ grounding      per source: ran · skippedReason · error · pool · cutoff · retrievers · items[]
│                 item: id · kind (chunk|catalog|map) · title · score · rank · state · blockId · omittedReason · path · catalog
│                       chunk place: parentId · level · position · contentHash · retriever   tool-surfaced: toolCallId
├─ history        the windowed history as sent — parts sized (text, tool_call, tool_call_response, compaction), never quoted
├─ toolset        the tool definitions as the model received them (description + JSON schema)
├─ modelCalls     one per provider request: attempt × step · usage · request outline (blockIds · historyCount · toolResultIds · toolsOffered) · response outline
├─ toolExecutions one per tool execute: input · model-facing output (post-compaction, ≤ 8 KB) · status · duration · compaction ref
├─ attempts       the provider rotation: started | ok | rotated | failed | cancelled, with the error kind
├─ citations      { itemId, source, match: path | quote | provider_source | unsurfaced }
├─ proposalId     the proposal a deskbot turn stopped on (read side: `proposal`, resolved by id — see Reading)
└─ createdAt · bodies (inline | persisted | redacted)

Five item states on every grounding item: available (the profile manifest's inventory — never a trace item) → considered (retrieved or ranked this turn) → included (in a model call's request) → executed (a tool execution returned it; toolCallId names which) → cited (the answer names its path or quotes it). Inclusion never claims influence. An item the prompt left out says why: below_cutoff (ranked past the top-N) or size_cap (cut by the block's size cap). An unsurfaced citation is a project path the answer named that nothing this turn produced — the answer's claim, not the turn's.

A chunk's place (2026-09-13). Tier 1 returns each hit with its parentId, level (section · paragraph), position, contentHash and the document's sourceUri (retrieval/tiers/contextual.ts); chunkPlace() (ai/capabilities/chunk-place.ts) records them on the item together with the retriever that produced it, and the source records the retrievers that ran. The read route resolves a recorded parentId to the parent as it stands now (parent: level, position, size) — current metadata, labelled so, never the turn's own account; a parent that is gone leaves it absent. The tiers the kernel has not taught yet (parent-child, graph) record no place: the inspector says "not recorded".

What each request carried. ModelCallRequest.toolResultIds names the tool calls whose results were in the request (the earlier steps' round trips, read from the prompt's tool-result parts by the middleware). Absent on a call recorded before the field existed — not recorded, which is what the graph then says — and empty when none. A tool-surfaced catalog row or desk chunk carries the toolCallId of the execution that returned it (the sinks in profile.ts take it from the tool's execute options).

TurnTraceSnapshot is the same shape with every key always present and the bodies withheld (block text, tool I/O, tool definitions): what rides the wire as message.metadata.trace. The client deep-merges metadata objects and replaces arrays, so a key that vanished would linger — hence "always all keys". TurnSummary is what a conversation's turn list carries so a reloaded thread renders its citation chips without the full trace.

Recording

One author: src/lib/server/ai/trace/recorder.ts (createTurnRecorder). Each part of the turn reports what only it sees; the recorder folds it into one trace.

The middleware and the step hook each open/close the same call record by order (one open call at a time); when the middleware could not wrap the model (a gateway id, a v2 provider, a test mock), callEnd() opens the record itself without the request outline. The hook names are the SDK's; the recorder's are the lifecycle's, so the v7 renames touch the orchestrator's adapter lines only.

profileVersion hashes the stable blocks' text and the tool definitions: identical across turns of one profile, so together with cacheReadTokens it shows whether the prefix is being cached.

Streaming

recorder.subscribe(flush): every change schedules one message-metadata frame per 16 ms burst (METADATA_FLUSH_MS), carrying { trace: recorder.snapshot() } — plus harness (the PlanCard) on the deskbot. Nothing else rides beside the trace. Explicit drains keep the order exact: before the first model frame, before finish. Nothing is written once the message is closed (closed), so no frame ever lands after finish or after an error frame — that would open a second, empty message on the client.

Client consumers (composites/chatbot/turn-progress.ts): turnProgress() reads the status row's stage from the snapshot (retrieving → catalog → generating); citedCatalogSources() turns cited catalog items into CitationChips; turnFinished() says when the snapshot's last attempt has ended — the moment the showcase can open the turn from its persisted row.

Persistence

Written once, in one transaction (saveTurnTrace), inside afterText after the answer row is backfilled and before finish; a turn that ended on an error frame is written from the stream's onError. Insert-only: a jsonb column updated per step would rewrite its whole TOAST value each time.

ai.message.parts stores the assistant message's parts (tool parts included, rebuilt from the trace) so a reloaded thread renders the tool rows the answer was built on. The dead conversation_step.retrieval_events and message.context columns are gone. Conversation totals are summed from model_call.

Retention (retention/schedule.ts, jobs/ai-telemetry-retention.ts), the mcp.call_log pattern: redact the bodies at 30 days (ai-turn-bodies: turn.blocks, turn.history, turn.toolset, model_call.request, tool_call.result; redacted_at stamped), delete the rows at 180 days (ai-turns, ai-model-calls, ai-tool-calls). Conversation delete cascades; user delete cascades through the conversation. The privacy report counts turns.

Reading

The turn records the proposal's id only: the row keeps moving after the turn (approved → executing → executed | failed, or rejected, or expired), so the trace shows where it stands when read, next to the awaiting_decision outcome the turn ended on.

The owner sees their own turn entirely: prompt bodies, desk context, tool I/O, the chunk text they were shown. The dev/admin gate on prompt text is gone (decision D1). Provider internals (keys, raw errors) are never recorded.

The turn inspector (Phases 3 and 5) and the turn graph (2026-09-13)

/showcases/ai/chatbot#orchestration and /showcases/ai/deskbot#orchestration open ONE turn of their surface — a committed fixture signed-out, any of the viewer's own turns signed in — and render it three ways over one selection: the turn graph, the tree (the accessible alternative) and the timing waterfall. Selecting anywhere opens the same detail pane, which takes space only while something is selected. The turn line above them names the question, the outcome, the calls, the tools, the citations and the provider.

The turn graph

Three fixed columns ($lib/showcases/ai/turn-graph.ts, the pure projection; its geometry in turn-graph-layout.ts; rendered by _components/turn-graph/* through the shared FlowDiagram wrapper on a desktop canvas and as three stacked lists under 768 px):

Every edge is a recorded relation; the graph never infers one. containment is where a record sits; the data-flow kinds are inclusion (item → block, by blockId), request (block → call by request.blockIds; the history → every call whose request counted messages; the last ok call → the answer), execution (call → tool execution, by modelCallId), result (execution → a later call only when that call's toolResultIds names it; execution → an item only when the item's toolCallId does) and citation (answer → item, for a citation that resolved; an unsurfaced citation is a count on the answer card, never an edge). A fact the record does not carry renders as "not recorded" (flags.unknown). Folding lifts an edge onto the nearest visible ancestor and merges what coincides (×n). The model-call selector marks what the chosen request was made of and which tools it offered; declared, offered and called stay separate facts.

Geometry is a pure function of the visible graph (layoutTurnGraph, routeEdges) — fixed sizes per kind and variant, columns at fixed x, nothing measured, so a test asserts it:

• A node with visible members is a group: a header row and its members stacked inside, each depth stepping in by the group's padding and tinted deeper (Source › Document › Section › Chunk; System prompt › Block). The nesting is the containment — a containment edge is never drawn on the canvas; the list writes nothing for it either.
• No edge crosses a card. Between adjacent columns an edge takes a track in the gutter (its one vertical run; runs that overlap in y take different tracks, 8 px apart). Inside a column an edge runs down that column's rail (a shared line just left of it, read as a bus with taps — the prompt and the history into every call, the last call into the answer). Two columns apart (a tool's result surfacing an item) an edge takes a detour beneath the columns. Data flow carries an arrowhead; containment is a box in a box; the legend under the canvas stays in view.
• Expanding never refits: the canvas grows with the graph (to 88 vh) and the zoom stays where the reader had it. A detail pane opening or closing refits to the highlighted subgraph, never below a readable floor (0.8). Fit everything is the reader's own button on the canvas, and the only fit allowed below that floor. Every node carries its size as measured: the flow counts a node initialized only once measured, and a node object the parent replaced would otherwise wait for a resize that never comes, leaving every queued fit unresolved.

On the chatbot page the inspector follows the thread above it (TurnInspectorState.follow): a running turn is on the page from its streamed snapshot (inspectedFromSnapshot, bodies pending), a finished one — or a resumed thread's last turn, which carries only its summary — is read from its persisted trace (a 404 is read again briefly: the row lands before the finish frame, but the client is not made to depend on it). A turn picked by hand, or opened from an "Inspect this turn" link, switches following off until the viewer asks for it again. A live turn that is loading, failed or not picked shows exactly that; the fixture never stands in for it. profileDrifted() compares the recorded identity, guidance and tool definitions with today's profile fact by fact — the two version hashes are built from different inputs and never agree.

The tree walks the five states in commitment order:

Every node opens content, never a count. The spine (#spine), the guard chain, the prompt tape and the deskbot's approval lifecycle all render the same trace — spineOf() derives the band statuses from the recorded facts (the deskbot's gate band is the door: skipped when no proposal, active while pending, not-taken when the human rejected it or let it expire, done otherwise); a trace exists only past the guard, so the guard chain of a turn that exists has passed. The projection is pure ($lib/showcases/ai/inspector.ts): the fixtures, a live turn and the recording script share it.

Sources. recorded: scripts/ai/record-turn-fixture.ts reads a real dev turn through the same owner-guarded routes the page uses — with the session cookie (--cookie), or from the four responses saved out of a signed-in browser (--from <dir>; the cookie is httpOnly, so the page's own fetches are the door that copies no secret) — renames every id with scrubTurn() (demo_*; the leak gate refuses UUIDs, createId shapes and key shapes) and writes the fixture module (--surface deskbot --out … for the deskbot's two). authored: the hand-written stand-in a fixture is until a turn has persisted — its identity, guidance, tool definitions and the composer's own blocks (catalog map and current page; permissions, workspace, desk context and layout) are the profile's real texts; its ranking, timings, model calls, plan and receipts are not, and the provenance strip says so. The deskbot page carries two: the plan halt (proposed through desk_propose_plan, then approved and run — receipts and all) and the sentinel denial (a gated tool refused before anyone was asked; the proposal is still pending). live: the viewer's own turn, fetched by id — from the picker, from "Inspect your latest Vely turn" on the chatbot page (its ids ride on the streamed snapshot), or from the "Inspect this turn" link every finished answer carries in Vely and in the desk's chat panel (?conversation=…&turn=…#orchestration; the owner-guarded routes decide what the ids may show).

The probe (POST /api/ai/context-probe, ContextProbe.svelte, the galaxy) is gone: it was a second run reporting ids and counts. The frame scrubber (TracePlayer, reduceTurn, the SurfaceReplay frame fixtures) is gone from both pages (decision D4, then Phase 5): a recorded turn is read, not replayed.

What it closes

• P5 of the plan: the trace was ephemeral, partial and gated.
• Chatbot tool calls are persisted; reloaded threads show tool rows and citation chips.
• The fabricated vectorHits/bm25Hits split and the degraded pipeline:chunks event are gone with the pipeline:* events; the recorder is the only trace author.
• P3: the prompt tape reads a turn's recorded blocks; the PROMPT_BLOCKS mirror and its text-scan drift test are gone.
• P6: the showcase shows real turns — recorded, or the viewer's own — never a simulation.
• The docs candidate pool is recorded from the real turn (12 candidates, top 4 in the prompt), so "what ranked below the cutoff" no longer needs a second run.

Not recorded

Citation quote matching (a verbatim span of a grounding body in the answer) has a place in the contract and no producer yet; provider_source waits for a provider that returns grounding metadata. The approval replay (POST /api/ai/proposals/[id]/approve) is a separate request and leaves no model call — its account is the proposal's receipts. The parent-child and graph tiers return no chunk place yet (RankedChunk leaves it absent), so a deskbot turn's desk chunks carry no ancestry. A model call recorded before 2026-09-13 has no toolResultIds: the graph draws no result edge into it and says so.