How v10r keeps a document a user ingested from steering the assistant that reads it — the "lethal trifecta" of prompt injection, and the code that holds each of its three legs.
Untrusted-content boundary
Problem. An assistant that (1) reads content someone else could have written, (2) can reach private data, and (3) can send something out — a fetch, an email, an image URL in its answer — can be made to exfiltrate that data by instructions hidden in the content. No detector reliably spots the instructions; the defence that holds is structural: bound the content, and take away a leg. The deskbot meets leg 1 on every turn — open panels, file reads and knowledge search are all user-controlled text, file names included.
Method. One rule per leg, each enforced in one place:
| Leg | Rule | Where |
|---|---|---|
| 1. Untrusted input | Every tool declares resultOrigin (system · user). A user result reaches the model as one escaped <untrusted-content source="…"> element — through AI SDK toModelOutput, so the loop's own value (approval sentinels, desk effects, the turn trace) stays raw. Results replayed from earlier turns get the same wrapping, failing closed on any shape (a compacted projection, an error, a multi-part output). Prompt blocks (<desk-context>, <retrieval-context>) were already escaped |
ai/untrusted-content.ts (withUntrustedBoundary in composeTurn, boundHistoryToolResults in the orchestrator); TOOL_MANIFEST in types/ai-tools.ts |
| 2. Exfiltration | Every tool declares reach (internal · external). An external tool must be approval-gated (requiresApproval(risk)). None exists today; because the label is self-declared, the gate also scans the tool modules and fails on a fetch, socket or delivery-channel import while every tool says internal |
untrusted-content.gate.test.ts |
| 3. Output channel | Model output renders with no <img> (the automatic beacon) and no link to a host outside TRUSTED_LINK_HOSTS; only http/https |
utils/markdown.ts (untrusted: true is the default) |
The data-not-instructions rule (profile/shared-rules.ts) names the tag, so the model is told
what the boundary means. That sentence is guidance; the escaping is the guarantee.
What is not claimed. An error the AI SDK raises while running a tool (invalid input) reaches the model as its own error-text, outside the boundary; it echoes model-produced input, not stored content. Whether a model obeys text inside the boundary is not testable and
not tested. Injection detectors and classifiers are not used: they are probabilistic, and a
missed detection on a turn with all three legs is the whole incident.
Invariants.
- Every manifest tool declares
resultOriginandreach; a tool nobody declared isuser. - No
user-origin result reaches the model unescaped, in the turn that ran it or a later one. - No tool with
reach: 'external'runs without a human-approved proposal. - Model output renders no image and no link to an unlisted host.
Adding a tool. Declare resultOrigin honestly — anything that echoes user text, even a
name, is user. A tool that sends anything outside the deployment is external and must be
write or destructive risk, so it goes through the approval gate.
Do not use the wrapper as the only defence on a surface that also has an unapproved
external tool: wrapping lowers the odds, removing a leg removes the attack.
Related: threat-model.md (the ingested-document adversary), harness-lens.md (the approval gate).