$ cat projects/dottr-agent.md
mode: exploit· Dottr· 2026 — presentreward: in production

An agent that has to be honest about what it did

The Claude-on-Bedrock agent behind Dottr's chat and email: dozens of tools, a layered honesty guard, an eval harness that replays real flows, and a streaming design that took time to first token from tens of seconds to under two.

Claude on Bedrocktool callingevalsprompt cachingstreaming
dottr.ai ↗
$ ./replay honesty-guard

The reply streams right away. The guard reads it as it arrives and checks every claim against records the system owns. A claim with no record behind it is struck out and corrected.

you
Send the NDA to Priya at Northwind for signature.
agent
guardidle

    Simulated: names and data are invented and the timing is slowed so it can be read.

    ## Problem

    A contract-intelligence agent reads, fills, routes and signs documents for paying customers. The demo is easy. The hard part is an agent that says it sent something it did not, or takes an action nobody authorised, or makes the user wait so long the product feels broken.

    ## How it is built

    • ›Tool calling across dozens of tools, with context assembled to keep the prompt cache warm so long conversations stay fast and cheap.
    • ›A layered honesty guard: before the user sees a claim like 'sent to the counterparty', the claim is checked against records the system actually owns. If the check fails, the claim is retracted.
    • ›An eval harness built from recorded real flows, replayed in CI against every prompt and model change, so a regression in planning or tool use fails a build instead of reaching an inbox.
    • ›Streaming with concurrent auditing: the reply starts immediately while the guard runs alongside it. Time to first token went from tens of seconds to under two, without giving up the guarantee.

    ## Decisions I would defend

    • ›Guard after, not before. Blocking the reply on the audit was correct and unusable. Streaming and retracting is correct and fast.
    • ›Replay real flows, not synthetic prompts. The failures that matter come from what customers actually do, and a harness of invented cases never found them.
    • ›Gate deploys on the suite. Once the harness existed, letting a red build ship would have thrown away its whole value.