What I tried
Every time you message Sam, it sends a model a lot: who Sam is, what it knows about you, and the conversation so far. Model providers let you mark points in that prompt. Everything before a mark is stored, and the next message reads it back for about a tenth of the price.
Sam had three marks. The last one sat on Sam’s previous reply, so it covered the whole conversation. I tested each mark against the routes Sam’s messages actually take, with the same prompt, twice in a row, and read what each provider said it had stored and read.
What I found
The router in front of the models quietly dropped the last mark. It only keeps marks on a message’s text, and Sam’s reply carried its mark on the message itself. The two earlier marks worked, which is why the bill looked roughly right and nobody noticed: the conversation, the part that grows, was paid in full every single time.
Then the twist. Fixing the mark made one route worse. That provider only reads the cache at the last mark; if the last mark doesn’t match, it reads nothing at all, not even the parts that did. The other route reads everything the previous message stored, which is how caching is meant to work.
One more leak: after ten quiet minutes the router could send the next message to a different provider, where nothing was stored yet.
What I changed
The mark now goes where the router keeps it. Sam’s chat is pinned to route B, with route A as the fallback, so a pause can’t land it on a cold cache. And every message now records which provider answered and what it stored, so the next leak shows up in a chart instead of a bill.
What I’d do next
Watch the long turns. A provider only looks back about twenty steps for an earlier mark, so a turn where Sam uses a lot of tools can push the last mark out of reach. If the charts show misses after busy turns, the fix is one more mark in the right place.