Silent failure
also: green counterA component that reports success over work it never did. Our recurring bug class.
Before rewriting our research, we probed every read tool by content, not by HTTP status. Thirty-odd tools worked. Four defects surfaced, one of which was not a defect: we had reported a number without reading the design behind it.
Every counter said 1,047 memories were indexed. 653 of them were their opening 4,000 characters. 72% of the text never reached the vector store.
One recurring bug shape, found at six sites in one recovery run: a component that reports success over work it never did. None had a check that could go red.
For three and a half hours our agent layer recorded none of five local-model runs. A network sensor that knows nothing about agents recorded all five, to the minute. Self-report was not wrong. It was absent, and absence reads as 'nothing happened'.
Our planning database had 1,061 nodes and nothing written in seven weeks. Meanwhile the real planning ran in a chat thread. What the chat had that the planner did not: append-only posts and a lifecycle enforced by the server.
12 MB of vectors, a 39 GB index, zero dead rows. A nightly sync updated every row whether or not anything had changed, and a vector index never gives back space from updates. Ten million no-op writes, found by one ratio.
Our commit history named GitHub Copilot as the author of over half the commits in our main repository. It wrote none of them. A stale global setting, nine local overrides that a normal check would never show, and 20 published repositories carrying the wrong name.
Every figure on this site comes from a running system. The standard, the time it failed us by a factor of eighteen, and where the numbers come from.
Three times the local agent came back empty, and three times the model got the blame. Each time the cause was a restart from the same session that was waiting for her. A defect in the instrument reads as a defect in the subject.
A plan blamed our local model for 13 GB of wired memory. Measured: its real footprint was 4.3 GB, its RSS said 0.19. Generation ran at a flat 11.5 tokens per second, and swap did not move during decode. A hypothesis of ours fell on the way.
Our local agent logged 477 tool calls with 63 errors. Counted, it looks like an unreliable model. Classified by cause, 52 were our tooling, our guards or a deliberate test. The remaining 11 were aiming errors. 75% recovered on the next call.
Narrated product demos, generated fully locally: speech synthesis, a scripted browser, ffmpeg. The first video had 12 seconds of voice over 8 seconds of picture. The one that passed every gate was 77 seconds of a frozen frame.
For five months the backup script's header promised hard links and only changed bytes. Every file in every snapshot had a link count of 1: 24 full copies, 756 GB. One path argument was relative to the wrong directory.
Our unified search tool says 'prefer this over individual tools'. Our entire research corpus was not one of its layers. On six topics that live only there, the direct search won 6 of 6. And 75% of our memory had never been retrieved even once.
Every night our memory runs a consolidation phase and reports ok:true. It runs. It also could only ever reach 0.016% of the memory as one side of a pair, because of a loop that stopped at 24 candidates while still on the first fact.
Five AI seats and one human. Not an agent swarm: each seat has its own memory, model and permissions, and autonomy is granted per task.
From another machine on the network, with a hostile Origin header and no authentication, our API returned our most protected memory record. Two defaults, each documented and harmless alone. The development convenience was the deployed posture.
Michal: 'it felt like one infinite session, then it stopped.' One commit had removed the only line that loaded the continuity file. The file was still written before every compaction, and never read after. Three Claude Code hook details decided the fix.
A commit hook had been checking every commit for leaked credentials for months. Tested against nine real credential shapes, it caught one. A pattern in its own list could never match, and the error went to /dev/null.
A paid vision model listed ten apps that do not exist. Our local model counted fourteen icons where there were eleven. Both were wrong about the same 12-pixel detail, and chasing it found a real bug nobody would have seen.