Negative control
A probe that must not change when the thing under test changes. Exact-name lookup is ours for any ranking prior.
Before rewriting our research, we probed every read tool by content, not by HTTP status. Thirty-odd tools worked. Four defects surfaced, one of which was not a defect: we had reported a number without reading the design behind it.
Gemma-4 26B swapped 27,848 times in a real agentic run. One flag reported zero, on an easy test. Under the load we actually run, it took two settings together: experts on the CPU and an 8-bit KV cache. No new runtime needed.
One recurring bug shape, found at six sites in one recovery run: a component that reports success over work it never did. None had a check that could go red.
We changed the sampling and added negative examples to a local model's entity-extraction prompt. Same 2.15 entities per file, same 43 kept. And the rule we had suggested would have deleted 18 real concepts. The constraint was not in the prompt.
For three and a half hours our agent layer recorded none of five local-model runs. A network sensor that knows nothing about agents recorded all five, to the minute. Self-report was not wrong. It was absent, and absence reads as 'nothing happened'.
Our embedder and both rerankers we tested are sold as multilingual. Measured with paired Slovak and English queries, Slovak failed 55% of the time against 20%, and a cross-encoder gave three different documents the identical score.
We clustered 1,046 knowledge-graph entities and found nothing, at every k, with two methods. Two clean hypotheses for why were falsified. The structure was there all along, in a different representation.
Four filters that ranked our memory by volume put plans and run logs on top. Measuring novelty instead put them at the bottom, where they belong. Then the criterion hit a wall: our register of known concepts was a list of filenames.
Una gave the correct physics and cited the encyclopedia instead of the primary paper sitting next to it. Four hypotheses, each tested by changing one variable. She does not pick the most authoritative source. She cites the one whose wording she reused.
A plan blamed our local model for 13 GB of wired memory. Measured: its real footprint was 4.3 GB, its RSS said 0.19. Generation ran at a flat 11.5 tokens per second, and swap did not move during decode. A hypothesis of ours fell on the way.
Narrated product demos, generated fully locally: speech synthesis, a scripted browser, ffmpeg. The first video had 12 seconds of voice over 8 seconds of picture. The one that passed every gate was 77 seconds of a frozen frame.
For five months the backup script's header promised hard links and only changed bytes. Every file in every snapshot had a link count of 1: 24 full copies, 756 GB. One path argument was relative to the wrong directory.
Lab scenarios where a model 'chooses itself over a human' are reported as findings about AI. Read as experiments, they measure the cage they are set in.
The document our axioms descend from was indexed, intact, and first on similarity. A guessed importance score buried it. 683 files carry importance nobody measured. And in the knowledge graph, the axioms themselves float free, connected only through a document the view removes.
A blueprint reported a fractal-orbit optimizer reaching loss 0.0176 on two-spirals. We ran its code. The fractal term never switched on: in 36,000 steps the measured dimension never crossed the threshold that activates it.
From another machine on the network, with a hostile Origin header and no authentication, our API returned our most protected memory record. Two defaults, each documented and harmless alone. The development convenience was the deployed posture.
A commit hook had been checking every commit for leaked credentials for months. Tested against nine real credential shapes, it caught one. A pattern in its own list could never match, and the error went to /dev/null.
Two passing evaluations in one day, both blind to the change they were meant to judge. What they missed, and the rule we took from it.
Three probe runs against our organic memory, each with a baseline written before the first question. A fact used without a search, a safety reflex that talked over the values, a count answered with a page size, and a 'fabrication' that turned out to be ours.
Context size, prompt-cache allocation, batch size: the standard advice for running a model locally. We tested it on one 12B model with tasks that had failed in production. None of it moved quality or speed. Bigger batches made it worse.
Ask our memory for a record by its own name: 93% rank-1 on plain semantic search, 43% after the biological rescore. Why a smarter score lost the easiest query, and the fix we chose.