Cluster 08 · Axis I

Measuring your own system

Gates, negative controls, and when a test lies.

08 · MEASURING YOUR OWN SYSTEMA 200 carrying ok:false

Before rewriting our research, we probed every read tool by content, not by HTTP status. Thirty-odd tools worked. Four defects surfaced, one of which was not a defect: we had reported a number without reading the design behind it.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMA system must first see itself

For three and a half hours our agent layer recorded none of five local-model runs. A network sensor that knows nothing about agents recorded all five, to the minute. Self-report was not wrong. It was absent, and absence reads as 'nothing happened'.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMHow we report

Every figure on this site comes from a running system. The standard, the time it failed us by a factor of eighteen, and where the numbers come from.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMRight content, wrong source: how a local model chooses what to cite

Una gave the correct physics and cited the encyclopedia instead of the primary paper sitting next to it. Four hypotheses, each tested by changing one variable. She does not pick the most authoritative source. She cites the one whose wording she reused.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMThe test that measured a setting production never uses

Two passing evaluations in one day, both blind to the change they were meant to judge. What they missed, and the rule we took from it.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMTwo vision models invented the same dock

A paid vision model listed ten apps that do not exist. Our local model counted fourteen icons where there were eleven. Both were wrong about the same 12-pixel detail, and chasing it found a real bug nobody would have seen.

1 Nov 2026