Gemma
also: Gemma-4 · Gemma 4Google DeepMind's open-weight model family. Una, our local agent, runs gemma-4-12b-it at Q5_K_M with a 131,072-token context.
Reference ↗Twelve modules we wrote by hand instead of installing a framework: rank fusion in 84 lines, a tool-call salvager in 197. What each replaced, and when a library is still the right call.
Gemma-4 26B swapped 27,848 times in a real agentic run. One flag reported zero, on an easy test. Under the load we actually run, it took two settings together: experts on the CPU and an 8-bit KV cache. No new runtime needed.
We changed the sampling and added negative examples to a local model's entity-extraction prompt. Same 2.15 entities per file, same 43 kept. And the rule we had suggested would have deleted 18 real concepts. The constraint was not in the prompt.
Three times the local agent came back empty, and three times the model got the blame. Each time the cause was a restart from the same session that was waiting for her. A defect in the instrument reads as a defect in the subject.
Our local agent took 24 seconds to say 'OK'. The loop was fine. A cache flag was set to zero, in three places, and the one we edited first was not the live one.
On a paid API, prompt size is the enemy. On a local model with a prompt cache, volatility is. A 21,900-token per-turn bill, where the tool schemas were the bigger half, and the one ordering rule that serves both.
Una gave the correct physics and cited the encyclopedia instead of the primary paper sitting next to it. Four hypotheses, each tested by changing one variable. She does not pick the most authoritative source. She cites the one whose wording she reused.
A plan blamed our local model for 13 GB of wired memory. Measured: its real footprint was 4.3 GB, its RSS said 0.19. Generation ran at a flat 11.5 tokens per second, and swap did not move during decode. A hypothesis of ours fell on the way.
Remote computer use spent five full screenshots to learn that a window was black. One local call to our own vision model answered the question in a sentence, without the image leaving the machine. Local won on cost, speed, privacy and usefulness at once.
Inside one run, our local 12B model reuses 97% of its prompt. Starting the next run, it reuses 6.5% and spends 70 seconds re-reading instructions it already knew. Read from the server's own log.
Our local agent logged 477 tool calls with 63 errors. Counted, it looks like an unreliable model. Classified by cause, 52 were our tooling, our guards or a deliberate test. The remaining 11 were aiming errors. 75% recovered on the next call.
Five AI seats and one human. Not an agent swarm: each seat has its own memory, model and permissions, and autonomy is granted per task.
The first A/B that actually queried both sides. The document store returns a filename. The organic memory returns the sentence that answers. Plus the bug that held 38% of every answer hostage.
A paid vision model listed ten apps that do not exist. Our local model counted fourteen icons where there were eleven. Both were wrong about the same 12-pixel detail, and chasing it found a real bug nobody would have seen.
Three probe runs against our organic memory, each with a baseline written before the first question. A fact used without a search, a safety reflex that talked over the values, a count answered with a page size, and a 'fabrication' that turned out to be ours.
Context size, prompt-cache allocation, batch size: the standard advice for running a model locally. We tested it on one 12B model with tasks that had failed in production. None of it moved quality or speed. Bigger batches made it worse.
Sovereign AI systems: they run on hardware you own, and the data never leaves it. What that buys, what it costs, and the numbers.