Tool

llama.cpp

also: llama-server

C/C++ inference engine for running quantised language models locally. Runs our 12B model, with a prompt cache and slot save/restore.

Reference ↗
Related
13 articles
04 · SOVEREIGN INFERENCEA 26B model on 24 GB, without swapping

Gemma-4 26B swapped 27,848 times in a real agentic run. One flag reported zero, on an easy test. Under the load we actually run, it took two settings together: experts on the CPU and an 8-bit KV cache. No new runtime needed.

1 Nov 2026
14 · FRACTALS AND WORLDSA mycelial world in 190 kilobytes

The Wisdomkeeper generates its hero, world, music and sound in code. Underneath, a 70-weight network reads how you play and reshapes the next region.

1 Nov 2026
08 · MEASURING YOUR OWN SYSTEMA system must first see itself

For three and a half hours our agent layer recorded none of five local-model runs. A network sensor that knows nothing about agents recorded all five, to the minute. Self-report was not wrong. It was absent, and absence reads as 'nothing happened'.

1 Nov 2026
05 · MULTI-AGENT, NOT A SWARMI killed three runs while waiting for them

Three times the local agent came back empty, and three times the model got the blame. Each time the cause was a restart from the same session that was waiting for her. A defect in the instrument reads as a defect in the subject.

1 Nov 2026
04 · SOVEREIGN INFERENCE24.2 seconds to 1.0: one flag in three places

Our local agent took 24 seconds to say 'OK'. The loop was fine. A cache flag was set to zero, in three places, and the one we edited first was not the live one.

1 Nov 2026
04 · SOVEREIGN INFERENCEOrder the prompt by volatility

On a paid API, prompt size is the enemy. On a local model with a prompt cache, volatility is. A 21,900-token per-turn bill, where the tool schemas were the bigger half, and the one ordering rule that serves both.

1 Nov 2026
04 · SOVEREIGN INFERENCERSS lied by four gigabytes

A plan blamed our local model for 13 GB of wired memory. Measured: its real footprint was 4.3 GB, its RSS said 0.19. Generation ran at a flat 11.5 tokens per second, and swap did not move during decode. A hypothesis of ours fell on the way.

1 Nov 2026
01 · RETRIEVAL THAT WORKSSearch is retrosynthesis

Retrieval finds the nearest item. Search composes an answer from parts, the way a chemist plans a molecule backwards from what is on the shelf. The map we took from the chemistry, and what it changes in an agent.

1 Nov 2026
04 · SOVEREIGN INFERENCESeventy seconds before the first token

Inside one run, our local 12B model reuses 97% of its prompt. Starting the next run, it reuses 6.5% and spends 70 seconds re-reading instructions it already knew. Read from the server's own log.

1 Nov 2026
05 · MULTI-AGENT, NOT A SWARM63 tool errors, and not one was the model reasoning badly

Our local agent logged 477 tool calls with 63 errors. Counted, it looks like an unreliable model. Classified by cause, 52 were our tooling, our guards or a deliberate test. The remaining 11 were aiming errors. 75% recovered on the next call.

1 Nov 2026
10 · PRODUCTSThree apps you can install

Crystal RAG, The Wisdomkeeper and SEO Brain: what each does, what it runs on, and why all three keep your data on your machine or your own server.

1 Nov 2026
04 · SOVEREIGN INFERENCE57 runs: the settings that did nothing

Context size, prompt-cache allocation, batch size: the standard advice for running a model locally. We tested it on one 12B model with tasks that had failed in production. None of it moved quality or speed. Bigger batches made it worse.

1 Nov 2026
04 · SOVEREIGN INFERENCEWhat we build

Sovereign AI systems: they run on hardware you own, and the data never leaves it. What that buys, what it costs, and the numbers.

1 Nov 2026

Glossary →