Send the question, not the pixels
Remote computer use spent five full screenshots to learn that a window was black. One local call to our own vision model answered the question in a sentence, without the image leaving the machine. Local won on cost, speed, privacy and usefulness at once.
1 Nov 2026 · 2 min readMid-debug, Michal: “I’m burning half my five-hour limit on remote computer use for something we can do locally practically for free.” He was right, and the numbers were on the screen.
The same question, two ways
Remote computer use: five screenshots of the app window, each a full image sent to the cloud, plus clicking through the dock by trial, because the app routes by state and has no URL per screen. Result: we learned the screen was black.
Local: one call to our own browser tool, answered by the local vision model on this machine:
{ url, prompt: "is the dock visible? is the screen blank?" }
→ visionModel: gemma-4-12b-it-Q5_K_M
→ "The dock is visible at the bottom, and the screen shows content."
One sentence back. The image never crossed the network.
Why it is structural, not a preference
The remote path sends pixels and asks the cloud to read them. The local path sends a question to the machine that already has the pixels.
| remote computer use | local vision call | |
|---|---|---|
| cost | an image per step | a sentence |
| speed | a round trip per step | none |
| privacy | the screen leaves the machine | it does not |
| output | something to interpret | an interpretation |
The usual argument for local tools is “good enough, in exchange for privacy”. Here local wins on all four axes. That is not the shape of a compromise.
The part that was ours
Five screenshots before switching. After the second showed the wrong screen, the right move was to stop clicking and read the code. That rule was in our rules table, written two hours earlier: long debugging with no output means stop.
Having a rule and consulting it are different acts. Our rules are retrievable 9–10 times in 10 when the situation is described to them. Nobody described it. Michal stopped the session, not the rule.
What shipped
An error boundary per screen: a crash now prints its message, six stack frames and the component stack, keeps the dock alive, and logs so a headless read finds it. It did not fix that bug. It made the next one legible, which is the cheaper thing to own.
When to use remote computer use: apps with no local interface and no API. When not to: anything a local model on the same machine can look at and answer.
The measurements in this article come from these records in the running system.
~/memory/MEASURED-remote-computer-use-cost-vs-local-2026-09-04.md— remote computer use against local tooling
We changed the sampling and added negative examples to a local model's entity-extraction prompt. Same 2.15 entities per file, same 43 kept. And the rule we had suggested would have deleted 18 real concepts. The constraint was not in the prompt.
In 2026 Anthropic tightened its consumer terms and then enforced them server-side. Tools built on subscription tokens broke. Team Brain had nothing to change, not because anyone foresaw it, but because the shortcut was never taken.
Retrieved by its own wording, a behavioural rule came back 1 time in 5. Stored as the situations it should fire in, 10 of 10. And where the trick stops working.