Local-First AI: Keep the Model Off the Critical Path
Local-First AI: Keep the Model Off the Critical Path
Most AI tools you use today have a hard dependency buried in them: they don’t work without a round trip to someone else’s server. Close the laptop lid on a flight, drop to one bar of signal, or hit a provider outage, and the “intelligent” part of the software goes dark. We’ve come to treat that as the natural cost of good models. It isn’t. The argument of this piece is narrow and, I think, load-bearing: a trustworthy agent should run local-first — against a model on your own machine — with hosted models available on tap, not sitting on the critical path. Local-first AI isn’t a nostalgia trip or a privacy garnish. It’s continuous with everything that makes an agent worth trusting in the first place.
What “local-first” actually means
Local-first is a term of art, and it’s worth being precise about it because the marketing version has already been diluted. It does not mean “we cache some things for offline.” It means the primary copy of your work and the primary loop of the software live on your machine, and the network is an enhancement you reach for — not a lifeline you can’t function without.
Applied to an agent, local-first means the default reasoning happens against a model that runs where your files are. The agent can read your directory, draft the change, and show you the result without any of it leaving the machine. When you want a frontier hosted model — for a hard problem where the biggest model genuinely earns its keep — that’s a capability you invoke, deliberately, for that task. The hosted model is on tap. It is not the tap.
The distinction between “on tap” and “on the critical path” is the whole ballgame. On the critical path means nothing works unless the request reaches the server: no connection, no agent. On tap means the local path is the default and the hosted call is an escalation you opt into. Same access to big models. Completely different failure mode, and completely different trust posture.
The trust asterisk when everything ships to a server
Here’s why this isn’t a side preference. If you care about agent trust — least privilege, containment, accountability, a boundary you can actually see — then where the model runs is not orthogonal to that. It’s part of it.
Consider what it means when the default is “ship the context to a hosted model.” To do useful work, the agent sends your files, your prompts, and often your surrounding context off the machine. Now every trust property you wanted has a footnote. Least privilege? Except for everything we transmitted. Containment? Except that the interesting part happened somewhere you can’t inspect. Accountability? Except the reasoning occurred on infrastructure you don’t control and can’t replay. None of those asterisks is necessarily catastrophic, but they are real, and they compound. A system whose every consequential thought happens off-box is a system whose trust boundary is drawn around someone else’s data center.
Keeping the default local collapses the asterisks. When the model runs where the files are, the trust boundary is the machine — a thing you own, can inspect, and can reason about. That’s the same argument that puts agent trust at the OS layer: trust is a property of the environment the agent runs in, and “the environment” includes where the intelligence physically executes. Local-first is what that principle looks like when you follow it down to the model.
Hosted-on-tap versus hosted-required
I want to be careful not to overclaim, because the honest position isn’t “local models for everything, forever.” Local models have gotten dramatically more capable, and for a large and growing share of everyday agent work they’re more than enough. But the frontier hosted models are still the frontier, and there are tasks where you’ll want them. Pretending otherwise would be the same dishonesty as pretending the network never fails.
The point is the arrangement, not abstinence. Hosted-required means the vendor decided, on your behalf, that your work always transits their servers — and you inherit their latency, their outages, their pricing, and their view of your data as the price of admission. Hosted-on-tap means you decided, for a specific task, that this one is worth an outbound call. The first is a dependency. The second is a choice. An agent-native system should default to the choice.
The practical consequences fall out of that arrangement directly:
- Latency. The common case — read a file, make an edit, answer a question about your own project — runs at local speed, with no round-trip tax on every interaction.
- Cost. You’re not metering every keystroke of routine work against a per-token bill. The expensive models get called when they’re worth it.
- Sovereignty. Your default working set stays on your hardware. What leaves does so because you sent it, for a reason, not because the software had no other mode.
- Continuity. The agent keeps working through a dead connection or a provider outage, because the default path never depended on the connection being up.
The takeaway
Local-first AI is easy to file under “privacy feature” and move on. That undersells it. Where the model runs determines whether your trust boundary is a machine you control or a server you don’t; whether the agent survives a lost connection or dies with it; whether routine work is fast and free or metered and remote. Those aren’t separate wins — they’re the same design decision seen from four angles.
That decision is one of the load-bearing properties of an agent-native operating system: a local model as the default reasoning loop, hosted frontier models on tap for when they earn it, and a trust boundary that stays somewhere you can point at. We’re building it in the open and we’re phase-honest about the road — the alpha replaces the desktop session, not the kernel, and the full picture is a long build. But the direction is settled. Keep the model off the critical path. Reach for the big ones on purpose. Don’t let the default be “someone else’s server or nothing.”