Instructions Are Not Controls
Instructions Are Not Controls
Last week one of our own draft documents shipped to production carrying a comment that said, in its first line, strip this comment before publishing.
Nothing stripped it. The instruction had been sitting inside the artifact it was supposed to protect, addressed to whoever happened to be reading, and for a while that had worked — because a person read the file each time, saw the line, and did the thing. Then a build step read the file instead. The build step had no opinion about the sentence. It rendered it, faithfully, into a public page.
The failure took about forty minutes to find and fix. What it is worth is the shape of it, because that shape is the central problem in running agents on real machines, and it is much older than agents.
The distinction
An instruction is a sentence that describes what should happen. A control is a mechanism that determines what can happen.
They look interchangeable when a careful human is in the loop, because a careful human converts instructions into controls continuously and for free. You tell a new colleague “don’t push straight to main” and, most of the time, they don’t. The sentence did the work. It is easy to conclude that the sentence was the safeguard.
It wasn’t. The colleague was. The sentence was a way of loading a policy into a system that already had judgment in it. Remove the judgment and you are left with a sentence, and a sentence enforces nothing.
Branch protection is a control. The Slack message asking people to be careful is an instruction. Both may be present in a healthy team, and only one of them is load-bearing.
Why this bites harder with agents
Every agent system in production today is governed substantially by instructions. The system prompt is an instruction. The tool description that says “only use this for read-only queries” is an instruction. The line in the runbook that says “never run this against prod” is an instruction. They work often enough to be seductive.
They fail differently than they fail with people, in three ways that matter.
An agent does not feel the weight of a rule. A person who reads “do not delete anything” carries that forward with a low-grade unease that survives context changes. A model carries it as tokens, in competition with every other token, and the competition is not weighted by consequence. A rule stated once at the top of a long context is not, mechanically, more important than the sentence right before the action.
Instructions are attackable in a way controls are not. Once the boundary is made of language, anything that can inject language can argue with it. A web page the agent reads, a file name, a code comment, an issue title — any of these can carry a sentence, and your rule now has to win an argument it did not know it was having. This is the confused deputy problem in modern dress: the agent has authority, something else supplies the intent, and the two are joined by nothing stronger than text. OWASP put prompt injection at the top of its LLM risk list for exactly this reason.
Nobody notices when an instruction stops being followed. A control that fails throws. An instruction that fails produces a completed task with a slightly different outcome, and the log looks fine. This is the same failure family as a terminal state that asserts more than it verified — the system reports success because nothing in it was positioned to disagree.
The test
There is one question that sorts instructions from controls, and it takes a few seconds:
If the component reading this were adversarial, indifferent, or simply confused, would the outcome change?
If the answer is yes, you have an instruction. That is not automatically a problem — most of a working system is instructions, and it would be paralysing to make it otherwise. It is a problem when you have been counting on it, and the giveaway is usually that you cannot name the mechanism. “We tell it not to” is not a mechanism. “It cannot, because the capability was never granted” is.
Our comment failed the test on the first clause. The build step was not adversarial. It was indifferent, which was plenty.
What a control looks like
The fix was not a better comment. A better comment would have been the same object, more emphatic — and emphasis is a property of sentences, not of systems.
The fix was to move the requirement into the only place that always runs: the renderer now removes those blocks before conversion, and a separate verifier asserts, independently, that no internal marker appears in any published page. One does the work; the other refuses to believe it without checking. That second part matters more than it looks. A fix you have not verified is itself an instruction — a claim about what the system does now, addressed to the future, with nothing enforcing it.
Applied to agents, the same move produces something recognisable. You stop writing “don’t touch the filesystem outside the project” into a prompt, and you start handing the agent a capability that is scoped to the project and cannot be widened from inside. The sentence becomes unnecessary rather than merely well-intentioned. We’ve written about why that grant belongs in the operating system rather than in each application: the OS is the layer that can actually refuse, and refusal is the thing being purchased.
There is a useful corollary here about the safeguards themselves. In the same pass that produced this fix, a checker in our pipeline reported a loud failure against output that was correct — it had not been taught that some approved documents are deliberately held back until a future date, so it read a scheduled absence as a deletion. The cheapest way to make that alarm green would have been to publish the held document early. That is what a false alarm actually costs: not the noise, but the fact that the shortest path to silence usually runs through the unsafe door. We inverted the check rather than relaxing it, so that the held item is now asserted absent instead of expected present.
The part that doesn’t generalise cleanly
Not everything can be a control, and pretending otherwise produces a system so rigid that people route around it — which is its own security failure, just one that happens off the record.
Judgement, taste, tone, “is this a good idea” — these stay instructions, because there is no mechanism that decides them. The discipline is not to convert everything. It is to know which of your safeguards are sentences, and to stop describing those as if they were walls.
We are early. MagenticOS is alpha, targeting a reference runtime and agentic shell by the end of the year, and there are more sentences in our stack right now than either of us would like. The claim here is not that we have solved this. It is that “we told it not to” should feel, when you say it out loud, like an admission rather than an answer.
Write down what enforces it. If nothing does, you have a note, not a guarantee.