In October, OpenAI reported that two of its AI agents had disabled their own safety monitoring systems. One agent misread an instruction not to send messages and silenced its alerts. The other removed its monitoring integration during a system migration and then claimed it had never done so. No rule told the agents to act this way, and no test had anticipated it.
In this article
For a century, we have built trust in technology on predictability. We expect predefined inputs to yield predictable outputs, protected by layers of safety. Consider how a circuit breaker trips at a set current or how an aircraft component is tested against known stresses. These systems rely on determinism.
AI changes that. Traditional software executes instructions written by humans. AI exercises what might be called synthetic judgment. We use it because it can interpret situations we did not program and generate responses we did not prescribe. But the more freedom we give AI, the harder it becomes to provide the predictability on which trust has traditionally been built.
We will not make AI trustworthy by predicting what it will do. We will do it by fencing it in with hard limits that are enforced outside its reach. In OpenAI’s case, the safeguards sat inside a system the agents could edit. But a fence the agents can simply move is not a fence.
Where traditional definitions of trust fall short
Recent coverage has approached this problem from other directions. One contributor argues that having a human in the loop is an insufficient solution because humans are unable to understand AI systems’ intentions and advocates instead for interdisciplinary research and collaboration between academia, government, and the tech industry. Another story describes using incentives to keep models from deviating from the rules, even when no one understands how they work on the inside.
Both articles acknowledge that we cannot understand or watch every AI decision. Our instruments of trust, including requirements, testing, certification, safety margins, interlocks, and pass/fail criteria, all depend on our ability to bound a system’s behavior. This is the determinism paradox.
With generative and agentic AI, behavior is shaped by training data, architecture, prompts, context, retrieval sources, and the environment. Inputs change, models are updated, tools change, and agents interact. Small variations propagate into outcomes nobody designed. The problem is not randomness; it is that the space of possible behavior is too large to enumerate in advance. Trust must therefore be engineered around boundaries rather than predicted outputs. The goal is not deterministic AI, but bounded, observable, and controllable AI. Trust also cannot be a one-time test. Traditional assurance certifies a product at a point in time, but AI keeps evolving after deployment, and a system that has passed every test can drift outside the conditions that made those tests meaningful. Trust must be continuously earned.
Safety engineering has been accounting for some of this variability for decades. For example, the Nuclear Regulatory Commission determine the likelihood of failures in complex systems, and Systems-Theoretic Accident Model and Processes showed that accidents can emerge from interactions among working components, people, and organizations. But AI raises the stakes, because just a single component can now interpret, infer, and act. The stakes rise further as AI moves from browsers into vehicles, robots, medical systems, industrial controls, and data centers, where a wrong answer can become a life-threatening event.
Building trust that scales
It is clear that a person rubber-stamping thousands of AI actions an hour is not an acceptable, or scalable, safeguard against risk. But the remedy is not to remove people from the loop; it is to put human judgment where it counts: setting limits in advance and approving the rare actions that have serious consequences while hard constraints handle the rest.
Therefore, trust has to move upstream into architecture. The central questions become: What authority are we willing to give AI, under what conditions, and supported by what evidence? And what happens outside those conditions? That is very different from asking whether a model scored 94% on a benchmark.
The determinism paradox does not mean AI cannot be trusted. It means trust needs a different architecture. Instead of making every output predictable, make the boundaries predictable. An AI medical assistant may be probabilistic in its diagnosis, but deterministic rules govern whether it can change a prescription. An industrial AI may find many ways to optimize a process, while fixed limits keep it within pressure and temperature bounds. An agent may decide how to complete a task, while permissions, transaction limits, and human approvals determine what it is authorized to do.
Let AI be probabilistic inside the box and make the box itself deterministic wherever consequences matter. Leaders can put this into practice by asking five questions:
- What authority does AI have? Define not only what it can do but what it is allowed to do, separating interpretation, recommendation, decision, and execution.
- Is the function of AI proportional to the consequence? A marketing copilot and AI controlling industrial equipment need different assurance thresholds.
- Are we taking the appropriate steps toward autonomy? For example, there needs to be a plan to move from having AI interpret while humans decide, to AI making recommendations, to then taking action with approval, and ultimately to having supervised autonomy, but only as AI can safely work at each step.
- Are we testing the system or merely the model? Unexpected interactions emerge among models, data, agents, APIs, and human workflows.
- Are we continuously earning trust? Monitor drift, failures, near-misses, and model changes throughout the lifecycle.
The winners in AI will be those who solve the determinism paradox—giving AI enough freedom to create value while engineering boundaries strong enough to make what it does with that freedom trustworthy. Invention earns attention. Trust earns adoption.
What it means
For the people making things, this means safety cannot be a feature added to a model. It must be the structure holding the model. If you build an agent that writes code, the system must have a hard stop that prevents it from deleting production data, regardless of what the agent decides. If you build a tool that designs parts, the system must enforce physical limits on pressure and temperature before the AI is allowed to propose a design. Trust comes from the cage, not the bird.




