Guardrails are model-side controls: system prompts, safety classifiers, moderation filters, and output constraints that shape what a model produces. Governance is action-side control: authorization, enforcement, and evidence around what an agent actually does to real systems. You need guardrails the moment you deploy a model. You need governance the moment the model's output starts triggering consequences, a payment, a deletion, an access grant, a message to a customer.
Different layers, different failure modes
| Dimension | Guardrails | Governance |
|---|---|---|
| Controls | Model inputs and outputs, which is to say text | Actions against real systems |
| Lives | Inside or beside the model | On the action path, between agent and system |
| Failure mode | Harmful, off-brand, or off-policy text | Unauthorized consequence: payment, deletion, access grant |
| Defeated by | Jailbreaks, prompt injection | Nothing the model says, because enforcement sits outside it |
| Evidence | Usually none beyond model logs | A verifiable record of every governed action |
| Typically owned by | The ML or product team shipping the model | Platform, security, and risk teams |
The uncomfortable fact about guardrails
Prompt injection sits at the top of the OWASP Top 10 for LLM applications, and it remains an open problem rather than a solved one[1]. The UK's AI Safety Institute reported that every leading model it evaluated remained vulnerable to basic jailbreaks[2]. None of this means guardrails are useless. It means their guarantees are statistical, and an agent with authority connected to untrusted input, which is any agent that reads tickets, emails, web pages, or documents, should be designed on the assumption that its model can sometimes be steered.
That assumption changes the design question. It stops being how do we prevent the model from ever trying something bad, and becomes what happens when it tries.
Governance assumes the model is compromised
A governance boundary evaluates the action, not the text. When an agent requests refund.create for $240, the boundary checks the agent's identity, its delegated authority, the applicable policy version, any required approval, and the budget state, then allows or denies. A successfully injected agent can ask for a bad action. The boundary's entire job is that asking is not getting: the action outside policy is denied, the credential the agent never held is never issued, and the attempt itself lands in the audit trail as evidence.
Use both, and match the control to the authority
Guardrails and governance are complements, not competitors. Guardrails keep content quality and safety high on every interaction, at low cost. Governance carries the weight wherever an agent holds authority over systems. Gartner has warned that applying one uniform governance regime across every agent is itself a failure mode for enterprise agent programs[3], and the point cuts both ways: a read-only research assistant does not need payment-grade controls, and a payout agent must not get by on a system prompt. A workable rule of thumb:
- The agent only converses: guardrails are enough.
- The agent reads internal data: guardrails plus scoped, least-privilege access.
- The agent writes, pays, provisions, or messages externally: full governance: authorization before, brokered credentials during, verifiable evidence after, with budgets in the decision.
Where FeirOS fits: FeirOS is the governance side of this split, a default-deny action boundary that works with any model and any framework, alongside whatever guardrails you already run. See FeirOS for organizations or open the live demo.
Sources
- OWASP GenAI Security Project, Top 10 for LLM Applications: LLM01 Prompt Injection.
- UK AI Safety Institute, Advanced AI evaluations: May update, 2024.
- Gartner, "Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure", May 2026.