The Governance Gap in Agentic AI
In one billing system I worked on, a stakeholder repurposed an existing attribute to serve a completely different reporting requirement. The change was never formalized and the original meaning was never retired. The accounting team continued using the attribute in its original sense while the new process used it differently, and the result was incorrect revenue figures that took time to trace back to a semantic disagreement nobody had documented.
That kind of problem is manageable when humans are in the loop and can recognize the inconsistency. It becomes harder to manage when an AI system is acting on the same ambiguous attribute at scale, because the system has no way to know that the attribute means different things to different parts of the organization. The standard response is to call the system an autonomous agent and treat the ambiguity as an implementation detail, but that framing obscures who is responsible for the meaning the system is acting on and what authority it was actually given.
I think the more useful framing is AI Delegates.
Using "delegate" marks a different set of commitments about what AI systems do, what they cannot do, and where human responsibility remains necessary. The argument here is meant for architects, AI designers, and governance professionals who need language that holds up when systems are audited, challenged, or revised. It builds on the companion article What I Mean by Knowledge, Information, and Semantics.
AI Delegates: What the Term Means
An AI Delegate is a computational system that executes delegated intent within clearly defined human authority, grounded in explicit conceptual schemasAn explicit model of the distinctions that matter in a domain, object types, fact types, roles, and constraints, that exists independently of any implementation and defines what terms mean within an organizational context., while ownership of purpose, meaning, and accountability remains human and externally defined.
Delegated intent. The system does not generate its own purposes; it receives intent from humans and acts within the scope of that intent. This sounds straightforward; however, most systems that claim to act on behalf of users are actually executing logic that encodes intent implicitly, buried in code where nobody can inspect or revise it without a development cycle. Delegation only means something when the intent being delegated is explicit enough to be named, tested, and changed independently of the implementation.
Human-defined authority. The boundaries of what the system may do are set by humans, not discovered by the system, and that authority is always conditional and revocable.
Grounded in conceptual schemas. The system operates over explicit conceptual distinctions rather than raw data or implicit assumptions. The discipline, borrowed from DietzJan Dietz is a Dutch computer scientist whose Enterprise Ontology theory models organizations in terms of essential commitments and transactions rather than implementation details. His DEMO methodology identifies what an organization essentially does, independent of how it is currently implemented., is to model only what is essential and what is consequential if misunderstood; a schema bounded that way stays stable and auditable as the domain evolves. Without that grounding, a system may produce correct-looking results while operating on meanings nobody ever agreed on, which is the kind of failure that is hardest to detect and most expensive to correct.
Without owning semanticsThe layer of meaning in the semiotic ladder where signs become associated with concepts and symbols refer to things. In this article, semantics refers to the structured set of conceptual distinctions defined in a schema, not to natural language meaning or AI inference., purpose, or accountability. The meanings the system uses were defined elsewhere, the purposes it serves were set by others, and responsibility for its outcomes remains with the people who designed and deployed it.
When an AI Delegate's behavior is explicitly governed by modeled goals, situations, and constraints, we call it an Intent-Driven AI Delegate. This is a more specific term for systems where intent is not just implicit in code but explicitly represented, inspectable, and testable independently of implementation.
What This Framing Preserves
The obvious objection is that rejecting autonomy limits what AI can do.
AI Delegates can still:
- Interpret ambiguous input and propose structured responses
- Query conceptual schemas and detect inconsistencies
- Execute complex workflows within defined boundaries
- Incorporate feedback and adjust behavior over time, within predefined constraints
- Coordinate with other systems and other delegates
The difference is that these capabilities operate within explicit constraints, with accountability that remains human.
The three-layer architecture described below has been implemented in a working prototype using Object-Role ModelingA conceptual modeling language that represents a domain as object types connected by fact types and roles, with explicit constraints. Used here to define the conceptual schema layer independently of any database or implementation technology. for the conceptual schema, an explicit intent registry, and a delegate that accepts natural language requests, interprets them through an LLM, and executes them through symbolic validation with a full audit trail.
The appendix sketches a set of levels describing how AI systems can engage with meaning, from purely syntactic execution up to socially embedded coordination. The purpose is not to define a maturity ladder for all systems, but to clarify what kinds of semantic involvement carry different kinds of risk.
The Three-Layer Architecture
To make AI Delegates work in practice, we need an architecture that separates concerns cleanly. I think three layers are enough here, and each has a distinct responsibility.
Defines what distinctions matter in a domain. Contains object types, fact types, roles, constraints, and value types. In ORM terms, this is the conceptual schema. It captures meaning but does not determine action or control flow.
Defines why information is used and what is supposed to happen when certain conditions hold. Intent is where meaning becomes actionable, but still requires operational interpretation before execution. References the schema without altering it.
Decides how intents are carried out in practice. Interprets intents, queries conceptual schemas, applies logic and heuristics, and invokes actions. It consumes semantics rather than defining them, and receives intent rather than originating it.
The schema captures distinctions without determining behavior, intent connects meaning to purpose without blending them, and the agent executes within boundaries it did not set.
The Flow
- Conceptual Schema provides the vocabulary and constraints
- Intent specifies what should happen and why
- Agent determines how to fulfill the intent
- Feedback from outcomes may lead to schema or intent evolution, but only through human governance
This separation is important because it makes failures easier to diagnose and fix. When something goes wrong, you can determine whether the issue was semantic, intentional, or operational and if schema, intent, or agent fixes are required. Each correction targets a specific layer without cascading changes across the system.
This architecture also maps to the knowledge formula from the companion article: Knowledge = (Symbols + Structure + Shared Meaning) × IntentionFrom the companion article: knowledge is meaning multiplied by intention. Without a stated purpose the other components are present but inert. The formula shows that stripping out intention reduces the product to zero, not just less.. The conceptual schema provides the first three components while the intent layer provides intention and the agent layer operationalizes the combination. Without all three layers working together, you may have information, but you do not yet have accountable action.
Intent as a First-Class Concept
Most systems treat intent implicitly. Business logic encodes intent in code, but the intent itself is not visible or inspectable. When requirements change, you rewrite code rather than updating a versioned, modeled intent. The Intent-Driven approach models intent directly, rather than leaving it buried in code.
The Intent Meta-Model
At minimum, an intent model includes:
| Field | Description | Example |
|---|---|---|
| Intent | A named purpose | DetermineLegalAdulthood |
| Goal | What the intent aims to achieve | AssessEligibility |
| Situation | When the intent applies | ServiceAccessRequest |
| AgentRole | Who or what pursues the intent | EligibilityEvaluator |
| Outcome | What success looks like | EligibilityStatusDetermined |
| ConceptualElement | Which schema elements the intent references | Person, BirthDate |
Example: Age Eligibility
Consider a simple case: determining whether someone is old enough to access a service.
Conceptual Schema (excerpt):
Person has BirthDate Date
-- Constraint: each Person has exactly one BirthDate
Intent Specification:
Goal: AssessEligibility
Situation: ServiceAccessRequest
AgentRole: EligibilityEvaluator
Outcome: EligibilityStatusDetermined
References: Person, BirthDate
What is not present yet is a rule such as "≥ 18," an algorithm, or a calendar assumption. Those are operational details that come later and can be changed, tested, and redeployed without changing the intent.
Why This Matters
If the legal age threshold changes or jurisdiction varies, the intent remains valid; only the operationalization changes. That separation is what keeps change manageable: when a rule changes, you update the rule, not the purpose it serves.
It also enables multiple intents to share the same schema. "CalculateBenefitsEligibility" and "DetectEmploymentFraud" might both reference Employment and StartDate, but for different purposes. The semantics stay stable while the practical scenario changes, which is cleaner than duplicating definitions or embedding purpose directly into the schema.
LLMs as Intent Interpreters
Large language models fit into this architecture as interpreters, not authorities.
What LLMs Do Well
LLMs are good at interpreting ambiguous natural language and mapping it to known conceptual elements. Given a user request, an LLM can propose candidate intents, identify referenced schema elements, and detect missing context.
What LLMs Cannot Do
What LLMs cannot do is govern or enforce constraints. They do not verify that a requester has authority to invoke an action and they do not reliably know when they are outside their competence. An LLM can propose that a request maps to a particular intent but it cannot confirm that the schema supports that intent, that the data exists, or that the person asking has the right to ask.
In the prototype implementation, the LLM receives a natural language request, extracts structured parameters, and hands them to the delegate. The delegate then validates the intent, checks the schema, executes within its defined authority, and writes a complete audit record. The LLM never touches governance and does not need to.
In an Intent-Driven architecture, LLMs may propose intent mappings but may not enact schema changes and must surface ambiguity rather than resolve it silently. Their outputs remain useful because those outputs still pass through symbolic validation before anything consequential happens.
The Human Accountability Principle
One principle runs through everything here: responsibility and accountability must remain human.
On the semiotic ladderA framework developed by Ronald Stamper and adopted by Jan Dietz that describes how meaning builds in layers, from physical signals at the bottom through syntax, semantics, and pragmatics, up to social commitments at the top., this principle belongs to the social level: norms, roles, and shared commitments. The three-layer architecture addresses the semantic and pragmatic levels. Human accountability addresses the point at which meaning is embedded in an organization with clear lines of responsibility.
Authority Is Delegated
An AI Delegate may have authority to query customer records within a defined situation but it does not have authority to redefine what "customer" means or grant itself access beyond that situation. Authority in this framework is always derived from human decision, conditional on the situation, and revocable; it is never possessed by the system itself.
Responsibility Cannot Be Delegated
AI can execute decisions but cannot accept blame, justify intent, or be penalized. Responsibility requires the capacity for commitment, and commitment is a social act that only people can perform.
Dietz's distinction between forma, informa, and performaDietz's three categories of human activity: forma covers documental acts such as storing and transmitting; informa covers informational acts such as computing and reasoning; performa covers ontological acts such as making commitments and decisions. Only performa involves genuine social responsibility. clarifies why. Forma and informa cover implementation and information handling; performa covers commitments and social acts. AI Delegates operate at the forma and informa levels, which is precisely why responsibility cannot be delegated along with the task.
The Human Accountability Test
An AI action is valid only if responsibility and accountability for its outcome remain human.
If you cannot identify a person who is responsible for the outcome, the delegation has failed and should be treated as a design error. This test does not prevent AI from acting but it ensures that when AI acts, someone is answerable.
Conclusion: From Autonomy to Delegation
The difference between "autonomous agent" and "AI delegate" is not just a matter of terminology; it determines what questions get asked at audit time.
"Autonomous agent" points toward a system that originates its own intent and bears its own responsibility. Neither of those holds up in practice, and when something goes wrong the accountability gap is the first thing an auditor will find.
Regulators, customers, and governance teams will ask four questions: what the system did, why it did it, who authorized it, and what changes if the rule changes. A delegate architecture answers all four, because intent is modeled explicitly, schema is separate from logic, and every decision traces back to a human actor.
The architecture described here has been implemented in a working prototype. The financial product eligibility demo shows the pattern in practice: an LLM interprets a natural language request, a jurisdiction-aware conceptual schema routes the decision, and a delegate produces an audit trail for the outcome. The indeterminate case, where required fields are missing and the system surfaces the gap rather than guessing, makes the governance layer easiest to see.
If you are building AI systems that will face regulatory scrutiny, high-stakes decisions, or the need to explain outcomes to people who were not in the room when the system was designed, the delegate framing is worth considering; not because it limits what AI can do, but because it makes what AI does easier to justify, audit, and revise.
Terms marked with a dotted underline include hover definitions for readers unfamiliar with the companion article.