The Governance Gap in Agentic AI

The term "agentic AI" is everywhere in enterprise discussions right now, and for good reason. The capability is real. AI systems can interpret requests, execute multi-step workflows, query data sources, and produce outputs that used to require a human in the loop. That is genuinely useful and worth building on.

The problem is not the capability. The problem is the governance gap that the language creates. When we call a system an "autonomous agent," we imply it originates its own intent. When we call it a "decision-maker," we imply it bears responsibility for outcomes. When we say it "acts on behalf of" someone, we blur the line between delegation and replacement. None of those implications hold up when something goes wrong, and in enterprise settings, something always eventually goes wrong.

The result is systems that are hard to audit, hard to correct, and hard to defend to regulators or customers, not because the AI failed technically, but because nobody defined who was actually responsible for what it did. This article proposes a framing that closes that gap without sacrificing capability: AI Delegates.

The shift from "agent" to "delegate" is not cosmetic. It encodes a different set of commitments about what AI systems do, what they cannot do, and where humans remain necessary. This article is written for architects, AI designers, and governance professionals who need a framework they can apply. It builds on the companion article What I Mean by Knowledge, Information, and Semantics.

AI Delegates: What the Term Means

An AI Delegate is a computational system that executes delegated intent within clearly defined human authority, grounded in explicit conceptual schemas, while ownership of purpose, meaning, and accountability remains human and externally defined.

This definition has several parts worth unpacking.

Delegated intent. The system does not generate its own purposes. It receives intent from humans and acts within the scope of that intent. Delegation implies that authority is granted, not assumed.

Human-defined authority. The boundaries of what the system may do are set by humans, not discovered by the system. Authority is scoped, conditional, and revocable.

Grounded in conceptual schemas. The system operates over explicit conceptual distinctions, not raw data or implicit assumptions. It uses terms correctly because those meanings are modeled and referenced at runtime, not because the system infers or invents them.

Without owning semantics, purpose, or accountability. The system uses meanings it did not define, pursues purposes it did not originate, and produces outcomes for which it does not bear responsibility.

When an AI Delegate's behavior is explicitly governed by modeled goals, situations, and constraints, we call it an Intent-Driven AI Delegate. This is a more specific term for systems where intent is not just implicit in code but formally represented, inspectable, and testable independently of implementation.

What This Framing Preserves

Rejecting autonomy does not reduce capability or limit practical use. AI Delegates can still:

The difference is that these capabilities operate within explicit constraints, with clear lines of authority, and with accountability that remains human.

This is not a theoretical proposal. The three-layer architecture described below is implemented in a working prototype at knowledge-foundation.ai, using Object-Role Modeling for the conceptual schema, an explicit intent registry, and a delegate that takes natural language requests, interprets them through an LLM, and executes them through symbolic validation with a full audit trail. You can open the live prototype directly at the financial product eligibility demo. The architecture described here maps directly to that implementation.

The appendix describes a maturity model for how AI systems can engage with meaning at increasing levels of responsibility, from purely syntactic execution up to socially embedded coordination.

The Three-Layer Architecture

To make AI Delegates work in practice, we need an architecture that separates concerns cleanly. I propose three layers, each with a distinct responsibility.

Layer 1
Conceptual Schema: Semantic Distinctions

Defines what distinctions matter in a domain. Contains object types, fact types, roles, constraints, and value types. In ORM terms, this is the conceptual schema. It captures meaning but does not determine action or control flow.

Layer 2
Intent: Pragmatic Commitments

Defines why information is used and what is supposed to happen when certain conditions hold. Intent is where meaning becomes actionable, but still requires operational interpretation before execution. References the schema without altering it.

Layer 3
Agent: Operational Interpretation

Decides how intents are fulfilled in practice. Interprets intents, queries conceptual schemas, applies logic and heuristics, and invokes actions. Does not define semantics; it consumes them. Does not originate intent; it receives it.

Key Principles

Schema captures distinctions, not behavior. Intent connects semantics to purpose without collapsing them. Agents execute within boundaries; they do not set boundaries.

The Flow

  1. Conceptual Schema provides the vocabulary and constraints
  2. Intent specifies what should happen and why
  3. Agent determines how to fulfill the intent
  4. Feedback from outcomes may lead to schema or intent evolution, but only through human governance

This separation matters because it makes failures easier to diagnose and fix. When something goes wrong, you can determine whether the issue was semantic, intentional, or operational. Semantic errors require schema changes. Intent errors require intent changes. Operational errors require agent fixes. Each correction targets a specific layer without cascading changes across the system.

This architecture also maps directly to the knowledge formula from the companion article What I Mean by Knowledge, Information, and Semantics: Knowledge = (Symbols + Structure + Shared Meaning) × Intention. The conceptual schema provides the first three components. The intent layer provides intention. The agent layer operationalizes the combination. Without all three layers working together, you have information but not actionable knowledge.

Intent as a First-Class Concept

Most systems treat intent implicitly. Business logic encodes intent in code, but the intent itself is not visible or inspectable. When requirements change, you rewrite code rather than updating a versioned, modeled intent. The Intent-Driven approach makes intent explicit by modeling it directly, rather than leaving it buried in code.

The Intent Meta-Model

At minimum, an intent model includes:

Field Description Example
Intent A named purpose DetermineLegalAdulthood
Goal What the intent aims to achieve AssessEligibility
Situation When the intent applies ServiceAccessRequest
AgentRole Who or what pursues the intent EligibilityEvaluator
Outcome What success looks like EligibilityStatusDetermined
ConceptualElement Which schema elements the intent references Person, BirthDate

Example: Age Eligibility

Consider a simple case: determining whether someone is old enough to access a service.

Conceptual Schema (excerpt):

Person
Person has BirthDate Date
-- Constraint: each Person has exactly one BirthDate

Intent Specification:

Intent: DetermineLegalAdulthood
Goal: AssessEligibility
Situation: ServiceAccessRequest
AgentRole: EligibilityEvaluator
Outcome: EligibilityStatusDetermined
References: Person, BirthDate

Notice what is not present: no rule yet, no "≥ 18," no algorithm, no calendar assumption. Those are operational details that come later and can be changed, tested, and redeployed without changing the intent.

Why This Matters

If tomorrow the legal age changes, or calendar systems differ, or jurisdiction varies, the intent remains valid. Only the operationalization changes. This separation creates resilience. The intent layer absorbs change that would otherwise ripple through the entire system. You update the rules, not the purpose.

It also enables multiple intents to share the same schema. "CalculateBenefitsEligibility" and "DetectEmploymentFraud" might both reference Employment and StartDate, but for different purposes. Same semantics, different pragmatics. No duplication, no ontology explosion.

LLMs as Intent Interpreters

Large language models fit into this architecture as interpreters, not authorities.

What LLMs Do Well

LLMs are good at interpreting ambiguous natural language and mapping it to known conceptual elements. Given a user request, an LLM can propose candidate intents, identify referenced schema elements, and detect missing context. This is useful because it connects how humans speak to how systems are structured.

What LLMs Cannot Do

What LLMs cannot do is govern. They do not enforce constraints. They do not verify that a requester has authority to invoke an action. They do not reliably know when they are outside their competence. An LLM can propose that a request maps to a particular intent. It cannot confirm that the schema supports that intent, that the data exists, or that the person asking has the right to ask.

This is not a criticism of LLMs. It is a description of where they belong in the architecture. In the prototype implementation, the LLM receives a natural language request, extracts structured parameters, and hands them to the delegate. The delegate then validates the intent, checks the schema, executes within its defined authority, and writes a complete audit record. The LLM never touches governance. It does not need to.

In an Intent-Driven architecture, LLMs may propose intent mappings but may not enact schema changes, may not collapse conceptual distinctions, and must surface ambiguity rather than resolve it silently. That makes them fast, flexible, and safe to swap out, because their outputs always pass through symbolic validation before anything happens.

The Human Accountability Principle

Throughout this framework, one principle remains constant: responsibility and accountability must remain human.

This is not a limitation. It is a requirement for operating AI systems responsibly at scale. In terms of the semiotic ladder from the companion article, this principle represents the social level: norms, roles, and shared commitments. The three-layer architecture (schema, intent, agent) addresses the semantic and pragmatic levels. Human accountability addresses the social level, where meaning becomes embedded in a community with clear lines of responsibility. For a business-facing treatment of that organizational grounding, see Enterprise Ontology for AI Reinvention.

Authority Is Delegated

Authority is not possessed by AI systems. It is delegated, scoped, and revocable. An AI Delegate may have authority to query customer records within a defined situation. It does not have authority to redefine what "customer" means, to access records outside that situation, or to grant itself additional capabilities.

Delegation is always:

Responsibility Cannot Be Delegated

An AI system can execute a decision. It cannot bear blame for the outcome. It cannot justify intent. It cannot accept sanctions. This is not a statement about AI capability. It is a statement about what responsibility means. Responsibility requires the capacity for commitment, and commitment is a social act that only persons can perform.

Jan Dietz's PSI theory from Enterprise Ontology makes this precise. Dietz distinguishes between forma (technical execution, the how) and performa (commitments and social acts, the what and why). AI Delegates belong to the forma layer. They support human action; they do not replace it. The related article Enterprise Ontology for AI Reinvention develops that connection in more organizational terms.

The Human Accountability Test

A useful test for any AI system design:

An AI action is valid only if responsibility and accountability for its outcome remain human.

If you cannot identify a human who is responsible for the outcome, the delegation has failed and should be treated as a design error. Either the intent was not properly specified, the authority was not properly scoped, or the governance is missing. This test does not prevent AI from acting. It ensures that when AI acts, someone is answerable.

Conclusion: From Autonomy to Delegation

The language we use shapes what we build. "Autonomous agents" points toward systems that replace human judgment. "AI Delegates" points toward systems that extend it, within boundaries that humans define, with accountability that humans retain.

The practical difference shows up at audit time. With a delegate architecture, you can answer the questions that regulators, customers, and internal governance teams will eventually ask: What did the system do? Why did it do it? Who authorized it? What would change if the rule changed? Those questions have clear answers when intent is modeled explicitly, when the schema is separate from the logic, and when every decision is attributed to a human actor.

The framework described here is implemented in a working prototype. The financial product eligibility demo shows this architecture running: an LLM interprets a natural language request, a jurisdiction-aware conceptual schema routes the decision, and a delegate produces a full audit trail for every outcome. The Indeterminate status, where required fields are missing and the system surfaces the gap rather than guessing, is where the governance layer becomes most visible. The next step is extending the architecture to more complex scenarios: competing intents over the same schema, and multi-delegate coordination. That is where the separation of schema, intent, and execution stops being a design preference and starts being a practical necessity.

If you are building AI systems that will face regulatory scrutiny, high-stakes decisions, or the need to explain outcomes to people who were not in the room when the system was designed, the delegate framing is worth adopting. Not because it limits what AI can do, but because it makes what AI does defensible.


G. Sawatzky works at the intersection of formal conceptual modeling and AI systems. For foundational definitions of knowledge, information, and semantics, see the companion article What I Mean by Knowledge, Information, and Semantics.