Reimagining Enterprise Infrastructure for Agentic AI

For the past few years, AI in organisations has mostly followed one pattern: a user supplies context, a model generates an output, and the application decides what happens next. That pattern is well understood, and its risks are essentially quality problems.

Agents, however, independently accomplish tasks on users' behalf. They plan toward a goal, reason, maintain memory and take action on their own. So when an agent gets something wrong, the consequences are far worse than a wrong answer from an AI model plugged into an existing workflow.

An inaccurate agent can alter a payment, a security setting, a customer account or a live system, turning a mistake into an operational and security incident. The central shift is therefore to treat an AI agent as a new kind of semi-trusted digital worker operating across the enterprise.

Agentic AI, where multiple agents work together, multiplies that risk. Agents therefore need identity, permissions, budgets, a lasting memory of their work, limits on what they can reach, a secure place to work and a traceable record of every interaction with core business systems.

As autonomy rises, the limiting factor shifts from the model's intelligence to the enterprise's ability to control what the agent can see, decide, communicate, and change.

Every infrastructure layer changes

Plugging an AI model to automate one aspect of a workflow doesn't require teams to change the entire architecture. Since agentic AI automates the entire workflow, agents require the entire architecture to be accessible.

For instance, identity systems built for people and service accounts must now handle authority passed from a user to an agent to a tool, using short-lived credentials. Monitoring must capture an agent's entire chain of behaviour, not just server performance and response times.

Memory deserves particular attention. A vector database (the searchable knowledge store most AI systems use) is not a universal answer. Enterprises have to separate a task's working state, reference knowledge, the agent's long-term memory, business-system data and audit evidence, each with its own lifecycle, data residency and deletion rules.

Persistent memory must be governed like any other data asset, with clear provenance, ownership, retention and access control, or it quietly becomes an ungoverned shadow data store.

The target architecture: federated in logic, centralised in control

Two extremes fail predictably.

1. One giant enterprise "super-agent" gathers too many privileges, so a single failure can cause catastrophic damage.
2. Dozens of independent departmental agent setups produce inconsistent governance, duplicated infrastructure and behaviour nobody can audit.

The recommended pattern is federated in business logic but centralised in control. Business units own their agents, tools, data and success metrics; a shared enterprise agent platform supplies model access, identity, policy enforcement, tool access, secure workspaces, connection gateways, monitoring, audit storage, testing and cost controls.

This mirrors how enterprise IT moved from one central integration hub to many independent services governed by central identity and API management, except that agents demand considerably more control over behaviour.

The architecture is easiest to understand as four layers, or "planes", shown below:

• A control plane that sets the rules (registries, identity, policies, budgets, testing gates)‍
• A cognitive/data plane that does the thinking (orchestration, model gateway, retrieval, memory)
• An execution plane that does the work (gateways to business systems and tools, sandboxes, event handling, workflow engine)
• An evidence plane that keeps the record (traces, policy decisions, tool records, costs, approvals).

The design deliberately places policy outside the AI model's reasoning loop. A model can propose that an action is safe; it must never be the sole authority on whether that action is permitted.

Why existing applications don't disappear

Agentic AI is sometimes pitched as the end of applications and microservices (the small, dedicated software services behind most modern systems). That is an unsafe interpretation. Microservices answer "how is this operation carried out reliably?"; agents answer "which operations should I use, and in what order, to reach this goal?"

The agent should never carry out a money transfer itself. It should call a transfer service whose fixed, tested code checks account ownership, limits, fraud controls, duplicate requests and transaction rules.

Thinking and doing must stay separate: models decide what to attempt, while proven systems remain the final authority for payments, identity changes, safety controls and regulated decisions. This separation also limits the damage from AI hallucination, because the systems that change records enforce their own rules regardless of what the model believes.

The major AI and cloud providers are converging on exactly this split:

1. ‍OpenAI's 2026 Agents SDK explicitly separates the agent's reasoning from the sandboxed computing where it works‍
2. Google's Gemini Enterprise Agent Platform names agent runtime, identity, gateway, registry and observability as platform services‍
3. Microsoft Foundry and AWS AgentCore offer equivalent building blocks

Protocols become the connection points

Two open protocols, or shared technical standards, now define the connection points of this architecture.

1. ‍MCP (Model Context Protocol) governs how an agent connects to tools and data.‍
2. A2A (Agent2Agent) governs which other agents an agent may collaborate with.

They are complementary, not competing. But neither protocol is a security policy: enterprises should support both through controlled gateways, wrapped in identity checks, approved lists, content inspection and audit, never as uncontrolled direct connections between systems.

Bounded autonomy is the destination

Benchmarks keep the hype honest. In the original WebArena evaluation, the strongest GPT-4-based agent completed just 14.4% of realistic web tasks, against 78.2% for humans. OSWorld 2.0, released in June 2026, deliberately stretched tasks to hundreds of steps, a reminder that research is only now moving from short demonstrations toward sustained autonomous work. Leading-edge capability should not be confused with everyday reliability.

For most enterprises, bounded autonomy is the economically and operationally sensible destination for the next several years. The autonomy ladder in below offers a practical framework:

• Reading information can be fairly autonomous;
• Changes that can be undone should be bound by policy; and
• Irreversible, high-impact or regulated actions should trigger fixed policy checks or human approval.

The strategic sweet spot for mainstream work is A2–A3: enough autonomy to remove workflow friction without giving an uncertain model unrestricted authority. Anthropic's 2026 research on real agent usage supports this: experienced users shift from approving every action toward monitoring and stepping in selectively. That argues for oversight matched to risk rather than a universal "human approves every step" rule.

Crucially, risk comes from capability × permission × environment × consequence. Two agents using the same model carry radically different risk if one can only search product manuals while the other can approve purchases. Governance must therefore follow the agent deployment (its owner, tools, data, autonomy level and maximum possible impact), not the model.

Security: a wider attack surface, formally recognised

Agentic systems widen the attack surface because untrusted content can now influence software that holds tools and permissions. OWASP, the leading application-security body, now treats reasoning, memory, tools, identity, human oversight and interactions between agents as distinct points of attack.

The multi-agent leakage risk is proven, not hypothetical: AgentLeak, a 2026 benchmark, found that checking only final outputs misses violations that occur inside the conversations between agents. Monitor the hand-offs, not just the answer.

Zero-trust identity is the backbone. An agent should never simply reuse a user's login session or a broad back-end system account. The safer chain, human → delegated agent identity → task-specific permission → downstream system, lets each system answer two separate questions: can this agent use this capability at all, and can it do so on behalf of this particular user?

And wherever an agent can write code, browse unknown websites or run system commands, sealed, disposable workspaces (sandboxes) with no stored credentials, restricted internet access and automatic deletion are mandatory, not optional extras.

From MLOps to AgentOps

Operationally, agentic AI extends MLOps (the practice of running machine-learning models in production) into what is best called AgentOps. That means versioning not just models but prompts, tool connections, agent workflows and policies; testing whole sequences of actions rather than single outputs; tracing hand-offs between tools and agents; replaying incidents; and setting service-level objectives (SLOs) specific to agents.

Below, we show what a governed agent run looks like with these controls in place, with every step feeding an evidence record that can be added to but never altered.

Anatomy of a governed agent run, with policy enforcement outside the model loop.

Traditional availability metrics are not enough, because an agent can be perfectly "up" while doing the wrong thing. Agent SLOs need outcome, safety and cost measures alongside system health:

Incident response also gains new options: an agent kill switch that pauses runs without shutting down the business application; containment (freezing memory updates, quarantining suspect information); and replay, which reconstructs a run from stored evidence without repeating its real-world effects.

The economics: pay per successful task, not per token

Agent cost is not the same as AI token cost (tokens are the units AI providers bill by). A useful planning equation divides all costs (model, data retrieval, tools, sandboxes, storage, network, monitoring and human oversight) by verified successful tasks. A task involving five agents, twelve tool interactions and several rounds of checking consumes far more tokens than its final answer suggests, and a cheap model needing six retries can cost more than a capable model that succeeds once.

Current list prices span a wide range, roughly 0.75–5 per million input tokens and 3.75–25 per million output tokens across major providers, but as a rough, illustrative planning assumption, enterprises may need to budget a further 30–150% for retrieval, sandboxing, monitoring and third-party tools.

On that assumption, a departmental deployment of 100,000 tasks per month could run in the order of 15k–180k in variable technology cost, and a large enterprise at one million tasks, 225k–2.5m. Actual costs will vary widely with workload, model choice and architecture.

The lesson is that token price should not dominate platform selection: a small increase in task success, or fewer unnecessary loops, outweighs large per-token differences. Cost controls belong at four levels (per run, per agent, per department and enterprise-wide), with routine work routed to cheaper models and premium reasoning reserved for steps where measured success justifies it.

A roadmap where authority grows with evidence

The migration path matters as much as the destination. Agents should be given more authority to act only as monitoring and testing maturity increases, following the phased approach below.

Good first pilots share a profile: high manual effort, clear completion criteria, good digital data, well-defined system connections, reversible consequences and enough volume to measure the benefit. Poor first pilots include vague executive decision-making, safety-critical controls and workflows that require unrestricted administrator access. A reasonable pilot scorecard targets 85–95% task completion on a standard test set, zero material policy violations, a 20–40% reduction in human effort, over 99% trace completeness for actions that change records, and 100% successful rollback in test scenarios.

Conclusion

The winning pattern is not "give every application an autonomous agent". It is an enterprise agent control plane: a central control layer through which increasingly autonomous agents operate safely on top of well-designed, reliable business systems, with identity, policy, sandboxing, resilience, auditability and testing built in as foundations alongside computing, storage and networking.

The broader shift is from treating AI as a model-hosting problem to treating agentic AI as a systems, identity, security and operations problem in which the AI's reasoning is one component. The enterprises that benefit most will not be those that deploy the most autonomous agents fastest. They will be those that build an infrastructure layer in which autonomy can be increased deliberately, measured continuously and revoked immediately, as evidence, business value and risk tolerance justify it.