The Agentic AI Stack - An Introduction

1. Introduction

Agentic AI should be understood not as a single model or application, but as a technology stack that enables AI systems to reason about goals, access relevant information, take actions, observe their consequences, and adapt until an outcome is achieved. Unlike a conventional chatbot, which primarily maps an input to a generated response, an agent can determine what needs to happen next, invoke tools and enterprise systems, and operate across multiple steps to complete a task (Yao et al., 2023; Microsoft, 2026a).

This distinction has important architectural consequences. The foundation model provides reasoning and language capabilities, but it is only one component of the overall system. A production-grade agent requires an agent runtime to manage state and orchestration, tools and integrations to interact with systems of record, data and knowledge services to provide trusted context, and a control plane to enforce identity, security, policy, observability and human oversight. Microsoft similarly describes modern agent architectures as comprising interfaces, foundational infrastructure, orchestration, language models and tool-calling capabilities (Microsoft, 2026b).

The reference architecture therefore separates the Agentic AI Stack into six complementary layers:

1. Experience & access — where users, applications and business processes interact with agents, supported by interfaces, APIs, identity and permissions.

2. Agent intelligence — the reasoning and context layer, comprising foundation models, model routing, instructions, context assembly, memory and planning.

3. Agent runtime — the execution control loop that translates decisions into governed actions, manages state, orchestrates tools, handles failures and coordinates multiple agents where required.

4. Tools & enterprise integration — the action layer through which agents interact with APIs, databases, SaaS platforms, code environments, browsers and human workflows.

5. Data & knowledge — the information layer that provides trusted enterprise data, searchable knowledge, retrieval, memory, permissions and provenance.

6. Control plane & operations — the governance layer that makes agents deployable and manageable through security, policy enforcement, evaluation, observability, cost controls and human oversight.

The most important architectural principle is to separate intelligence from execution. The model can propose a plan or an action, but it should not have unrestricted authority to execute that action. The runtime and control plane determine whether the action is permitted, validate its inputs, execute it through an approved tool, and verify the resulting state. This separation allows organizations to combine the flexibility of probabilistic models with the reliability and control of deterministic enterprise systems (Microsoft, 2026a; NIST, 2024).

At the centre of the stack is therefore a closed execution loop:

Goal → Observe → Reason → Act → Verify → Repeat → Complete or Escalate

This pattern is consistent with research on agentic reasoning and acting, where language models interleave reasoning with actions that retrieve information or interact with external environments (Yao et al., 2023). In an enterprise setting, the loop becomes more controlled: every significant action should be subject to appropriate identity, permissions, policy checks, logging and, where necessary, human approval.

Data and knowledge play an equally important role. Agents need access to current, authoritative information rather than relying solely on knowledge encoded in model parameters. Retrieval-Augmented Generation (RAG) addresses this by retrieving relevant information from an external knowledge source and providing it to the model as context. The original RAG research demonstrated that combining model-based knowledge with retrieved external knowledge can improve factuality and specificity while making external knowledge easier to update and inspect (Lewis et al., 2020). In enterprise architectures, retrieval should additionally respect data permissions and preserve source provenance.

The stack should consequently distinguish between knowledge and action. Retrieval is primarily used to help an agent understand policies, documentation, procedures and other information; tools and APIs are used to change systems or perform work. This distinction reduces the temptation to ask the model to invent or infer operational data and instead gives it controlled access to the systems that hold the authoritative answer.

Finally, enterprise agentic AI must be designed for failure, uncertainty and oversight. Models can produce incorrect reasoning, tools can fail, data can be incomplete, permissions can change, and workflows can be interrupted. Production architectures therefore need retries, timeouts, checkpointing, verification, evaluation, audit trails and escalation paths. NIST's Generative AI Profile similarly emphasizes managing risks across the AI lifecycle rather than treating model performance as the sole measure of system trustworthiness (Autio et al., 2024).

The practical implication is that organizations should not begin by asking, “Which AI model should we use?” They should begin by asking, “What work should the agent perform, what information and systems does it need, what actions may it take, and what controls are required to make those actions reliable and accountable?”

The Agentic AI Stack provides the architecture for answering those questions. It connects models to context, context to decisions, decisions to governed actions, and actions to verified business outcomes.

2. The Azure AI Stack

Microsoft Azure provides a broad set of services for building, deploying and operating enterprise AI agents. Its current architecture is increasingly centred on Microsoft Foundry, which brings together models, agent development, tools, evaluation and operational capabilities, while surrounding Azure services provide identity, data, integration, compute and governance (Microsoft, 2026a).

The Azure approach can therefore be understood as an implementation of the Agentic AI Stack rather than as a separate architecture. The same fundamental layers remain: experience and access, agent intelligence, runtime and orchestration, tools and enterprise integration, data and knowledge, and the control plane.

2.1. Experience & access

The experience layer is typically delivered through web applications, Microsoft applications, custom applications, APIs, voice interfaces or business workflows. Azure API Management can provide a controlled entry point for APIs, while Microsoft Entra ID provides identity and access management.

For enterprise agents, identity should be propagated through the execution chain wherever possible. The objective is not simply to authenticate the user at the front door, but to ensure that downstream actions are performed within an appropriate security context.

Typical capabilities include:

• Microsoft Entra ID for identity and authentication

• Role-based access control and managed identities

• API Management for controlled API exposure

• Web and application interfaces

• Power Platform for business-facing experiences and workflows

2.2. Agent intelligence

Azure provides access to a range of foundation models through Microsoft Foundry and associated model services. The model layer provides the agent with language understanding, reasoning, generation and tool-selection capabilities.

The architecture should nevertheless avoid coupling the agent too tightly to a single model. Model selection can be based on capability, latency, cost, availability, data requirements or policy.

The intelligence layer typically includes:

• Foundation models

• Model selection and routing

• Prompt and instruction management

• Context assembly

• Structured outputs and tool calling

• Agent memory and state

Microsoft Foundry provides a central environment for working with models and agents, while Azure services can provide the surrounding infrastructure required for production deployment (Microsoft, 2026a).

2.3. Agent runtime & orchestration

The runtime is where an Azure agent becomes an operational system rather than simply an LLM application.

Microsoft Foundry Agent Service provides managed capabilities for building and deploying agents, including tool use and orchestration. For more complex workflows, developers can combine agent services with Azure Functions, Logic Apps, Durable Functions, event-driven services or custom orchestration frameworks.

The runtime is responsible for the execution loop:

Goal → Observe → Reason → Act → Verify → Repeat

It maintains state, invokes tools, handles failures and determines when a task is complete or needs escalation.

For long-running or business-critical processes, durable workflow technologies are particularly important because they allow execution to survive interruptions rather than relying on a single model invocation.

2.4. Tools & enterprise integration

Azure agents become operationally valuable when they can interact with enterprise systems.

The tool layer can include:

• Azure Functions

• Logic Apps

• REST APIs

• API Management

• Microsoft Graph

• MCP-based tools

• Database queries

• Custom application services

• Power Platform connectors

• Controlled code execution

This layer should be treated as a governed action surface. An agent should not receive unrestricted credentials to enterprise systems. Instead, tools should expose narrowly defined capabilities with explicit permissions, validation and logging.

For example, rather than giving an agent unrestricted access to an ERP (Enterprise Resource Planning e.g. SAP, Oracle, and Microsoft Dynamics 365) system, the architecture might expose separate tools such as:

get_customer()
check_order_status()
create_service_request()
request_refund_approval()

This makes the agent's authority explicit and easier to govern.

2.5. Data, knowledge & memory

Azure offers multiple options for providing agents with enterprise context.

Azure AI Search can support enterprise search and RAG architectures, while Azure SQL, Cosmos DB, Azure Data Lake, Fabric and other data services provide access to structured and operational data.

A typical architecture separates:

• Systems of record — operational databases, CRM, ERP and other authoritative sources

• Knowledge retrieval — documents, policies, manuals and other unstructured information

• Agent state — current task state and intermediate results

• Long-term memory — information intentionally retained across interactions

• Provenance — information about where retrieved knowledge originated

Access control must extend into retrieval. An agent should not retrieve information simply because it exists in an index; retrieval must respect the user's or agent's authorization context.

2.6. Control plane & operations

Azure's enterprise strength is particularly visible in the surrounding control plane.

Relevant services include:

• Microsoft Entra ID

• Azure RBAC and managed identities

• Azure Key Vault

• Azure Policy

• Microsoft Purview

• Azure Monitor

• Application Insights

• Microsoft Defender

• Microsoft Sentinel

• Azure Cost Management

• Microsoft Foundry evaluation capabilities

These services provide the mechanisms required to monitor, secure and govern an agent throughout its lifecycle.

Observability should capture more than application errors. For an agentic system, useful telemetry includes:

user request → model invocation → retrieved context → decision → tool call → tool result → verification → final outcome

This makes it possible to understand not only whether an agent failed, but why it failed and what it actually changed.

2.7. A representative Azure mapping

The Azure implementation of the Agentic AI Stack can be understood as a set of complementary capabilities, each supporting a specific part of the agent lifecycle.

Experience & access form the entry point into the system. Web applications, APIs and Power Platform provide the interfaces through which users and business processes interact with agents, while Microsoft Entra ID provides identity and access control.

Agent intelligence provides the reasoning and context capabilities. Microsoft Foundry and Azure-hosted models provide the foundation models that interpret goals, reason over context, generate plans and select appropriate actions.

Runtime & orchestration turns those decisions into controlled execution. Microsoft Foundry Agent Service, Durable Functions, Logic Apps and custom orchestration components manage state, coordinate steps, invoke tools, handle retries and support long-running workflows.

Tools & integration give agents the ability to perform real work. Azure Functions, API Management, Logic Apps, Microsoft Graph and MCP-based tools expose controlled capabilities that allow agents to interact with enterprise applications, APIs and services.

Data & knowledge provide the information agents need to make grounded decisions. Azure AI Search supports retrieval and RAG, while Azure SQL, Cosmos DB, Microsoft Fabric and Data Lake services provide access to structured and enterprise data.

Memory & state maintain the information required across an agent's execution. Cosmos DB, Azure SQL and durable workflow state can store task state, intermediate results, conversation context and other information that needs to persist across steps or sessions.

Security provides the controls around identity, access and execution. Microsoft Entra ID, RBAC, Managed Identity, Key Vault and Microsoft Defender help ensure that agents operate with appropriate permissions and that credentials, resources and execution environments are protected.

Governance establishes the policies and controls under which agents operate. Azure Policy, Microsoft Purview, audit capabilities and compliance controls help enforce organizational requirements around data, security, access and responsible use.

Observability provides visibility into what agents are doing and why. Azure Monitor, Application Insights and Microsoft Foundry's evaluation capabilities can capture execution telemetry, monitor performance, evaluate agent behaviour and help identify failures or regressions.

Finally, operations provide the lifecycle management required to run agents reliably at scale. Azure DevOps, GitHub, deployment pipelines and Cost Management support versioning, testing, deployment, rollback, operational control and cost management.

Taken together, these capabilities create an Azure implementation of the Agentic AI Stack in which models provide intelligence, data provides context, tools provide action, the runtime controls execution, and the Azure platform provides the security, governance and operational foundations required for production use.

2.8. Azure architectural guidance

The Azure stack is strongest when organizations treat Microsoft Foundry and Azure services as parts of a broader enterprise control architecture, rather than allowing an agent platform to become a new isolated technology silo.

A practical Azure implementation should therefore:

1. Start with a defined business workflow.

2. Establish identity and authorization before exposing tools.

3. Keep model reasoning separate from deterministic execution.

4. Use Azure AI Search and RAG for knowledge retrieval.

5. Expose enterprise capabilities through narrow, governed tools and APIs.

6. Use durable orchestration for long-running processes.

7. Instrument the complete agent execution path.

8. Introduce human approval for high-impact actions.

9. Evaluate agents against business outcomes, not only response quality.

10. Version models, prompts, tools, policies and workflows together.

The resulting architecture is not simply “Azure + an LLM.” It is a controlled execution environment in which models provide intelligence while Azure provides the identity, integration, data, runtime and operational foundations needed to turn that intelligence into enterprise work.

3. The AWS Agentic AI Stack

Amazon Web Services approaches agentic AI through a similarly layered architecture, centred on Amazon Bedrock for foundation models and managed agent capabilities, surrounded by AWS services for identity, orchestration, integration, data, security and operations.

The key architectural principle is the same: the model supplies intelligence; the surrounding AWS services provide the mechanisms through which that intelligence can safely become action.

3.1. Experience & access

AWS applications can expose agents through web applications, APIs, contact-centre experiences, enterprise applications and event-driven workflows.

Common building blocks include:

• Amazon API Gateway

• Amazon Cognito

• AWS IAM

• Application Load Balancer

• Amazon Connect

• Custom web and mobile applications

AWS Identity and Access Management (IAM) provides the foundation for controlling what identities, roles and services can access.

As with Azure, authentication should not be treated as a one-time front-door control. Permissions need to follow the execution path into data stores, APIs and tools.

3.2. Agent intelligence

Amazon Bedrock provides access to foundation models from Amazon and other model providers through a managed AWS service. This allows organizations to select models according to capability, performance, cost and workload requirements.

The intelligence layer can include:

• Foundation models through Amazon Bedrock

• Model selection and routing

• Prompt management

• Context assembly

• Structured outputs

• Tool calling

• Knowledge retrieval

• Agent memory and session state

This model-provider abstraction can be valuable where organizations want to avoid designing their architecture around a single model vendor.

3.3. Agent runtime & orchestration

Amazon Bedrock provides managed capabilities for building agents that can reason over a task, retrieve information and invoke defined actions.

For more complex enterprise workflows, AWS Step Functions provides durable orchestration, while AWS Lambda, Amazon EventBridge, Amazon SQS and other services can provide execution and event-driven coordination.

The division of responsibility is important:

Bedrock provides the agent intelligence and managed agent capabilities; deterministic AWS services provide workflow control and execution.

For example, an agent might determine that a customer refund should be initiated, but a Step Functions workflow could enforce the required approval, validation and financial-system interactions.

This creates a useful separation between probabilistic decision-making and deterministic business process execution.

3.4. Tools & enterprise integration

AWS agents can interact with enterprise systems through APIs, Lambda functions, application services, databases and external services.

Typical components include:

• AWS Lambda

• Amazon API Gateway

• AWS Step Functions

• EventBridge

• Amazon SQS/SNS

• REST and GraphQL APIs

• Custom application services

• MCP-compatible tool integrations

• Controlled compute environments

As with the broader reference architecture, tools should expose specific business capabilities rather than unrestricted system access.

For example:

get_customer_account()
retrieve_invoice()
check_payment_status()
submit_refund_request()

Each operation can then have its own authorization, validation, logging and approval requirements.

3.5. Data, knowledge & memory

AWS provides several complementary services for agent data and knowledge architectures.

Amazon Bedrock Knowledge Bases can support RAG-based retrieval, while services such as Amazon S3, Amazon Aurora, Amazon DynamoDB, Amazon OpenSearch Service and AWS databases provide different forms of enterprise data storage and retrieval.

A typical architecture distinguishes:

• Enterprise systems of record

• Document and knowledge repositories

• Retrieval indexes

• Agent session state

• Long-term memory

• Source provenance and access metadata

The same principle applies as in Azure: RAG should provide grounded knowledge; enterprise tools should provide controlled access to systems and actions.

3.6. Security, governance & operations

AWS provides a mature set of services for controlling the operational environment around agents.

Relevant capabilities include:

• AWS IAM

• AWS KMS

• AWS Secrets Manager

• AWS CloudTrail

• Amazon CloudWatch

• AWS Config

• AWS Organizations

• AWS Security Hub

• Amazon GuardDuty

• Amazon Macie

• AWS WAF

• AWS Control Tower

These services can provide identity, secrets management, security monitoring, compliance controls and auditability.

For an agentic workload, CloudTrail and application telemetry should be combined with agent-level tracing so that organizations can reconstruct the execution path:

request → model → retrieved context → decision → tool → system change → verification

This is essential for investigating failures and demonstrating accountability.

3.7. A representative AWS mapping

The AWS implementation of the Agentic AI Stack is a set of complementary services that support each layer of the agent lifecycle, from user interaction and model reasoning through orchestration, enterprise integration, data access, security, governance, and operations.

Experience & access provides the entry points through which users, applications, and business processes interact with agents. Web and mobile applications can connect through Amazon API Gateway, while Amazon Cognito provides application identity and access management. Amazon Connect can provide conversational and voice-based experiences where agents are integrated into contact-centre workflows.

Agent intelligence is provided primarily through Amazon Bedrock and its foundation models. Bedrock provides access to different model capabilities while supporting the development of applications that require reasoning, structured outputs, tool calling, retrieval, and other agent capabilities. This layer provides the intelligence that interprets objectives and proposes the next action.

Runtime & orchestration controls how those decisions are executed over time. Amazon Bedrock Agents can provide managed agent orchestration, while AWS Step Functions and AWS Lambda can be used to implement more deterministic, stateful, and controlled workflows. This separation allows the model to determine what may need to happen while the runtime determines how and whether that action is actually executed.

Tools & integration connect agents to the systems where business work is performed. Lambda functions can expose controlled business operations, API Gateway can provide governed API access, and EventBridge can connect agents to event-driven business processes. Existing enterprise APIs and services can also be exposed as tools, including through MCP where appropriate. Tools should be narrowly scoped, authenticated, authorised, and designed around specific business capabilities rather than giving an agent unrestricted system access.

Data & knowledge provides the information agents need to make grounded decisions. Amazon Bedrock Knowledge Bases can support retrieval-augmented generation and knowledge retrieval, while Amazon S3 provides durable object storage. Amazon OpenSearch can support search and retrieval workloads, while Aurora and DynamoDB provide access to structured operational data. The appropriate source should be selected according to whether the agent needs retrieved knowledge, transactional data, or access to a system of record.

Memory & state maintains the information required across agent steps and longer-running processes. DynamoDB and Aurora can store application or agent state, while other application state stores can be used where appropriate. State should be explicitly designed around the lifecycle of the task, with clear policies for retention, access, correction, and deletion rather than treating all historical context as permanent memory.

Security establishes the identity, access, and protection mechanisms around the agent and its tools. AWS Identity and Access Management (IAM) provides authentication and authorisation controls, while AWS Key Management Service (KMS) protects encryption keys and AWS Secrets Manager manages sensitive credentials. AWS WAF can protect web-facing entry points. Agent permissions should follow least-privilege principles, with access granted to specific tools and resources rather than to broad AWS environments.

Governance provides the organisational controls required to operate agents consistently across an AWS environment. AWS Organizations and Control Tower can support account and environment governance, while AWS Config provides configuration and compliance monitoring and AWS CloudTrail provides an audit record of AWS API activity. Governance should also cover agent configurations, models, tools, policies, prompts, and deployment changes.

Observability provides visibility into what agents are doing and why. Amazon CloudWatch can collect logs, metrics, alarms, and operational telemetry, while CloudTrail provides an audit trail of AWS API activity. Application-level tracing should connect the agent's reasoning steps, tool calls, workflow execution, errors, latency, and outcomes so that an agent can be diagnosed as an end-to-end business process rather than simply as a model invocation.

Security operations provide additional protection against threats and inappropriate data access. AWS Security Hub can centralise security findings, GuardDuty can detect suspicious activity and threats, and Macie can help identify and protect sensitive data stored in AWS environments. These services complement the agent-specific controls applied at the identity, tool, data, and runtime layers.

Event-driven execution enables agents to respond to business events and operate asynchronously. Amazon EventBridge can route events between services and applications, while Amazon SQS provides durable message queuing and SNS supports event and notification fan-out. This pattern is particularly useful when agent workloads need to be decoupled from user requests, retried safely, or triggered by changes in enterprise systems.

Compute provides the execution environment for agent applications, tools, and supporting services. AWS Lambda is appropriate for event-driven and short-running workloads, while containers running on Amazon ECS or Amazon EKS provide greater control for longer-running or more specialised workloads. Other AWS compute services can be selected according to workload, scalability, networking, performance, and operational requirements.

Taken together, these capabilities provide an AWS implementation of the Agentic AI Stack in which Bedrock and foundation models provide intelligence, data and knowledge provide context, tools and APIs provide action, orchestration controls execution, and AWS security, governance, observability, and compute services provide the production foundation. The architecture should remain modular: models can evolve independently of business tools, orchestration can enforce deterministic controls around model decisions, and enterprise systems remain protected behind explicit identity, policy, and integration boundaries.

3.8. AWS architectural guidance

A strong AWS implementation should apply the same architectural discipline as the reference stack:

1. Define the business outcome before selecting the model.

2. Use Bedrock to provide model intelligence without making it the execution authority.

3. Keep deterministic workflows in services such as Step Functions.

4. Expose enterprise capabilities through narrowly scoped tools and APIs.

5. Use Knowledge Bases and RAG to ground model responses in enterprise information.

6. Enforce IAM permissions at every meaningful access boundary.

7. Use event-driven architecture for asynchronous and long-running work.

8. Verify business outcomes after tool execution.

9. Capture sufficient telemetry to reconstruct agent decisions and actions.

10. Require human approval where actions are financially, operationally or legally consequential.

The resulting AWS architecture follows the same fundamental pattern as Azure: models reason, retrieval supplies context, tools perform actions, orchestration controls execution, and the cloud control plane enforces security, governance and observability.

4. Azure and AWS: the common architectural pattern

Azure and AWS take different approaches to implementing agentic AI, but the architectural patterns they enable are remarkably similar. The differences are primarily in the services used to implement each capability, the way those services integrate with the wider cloud platform, and the strengths of each provider within an enterprise technology estate.

At a high level, the two implementations can be represented as:

Azure:
Microsoft Foundry + Azure services + Microsoft Entra + Azure data and integration services

AWS:
Amazon Bedrock + AWS services + IAM + AWS data and integration services

Neither platform should therefore be evaluated simply by comparing individual AI services. An agent is not a model endpoint. It is a coordinated system in which models, context, memory, tools, orchestration, identity, data, security, and operational controls work together.

The common architectural pattern is:

Model → Context → Decision → Governed Tool → Verification → Outcome

The model provides reasoning capability. Context provides the information required to make a decision. The agent determines the next step, but a governed runtime and tool layer control what the agent is actually permitted to do. The resulting action is verified against the real state of the system, and the outcome is either returned, followed by another step, or escalated to a human.

This pattern is common to both platforms. The strategic difference lies in how effectively each cloud can implement it within the organisation's existing environment.

4.1. Azure and AWS are implementation choices, not different agent architectures

The most important conclusion is that Azure and AWS should not be treated as fundamentally different approaches to agentic AI.

Both provide capabilities for:

• model access and routing;

• agent orchestration and runtime execution;

• tool and API integration;

• enterprise data and knowledge retrieval;

• persistent state and memory;

• identity and access control;

• security and policy enforcement;

• observability and evaluation;

• event-driven and asynchronous execution;

• scalable compute; and

• governance and operational management.

The architectural principles therefore remain constant regardless of cloud provider.

The model should not be the execution authority.

The model proposes an action; deterministic platform services, policies, permissions, and business rules determine whether and how that action can execute.

Tools should be treated as controlled interfaces to business capabilities.

Agents should not receive unrestricted access to enterprise systems. They should interact through narrowly defined, authenticated, authorised operations.

Knowledge and actions should remain distinct.

Retrieval provides information; tools perform actions. Combining the two without clear controls increases the risk of unintended behaviour.

State should be explicit.

Long-running agents require durable state, checkpoints, retries, and recovery mechanisms rather than relying entirely on conversational context.

Verification is part of execution.

A successful API call does not necessarily mean a successful business outcome. Agents should verify the actual state after important actions.

These principles apply equally to Azure and AWS. The choice of platform determines the services used to implement them, not the underlying architecture.

4.2. Where Azure has advantages

Azure's principal advantage is its strong alignment with organisations already invested in the Microsoft ecosystem. Where Microsoft 365, Entra ID, Power Platform, Dynamics, Azure data services, Microsoft security tooling, and Microsoft developer technologies are already strategic components of the enterprise estate, Azure can provide a relatively coherent path from existing systems to agentic applications.

Microsoft identity integration is a significant advantage in Microsoft-centric environments. Entra ID can provide a common identity foundation across users, applications, agents, and enterprise resources. This can simplify the implementation of authentication, role-based access, managed identities, and conditional access policies.

Microsoft enterprise integration is another major strength. Organisations using Microsoft 365, Dynamics, Teams, SharePoint, Power Platform, and other Microsoft services can often connect agentic workflows to existing business processes without introducing an entirely separate integration architecture.

Power Platform can also reduce the barrier to introducing agentic capabilities into business workflows. Agents can be incorporated into existing low-code and workflow environments, allowing some use cases to be developed closer to the business process rather than exclusively through traditional software engineering.

Azure's data ecosystem provides strong integration with enterprise data platforms, including Azure SQL, Azure Data Lake, Microsoft Fabric, and Azure AI Search. This can be particularly valuable where organisations already operate a Microsoft-based data estate and want agents to work against governed enterprise information.

Security and governance integration is another potential advantage. Organisations already standardised on Microsoft's security and compliance ecosystem can align agentic workloads with existing identity, policy, monitoring, data governance, and security operations processes.

However, these advantages are strongest when the organisation is already significantly invested in Microsoft technologies.

Azure can become less compelling where an organisation has a predominantly heterogeneous or non-Microsoft estate, particularly if adopting Azure introduces additional integration boundaries rather than reducing them. Microsoft services can also create a strong platform dependency, meaning that architectural decisions made for convenience in the short term may increase switching costs over time.

There is also a risk of treating the Microsoft ecosystem itself as the architecture. A sound agentic architecture should remain vendor-neutral at its conceptual level, even when Azure is the selected implementation platform.

4.3. Where AWS has advantages

AWS's principal advantage is the breadth and maturity of its cloud-native infrastructure and its strong position in organisations that have built their technology estate around AWS services.

Amazon Bedrock provides access to foundation models while fitting into the broader AWS environment. This enables organisations to construct agentic systems using AWS-native identity, compute, storage, networking, security, eventing, and operational services.

AWS's composability is a significant architectural strength. Agents can be assembled from relatively independent services such as Bedrock, Lambda, Step Functions, EventBridge, SQS, API Gateway, DynamoDB, S3, and OpenSearch. This makes AWS particularly well suited to organisations that prefer building explicit architectures from loosely coupled cloud services.

Event-driven architecture is another strong area. EventBridge, SQS, and SNS provide established patterns for asynchronous execution, messaging, retries, and decoupling. This is valuable for agents that operate in response to enterprise events rather than only through synchronous conversational interactions.

AWS's compute ecosystem also provides considerable flexibility. Agent runtimes and tools can be implemented using Lambda, containers, ECS, EKS, or other AWS compute services. This supports a broad range of workloads from lightweight event-driven functions through to complex, long-running or specialised applications.

AWS's security and operational ecosystem is similarly extensive. IAM, KMS, Secrets Manager, CloudTrail, CloudWatch, Config, Security Hub, GuardDuty, and related services allow agentic applications to be incorporated into established AWS security and operations practices.

AWS can therefore be particularly attractive to organisations with an existing AWS operating model, a cloud-native engineering culture, and a preference for composable infrastructure.

The principal disadvantage is that this flexibility can also increase architectural complexity. AWS provides a very large number of services and configuration options, and constructing an agentic platform from those components can require substantial architecture and engineering capability.

The organisation may need to make more explicit decisions about orchestration, state, integration, security boundaries, observability, and operational patterns. This provides flexibility but can also result in inconsistent implementations if a common enterprise architecture is not established.

AWS can also create significant platform dependency when an agent becomes deeply integrated with AWS-native services. As with Azure, this is not inherently a problem, but it should be recognised as an architectural consequence.

4.4. Azure versus AWS: the practical trade-off

The comparison is therefore less about which platform has the “better” AI stack and more about which platform provides the better system-level fit for the organisation.

For organisations with an established Microsoft technology estate, Azure has a clear advantage. Existing investments in Microsoft Entra, Microsoft 365, Dynamics, Power Platform, Azure data services, and Microsoft security capabilities can be incorporated directly into the agentic architecture. An organisation with an established AWS estate has the equivalent advantage on AWS, where existing IAM, data, integration, compute, and cloud-native services can be reused. Selecting the alternative platform may introduce additional integration, operational complexity, and skills requirements without providing a corresponding architectural benefit.

Enterprise identity is strong on both platforms, but each is naturally aligned with its respective ecosystem. Azure benefits from deep integration with Microsoft Entra and the wider Microsoft identity model, while AWS provides extensive integration through IAM and related identity and access services. The preferred platform is therefore often the one that aligns most closely with the organisation's existing identity architecture and security operating model.

The same pattern applies to Microsoft 365, Dynamics, and Power Platform. Azure has a strong advantage where agents need to operate across these services and existing business processes. AWS can integrate with these environments, but doing so may require additional APIs, integration services, or engineering effort.

Both platforms provide strong cloud-native composability, allowing agentic solutions to be assembled from independent services. AWS provides particularly extensive service choice, giving engineering teams considerable flexibility in how they construct runtimes, integrations, event flows, and supporting infrastructure. Azure provides similar flexibility while offering a potentially more integrated experience for organisations already operating within the Microsoft ecosystem.

For event-driven architecture, both platforms are capable, with AWS having a particularly mature and extensive set of services for event routing, messaging, asynchronous processing, and distributed workflows. Azure provides strong equivalents and can be especially effective where event-driven agents need to participate in Microsoft business and workflow environments.

Both platforms provide strong data and knowledge integration, but their relative advantage depends heavily on the existing data estate. Azure can be particularly effective where enterprise data is already centred on Microsoft technologies such as Azure SQL, Fabric, Data Lake, or Azure AI Search. AWS provides a similarly strong foundation where data is primarily held in AWS-native services such as S3, OpenSearch, Aurora, and DynamoDB.

In terms of developer flexibility, both platforms are strong. AWS generally provides a very broad set of composable services and infrastructure options, giving experienced engineering teams substantial architectural control. Azure provides comparable flexibility while also offering strong integration with Microsoft's development, application, data, and low-code ecosystem.

Low-code business integration is a more significant differentiator. Azure has a strong advantage through Power Platform, particularly for organisations that want agents to participate directly in existing business workflows and enable business teams to build or extend automation. AWS tends to be more engineering-oriented, although it provides extensive APIs, workflow, event, and application services for building equivalent solutions.

Both platforms have broad operational service portfolios, although AWS has an exceptionally extensive range of infrastructure and platform services. This breadth is a strength for organisations that require fine-grained architectural control, but it can also increase the number of design and operational decisions that need to be governed.

This leads to an important trade-off around architectural simplicity. Azure can provide a simpler implementation in a Microsoft-centric organisation because identity, productivity applications, business systems, data, security, and workflow services may already share a common platform. AWS can require more explicit service composition, particularly when constructing the complete agent runtime and surrounding control architecture from individual cloud services. However, that additional composition also provides flexibility and can be an advantage for organisations with strong cloud engineering capabilities.

Both platforms offer strong platform flexibility, but neither should be considered inherently portable. Agentic applications that make extensive use of native identity, orchestration, data, security, monitoring, and integration services will inevitably develop some dependency on the underlying cloud platform.

The risk of service sprawl is therefore a consideration for both platforms. Azure presents a moderate risk where organisations adopt a growing collection of Microsoft services without establishing clear architectural boundaries. AWS presents a potentially higher risk because of the breadth of its service catalogue and the number of ways a given capability can be implemented. In both cases, enterprise architecture standards, approved patterns, reusable components, and central governance are important to prevent each agent from becoming its own bespoke platform.

Vendor lock-in is material on both platforms. This should not automatically be treated as a reason to avoid native services. The greater concern is unmanaged dependency. Organisations should deliberately decide which elements need to remain portable—such as business APIs, tool contracts, agent interfaces, data access patterns, evaluation suites, and policy definitions—and where deeper use of cloud-native capabilities provides sufficient value to justify platform dependency.

The resulting best fit is therefore contextual rather than universal. Azure is generally the stronger fit for Microsoft-centric enterprises, particularly where Microsoft identity, productivity applications, business applications, data services, and Power Platform are already strategic components of the technology estate. AWS is generally the stronger fit for AWS-centric and cloud-native enterprises, particularly where the organisation values composable infrastructure, extensive service choice, event-driven architectures, and a strong engineering-led operating model.

The decision should therefore be based on enterprise alignment rather than AI feature comparison. The platform that best connects the agent to the organisation's existing identity, data, workflows, systems of record, engineering capabilities, security controls, and governance model will generally provide the strongest foundation for enterprise-scale agentic AI.

This comparison should not be interpreted as a universal ranking. The same characteristic can be an advantage or disadvantage depending on the organisation.

For example, AWS's broad service catalogue may be an advantage for an experienced cloud engineering organisation that wants architectural control. The same breadth may be a disadvantage for an organisation seeking a highly integrated platform with fewer implementation decisions.

Similarly, Azure's integration with Microsoft business applications may dramatically simplify an agentic workflow in one organisation while providing little benefit to another organisation whose systems of record are predominantly running on AWS, Salesforce, SAP, or other platforms.

4.5. The real decision criteria

The strategic platform decision should therefore begin with the organisation's existing architecture rather than with a benchmark of foundation models.

The first question should be:

Where does the work already happen?

Identify the systems of record, business applications, data platforms, identity providers, workflow engines, APIs, and operational processes that the agent must interact with.

The second question should be:

Where does the organisation already have engineering and operational capability?

An organisation with established Azure skills, Microsoft security controls, Entra expertise, and Microsoft data services has a different starting point from an organisation with an AWS-native engineering platform, Kubernetes expertise, and AWS-based security operations.

The third question should be:

Where can the agent operate safely?

The platform must support the required identity model, least-privilege access, secrets management, policy enforcement, data protection, auditability, human approval, and security monitoring.

The fourth question should be:

Where can the agent operate reliably?

The architecture needs durable state, retries, timeouts, checkpoints, event handling, observability, failure recovery, and controlled execution. A model that performs well in a demonstration is insufficient if the surrounding system cannot reliably complete the business process.

The fifth question should be:

Where can the organisation govern the agent at scale?

Enterprise agentic AI requires governance of models, prompts, tools, data access, agent configurations, versions, costs, evaluations, security events, and human approvals. Governance should be designed into the platform rather than added after deployment.

Finally, the organisation should ask:

What is the cost of remaining portable?

Using cloud-native capabilities can accelerate delivery and improve integration, but deep adoption of platform-specific services increases switching costs. The appropriate response is not to avoid native services altogether. Instead, organisations should identify which architectural interfaces should remain portable—such as agent definitions, tool contracts, business APIs, data access patterns, evaluation suites, and policy models—while allowing infrastructure implementations to remain cloud-specific where this provides genuine value.

4.6. A common reference architecture across both clouds

The common architecture can therefore be expressed independently of the provider:

Experience → Identity → Agent Intelligence → Runtime → Context & Memory → Governed Tools → Enterprise Systems → Verification → Observability & Governance

Azure and AWS then provide different implementations of those capabilities.

The important architectural boundary is between probabilistic intelligence and deterministic control.

The model operates probabilistically. It interprets intent, reasons over context, proposes plans, selects tools, and generates responses. The surrounding platform should provide deterministic controls over identity, permissions, tool contracts, business rules, state transitions, approvals, logging, and execution.

This leads to a useful abstraction:

Model → Context → Decision → Governed Tool → Verification → Outcome

The model determines what it believes should happen. The runtime determines what is allowed to happen. The tool executes the approved operation. The enterprise system becomes the source of truth for the resulting state. Verification determines whether the intended outcome actually occurred.

This pattern should remain stable even if the underlying model, agent framework, cloud provider, or individual services change.

4.7. Architectural recommendation

For most organisations, the recommended approach is therefore not to select Azure or AWS because one has a universally superior agent platform.

Instead:

1. Start with the business workflow. Define the task the agent must complete and the measurable outcome required.

2. Map the systems of record. Identify the data, applications, APIs, and operational systems the agent must access.

3. Map the identity and security model. Determine how users, agents, tools, and services will be authenticated and authorised.

4. Define the tool boundary. Expose business capabilities through controlled APIs and tools rather than providing unrestricted system access.

5. Define the runtime. Establish state, orchestration, retries, verification, escalation, and human approval.

6. Select the cloud implementation. Choose the platform that provides the strongest fit with the existing estate, skills, security model, data architecture, and operating model.

7. Keep the architectural interfaces stable. Separate business capabilities and agent contracts from cloud-specific implementation details where portability has strategic value.

8. Measure the outcome. Evaluate task completion, reliability, safety, latency, cost, and business value—not simply model quality.

The result should be a cloud-aligned but architecturally coherent Agentic AI Stack. Azure and AWS can both provide the infrastructure required to make the agentic loop reliable, secure, observable, and operationally accountable. The better choice is the platform that reduces integration complexity, fits the organisation's operating model, provides the required controls, and enables the target workflows to be delivered and governed at scale.

The strategic question is therefore not:

"Which cloud has the better AI service?"

It is:

"Which platform provides the best environment for our agents to access the right context, take the right actions, verify the outcome, and operate safely at enterprise scale?"

5. References

Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P. and Roberts, K. (2024) Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1 Gaithersburg, MD: National Institute of Standards and Technology.

Lewis, P. et al. (2020) ‘Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks’, Advances in Neural Information Processing Systems, 33.

Microsoft (2026a) What is Microsoft Foundry Agent Service? Microsoft Learn.

Microsoft (2026b) Agent architecture components. Microsoft Learn.

Yao, S. et al. (2023) ‘ReAct: Synergizing Reasoning and Acting in Language Models’, International Conference on Learning Representations (ICLR).

Contact

Reach out via email for inquiries.

Email

Subscribe to newsletter

info@grcadvisory.ch

© 2025. All rights reserved.