Fundamentals of AI Architecture

AI architecture is evolving from building better models to engineering governed intelligent systems—where models, data, context, memory, tools, infrastructure, security, evaluation, and human oversight work together to deliver trustworthy enterprise capabilities.

Sanchez P.

9/23/2026167 min read

Abstract

Artificial intelligence (AI) architecture is increasingly becoming a systems-engineering discipline rather than a problem confined to the selection or development of machine-learning models. The emergence of large language models (LLMs), foundation models, retrieval-augmented generation (RAG), tool-using agents, and increasingly autonomous AI systems has expanded the architectural problem from model inference to the coordinated design of data, context, memory, orchestration, tools, security, evaluation, observability, infrastructure, and governance. This paper develops a conceptual framework for the fundamentals of contemporary AI architecture by tracing these developments to their underlying research and technical foundations and examining their implications for enterprise systems.

The paper argues that an enterprise AI system should be understood as a socio-technical system in which the foundation model represents one component within a broader architectural environment. Reliable and useful AI behaviour emerges from the interaction between the model and that environment. Data and knowledge systems provide authoritative information; retrieval mechanisms provide task-relevant context; memory provides state and persistence; orchestration coordinates execution; tools and APIs provide controlled access to external capabilities; infrastructure provides the runtime environment; evaluation establishes evidence of system performance; security constrains information and execution boundaries; observability provides evidence of operational behaviour; and governance establishes accountability, control, and lifecycle management. Agentic systems further introduce iterative planning, action, observation, and feedback, transforming AI applications from predominantly stateless inference services into stateful and adaptive software systems.

The analysis synthesises these developments into seven fundamental architectural principles: separation of intelligence from system control; externalisation of dynamic knowledge and context; bounded autonomy; modularity and interoperability; security and governance by design; continuous evaluation and observability; and accountable human and organisational authority. Together, these principles establish a framework for designing enterprise AI systems in which model capability is combined with explicit architectural boundaries, controlled execution, continuous evidence, and appropriate accountability.

The paper concludes that the fundamental architectural unit of enterprise AI should not be the model in isolation but the AI-enabled business capability. Sustainable value depends on the interaction of model capability with data quality, context, workflow integration, control, infrastructure, and organisational adoption. The objective of AI architecture is therefore not maximum autonomy, but trustworthy capability within an appropriately controlled system.

Keywords: artificial intelligence architecture; AI systems engineering; foundation models; large language models; generative AI; agentic AI; retrieval-augmented generation; AI governance; AI security; enterprise architecture.

1. Introduction

The architecture of artificial intelligence (AI) has undergone a significant transformation with the emergence of foundation models, large language models (LLMs) and increasingly agentic forms of artificial intelligence. Earlier generations of enterprise AI were typically implemented as relatively bounded computational systems in which a model received defined inputs, performed a classification, prediction or optimisation task, and returned an output. Although such systems could be highly sophisticated, their functional boundaries were generally established in advance by the data, model, workflow and decision logic designed by their developers. Contemporary generative AI systems introduce a fundamentally different architectural paradigm. They can retrieve information from external knowledge sources, interact with application programming interfaces (APIs), invoke software tools, maintain state and memory, generate executable artefacts, evaluate intermediate results and coordinate multi-step activities. AI is therefore increasingly becoming an active component of enterprise processes rather than a discrete analytical function embedded within them.

This development changes the fundamental architectural question. The central concern is no longer simply which model provides the highest level of performance for a particular task, but how models, data, knowledge, software tools, infrastructure, workflows, security controls and human oversight can be integrated into a coherent and controllable intelligent system. The distinction is important because the capabilities of an AI model do not, in themselves, determine the capabilities, reliability or risk profile of the system in which that model operates. The architecture surrounding the model determines what information it can access, what actions it can perform, how those actions are constrained, how outputs are evaluated and how failures are detected and managed. AI architecture must therefore be understood as a systems-engineering discipline concerned with the design of the complete computational and organisational environment within which artificial intelligence operates.

The technical foundations of this transition can be traced to several developments in machine learning and AI research. Vaswani et al. (2017) introduced the Transformer architecture, establishing the attention-based architecture that subsequently became fundamental to the development of modern large-scale language models. Subsequent research demonstrated that increases in model parameters, training data and computational resources could produce predictable improvements in language-model performance and increasingly general capabilities (Kaplan et al., 2020). Brown et al. (2020) subsequently demonstrated that sufficiently large language models could perform a broad range of tasks through prompting and few-shot learning, without requiring a separate task-specific model for each application. These developments contributed to the emergence of foundation models: large pretrained models capable of supporting multiple downstream applications and domains rather than being developed exclusively for a single narrowly defined task (Bommasani et al., 2021).

The increasing generality of foundation models, however, does not remove the need for application and enterprise architecture. A model's knowledge is substantially determined by its training data and parameters, while its behaviour remains probabilistic and dependent on the context provided at inference time. More importantly, a model cannot be assumed to possess reliable access to current, organisation-specific or dynamically changing information. These limitations create a fundamental architectural distinction between knowledge encoded within the model and knowledge maintained within external systems. Retrieval-augmented generation (RAG) provides one important response to this problem by combining the generative capabilities of a pretrained model with information retrieved from an external knowledge source (Lewis et al., 2020). This approach establishes a pattern in which models can be separated from the authoritative information on which enterprise applications depend, allowing knowledge to be updated and governed independently of model retraining.

The same architectural principle becomes more significant as AI systems move from generation towards action. Agentic AI extends the role of the model by enabling iterative interaction with an external environment. Rather than producing a single response from a fixed context, an agent can reason about a task, select an action, observe its result and use that information to determine subsequent actions. The ReAct framework illustrates this development by combining reasoning and action within an iterative process in which external interactions can inform subsequent reasoning (Yao et al., 2023). Reflexion extends this approach by introducing feedback and verbal reflection that can be retained as episodic information and used to improve subsequent attempts (Shinn et al., 2023). Voyager demonstrates a further progression towards persistent, open-ended interaction through the acquisition and reuse of skills within an external environment (Wang et al., 2023).

These developments are architecturally significant because they introduce capabilities that cannot be adequately represented by the model alone. An agentic system requires mechanisms for managing state, memory, context, tool access, permissions, execution, feedback and termination. It must also operate within defined security and governance boundaries. The transition from generative AI to agentic AI therefore represents not simply an increase in model capability, but a change in the nature of the system being engineered. The architectural object shifts from an AI model that produces an output to an intelligent system capable of participating in, and potentially acting upon, an organisational environment.

This distinction is particularly important in enterprise contexts. Once AI systems are connected to organisational data, applications and business processes, their behaviour becomes subject to the same requirements that govern other critical information systems, including security, reliability, data governance, auditability, resilience and accountability. At the same time, their probabilistic and adaptive characteristics introduce additional requirements for evaluation, monitoring and human oversight. The NIST AI Risk Management Framework consequently treats governance and risk management as lifecycle-wide activities rather than controls that can be applied only after an AI system has been developed (NIST, 2023). Similarly, ISO/IEC 42001 establishes an organisational management-system approach to the governance and continual improvement of AI systems (ISO, 2023).

The architecture of AI should therefore be understood as extending beyond the model itself. It encompasses the mechanisms through which intelligence is connected to data and knowledge, contextualised through retrieval and memory, operationalised through tools and APIs, coordinated through workflows and agents, executed through infrastructure, and constrained through security, governance and human oversight. The resulting system is inherently socio-technical: its behaviour emerges not only from the statistical properties of the underlying model but also from the data, interfaces, policies, workflows and organisational structures within which that model operates.

The fundamental architectural challenge is consequently to transform increasingly capable models into reliable, secure, controllable and valuable intelligent systems. AI architecture is therefore best understood not as the selection or deployment of an individual model, but as the systematic design of the computational, informational and organisational environment in which artificial intelligence can operate effectively and accountably.

2. From Model-Centric AI to AI Systems Engineering

The foundation of contemporary AI architecture remains the AI model, but the architectural significance of the model has changed substantially with the emergence of foundation models and generative AI. The introduction of the Transformer architecture by Vaswani et al. (2017) established an attention-based approach to sequence modelling that provided an important technical foundation for the subsequent development of large-scale language models. Unlike earlier architectures that relied primarily on recurrent or convolutional mechanisms, the Transformer enabled highly parallelised processing and provided an architecture that could scale effectively with increasing computational resources and training data (Vaswani et al., 2017).

The subsequent development of increasingly large language models reinforced the importance of scale as a driver of model capability. Kaplan et al. (2020) identified systematic relationships between model performance, model size, dataset size and computational resources, demonstrating that performance improvements could be achieved predictably through appropriate scaling of these factors. Brown et al. (2020) further demonstrated that sufficiently large language models could perform a broad range of tasks through prompting and few-shot learning, without requiring task-specific parameter updates. These developments fundamentally changed the economics and architecture of AI application development. Rather than constructing a separate model for every narrowly defined task, organisations could increasingly build applications around general-purpose models capable of supporting multiple forms of reasoning, generation and transformation.

This development encouraged a model-centric conception of artificial intelligence in which increasing model scale and capability appeared to provide an increasingly general solution to a wide range of computational problems. The emergence of foundation models reinforced this perspective by establishing pretrained models as reusable technological foundations upon which multiple downstream applications could be constructed (Bommasani et al., 2021). However, while model capability is a necessary component of contemporary AI systems, it is not sufficient to constitute an enterprise AI architecture.

The distinction becomes apparent when AI is moved from experimentation into production. A production AI system must operate within an environment in which the model's access to information, ability to interact with other systems, authority to perform actions and exposure to risk are explicitly defined. The architectural problem therefore extends beyond model selection and inference. It encompasses the sources from which information is obtained, the mechanisms through which relevant context is assembled, the tools and enterprise systems that the AI can access, the permissions associated with those interactions, the management of application and agent state, and the mechanisms through which outputs and actions are evaluated. It must also address how failures are detected and contained, how security boundaries are enforced, how system behaviour is recorded and audited, and where responsibility for consequential decisions ultimately resides.

This distinction can be expressed in terms of the relationship between inference and system behaviour. A model produces an inference or generated output in response to a given input and context. An AI system, by contrast, establishes the conditions under which that inference is produced, determines what information and capabilities are available to the model, constrains what may happen as a consequence of the output, and provides mechanisms for monitoring and correcting system behaviour. The model is therefore a component of the system rather than a complete representation of the system itself.

The significance of this distinction becomes particularly clear when external knowledge and tools are introduced. Retrieval-augmented generation separates the model from external knowledge by allowing relevant information to be retrieved from controlled sources at inference time (Lewis et al., 2020). Agentic architectures extend this principle further by enabling models to interact iteratively with external environments. ReAct demonstrates how reasoning can be combined with actions that provide additional information or produce changes in an environment (Yao et al., 2023), while Reflexion illustrates how feedback and retained experience can influence subsequent behaviour (Shinn et al., 2023). The resulting architecture is no longer adequately described as a model receiving an input and producing an output. It is a system capable of repeatedly interpreting context, selecting actions, observing their consequences and adapting its subsequent behaviour.

This transition introduces a corresponding expansion in the architectural responsibility of the enterprise. Once AI can access organisational data or invoke external capabilities, the system becomes subject to requirements that extend beyond conventional model performance. Access to data must be governed; interactions with enterprise applications must be authorised; actions must be appropriately bounded; system behaviour must be observable; and consequential activities may require human review. Security therefore becomes an architectural property rather than merely a characteristic of the underlying model. The OWASP Top 10 for Large Language Model Applications identifies, among other risks, prompt injection, sensitive information disclosure, supply-chain vulnerabilities, improper output handling and excessive agency, illustrating how risks can emerge from the interaction between a model and its surrounding application architecture rather than from the model alone (OWASP, 2025).

The same principle applies to governance. AI systems cannot be governed effectively through model documentation alone because many important risks arise from the way a model is deployed and used. The NIST AI Risk Management Framework positions governance as a cross-cutting function spanning the AI lifecycle, linking organisational accountability with the identification, measurement and management of AI risks (NIST, 2023). ISO/IEC 42001 similarly establishes an organisational management-system approach for governing AI and promoting continual improvement throughout the lifecycle of AI systems (ISO, 2023). These approaches reinforce the architectural proposition that governance must be reflected in system design and operational controls rather than treated solely as an external compliance activity.

The transition can therefore be understood as a progression from model-centric AI towards systems-centric AI. In the earliest stage, the principal architectural concern is the model and its inference capabilities. As applications become more sophisticated, the model is combined with external context and knowledge. The addition of tools and APIs allows the system to interact with enterprise environments, while memory and orchestration allow it to maintain state and execute multi-step processes. With agentic architectures, these capabilities can be coordinated dynamically rather than through entirely predetermined workflows. At enterprise scale, however, this progression must be complemented by security, governance, evaluation, observability and human accountability. The resulting system is not simply an increasingly capable model but a governed intelligent system embedded within a broader organisational architecture.

This perspective also changes the role of AI systems engineering. The objective is no longer simply to maximise model performance, but to engineer the interaction between models and their operating environment. System quality becomes dependent upon the combined performance of the model, data, context, retrieval mechanisms, tools, orchestration, infrastructure and control mechanisms. A highly capable model operating on poor-quality data may produce unreliable results; an accurate model with inappropriate permissions may create unacceptable operational risk; and a technically sophisticated AI application that is disconnected from organisational workflows may provide limited practical value.

The shift from model-centric AI to AI systems engineering therefore represents a change in the unit of architectural analysis. The relevant object is no longer the model considered in isolation, but the complete socio-technical system in which intelligence is embedded. This system encompasses computational components, information resources, interfaces, workflows, security controls, governance mechanisms and human actors. The architectural challenge is consequently to integrate these components in a manner that enables AI capabilities to be exploited while maintaining appropriate boundaries around information, authority, risk and accountability.

For enterprise AI, the fundamental principle follows directly: model capability provides the potential for intelligence, but architecture determines how that intelligence can be used, constrained, observed and governed. AI systems engineering is therefore the discipline through which model capabilities are transformed into reliable organisational capabilities.

3. The Layered AI Architecture

Contemporary AI systems are best understood as layered socio-technical architectures rather than as individual models or applications. As AI capabilities become integrated with external knowledge sources, software tools, enterprise applications and human decision-making processes, the functionality of the overall system becomes distributed across multiple architectural concerns. These concerns can be separated conceptually into distinct layers, each with a defined responsibility and controlled interfaces to the layers above and below it. Such separation does not prescribe a particular technology stack or implementation pattern. Rather, it provides an architectural framework for reasoning about the dependencies, boundaries and control mechanisms that collectively determine how an AI system behaves.

At the foundation of this architecture is the infrastructure and runtime layer. AI systems depend upon computational resources, storage, networking, execution environments and deployment mechanisms through which models and applications operate. The increasing scale of foundation models has made computational infrastructure an important determinant of AI system capability, cost and operational characteristics (Kaplan et al., 2020). Infrastructure therefore provides more than a passive execution environment. It establishes the availability, scalability, isolation and resilience characteristics within which higher-level AI components function.

Above the infrastructure lies the data and knowledge layer, which contains the information resources upon which AI applications depend. These resources may include structured enterprise data, documents, databases, knowledge repositories and other information sources. The distinction between model parameters and externally maintained knowledge is architecturally significant. Foundation models encode substantial information within their learned parameters, but applications frequently require access to information that is current, domain-specific, proprietary or subject to organisational governance. The use of external knowledge sources therefore allows information to be managed independently from model parameters and updated without retraining the underlying model (Bommasani et al., 2021; Lewis et al., 2020).

The context, retrieval and memory layer provides mechanisms through which relevant information is selected and made available to the model during execution. Retrieval-augmented generation demonstrated the value of combining parametric knowledge within a pretrained model with non-parametric knowledge retrieved from an external source (Lewis et al., 2020). This establishes an important architectural separation between the model responsible for generation and the systems responsible for supplying authoritative or dynamically changing information. Memory introduces a related but distinct capability. Whereas retrieval provides access to relevant external knowledge, memory can preserve information about previous interactions, tasks or experiences and make that information available to subsequent stages of execution. Research on language agents, including Reflexion, illustrates how retained experience can influence subsequent behaviour and improve performance across iterative interactions (Shinn et al., 2023).

The next layer comprises the reasoning and model capabilities of the system. This layer includes foundation models, language models and other machine-learning components that transform inputs and contextual information into predictions, representations, generated content or decisions. The Transformer architecture provided an important foundation for modern large-scale language models (Vaswani et al., 2017), while subsequent research demonstrated that scaling model size, training data and computational resources could produce increasingly general capabilities (Kaplan et al., 2020). Brown et al. (2020) further demonstrated that large language models could perform diverse tasks through prompting and few-shot learning. Nevertheless, the model layer should not be treated as synonymous with the AI system as a whole. Model capability is exercised within an environment established by the surrounding architectural layers, which determine what information the model receives, what actions can follow from its outputs and what controls apply to its operation.

The agent and orchestration layer coordinates these capabilities into processes that can extend beyond a single inference. Agentic systems can interpret objectives, determine intermediate steps, invoke tools, observe results and use those results to inform subsequent actions. The ReAct framework illustrates this pattern by integrating reasoning with external actions in an iterative process (Yao et al., 2023). Other research has explored mechanisms through which agents can retain experience, acquire skills and operate over extended sequences of interaction (Shinn et al., 2023; Wang et al., 2023). Architecturally, orchestration is therefore responsible for coordinating models, context, memory and tools rather than simply generating content. This layer is also where autonomy must be bounded: the system's ability to select actions does not imply unrestricted authority to execute them.

The tools and enterprise integration layer connects AI capabilities to the operational environment of the organisation. Tools may include APIs, databases, search services, software applications, execution environments and other machine-accessible capabilities. This layer transforms AI from a system that primarily generates information into one that can interact with external systems and potentially produce operational effects. The distinction is consequential because tool access introduces permissions, security boundaries and potential side effects that do not exist when a model is limited to generating text. The OWASP Top 10 for Large Language Model Applications identifies excessive agency and other application-level risks as significant concerns, illustrating that the security of an AI system depends partly on how model outputs are connected to external capabilities (OWASP, 2025).

Above these technical capabilities sits the interaction and experience layer, through which users and other applications interact with the AI system. This layer encompasses interfaces, conversational interactions, application workflows and mechanisms through which AI-generated information or actions are presented to users. Its architectural importance arises from the fact that the consequences of AI behaviour are mediated through human and application interactions. An AI system may therefore require different controls depending upon whether its output is presented as informational assistance, used to support a human decision or automatically propagated into another operational system. Human interaction is consequently not merely an interface concern but part of the system's control architecture.

At the highest level is the human and application layer, representing the organisational context in which AI capabilities are ultimately consumed and acted upon. AI systems operate within business processes, organisational responsibilities and decision structures rather than in isolation. Where AI contributes to consequential decisions or actions, responsibility must therefore remain connected to appropriate human or organisational authority. The NIST AI Risk Management Framework emphasises governance, accountability and lifecycle risk management as integral elements of trustworthy AI systems rather than controls that can be applied only after technical development has been completed (NIST, 2023). ISO/IEC 42001 similarly approaches AI governance as an organisational management system involving defined responsibilities, risk management and continual improvement (ISO, 2023).

The layered architecture is complemented by cross-cutting control capabilities that apply across the entire system rather than belonging to a single technical layer. Security is required at every boundary, from infrastructure and data access through model invocation, tool execution and user interaction. Governance establishes the policies, responsibilities and decision rights that determine how AI systems may be developed, deployed and operated. Evaluation provides mechanisms for assessing whether models and systems perform adequately for their intended purposes, including the identification of limitations and failure modes. Holistic evaluation research demonstrates the importance of assessing language models across multiple dimensions rather than relying on a single performance metric (Liang et al., 2022). Observability provides the operational visibility required to understand system behaviour, detect failures and support investigation and improvement.

These cross-cutting capabilities are particularly important because failures in AI systems frequently arise at the interfaces between layers rather than exclusively within individual components. A retrieval system may provide inaccurate or inappropriate context to an otherwise capable model. A model may produce a technically plausible output that is unsafe when passed directly to an external tool. An agent may operate correctly within its defined reasoning process but possess excessive permissions in the environment in which it executes. Similarly, a technically accurate model may provide limited organisational value if it is disconnected from the workflow in which its output is required. AI system reliability is therefore an emergent property of the interaction between components rather than a characteristic that can be attributed exclusively to the model.

This systems perspective also creates a strong architectural rationale for explicit responsibilities and controlled interfaces. Each layer should have a clearly defined purpose, while the interfaces between layers should establish what information, capabilities and authority can pass between them. Such separation supports modularity because individual components can be evaluated, replaced or upgraded without requiring the entire system to be redesigned. It also supports security and governance by creating identifiable control boundaries. The principle is consistent with broader research on machine-learning systems, which has shown that the complexity and technical debt of production AI systems can arise from the interaction of data, models, configuration, dependencies and surrounding infrastructure rather than from the learning algorithm alone (Sculley et al., 2015).

The layered architecture should therefore not be interpreted as a rigid sequence through which every AI system must be implemented. Some systems may combine layers, omit particular capabilities or introduce additional architectural components according to their purpose and risk profile. A simple predictive application may require neither an agentic orchestration layer nor persistent memory, whereas a complex enterprise agent may require multiple retrieval systems, specialised models, workflow engines and tool interfaces. The value of the layered approach lies instead in providing a conceptual separation of concerns through which the architecture can be analysed independently of any particular vendor, deployment model or technology.

The resulting principle is that AI architecture should be designed around explicit functional boundaries, controlled interfaces and cross-cutting controls. Models provide computational intelligence, but they operate within a broader architecture that determines how information is supplied, how context is constructed, how actions are selected and executed, how users interact with the system, and how behaviour is secured, evaluated, observed and governed. The layered architecture therefore provides a foundation for moving from an understanding of AI as a model towards an understanding of AI as an engineered and governable system.

4. Foundation Models and the Intelligence Layer

Foundation models constitute the principal reasoning, representation and generation component of many contemporary AI systems. Their emergence has changed the architecture of AI application development by separating the development of general-purpose model capabilities from the implementation of individual applications. Rather than training a dedicated machine-learning model for every application, organisations can increasingly consume general-purpose models and adapt their behaviour through prompting, retrieval, fine-tuning and interaction with external tools (Bommasani et al., 2021). This creates an architectural abstraction in which intelligence becomes a reusable computational capability rather than an application-specific component.

The technical foundations of this development can be traced to the Transformer architecture introduced by Vaswani et al. (2017). The Transformer provided an attention-based architecture that could be trained efficiently at scale and subsequently became a foundation for large language models. Research into scaling laws demonstrated systematic relationships between model performance, model size, training data and computational resources, providing evidence that increasing scale could produce predictable improvements in model capability (Kaplan et al., 2020). Brown et al. (2020) subsequently demonstrated that sufficiently large language models could perform a wide range of tasks through prompting and few-shot learning. These developments established the technical basis for treating a single model as a general-purpose intelligence resource capable of supporting multiple applications.

The architectural significance of foundation models lies therefore not only in their capabilities but also in their position within the application stack. A foundation model can provide common capabilities across multiple applications while the surrounding architecture determines how those capabilities are contextualised and operationalised. Prompting can shape model behaviour at inference time; fine-tuning can adapt a model to particular domains or behaviours; retrieval can provide access to external knowledge; and tools can extend the model's ability to interact with external systems (Bommasani et al., 2021; Lewis et al., 2020; Yao et al., 2023). The model consequently becomes one component in a larger computational system rather than the complete implementation of an AI capability.

This separation creates an important architectural distinction between intelligence and application logic. Application logic determines how a particular business process is implemented, while the foundation model provides general-purpose capabilities that can be invoked by that process. Maintaining this separation can reduce coupling between the application and the underlying model and can allow model capabilities to evolve independently of business-process logic. It also creates the possibility of using different models for different tasks, according to requirements such as capability, latency, cost, privacy or deployment constraints.

The abstraction is nevertheless imperfect. Although foundation models can be consumed through relatively standard interfaces, their behaviour is not necessarily interchangeable. Different models can respond differently to identical prompts, exhibit different strengths and weaknesses, impose different context or output constraints, and produce different operational costs and latency characteristics. Model behaviour may also change between versions as providers modify training procedures, alignment methods, inference systems or other components. Consequently, an application that depends on a foundation model retains a substantive dependency even when the model is accessed through an abstraction layer.

This creates a form of model dependency risk analogous to dependencies found elsewhere in enterprise technology architectures. An application may be technically decoupled from the internal implementation of a model while remaining operationally dependent upon its availability, performance characteristics and behavioural properties. Changes to the model can therefore affect downstream applications even when the application's own code has not changed. Model versioning, provider changes and configuration changes can alter outputs, latency, cost or safety characteristics and may consequently affect the behaviour of the overall AI system.

The architectural response should be to treat the model interface as a controlled dependency boundary. Applications should interact with models through explicit interfaces that define the expected inputs, outputs and operational characteristics of the model service. Where appropriate, the architecture should separate application logic from provider-specific implementation details so that model changes do not propagate unnecessarily throughout the application. This principle does not imply that all models can or should be made fully interchangeable. Rather, it establishes a degree of abstraction that allows model dependencies to be identified, tested and managed explicitly.

Model selection should therefore be based on system requirements rather than model capability in isolation. Relevant criteria may include task performance, reasoning capability, domain suitability, latency, throughput, context capacity, cost, privacy characteristics, deployment requirements and security considerations. A model that performs well on a general benchmark may not necessarily provide the appropriate characteristics for a particular enterprise application. Evaluation should consequently be performed in relation to the intended use case and operating environment. Holistic evaluation research reinforces this principle by demonstrating the importance of assessing language models across multiple dimensions rather than relying on a single measure of performance (Liang et al., 2022).

Evaluation must also extend beyond the model itself. The quality of an AI system depends on the interaction between the model and its surrounding architecture, including the prompts, retrieved information, tools, workflows and controls through which the model operates. A model that performs adequately in isolation may behave differently when embedded within an agentic workflow or supplied with dynamically retrieved information. System-level evaluation is therefore required to determine whether the combined architecture satisfies its intended functional, operational and risk requirements.

The same principle applies to security and governance. Foundation models may be exposed to risks associated with their inputs, outputs, surrounding applications and connections to external systems. The OWASP Top 10 for Large Language Model Applications identifies risks such as prompt injection, sensitive information disclosure, improper output handling and excessive agency, illustrating that model-related risks cannot be addressed exclusively within the model itself (OWASP, 2025). Security controls must therefore be applied across the interfaces through which the model receives information, produces outputs and interacts with external capabilities.

Foundation-model abstraction also introduces considerations of provider and supply-chain dependency. Organisations that rely on externally provided models may become dependent upon a particular provider's availability, pricing, technical interfaces, geographic deployment options and service policies. The degree of dependency becomes particularly significant when an AI system is embedded in a critical business process. Architectural resilience may therefore require mechanisms such as provider abstraction, alternative model configurations, locally deployable models or defined fallback mechanisms, depending on the criticality of the application and the feasibility of alternative implementations.

Fallback strategies, however, should not be understood simply as maintaining a second model that can be substituted whenever the primary model becomes unavailable. Different models may exhibit materially different capabilities and behavioural characteristics. A fallback model must therefore be evaluated against the requirements of the specific application, and transitions between models may require changes in prompting, context management, output validation or workflow controls. Model substitution is consequently an architectural and operational concern rather than merely a configuration change.

Version management is similarly important. Because changes to foundation models can alter application behaviour without corresponding changes in application code, model updates should be subject to appropriate change-management processes. Depending on the criticality of the application, this may include version identification, regression testing, performance evaluation, security testing, monitoring of behavioural changes and controlled deployment. Such practices extend established software-engineering principles into an environment in which the behaviour of a major computational dependency may change even when the surrounding application remains unchanged.

The governance implications are consistent with broader AI risk-management principles. The NIST AI Risk Management Framework emphasises the need to identify, measure and manage risks throughout the AI lifecycle, while ISO/IEC 42001 establishes an organisational management-system approach to governing AI systems and supporting continual improvement (NIST, 2023; ISO, 2023). Within this context, the foundation model should be treated as a governed component whose selection, configuration, evaluation, deployment and replacement are subject to defined organisational responsibilities and controls.

The resulting architectural principle is that foundation models should provide intelligence without becoming the architectural centre of gravity. The model should be sufficiently abstracted that its role, dependencies and operational characteristics are explicit, while the surrounding system retains control over data, context, permissions, tools, workflows, evaluation and governance. This allows organisations to benefit from improvements in model capability without unnecessarily coupling business processes to a particular model, provider or version.

The objective is therefore not to eliminate dependence on foundation models, which would often be impractical, but to make that dependence visible, bounded and manageable. A mature AI architecture treats the foundation model as a replaceable or evolvable intelligence component within a controlled system. Its value derives not simply from what the model can generate or reason about, but from how effectively the surrounding architecture can contextualise, constrain, evaluate and operationalise those capabilities.

5. Context Engineering and Retrieval

A fundamental architectural limitation of contemporary language models is that the information encoded within their parameters cannot be assumed to be current, complete, organisation-specific or authoritative for a particular application. Foundation models acquire substantial knowledge during training, but their parametric representations are fixed at inference time unless the model is retrained or otherwise updated. This creates an architectural distinction between knowledge embedded within a model and knowledge that must remain dynamically accessible through external information systems (Bommasani et al., 2021).

This distinction provides the foundation for context engineering, understood broadly as the systematic design of the information and state made available to a model at inference time. Context includes not only retrieved documents, but also instructions, conversation history, task state, structured data, tool outputs and other information required for the model to perform a particular task. The architectural objective is therefore not simply to provide the model with more information, but to provide the right information, in the right form, at the right point in the execution process.

Retrieval-augmented generation (RAG) represents one of the most important architectural patterns for achieving this objective. Lewis et al. (2020) demonstrated an architecture that combines a pretrained generative model with an external retrieval mechanism, allowing relevant information to be retrieved from a non-parametric knowledge source and incorporated into generation. Rather than requiring all relevant information to be encoded within model parameters, the system can access external knowledge dynamically during inference.

The architectural significance of RAG extends beyond the introduction of a search mechanism. It establishes a separation between parametric knowledge and externally maintained knowledge. Parametric knowledge is represented within the learned parameters of the model, whereas external knowledge is maintained in information systems that can be updated, governed and accessed independently of the model. This separation allows organisations to treat certain forms of enterprise knowledge as a managed information resource rather than as an immutable component of the model itself.

The distinction is particularly important for enterprise applications because organisational knowledge is dynamic. Policies, procedures, regulations, contracts, product information, customer records and operational data may change independently of the model's training cycle. External retrieval allows such information to be updated without requiring the underlying model to be retrained. It also permits different applications or user groups to retrieve different information according to their context and authorised access.

Retrieval can additionally strengthen the relationship between AI outputs and their underlying sources. Where retrieved content is associated with identifiable documents, records or other authoritative sources, the system can preserve information about the origin of the context supplied to the model. Such provenance can support verification, auditing and user confidence, although retrieval alone does not guarantee that generated outputs will accurately represent the retrieved material. Provenance must therefore be treated as an architectural capability that supports evaluation and accountability rather than as proof of correctness.

The same architecture can support information-security controls. If retrieval operates against governed information repositories, access decisions can be incorporated into the process through which information is made available to the model. This creates the possibility of applying organisational permissions to AI-mediated access rather than treating the model as an unrestricted gateway to enterprise information. Such controls are particularly important because the model may otherwise expose information that the requesting user or application is not authorised to access. The architectural boundary between the model and the underlying information source must therefore preserve the security and governance properties of the source system.

However, introducing retrieval does not eliminate the problem of information quality. It changes the nature of the dependency. In a RAG system, the quality of the generated response depends partly upon whether the retrieval mechanism identifies information that is relevant, authoritative, current and sufficient for the task. A highly capable language model can produce a coherent response from inadequate context, meaning that improvements in model capability do not necessarily compensate for weaknesses in the information architecture.

Retrieval quality is consequently determined by several interacting factors. These include the quality and structure of the underlying documents, metadata, document segmentation, indexing strategy, retrieval method, ranking mechanisms and the representation of the user's information need. Poorly structured or outdated information can result in inappropriate context being supplied to the model. Conversely, excessive retrieval can introduce irrelevant information, increasing contextual complexity and potentially reducing the model's ability to identify the information that matters. Context engineering is therefore fundamentally a problem of information selection and representation, not merely information retrieval.

The issue becomes more complex when enterprise information exists across heterogeneous sources. Organisational knowledge may be distributed across databases, document repositories, collaboration platforms, transactional systems, data warehouses and specialised applications. These sources may use different schemas, identifiers, access models and definitions. Reliable AI therefore depends upon an architecture capable of establishing relationships between these information resources and presenting relevant context in a form that the AI system can use effectively.

This creates a direct relationship between AI architecture and data architecture. AI applications depend upon the quality, accessibility, structure and governance of the information from which their context is constructed. Data quality therefore becomes an AI-system concern rather than merely a data-management concern. If the underlying information is incomplete, inconsistent, ambiguous or outdated, the AI system may reproduce or amplify those deficiencies even when the model itself is functioning as designed.

AI-ready data architecture consequently requires more than the presence of a vector database or semantic search capability. It requires identifiable and authoritative sources of information, appropriate metadata, meaningful data definitions, lineage and provenance, access controls, versioning and mechanisms for assessing data quality. Depending on the application, organisations may require a combination of structured queries, keyword search, semantic retrieval, vector representations and other retrieval mechanisms. The architectural choice should follow the characteristics of the information and the requirements of the use case rather than assuming that a single retrieval technology is universally appropriate.

The use of vector representations is particularly important in many contemporary retrieval architectures because semantic similarity can support the discovery of conceptually related information even when the wording of the query and source material differs. However, semantic similarity is not equivalent to authority, correctness or relevance in a business context. A semantically similar document may be outdated, superseded or inappropriate for the user's purpose. Retrieval systems must therefore incorporate additional information about source authority, temporal validity, access rights and other contextual factors where these characteristics are material to the application.

The architecture should also distinguish between retrieval relevance and evidential validity. Retrieving a document that appears relevant does not establish that the information is correct, complete or applicable to the decision being supported. AI systems operating in consequential domains may therefore require mechanisms for identifying source provenance, document versions, effective dates and other evidence characteristics. These mechanisms can help users and downstream processes determine not only where information originated but also whether it should be relied upon for the particular task.

Context engineering also becomes increasingly important in agentic systems. An agent may retrieve information, invoke a tool, observe the resulting state and use that information to determine its next action. Context is therefore dynamic rather than fixed at the beginning of an interaction. Research on agentic architectures such as ReAct illustrates how external observations can become part of the context used for subsequent reasoning and action (Yao et al., 2023). The architecture must consequently manage not only document retrieval but also the accumulation, prioritisation and potential removal of information as the task progresses.

This introduces an additional concern: context must be governed as well as constructed. Information supplied to an AI system can influence subsequent reasoning and actions, making the provenance and trustworthiness of context relevant to system security. The OWASP Top 10 for Large Language Model Applications identifies prompt injection and related application-level risks that demonstrate how externally supplied content can influence model behaviour (OWASP, 2025). Context therefore constitutes part of the system's security boundary. External information should not automatically be treated as trusted instructions merely because it has been retrieved from an accessible source.

The resulting architecture requires a clear distinction between information that provides evidence, information that provides instructions, and information that represents state. A retrieved document may provide evidence about a business process without possessing authority to instruct the model to perform an action. A tool response may provide information about the current state of an external system without defining what the agent is permitted to do. Separating these functions helps prevent the contents of retrieved information from implicitly acquiring authority over the system's behaviour.

The broader implication is that reliable AI requires an information architecture capable of supporting authoritative, contextual, governed and traceable knowledge access. Retrieval should therefore not be treated as a technical add-on to a language model. It is an architectural mechanism through which organisational knowledge is connected to model capabilities, and its effectiveness depends upon the quality of the information systems surrounding it.

AI architecture is consequently inseparable from data architecture. The model provides the capability to interpret and generate information, while the data and knowledge architecture determines what information can be made available, under what conditions, with what provenance and subject to which controls. Reliable AI systems therefore require both capable models and appropriately engineered information environments.

The fundamental principle is that context should be treated as a governed architectural resource. Models should not be expected to contain all information required by an enterprise application. Instead, relevant knowledge should be maintained in appropriate external systems and made available through controlled mechanisms for retrieval, contextualisation and verification. In this architecture, the quality of AI depends not only on the intelligence of the model but also on the quality, governance and traceability of the information with which that intelligence is supplied.

6. Memory and State

Traditional software systems commonly maintain explicit application state through databases, session stores, caches and other persistence mechanisms. Early applications built around large language models were often comparatively stateless: an input was provided to the model, an output was generated, and the interaction ended without necessarily creating persistent information that could influence future executions. Agentic AI changes this architecture because systems that perform multi-step tasks, interact repeatedly with users or operate across extended workflows require mechanisms for maintaining and accessing state over time.

The distinction between context and memory is therefore important. Context refers to information made available to the model during a particular execution, whereas memory refers to information that is retained beyond the immediate execution and may subsequently be retrieved to influence future behaviour. A system may construct context from retrieved documents, current tool outputs and task instructions without storing those materials as persistent memory. Conversely, information that has been deliberately retained as memory may be incorporated into the context of a later interaction. This distinction provides an important architectural boundary between transient task information and persistent organisational or user information.

Memory can take several forms, each serving a different architectural purpose. The first is working memory, which contains the information required to perform the current task. This may include instructions, retrieved information, intermediate reasoning artefacts, tool outputs, task state and other information accumulated during an active interaction. Working memory allows an agent to maintain continuity across multiple steps without requiring every component of the task to be reconstructed from the beginning. In agentic systems, this state can change dynamically as the system observes new information and performs actions.

Working memory is particularly important for agentic execution because an agent may need to maintain a representation of the current task state while interacting with multiple external systems. The ReAct framework illustrates an architecture in which reasoning and external actions are interleaved, with observations from the environment contributing to subsequent reasoning (Yao et al., 2023). The resulting execution process depends upon retaining sufficient information about what has occurred, what remains to be done and what information has been obtained. Working memory therefore represents the transient state of an ongoing computation.

A second form is episodic memory, which concerns information derived from previous interactions, experiences or completed tasks that may be relevant to future behaviour. Episodic memory can allow an agent to retain information about previous attempts, outcomes or reflections rather than treating every new task as an entirely independent interaction. Reflexion provides an example of this principle by allowing language agents to generate verbal reflections that are retained and subsequently used to influence future attempts (Shinn et al., 2023). Such mechanisms demonstrate how retained experience can become an input to subsequent agent behaviour.

Episodic memory differs from simple conversation history because its purpose is not necessarily to preserve every previous interaction. Rather, the system may need to determine which experiences are sufficiently relevant or valuable to retain and how they should subsequently be retrieved. This introduces an architectural problem of memory selection and consolidation. Storing every interaction indefinitely may create excessive storage, retrieval noise, privacy exposure and governance complexity. Conversely, retaining too little information may prevent the system from learning effectively from previous interactions or maintaining continuity across tasks.

A third form is semantic memory, which represents relatively stable knowledge that can be retrieved independently of a particular interaction or episode. Semantic memory may include facts, concepts, organisational knowledge, user preferences or other information that is intended to remain useful across multiple tasks. In practical architectures, semantic memory may be implemented through structured databases, knowledge graphs, document repositories or vector-based retrieval systems. Its distinction from episodic memory is primarily functional: semantic memory represents knowledge that can be reused independently of the circumstances in which it was originally acquired, whereas episodic memory retains information about particular experiences or interactions.

These categories should not be interpreted as requiring three separate technologies. A single architectural system may support multiple forms of memory using combinations of databases, caches, retrieval systems, application state stores and other persistence mechanisms. The categories are useful because they distinguish different retention purposes, lifecycles and governance requirements. Working memory may exist only for the duration of a task, episodic memory may persist for a defined period, and semantic memory may require long-term maintenance and explicit ownership.

The architectural challenge is therefore not simply how to give an AI system more memory, but what information should be retained, for what purpose, for how long, and under whose authority. Memory creates persistence, and persistence creates responsibility. Once information survives beyond the immediate execution that generated it, the system must establish how that information is stored, accessed, modified, corrected and ultimately deleted.

This makes AI memory fundamentally a data-governance concern as well as a technical capability. Persistent memory may contain personal information, confidential business information, user preferences, records of previous decisions, internal organisational knowledge or information about interactions with customers and employees. The security and governance requirements associated with such information therefore cannot be separated from the architecture of the memory system itself.

Access control is particularly important because the ability to retrieve a memory can influence subsequent AI behaviour. A memory that was valid in one context may not be appropriate in another. For example, information associated with one user, business unit or process should not automatically become available to another context merely because it exists within the same memory infrastructure. Memory retrieval must therefore respect the authorisation boundaries that apply to the underlying information.

The system must also account for provenance and confidence. Persistent information may originate from user statements, retrieved documents, model-generated inferences, tool outputs or previous agent actions. These sources do not necessarily have equivalent evidential status. An architectural system that stores all generated information as if it were authoritative risks converting temporary model outputs into persistent organisational knowledge. Memory systems should therefore preserve sufficient provenance to distinguish, where relevant, between information supplied by an authoritative source and information generated or inferred by the AI system.

Retention and deletion are equally important. Not all information should be retained indefinitely, and some information may need to be removed when it is no longer required or when the conditions under which it was collected no longer apply. Memory architectures should therefore support explicit retention policies, deletion mechanisms, lifecycle management and, where appropriate, correction or updating of stored information. These requirements are consistent with the broader principle that AI systems should be governed throughout their lifecycle rather than treated as static technical artefacts (NIST, 2023; ISO, 2023).

Memory also introduces the possibility of state drift. Information that was accurate when stored may become outdated as organisational circumstances change. A persistent memory system can therefore preserve information beyond its period of validity. This creates a distinction between persistence and truth: storing information reliably does not guarantee that the information remains correct. Memory systems consequently require mechanisms for expiry, validation, refresh or re-evaluation where the underlying information is subject to change.

The problem becomes more significant in autonomous or agentic systems because memory can influence future actions without direct human intervention at every stage. A previous interaction or stored preference may affect how an agent interprets a later request, selects a tool or determines an action. Memory therefore becomes part of the system's effective decision context. Its design must consequently be considered alongside permissions, orchestration and tool access rather than as an isolated storage feature.

Memory can also contribute to system resilience and continuity. Persistent state may allow an agent to resume an interrupted task, maintain continuity across sessions or preserve information required for long-running processes. However, this benefit must be balanced against the risks created by persistent state. The more consequential the actions that depend upon memory, the greater the need for mechanisms to inspect, validate and override the information influencing those actions.

The resulting architectural principle is that memory should be treated as governed state rather than as unlimited AI recall. Working memory supports current execution; episodic memory can preserve relevant experiences; and semantic memory can provide persistent knowledge that remains available across interactions. Each form requires appropriate mechanisms for storage, retrieval, provenance, access control, retention and deletion.

Agentic AI therefore requires a deliberate architecture for state management. The objective is not to maximise the amount of information an AI system remembers, but to ensure that the information it retains is relevant, authorised, traceable, appropriately persistent and fit for future use. Memory becomes valuable when it improves continuity and system capability without allowing uncontrolled persistence to undermine security, privacy, data quality or organisational accountability.

7. Agentic Architecture: From Inference to Action

The defining architectural characteristic of agentic AI is the ability to operate through iterative interaction with an environment. A conventional generative AI application can often be understood as a bounded inference process in which an input is provided to a model and an output is returned. Agentic architectures extend this pattern by allowing the system to interpret an objective, determine intermediate steps, interact with external capabilities, observe the resulting state and use those observations to influence subsequent behaviour. The system therefore becomes an active participant in a computational environment rather than simply a generator of information.

The distinction is important because an agentic system introduces a control loop around model inference. The model may contribute reasoning, planning or action selection, but the overall system must also maintain state, execute actions, observe outcomes and determine whether further activity is required. ReAct provides an important illustration of this architecture by combining reasoning with external actions and allowing observations obtained through those actions to inform subsequent reasoning (Yao et al., 2023). The resulting process is iterative rather than linear: an action can change the information available to the system, and that new information can alter what the system does next.

The central architectural property introduced by this pattern is therefore feedback. In a conventional generative interaction, the quality of the response is largely assessed after generation. In an agentic system, the consequences of an action become part of the system's subsequent operating context. A search can return new information; an API call can change an external system; a software execution can produce an error or result; and a transaction can change the state of the environment. The agent must be capable of incorporating these observations into subsequent decisions.

Feedback consequently creates a distinction between generation and execution. The model may generate a proposed action, but another component of the architecture must determine whether and how that action is executed. This distinction is particularly important in enterprise environments because generated actions can have operational consequences. The architecture must establish which actions are permitted, which require validation, which require human approval and which are prohibited altogether. Agentic capability therefore does not imply unrestricted autonomy. Rather, useful enterprise agency requires autonomy to be bounded by explicit authority and control mechanisms.

ReAct demonstrates the basic relationship between reasoning and action, while subsequent research illustrates how agents can incorporate experience into longer-running behaviour. Reflexion extends the agentic pattern by enabling an agent to generate verbal reflections from feedback and retain those reflections for use in subsequent attempts (Shinn et al., 2023). The significance of this approach is architectural as much as behavioural: feedback can become persistent state that influences future execution rather than being discarded at the end of a single task.

Voyager provides another example of this progression. The system combines an agentic model with interaction in an external environment, executable code generation and a mechanism for retaining acquired skills for subsequent reuse (Wang et al., 2023). This demonstrates how an agent can move beyond individual action sequences towards the accumulation and reuse of capabilities. Such architectures reinforce the distinction between a language model and an agentic system: the agent emerges from the interaction between the model, environment, memory, tools and execution mechanisms.

An enterprise agent therefore requires substantially more than a capable language model. It requires an objective or task definition that establishes what the system is intended to accomplish. It requires state management so that the current condition of the task and environment can be maintained. It requires mechanisms for planning or sequencing actions, although the degree of explicit planning may vary between architectures. It requires access to tools and external systems through which actions can be performed. It requires execution mechanisms that translate proposed actions into controlled operations. It requires feedback mechanisms through which results and errors become available to subsequent processing. Finally, it requires termination conditions that determine when the system should stop.

The architecture must also distinguish between goal interpretation and action authority. A user may provide an objective without thereby granting the agent unrestricted permission to achieve that objective by any available means. The agent's task defines what it is attempting to accomplish, while its permissions define what it is authorised to do. This distinction is fundamental to secure agentic architecture. An agent may be capable of performing an action without being authorised to perform it, and a technically available tool does not necessarily constitute an appropriate capability for every agent or task.

Permissions should therefore be treated as an architectural property of tool and system integration. An agent interacting with enterprise systems may require access to databases, APIs, files, software applications or transactional systems. Those interfaces should expose only the capabilities necessary for the intended task, with appropriate authentication, authorisation and monitoring. The OWASP Top 10 for Large Language Model Applications identifies excessive agency as a significant risk category, highlighting the dangers that arise when applications grant AI systems excessive functionality, permissions or autonomy (OWASP, 2025).

The same principle applies to execution boundaries. A model may generate a command, query, transaction or piece of executable code, but the architecture should determine how that output is validated before it is allowed to affect an external environment. This may involve schema validation, policy checks, sandboxing, transaction limits, confirmation requirements or human approval depending upon the consequences of the action. The architecture should therefore distinguish between an AI system's ability to propose an action and its authority to execute that action.

Agentic systems also require explicit termination and failure-handling mechanisms. An iterative system can potentially continue generating actions as long as it receives new observations or continues to identify additional steps. Without appropriate termination conditions, this can result in unnecessary computation, repeated actions or uncontrolled interaction with external systems. Termination criteria may be based on task completion, predefined limits, confidence thresholds, time constraints, resource budgets, policy conditions or human intervention. These controls transform an open-ended interaction loop into a bounded computational process.

Error handling is similarly fundamental. Agentic systems operate in environments in which tools may fail, information may be incomplete, APIs may become unavailable and generated actions may produce unexpected results. A robust architecture must therefore distinguish between recoverable and unrecoverable failures and determine how the system should respond. Recovery may involve retrying an action, selecting an alternative tool, requesting additional information or escalating to a human operator. In more consequential contexts, failure may require the system to stop rather than attempt autonomous recovery.

This creates an important distinction between autonomy and reliability. Increasing the number of actions an agent can perform does not necessarily increase the reliability of the overall system. Each additional interaction with an external environment introduces opportunities for error, ambiguity and unintended effects. Agentic architectures must therefore manage the relationship between capability and control. The objective is not maximum autonomy but an appropriate level of bounded autonomy for the task and risk environment.

The feedback loop also changes the nature of evaluation. A conventional model can be evaluated by examining the quality of its outputs against defined benchmarks or task-specific criteria. An agentic system must additionally be evaluated across sequences of actions and environmental states. The relevant question becomes not simply whether the model produced an appropriate response, but whether the system pursued an objective appropriately, selected suitable actions, respected its permissions, responded correctly to observations and terminated in an acceptable state. Evaluation must therefore increasingly address trajectory-level behaviour rather than isolated model outputs.

Observability becomes correspondingly important. Because agentic behaviour emerges through sequences of decisions and actions, understanding system behaviour may require records of the task objective, model invocations, retrieved context, tool calls, outputs, observations, errors and state transitions. Such information supports debugging, evaluation, security monitoring and post-event investigation. It also provides a basis for determining why a particular action occurred and which components contributed to the resulting outcome.

Agentic architectures can also incorporate multiple specialised agents rather than relying upon a single general-purpose agent. In such systems, agents may divide responsibilities across planning, information retrieval, execution, verification or other functions. This can provide architectural modularity but also introduces additional coordination and communication boundaries. Each additional agent can create new dependencies, state-management requirements and security considerations. Multi-agent architecture should therefore be treated as a systems-design decision rather than as an automatic extension of agentic capability.

The transition from inference to action consequently represents a fundamental change in the unit of architectural analysis. A language model produces predictions or generated outputs; an agentic system establishes a closed or partially closed interaction loop in which those outputs can influence subsequent system behaviour and the state of an external environment. The resulting architecture requires explicit mechanisms for objectives, state, planning, tools, execution, feedback, permissions, error handling and termination.

The fundamental principle is therefore that agency is an architectural property, not simply a model capability. A sufficiently capable language model may provide the reasoning and generation capabilities required by an agent, but it becomes an agentic system only when those capabilities are embedded within mechanisms that allow the system to maintain state, interact with an environment, observe consequences and act iteratively within defined boundaries.

For enterprise AI, the objective should consequently be to engineer bounded and observable agency. The system should have sufficient autonomy to accomplish its intended tasks, while its authority, tool access, execution scope, resource consumption and ability to affect external systems remain explicitly controlled. Agentic architecture is therefore not primarily about enabling AI to act without humans; it is about designing the conditions under which AI can act appropriately, measurably and accountably.

7. Agentic Architecture: From Inference to Action

8. Orchestration

As AI systems become more complex, orchestration becomes a central architectural capability. A contemporary AI application may involve multiple models, retrieval mechanisms, memory stores, tools, enterprise applications, human approvals and execution environments. The system therefore requires mechanisms that determine how these components interact, in what sequence they operate, what information is passed between them and under what conditions execution should continue, pause, change direction or terminate. Orchestration provides this coordination function and, in doing so, connects individual AI capabilities into an executable system.

The importance of orchestration follows directly from the transition from model-centric to systems-centric AI. A foundation model can generate an answer or propose an action, but it does not by itself determine when information should be retrieved, which tool should be invoked, whether an action is authorised, how state should be maintained or when a task has been completed. These responsibilities belong to the surrounding application architecture. Orchestration therefore provides the mechanisms through which model capabilities are transformed into an operational process. This distinction is particularly important in agentic systems, where reasoning and action may occur iteratively in response to changing observations and intermediate results (Yao et al., 2023; Shinn et al., 2023).

At its simplest, orchestration determines the sequence and conditions of execution. It can determine which model is invoked for a particular task, when external information should be retrieved, which tools are available, how subtasks are decomposed, how state is passed between steps, when human intervention is required and how failures are handled. In more complex systems, orchestration may also coordinate multiple specialised agents and determine how their outputs are combined, validated or escalated. The orchestrator therefore functions as a control layer between individual components and the behaviour of the overall system.

This creates an important distinction between workflow orchestration and agentic orchestration. Workflow orchestration is generally based on predefined logic. The sequence of activities, decision points and permitted transitions is established in advance by the system designer. Such architectures are appropriate when the process can be specified reliably and when consistency, predictability and auditability are important requirements. Traditional workflow and business-process systems exemplify this architectural approach.

Agentic orchestration introduces greater dynamism. Rather than specifying every subsequent step in advance, the architecture can allow an agent or model to determine which action should be taken based on the current objective, context and state. ReAct provides a representative example in which reasoning and action are interleaved, allowing observations from external actions to influence subsequent decisions (Yao et al., 2023). Reflexion similarly illustrates architectures in which feedback and self-generated information can influence subsequent attempts (Shinn et al., 2023). In these architectures, the precise sequence of actions may not be known when execution begins.

The distinction should not, however, be interpreted as a choice between deterministic workflows and unrestricted autonomous agents. Enterprise AI systems will frequently require both forms of orchestration within the same architecture. A regulated business process may contain deterministic stages that must always occur in a defined sequence while allowing an AI agent to perform a bounded task within one of those stages. For example, an agent may investigate unstructured information or prepare a recommendation, while the surrounding workflow determines when that recommendation must be reviewed, approved, recorded or acted upon.

This hybrid architecture is particularly valuable because different parts of a business process can have different levels of uncertainty and risk. Where a process is highly predictable and its rules can be explicitly specified, deterministic orchestration can provide strong control, consistency and reproducibility. Where the process involves ambiguous information, open-ended investigation or dynamic problem solving, agentic execution may provide greater flexibility. The architectural question is therefore not whether an organisation should adopt workflows or agents, but which execution model is appropriate for each part of the process. This is consistent with the broader principle that autonomy should be matched to the nature and consequences of the task rather than maximised as an end in itself (NIST, 2023; ISO, 2023).

Orchestration also provides an important mechanism for decomposing complex tasks. Rather than asking a single model to perform an entire business process, an orchestrator can divide the task into distinct activities and assign each activity to an appropriate model, retrieval mechanism, tool or human actor. Such decomposition can improve modularity and make individual stages easier to evaluate and control. It can also allow different models to be selected according to the requirements of particular subtasks, rather than requiring one model to perform every function. Model routing and task-specific allocation can therefore become orchestration decisions rather than properties fixed at application design time (Kaplan et al., 2020; Bommasani et al., 2021).

However, decomposition introduces additional coordination complexity. Each boundary between subtasks creates an opportunity for information loss, incorrect interpretation or inconsistent state. When multiple agents are involved, their interactions create further dependencies concerning communication, authority and conflict resolution. Multi-agent architectures can therefore increase capability while simultaneously increasing the complexity of the system that must be governed (Wooldridge, 2009; Sculley et al., 2015). The architectural value of additional components should consequently be assessed against the coordination and control mechanisms required to manage them.

State management is another central orchestration responsibility. Multi-step AI processes require the system to maintain information about what has already occurred, what remains to be completed and what information is available for subsequent steps. This may involve working state, episodic interaction history, retrieved information and longer-term semantic memory. The orchestrator must determine which state is relevant to a particular stage of execution and ensure that only appropriate information is passed between components. Orchestration is therefore closely connected to context and memory architecture. The distinction between state that is required for execution and information that is authoritative for decision-making should also remain explicit, particularly where memory may contain stale, inferred or externally generated information (Lewis et al., 2020; Gebru et al., 2021).

Orchestration also provides a boundary between proposed actions and authorised actions. An agent may propose a tool invocation or external operation, but the orchestrator can determine whether that operation is permitted within the current workflow, identity and authority context. This provides an architectural location for policy checks, approval gates, transaction limits and other controls. The distinction is particularly important in systems where models can dynamically select tools, because the ability to formulate a tool request should not itself constitute permission to execute it (OWASP, 2025; NIST, 2023).

Human intervention can similarly be incorporated into orchestration as an explicit architectural state rather than as an informal fallback. A workflow may require human approval before a consequential action is executed, or it may escalate to a human when the system encounters uncertainty, an exception or a policy condition. This creates human-in-the-loop or human-on-the-loop control mechanisms in which human authority is deliberately integrated into the execution architecture. The appropriate degree of human involvement depends upon the consequences of failure, reversibility, uncertainty and organisational requirements rather than upon a universal requirement for human approval at every stage (NIST, 2023; ISO, 2023).

Orchestration should also distinguish between information, recommendation and execution states. An AI component may retrieve information, formulate a recommendation and subsequently propose an action without necessarily possessing authority to execute that action. By representing these states explicitly, the architecture can introduce approval or validation boundaries between increasingly consequential stages. This supports the broader separation between reasoning capability and execution authority established elsewhere in the architecture and prevents technical workflow progression from being mistaken for authorisation.

Termination is another essential orchestration function. Agentic processes can potentially continue generating actions as long as the system identifies additional tasks or receives new observations. The orchestrator should therefore establish explicit conditions under which execution terminates. These may include successful completion, predefined resource limits, time constraints, failure thresholds, policy violations, human intervention or other task-specific conditions. Termination transforms a potentially open-ended agentic process into a bounded computational activity. Explicit termination conditions are particularly important where agents can repeatedly invoke tools or generate further subtasks (Yao et al., 2023; Shinn et al., 2023).

A useful bounded-execution structure is:

task definition → action selection → policy check → execution → observation → state update → termination or next action.

Each iteration should remain subject to the system's authority, resource and policy boundaries. The existence of a feedback loop should therefore not imply unrestricted continuation. Instead, orchestration should establish the conditions under which the loop may continue and the conditions under which it must stop, escalate or transfer control.

Failure handling further distinguishes mature orchestration from simple task sequencing. AI systems operate in environments where models may produce invalid outputs, retrieval may fail, APIs may become unavailable, information may be incomplete and external systems may return unexpected results. An orchestrator can provide mechanisms for bounded retries, alternative execution paths, fallback components or human escalation. These mechanisms should be designed according to the consequences and reversibility of failure rather than assuming that autonomous recovery is always appropriate. A failed read operation may be safely retried, whereas repeating a transaction with external side effects may create a substantially different risk.

This requires orchestration to understand at least some of the semantic consequences of operations, rather than treating all tool calls as equivalent technical events. Retry policies, for example, should depend upon whether an operation is idempotent, whether it has already produced an external effect and whether the system can establish the current state before attempting another action. Orchestration therefore becomes part of failure containment rather than merely a mechanism for sequencing successful operations (Sculley et al., 2015; NIST, 2023).

Orchestration is closely related to observability and evaluation because it controls the sequence through which system behaviour emerges. Records of model calls, retrieved context, tool invocations, state transitions, policy decisions, approvals, errors and termination conditions can support system evaluation, debugging, security monitoring and auditability. In agentic systems, this execution history can be more informative than the final model output alone because the outcome may depend upon a sequence of intermediate decisions and environmental observations (Liang et al., 2022; Yao et al., 2023).

This makes orchestration an important source of execution evidence. A mature architecture should be capable of reconstructing not only what final output was produced, but also which components were invoked, which information was supplied, which actions were attempted, which controls were applied and where human intervention occurred. Such evidence supports evaluation and incident investigation while also making it possible to identify whether a failure originated in model reasoning, retrieval, state management, tool execution, policy enforcement or workflow design (NIST, 2023; ISO, 2023).

Orchestration can also act as a mechanism for policy-driven resource management. Model selection, context size, tool availability, execution duration and iteration limits can all be constrained according to workload, user role, risk level or operational conditions. This connects orchestration with infrastructure and cost architecture because execution decisions directly affect model calls, retrieval operations, latency and resource consumption. In agentic systems, resource limits therefore serve both economic and safety purposes (Kaplan et al., 2020; NIST, 2023).

The architectural significance of orchestration can consequently be understood as a form of controlled coordination. Models provide intelligence, retrieval provides contextual information, memory provides state, tools provide operational capabilities and humans provide authority and judgement. Orchestration determines how these capabilities are combined into an executable process and establishes the conditions under which that process can continue. It therefore represents an important control boundary between the individual components of an AI architecture and the behaviour of the overall system.

The objective should not be to maximise the degree of autonomous decision-making performed by the orchestration layer. Greater autonomy can increase flexibility, but it can also increase uncertainty, execution complexity and the potential consequences of error. The appropriate architectural objective is instead to establish the minimum level of autonomy necessary to accomplish the intended task effectively, while retaining explicit controls over authority, resources, information access, execution and consequential actions. This reflects the broader principle of bounded autonomy, in which increased capability does not automatically imply increased authority (NIST, 2023; ISO, 2023).

The resulting principle is that orchestration should match execution flexibility to process uncertainty and risk. Deterministic workflows should remain the default where requirements can be specified reliably and strong predictability is required. Agentic orchestration should be introduced where dynamic reasoning and adaptation provide genuine value. In many enterprise systems, the most appropriate architecture will therefore combine deterministic control structures with bounded agentic components.

This can be represented conceptually as a spectrum:

deterministic workflow → rule-guided workflow with AI components → bounded agentic stage → dynamically coordinated agentic process.

This spectrum should not be interpreted as a universal maturity ladder. More agentic execution is not inherently more advanced or desirable. The appropriate point depends on the uncertainty of the task, the consequences of failure, the reversibility of actions, the availability of reliable rules and the organisation's ability to observe and control execution.

Orchestration ultimately provides the mechanism through which AI capabilities become coordinated organisational processes. Its role is not simply to connect models and tools, but to establish the conditions under which intelligence can be applied with appropriate sequencing, authority, state management, feedback, human oversight, resource constraints and termination. Mature AI architecture therefore seeks not maximum autonomy, but appropriate autonomy within explicitly bounded, governed and observable execution environments.

9. Tools, APIs and Interoperability

An AI system becomes operationally significant when its capabilities can be connected to external systems and used to obtain information or perform actions within an organisational environment. A language model operating solely through a conversational interface primarily produces information, whereas an agent connected to databases, enterprise applications, software environments or transactional systems can interact directly with the environment in which organisational activities occur. The architectural boundary between the model and these external capabilities is therefore fundamental to the design of secure, governable and interoperable AI systems.

Tools can provide access to a wide range of capabilities, including databases, search services, enterprise applications, customer relationship management systems, document repositories, software-development environments, cybersecurity platforms and workflow systems. In financial and other transactional environments, tools may also provide access to systems capable of initiating or modifying transactions. The significance of these integrations is that they convert model outputs into potential operational effects. A generated response can be ignored, corrected or reconsidered; an executed tool call may change data, trigger a workflow or affect an external system. The transition from information generation to external action therefore represents a fundamental architectural boundary.

This distinction creates a central principle: models should not ordinarily receive unrestricted access to enterprise capabilities. External functionality should instead be exposed through controlled interfaces that define what the AI system is permitted to request and what the connected system is permitted to execute. This creates a separation between the model's ability to propose an action and the system's authority to perform that action. Agentic architectures consequently require explicit execution boundaries rather than treating technical connectivity as equivalent to operational authority (Yao et al., 2023; NIST, 2023; OWASP, 2025).

The use of explicit tools creates a corresponding separation between reasoning and execution. A model can determine that a particular capability is required and formulate a structured request, while a separate tool interface validates and executes that request. This makes it possible to apply authentication, authorisation, input validation, policy checks and monitoring independently of the model. It also reduces the need to embed direct enterprise-system access within prompts or model-specific application logic. The resulting architecture allows model capabilities to evolve without necessarily changing the underlying enterprise systems through which actions are performed.

This pattern is consistent with established principles of distributed software engineering, in which independent components interact through defined interfaces and contracts. AI architectures can apply the same principle by exposing enterprise capabilities through explicit interfaces rather than coupling models directly to implementation details of individual applications. This reduces dependency coupling and provides clearer boundaries for change, testing and governance. Such modularity is particularly important in machine-learning systems, where unmanaged dependencies can create substantial technical debt and maintenance complexity (Sculley et al., 2015).

The emerging Model Context Protocol (MCP) illustrates this movement towards standardised interfaces between AI applications and external tools and resources (Anthropic, 2024). Its architectural significance lies in the broader principle of establishing a standardised mechanism through which AI applications can discover and interact with external capabilities. A common protocol can reduce integration-specific coupling and make tools potentially reusable across different AI applications. MCP should nevertheless be understood as an interoperability mechanism rather than as a substitute for identity, authorisation, validation, security or governance controls. Standardising how a capability is exposed does not determine who should be permitted to use it or what operations should be authorised.

Interoperability is valuable because enterprise AI architectures are unlikely to depend indefinitely upon a single model, application or provider. Models may change, enterprise systems may be replaced and new tools may become available. If integrations are implemented through explicit contracts, individual components can potentially be changed without requiring the entire AI architecture to be redesigned. This creates a form of architectural modularity in which models, tools and enterprise applications can evolve at different rates. Explicit interfaces therefore support both technical interoperability and strategic flexibility.

Interoperability does not, however, imply unrestricted portability. Different models may interpret tool descriptions differently, impose different constraints on structured outputs or vary in their ability to generate valid tool calls. Tools may also have different authentication mechanisms, transactional semantics, resource constraints and error conditions. Effective interoperability therefore depends upon sufficiently precise interface contracts, explicit schemas and appropriate validation at the boundary between the AI system and the external capability. The interface must accommodate variation in model behaviour rather than assuming that all models will produce equivalent requests.

A tool interface should consequently define more than the name and purpose of a function. It should establish the identity and authority under which the operation is performed, the authentication mechanism required to establish that identity and the authorisation rules governing the permitted operation. It should also specify expected input and output structures, permitted operations, resource constraints, error conditions and relevant audit requirements. Where actions have material consequences, the interface or surrounding orchestration layer may additionally require explicit approval before execution. In this way, the tool interface becomes both a technical contract and an enforcement boundary.

Input and output schemas are particularly important because language models naturally produce flexible and probabilistic outputs, whereas enterprise systems generally require structured and validated requests. A tool interface can provide a formal contract that constrains the information passed between the model and the external system. Validation at this boundary can prevent malformed, incomplete or unexpected model outputs from being interpreted as valid operational instructions. Structured interfaces therefore provide an important mechanism for translating probabilistic outputs into controlled system interactions.

The interface should also define operational limits. Rate limits, transaction limits, execution timeouts and resource constraints can prevent an AI system from consuming excessive resources or repeatedly executing an operation because of an error or unexpected agent behaviour. Such limits are particularly important in agentic systems because the same tool may be invoked repeatedly as part of an iterative reasoning process. Operational limits consequently function not only as performance controls but also as safety mechanisms that constrain the possible consequences of erroneous or runaway behaviour (Yao et al., 2023; NIST, 2023; OWASP, 2025).

Error handling is another essential component of tool architecture. External systems can fail, return incomplete information, reject requests or become temporarily unavailable. The tool interface should therefore expose meaningful error states that allow the orchestration layer to distinguish between conditions that can safely be retried, conditions requiring an alternative action and conditions that should terminate execution or trigger human intervention. Treating every error as an instruction to retry can itself create operational risk, particularly where the requested operation has side effects. Retry behaviour should therefore be bounded and informed by the semantics of the underlying operation.

The architecture must also account for transactionality and side effects. A tool that performs a read operation generally presents a different risk profile from a tool that modifies records, sends communications, transfers funds or changes system configuration. Tool interfaces should therefore distinguish between informational and consequential operations and apply controls proportionate to their potential impact. The existence of a technical API connection should not automatically imply that an AI agent is authorised to perform every operation exposed by that API. Capability must remain distinct from authority.

This principle is closely related to the security risk of excessive agency. OWASP identifies excessive agency as a risk arising when applications grant language models excessive functionality, permissions or autonomy (OWASP, 2025). The architectural response is not necessarily to prevent tool use, but to establish least-privilege capability boundaries in which each agent receives only the tools and permissions necessary for its intended role. Tool access should therefore be explicitly associated with identity, purpose, context and authority rather than granted to an AI application as an undifferentiated capability set (NIST, 2023; ISO, 2023).

Auditability becomes increasingly important as tool interactions acquire operational significance. The system should, where appropriate, record which identity invoked a tool, which operation was requested, what relevant information was supplied, what result was returned, whether policy checks were applied and whether additional approval was required. These records can support monitoring, incident investigation, compliance processes, evaluation and reconstruction of system behaviour. Observability of tool interactions is therefore an integral component of trustworthy agentic architecture rather than an optional operational feature (NIST, 2023; Liang et al., 2022).

Tool discovery introduces another architectural consideration. As AI systems interact with increasingly large ecosystems of services, agents may require mechanisms for determining which capabilities are available and what those capabilities permit. Tool descriptions and metadata can assist discovery, but discovery mechanisms must not bypass authorisation. Knowing that a capability exists is not equivalent to being authorised to use it. The architecture should therefore maintain a clear distinction between capability discovery and capability authority. Discovery answers what can potentially be requested; authorisation determines what the particular agent or identity may actually perform.

This distinction is particularly important where tools are dynamically selected. An agent may identify several potentially relevant capabilities, but the availability of a tool should not itself constitute permission to invoke it. Tool selection should occur within an explicit policy and authority context. This enables the architecture to restrict certain tools to particular agents, users, workflows, environments or transaction classes while still allowing the broader system to discover that those capabilities exist.

Interoperability also creates a supply-chain consideration. A tool may itself depend upon third-party services, external data sources, libraries or other software components. Connecting an AI system to that tool can therefore extend the system's effective dependency boundary. Tool selection and integration should consequently consider not only functional capability but also security, reliability, provenance, availability and governance characteristics. The more consequential the operation performed through a tool, the greater the importance of understanding the dependencies behind that operation. This reflects the broader challenge of managing hidden and transitive dependencies in machine-learning and software systems (Sculley et al., 2015; NIST, 2023; OWASP, 2025).

Tool architecture should also distinguish between read, propose and execute capabilities. A tool that permits an agent to retrieve information need not provide the ability to modify that information. Similarly, an agent may be allowed to construct a transaction or workflow request without being authorised to commit it. This creates opportunities for staged execution in which increasingly consequential operations require increasingly strong validation or approval. Such separation reinforces the broader principle that authority should be proportional to the consequences of the operation.

The architectural objective is consequently not to maximise the number of tools available to an AI system. A large collection of loosely controlled capabilities can increase complexity and expand the system's attack, failure and coordination surfaces. Instead, tools should be selected according to clearly defined use cases and exposed through interfaces that make their capabilities, constraints and authority explicit. Tool proliferation should therefore be treated as an architectural dependency decision rather than as an automatic increase in system capability.

Tools and APIs consequently represent more than integration mechanisms. They constitute control boundaries between probabilistic intelligence and enterprise execution. The model can interpret objectives and propose actions, while the tool interface establishes the technical and organisational conditions under which those actions can be performed. This separation allows the flexibility of AI to coexist with the predictability, validation and control requirements of enterprise software.

The relationship can be expressed as a simple architectural chain:

Model proposes → tool interface validates → policy authorises → enterprise system executes → observability records → governance accounts.

This chain is particularly important because the model itself should not be regarded as the ultimate authority over the external system. The model contributes reasoning and intent; the tool interface translates that intent into a formally constrained request; policy and identity controls determine whether the request is permissible; and the enterprise system remains responsible for enforcing its own operational constraints. This creates multiple opportunities to prevent an erroneous or manipulated model output from becoming an uncontrolled external effect.

The fundamental principle is therefore that AI systems should interact with enterprise capabilities through explicit, governed and interoperable interfaces. Tool access should follow least-privilege principles, be constrained through formal contracts, be validated according to operational risk and remain observable throughout execution. Interoperability should reduce unnecessary coupling between models and applications without weakening the authority, security and governance boundaries of the systems being connected.

In this architecture, the tool interface becomes a critical point of control. It is where model-generated intentions are translated into structured requests, where identity and permissions can be enforced, where actions can be validated, where operational limits can be applied and where the consequences of AI interaction with the external environment can be observed. The result is an architecture in which AI can act across enterprise systems without granting the model unrestricted authority over those systems. The strategic objective is therefore not unrestricted AI connectivity, but controlled interoperability between probabilistic intelligence and governed enterprise execution.

10. Multi-Agent Systems

As AI systems become more complex, organisations may distribute responsibilities across multiple specialised agents rather than relying upon a single general-purpose agent. A multi-agent architecture can assign distinct computational roles to different agents, including planning, research, data analysis, compliance analysis, software engineering, review or assurance. The architectural rationale for such decomposition is that specialised components may be easier to constrain, optimise and evaluate for particular responsibilities than a single component responsible for an entire process. Multi-agent architecture therefore represents a form of system decomposition in which intelligence and authority are distributed across explicitly defined computational roles.

The conceptual foundations of multi-agent systems substantially predate contemporary large language models. Research in distributed artificial intelligence and multi-agent systems has examined how autonomous computational entities communicate, coordinate, cooperate and compete within shared environments (Wooldridge, 2009). Classical agent architectures generally conceptualised agents as software entities with defined objectives, capabilities, perceptions and interactions. Contemporary foundation models introduce a different implementation possibility: language models can provide flexible reasoning, information interpretation and communication capabilities through natural-language or structured interfaces. The underlying architectural concept of multiple interacting agents is therefore not new; what has changed is the range of capabilities that can be incorporated into individual agents.

The contribution of large language models to multi-agent architecture is consequently better understood as an expansion of the computational capabilities available to individual agents, rather than the invention of the multi-agent paradigm. An LLM-based agent can interpret natural-language objectives, reason over unstructured information, generate intermediate plans, invoke tools and communicate with other components through language-based or structured interfaces. Agentic approaches such as ReAct demonstrate how language-model reasoning can be coupled with actions and external observations, while systems such as Voyager illustrate how language models can coordinate longer sequences of tool-mediated activity (Yao et al., 2023; Wang et al., 2023). This creates a practical basis for architectures in which different agents perform complementary roles while potentially sharing models, specialised models, tools and information sources.

The primary architectural benefit of decomposition is bounded specialisation. A specialised agent can be given a narrower objective, a restricted set of tools and access to a particular information domain. A research-oriented agent, for example, may be responsible primarily for retrieving and synthesising information, while a review agent may check outputs against defined criteria. A planning agent may formulate a sequence of proposed actions without possessing authority to execute them. Such separation can reduce the scope of individual components and create clearer boundaries around their responsibilities. It can also allow different models, prompts, retrieval strategies or tools to be selected according to the requirements of particular tasks (Yao et al., 2023; Shinn et al., 2023).

Specialisation can also support modular evaluation and control. When an agent has a clearly defined function, its inputs, outputs and expected behaviour can be evaluated against more specific criteria. A compliance-oriented component, for example, may be assessed according to its ability to identify relevant requirements and provide appropriate evidence, while a software-engineering agent may be evaluated against code quality, testing and security criteria. This aligns with the broader principle that AI evaluation should operate at multiple levels, from individual components through to complete systems and workflows (Liang et al., 2022). Decomposition can therefore make complex AI processes more amenable to targeted evaluation, provided that component-level evaluation is complemented by assessment of interactions and end-to-end outcomes.

However, introducing multiple agents also creates additional architectural complexity. Each agent represents another component that must be configured, secured, monitored, evaluated and maintained. Communication between agents creates additional interfaces, and each interface introduces opportunities for information loss, ambiguity, inconsistent state, inappropriate instruction propagation or failure. These additional dependencies contribute to the technical complexity associated with machine-learning systems and can increase maintenance and operational burden over time (Sculley et al., 2015). A multi-agent system can therefore become more difficult to reason about than a comparable single-agent architecture even when each individual agent is relatively simple.

The central architectural challenge is coordination. Agents must have explicit definitions of their responsibilities, accessible information, permitted tools and authority boundaries. Where multiple agents contribute to a common objective, the architecture must determine how their outputs are combined, which component is responsible for resolving disagreement and how conflicts are handled. Coordination mechanisms may include an orchestrator, explicit workflow states, shared protocols, structured messages, approval gates or deterministic business rules. Without such mechanisms, a collection of specialised agents may produce inconsistent, redundant or conflicting behaviour rather than a coherent system outcome.

Communication should itself be treated as an architectural boundary. Agents may communicate through natural language, structured messages, shared state or explicit service interfaces. Natural-language communication provides flexibility and can support heterogeneous agents, but it may introduce ambiguity or make interface contracts less explicit. Structured interfaces can improve predictability, validation and interoperability, although they impose additional design constraints. The appropriate mechanism should therefore depend on the role of the interaction, the degree of coupling required and the consequences of communication failure. Explicit interface design is particularly important where AI components interact with external systems or where changes to one component can propagate across the architecture (Anthropic, 2024; Sculley et al., 2015).

Multi-agent architectures also introduce additional security boundaries. Each agent may require different access to data, tools and enterprise systems. A research agent may require read access to selected information sources but no transactional capability, whereas an execution-oriented agent may require access to a narrowly defined operational API. Treating all agents as having equivalent permissions would undermine the security benefits that specialisation is intended to provide. Least-privilege principles should therefore apply at the level of individual agents, identities and interfaces rather than only to the overall AI application (NIST, 2023; OWASP, 2025).

The architecture must also account for state and memory. Agents may maintain isolated state, share a common memory system or exchange information through an orchestration layer. Each approach creates different trade-offs. Shared state can facilitate coordination but can also introduce contention, information leakage, inconsistent updates and complex dependencies. Isolated state can strengthen boundaries but requires explicit mechanisms for transferring information between agents. The architecture should therefore define the ownership, provenance, access rights and authority associated with shared information rather than treating collective memory as an undifferentiated resource. This is particularly important where agent memory contains retrieved information, intermediate results or persistent records that may subsequently influence consequential actions (Lewis et al., 2020; Gebru et al., 2021).

A further consideration is the distinction between specialisation and duplication. Multiple agents are not inherently more capable than a single agent. If several agents perform substantially the same reasoning task using the same information and tools, decomposition may add communication and coordination overhead without producing a corresponding improvement in system performance. The introduction of an additional agent should therefore have an explicit architectural rationale, such as differentiated expertise, authority boundaries, independent validation, security isolation or resilience. Decomposition should create a meaningful separation of responsibility rather than merely distribute similar reasoning across additional components.

The same principle applies to resilience. Multiple agents can potentially provide redundancy, allowing one component to perform a function when another fails. However, logical multiplicity does not necessarily provide genuine fault independence. If multiple agents depend upon the same underlying model, information source, orchestration service, retrieval system or infrastructure, they may share common failure modes. Sculley et al. (2015) highlight the broader importance of dependency management in machine-learning systems, while agentic architectures introduce additional dependencies through tool use and interaction. Multi-agent resilience should therefore distinguish between logical diversity and architectural independence. Additional agents provide meaningful redundancy only where their relevant failure modes are sufficiently differentiated.

Coordination also creates the possibility of cascading failure. An error produced by one agent may become input to another agent, where it may be accepted as an apparently valid intermediate result and propagated further through the system. Similarly, a mistaken instruction may pass through several components before reaching an external execution boundary. Multi-agent architectures should therefore define validation and trust boundaries between agents rather than assuming that outputs from one AI component are inherently reliable inputs to another. This reinforces the need for explicit schemas, provenance, policy checks and, where appropriate, independent validation before consequential actions.

Evaluation becomes correspondingly more complex. Individual agents can be evaluated according to their specialised responsibilities, but the overall system must also be evaluated at the level of coordination and end-to-end outcomes. A collection of individually successful agents may still produce an unsuccessful system if information is transferred incorrectly, responsibilities overlap, coordination fails or the final output does not satisfy the overall objective. Multi-agent evaluation should therefore combine component-level evaluation, interaction-level evaluation and system- or workflow-level evaluation (Liang et al., 2022). Agentic evaluation may additionally need to consider action sequences, tool selection, intermediate observations, recovery behaviour and termination conditions rather than assessing only the final response (Yao et al., 2023; Shinn et al., 2023).

Observability is particularly important because failures may emerge from interactions rather than from a single component. An effective architecture should provide sufficient visibility into agent invocations, identities, messages, retrieved information, tool calls, state transitions, policy decisions, errors and hand-offs to reconstruct how a particular outcome was produced. As the number of interactions grows, this execution evidence becomes increasingly important for debugging, security investigation, evaluation and governance. Without adequate observability, determining whether a failure originated in an individual agent, an interaction, shared state, orchestration logic or an external dependency can become substantially more difficult (NIST, 2023; Liang et al., 2022).

There is also an important economic and operational consideration. Each additional agent may introduce additional model calls, context construction, retrieval operations, message exchanges and tool interactions. These activities can increase latency, computational consumption and operating cost. Model scaling research demonstrates the relationship between model capability and computational requirements, while agentic architectures can further multiply inference activity through iterative reasoning and action (Kaplan et al., 2020; Yao et al., 2023; Shinn et al., 2023). Multi-agent design should therefore demonstrate sufficient benefit in capability, specialisation, reliability or control to justify the additional execution overhead.

The appropriate architectural question is consequently not whether multiple agents are more advanced than a single agent, but whether decomposition produces a measurable systems-level benefit. Such a benefit may take the form of improved task performance, clearer responsibility boundaries, stronger security isolation, more effective independent evaluation, greater resilience or more precise human oversight. If the same outcome can be achieved through a simpler workflow, deterministic service composition or single-agent architecture, unnecessary decomposition may increase rather than reduce system risk and operational complexity.

This suggests that multi-agent architecture should be evaluated according to the coordination surface it creates. Every additional agent can introduce new communication paths, state dependencies, permissions, failure modes, evaluation requirements and operating costs. The relevant design question is therefore not simply how many agents a system contains, but whether each agent creates a sufficiently valuable architectural boundary to justify those additional dependencies. In this sense, decomposition should be treated as a systems-engineering decision rather than as a measure of AI sophistication.

Multi-agent systems should therefore be understood as an architectural pattern rather than an inherent feature of advanced AI. The decision to introduce multiple agents should follow from the characteristics of the problem, the need for genuine specialisation and the control requirements of the operating environment. It should not be driven solely by the availability of agentic technologies or by the perceived sophistication of a multi-agent design. Classical multi-agent research provides the conceptual foundation for distributed computational entities, while contemporary foundation models provide new mechanisms through which such entities can reason and communicate (Wooldridge, 2009; Bommasani et al., 2021).

The fundamental principle is that architectural decomposition should create more value than complexity. Multiple specialised agents can provide meaningful benefits when responsibilities are genuinely distinct and their interactions can be explicitly controlled. Where those conditions do not apply, a simpler single-agent or workflow-based architecture may provide clearer boundaries, lower operational overhead and easier governance.

A mature AI architecture therefore treats the number of agents as a design variable rather than a measure of architectural maturity. The appropriate architecture is the one that provides the required level of capability, specialisation, resilience and control while minimising unnecessary coordination, dependency and authority surfaces. Multi-agent design should consequently be justified not by the number of agents introduced, but by the measurable architectural value created by their separation.

11. Security Architecture

AI systems introduce a distinctive security architecture because they combine probabilistic computation with access to information, software interfaces and, increasingly, operational systems. The security problem therefore extends beyond protecting infrastructure and application code. It also concerns what information an AI system can access, how that information may influence its reasoning, what actions it can request, and which controls determine whether those actions are permitted. Security must consequently be considered as a property of the complete AI-enabled system rather than of the model in isolation.

The attack surface of an AI system spans multiple architectural layers. It may include prompts and instructions, retrieved documents and contextual information, embeddings and knowledge stores, model interfaces, agent memory, tools and APIs, external integrations, software and model supply chains, and outputs subsequently consumed by people or downstream systems. This creates dependencies across information, reasoning and execution boundaries. Security architecture must therefore protect the complete execution path, including the interfaces through which information enters the system and the points at which AI-generated outputs can produce external effects. OWASP identifies a range of risks specific to large-language-model applications, including prompt injection, sensitive information disclosure, supply-chain vulnerabilities, data and model poisoning, improper output handling and excessive agency (OWASP, 2025).

These risks demonstrate why conventional application-security controls remain necessary but are insufficient on their own. An AI system may process untrusted information as part of its context, generate instructions or code that are subsequently executed, or dynamically determine which tools to invoke. Security controls must therefore address both the information flowing through the system and the authority available to the system to act upon that information. This is particularly important because the probabilistic nature of model outputs means that security cannot depend solely on predicting or constraining what a model will generate. Instead, the architecture must constrain what generated outputs are permitted to do.

A central architectural principle is least privilege. Each model, agent, service and tool should receive only the permissions required for its defined function. Permissions should be associated with explicit identities, roles and execution contexts rather than being inherited simply because an AI component is technically capable of requesting an action. Least privilege should therefore apply not only to infrastructure and software services, but also to AI agents, retrieval mechanisms, memory stores and tool interfaces. This extends established security principles into architectures in which probabilistic components can dynamically formulate requests for information or action (NIST, 2023; OWASP, 2025).

The distinction becomes particularly important when AI systems interact with consequential enterprise processes. An agent responsible for preparing a payment instruction should not automatically have authority to execute the payment. Similarly, an agent analysing security events should not necessarily have permission to modify production security controls. The ability to formulate an action and the authority to perform that action are distinct architectural concerns. This separation reduces the extent to which an erroneous, manipulated or compromised AI component can directly produce consequential effects.

This leads to a fundamental distinction between reasoning authority and execution authority. A model or agent may determine that a particular action appears appropriate, but a separate control layer should establish whether the action is authorised, whether its parameters satisfy applicable policies, and whether additional approval is required. The resulting architecture places a policy and authorisation boundary between probabilistic reasoning and consequential execution. In conceptual terms, reasoning proposes; policy constrains; execution requires authority. This separation is consistent with broader principles of controlled agency, risk management and system-level governance (Yao et al., 2023; Shinn et al., 2023; NIST, 2023; ISO, 2023).

Such separation is closely related to established security principles including least privilege, separation of duties and defence in depth. An AI component should not be treated as a trusted actor merely because it operates within an otherwise trusted enterprise environment. Its requests should pass through identity, authentication, authorisation, validation and policy controls appropriate to the sensitivity of the requested operation. The architecture should therefore distinguish between the trust placed in the surrounding enterprise environment and the permissions granted to a particular AI component.

This distinction becomes increasingly important as systems become agentic and multi-agent. Different agents may require different permissions, information domains, tools and execution boundaries. A planning agent may need to formulate a proposed sequence of actions without being permitted to execute those actions. A research agent may require access to selected information sources but no transactional authority. A review or assurance agent may need visibility into proposed actions without possessing permission to modify the underlying system. Multi-agent architectures should consequently define authority at the level of individual functions, identities and interfaces rather than granting broad permissions to an AI system as a whole. Agent architectures based on iterative reasoning and tool use make these boundaries particularly significant because the system may generate and execute sequences of actions rather than a single isolated output (Yao et al., 2023; Wang et al., 2023; Wooldridge, 2009).

Context and memory require equivalent protection. Retrieved documents, tool outputs, previous interactions and persistent agent memories may contain sensitive information or instructions that influence subsequent behaviour. These resources should therefore be subject to appropriate access controls, provenance requirements, retention policies and trust boundaries. Information supplied to a model should not automatically be treated as an authorised instruction merely because it appears within the model's context. Retrieval systems can provide valuable external knowledge, but the information they return must remain subject to source authority, access controls and contextual validation (Lewis et al., 2020; Gebru et al., 2021). This is particularly important for prompt injection, in which untrusted content can attempt to influence an AI system's subsequent behaviour (OWASP, 2025).

The distinction between information and instruction is therefore an important security boundary. A document retrieved from an enterprise repository may contain legitimate business information without having any authority to redefine system behaviour. Likewise, content returned by an external source, tool or user should not automatically acquire the privileges of the system that processes it. Security architecture should explicitly identify trusted instruction channels and treat other contextual material as data subject to appropriate validation and policy. This provides an architectural mechanism for reducing the influence that untrusted content can exert on system behaviour.

Security must also extend to generated outputs. An AI-generated response may be informational, but it may alternatively contain executable code, database queries, configuration changes, workflow instructions or transaction parameters. Outputs that cross into consequential systems should therefore be validated and constrained before execution. Schema validation, parameter validation, policy checks, transaction limits, sandboxing and approval controls can provide additional boundaries between generated content and external effects (NIST, 2023; OWASP, 2025). The architectural boundary should distinguish between generated intent and authorised execution, rather than allowing generated content to become an action merely through technical connectivity.

These principles imply that security controls should be embedded throughout the AI execution path. Identity, authentication, authorisation, input and context validation, output validation, tool controls, policy enforcement, logging, monitoring and incident response should operate as coordinated architectural mechanisms. Security is therefore not a single gateway surrounding the model; it is a distributed control structure spanning data, context, models, agents, tools, infrastructure and downstream systems. This defence-in-depth approach is particularly important because failures can occur at different layers and may interact in ways that are not visible when security is considered only at the model boundary (OWASP, 2025; NIST, 2023).

Observability is particularly important within this architecture because AI-enabled execution may involve multiple model calls, retrieved sources, memory operations, tool invocations, intermediate outputs and hand-offs between agents. Security monitoring should provide sufficient evidence to reconstruct relevant execution paths, identify anomalous behaviour, investigate failures and determine which components, identities and authorities were involved in a consequential action. This connects AI security with the broader requirement for observability, evaluation and governance in AI systems. NIST's AI Risk Management Framework emphasises the need to identify, assess, manage and monitor AI-related risks across the system lifecycle (NIST, 2023), while agentic architectures make execution-path visibility increasingly important for understanding how an outcome was produced (Yao et al., 2023; Shinn et al., 2023).

Security telemetry must itself be treated as governed information. Logs and execution records may contain prompts, retrieved documents, personal or commercially sensitive information, tool parameters and system responses. Consequently, logging and monitoring mechanisms require their own access controls, retention policies and protection against unauthorised modification. Security architecture should therefore balance the need for sufficient evidence with principles of data minimisation, controlled access and appropriate retention. This reinforces the broader architectural principle that observability is not merely a monitoring function but part of the system's evidence and control infrastructure (NIST, 2023; ISO, 2023).

The resulting security model can be understood as a bounded-authority architecture. AI components may reason, retrieve information, formulate proposals and request actions, but their ability to access information or produce consequential effects is constrained by explicit identities, permissions, policy controls, validation mechanisms and, where appropriate, human approval. The objective is not to prevent AI systems from acting, but to ensure that their ability to act remains within a deliberately defined operating envelope. This is consistent with the broader principle that autonomy should be coupled to appropriate controls, monitoring and accountability (NIST, 2023; ISO, 2023).

A useful conceptual security chain is therefore:

Identity establishes who is acting → authorisation establishes what may be accessed → context controls establish what information may influence behaviour → policy establishes what is permitted → validation checks the proposed action → execution controls determine what can actually occur → observability records what happened → governance determines accountability and response.

This architecture also clarifies the relationship between AI capability and AI security. Increasing model capability may expand the range of tasks an AI component can perform, but capability should not automatically imply increased authority. Authority is an architectural decision determined by identity, permissions, policies, workflow requirements and risk. Consequently, a more capable model can operate within the same constrained execution boundary as a less capable model, provided that the surrounding controls remain appropriately designed and enforced.

The central principle is therefore that AI security is fundamentally an architectural property. The objective is not simply to secure the model, but to control the information, interfaces, identities, permissions, execution paths and external effects surrounding it. As AI systems become more capable and autonomous, the critical security question is not only what the model can generate, but what the overall architecture permits that generated capability to access and do. Enterprise AI security is consequently best understood as the engineering of controlled information flow and bounded authority across the complete AI execution lifecycle.

12. AI Safety and Guardrails

AI systems operate under conditions of uncertainty. Their outputs may be syntactically valid, linguistically convincing and internally coherent while nevertheless being incorrect, incomplete, unsafe or inappropriate for the context in which they are used. Consequently, correctness cannot be inferred from fluency or from the apparent confidence of an output. Safety architecture must instead provide mechanisms for detecting, constraining and managing undesirable behaviour across the complete AI system.

Guardrails are architectural mechanisms that constrain system behaviour within defined boundaries. They should not be understood as a single filtering mechanism surrounding the model. Rather, guardrails can operate at multiple points across the execution lifecycle, including before model invocation, during context construction and retrieval, during model and tool selection, at the boundary between generated outputs and external systems, and after model generation. This distributed approach is particularly important because AI system behaviour emerges from interactions among models, data, prompts, tools, workflows and surrounding infrastructure rather than from the model alone (Sculley et al., 2015; NIST, 2023).

Pre-invocation controls may include input validation, identity and access checks, rate limits, policy checks and classification of requests according to risk. Context controls can determine which information is retrieved, whether that information is authorised for the current task and whether untrusted content should be treated as data rather than as instructions. Execution controls can constrain which tools an agent may invoke, which parameters it may supply and under what conditions an action is permitted. Post-generation controls can validate outputs before they are passed to users, applications or transactional systems. This creates a series of control boundaries rather than relying on a single point of defence.

Guardrails can therefore take many forms, including input validation, policy enforcement, content filtering, schema validation, tool allow-lists, output validation, transaction limits, sandboxing, approval requirements, rate limits and runtime policy enforcement. Their appropriate placement depends on the risk associated with the operation. An informational response may require relatively limited controls, whereas an action affecting financial assets, production infrastructure, regulated records or sensitive personal information may require multiple independent controls and explicit approval. Risk-management frameworks such as NIST AI RMF provide a basis for relating controls to the context and potential impact of AI system behaviour (NIST, 2023).

This layered approach is important because no individual safeguard should be assumed to be complete. A model-level control may reduce undesirable outputs without preventing malicious content from manipulating retrieved context. An input filter may detect known patterns without preventing a compromised tool from producing unsafe results. A policy check may identify a prohibited action while failing to detect an unexpected combination of individually permissible actions. OWASP's treatment of risks in LLM applications illustrates the breadth of threats that can arise across prompts, application logic, data, tools and external interactions rather than exclusively within the model itself (OWASP, 2025).

Safety should therefore emerge from the interaction of multiple controls. The architecture should assume that individual controls may be incomplete, bypassed or rendered ineffective by changing conditions. Defence in depth is consequently more appropriate than reliance on a single safety mechanism. Where the consequences of failure are significant, independent controls can provide additional protection by reducing the probability that a single erroneous output, compromised component or failed policy check produces an unacceptable external effect.

Research into alignment has demonstrated that model training can improve adherence to intended behaviour. Ouyang et al. (2022), for example, demonstrated the use of supervised fine-tuning and reinforcement learning from human feedback to improve language-model instruction following. Bai et al. (2022) subsequently described Constitutional AI, in which explicit principles are used to guide model behaviour and AI-assisted feedback is incorporated into the training process. These approaches are important because they influence the behaviour of the model itself and can establish behavioural tendencies that are useful across applications.

However, model alignment should not be treated as equivalent to system safety. Alignment can improve the likelihood that a model behaves according to specified objectives or principles, but it does not establish the security and control properties of the surrounding system. A model may have been trained to follow appropriate instructions while still operating within an unsafe architecture. It may receive malicious or manipulated prompts, encounter untrusted retrieved content, interact with compromised external tools, expose sensitive information through inappropriate context, or possess permissions that exceed what is necessary for its task.

Conversely, an appropriately designed system can impose external controls that remain effective when the underlying model produces an unexpected or erroneous output. Identity controls, least-privilege permissions, tool restrictions, transaction limits, output validation, sandboxing, monitoring and human approval mechanisms do not depend on the model consistently behaving as intended. They provide architectural constraints around model behaviour.

The distinction can therefore be expressed as follows:

Model alignment influences how the model behaves; system safety constrains what the complete system can do.

This distinction is particularly important in agentic systems. As established in the preceding architecture, an agent can reason, select tools, maintain state and interact iteratively with its environment. The safety problem therefore extends beyond the content of individual model responses to include the sequence of actions generated by the system (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

A single action may appear permissible in isolation while a sequence of individually permissible actions produces an unintended outcome. For example, retrieving information, modifying a record and initiating an external communication may each be permitted under different circumstances, while their combination may produce an outcome that should have required additional authorisation. Guardrails must consequently operate at both the component level and trajectory level. This means evaluating not only whether individual actions are permitted, but also whether the sequence of actions remains within the system's authorised operating envelope.

This creates an important relationship between safety and the principle of bounded autonomy. An autonomous agent should not simply be given a general instruction and broad access to the systems it can technically reach. Its operating envelope should define the actions it may perform, the resources it may access, the conditions under which those actions are permitted and the circumstances that require escalation. Agentic architectures therefore require explicit boundaries around tools, state, permissions, transaction scope and termination conditions (Yao et al., 2023; NIST, 2023).

Safety controls should also be proportionate to the consequences of failure. Low-impact activities may be permitted within automated boundaries, while higher-impact activities may require stronger validation, transaction limits, dual control or human approval. This creates a graduated approach to autonomy in which the level of control corresponds to the potential effect of the system's actions rather than simply to the technical capabilities of the underlying model.

This principle also supports the distinction between capability and authority developed earlier. A model may be capable of generating a particular action, and an agent may be technically capable of executing it, without either being authorised to do so. Guardrails provide the architectural mechanisms through which capability is translated into permitted action. The system therefore establishes not merely what the model can produce, but what the overall architecture will permit that output to cause.

Runtime enforcement is particularly significant because many risks cannot be fully addressed during model training. Enterprise systems operate in changing environments: data changes, permissions change, tools are updated, threats evolve and new workflows are introduced. A behaviour that was acceptable under one set of conditions may become inappropriate under another. Runtime controls provide an additional layer of protection by evaluating actual execution conditions rather than relying solely on assumptions established during model development (NIST, 2023; ISO, 2023).

Runtime controls can include policy evaluation, access checks, content and output validation, tool restrictions, transaction thresholds, anomaly detection, approval gates and execution termination. These mechanisms are especially important where an AI system can produce external effects. The closer an AI system is to changing enterprise state or producing consequential external actions, the more important it becomes that safety controls operate immediately before and during execution rather than relying exclusively on model-level behaviour.

Safety architecture should also include mechanisms for detecting when guardrails themselves have failed or been bypassed. Logging, monitoring, evaluation, anomaly detection, incident response and periodic review allow organisations to determine whether controls are operating as intended. This connects safety directly to the observability and evaluation architectures developed elsewhere in this paper. A control that cannot be observed cannot readily be verified, while a control that is never evaluated may gradually become ineffective as the system, data or threat environment changes (Liang et al., 2022; NIST, 2023).

This creates a useful safety-control loop:

Define boundary → Enforce boundary → Observe behaviour → Evaluate control → Detect deviation → Escalate or correct

The loop is deliberately continuous. Safety is not established once when a model is deployed; it must be maintained as models, data, tools, policies and operating conditions change. This is consistent with the lifecycle orientation of AI risk management and management-system approaches to AI governance (NIST, 2023; ISO, 2023).

Guardrails must also account for untrusted information and context. Retrieval-augmented systems may introduce external documents, web content, enterprise documents or other information into the model context. The fact that information has been retrieved does not mean that it should be treated as an instruction or that it is authoritative for the current task. Data provenance, source authority and separation between information and executable instruction are therefore important safety boundaries (Lewis et al., 2020; Gebru et al., 2021). The architecture should distinguish between information that the model may use and instructions or permissions that the model is authorised to follow.

Similarly, tools should be treated as controlled execution interfaces, not simply as additional model capabilities. Tool access should be scoped according to the task, permissions should follow least-privilege principles, and consequential operations should have appropriate validation or approval mechanisms. This is particularly important where tools can modify enterprise state, communicate externally or trigger financial, operational or regulatory consequences. The security and governance of the tool boundary therefore form part of the safety architecture rather than being separate concerns.

The architecture should also account for failure containment. Where an AI system behaves unexpectedly, the consequences should be constrained to the smallest practical scope. Mechanisms such as sandboxing, isolated execution environments, scoped credentials, transaction limits, rate limits, timeouts and termination conditions can reduce the potential impact of erroneous behaviour. These mechanisms are particularly relevant to agentic systems, where repeated reasoning and tool invocation can otherwise create cumulative effects (Yao et al., 2023; Shinn et al., 2023).

The architectural distinction can therefore be stated clearly:

Model alignment is a property of the model, whereas system safety is a property of the complete socio-technical architecture.

Alignment can establish useful behavioural tendencies, but it cannot by itself provide identity management, least-privilege access, transaction controls, tool isolation, data protection, runtime monitoring, failure containment or human authority. These properties must be established by the architecture surrounding the model.

Guardrails should consequently be designed as part of the system architecture rather than added as an afterthought. Their purpose is not to eliminate uncertainty, which is generally impossible, but to ensure that uncertainty and model error remain within acceptable operational boundaries. The architecture should assume that models can be wrong, context can be incomplete, tools can fail, data can be manipulated and controls can encounter unexpected conditions.

The central principle is therefore that safe AI requires bounded behaviour, not merely well-behaved models. Model alignment contributes to safety, but architectural controls determine what the system can access, what it can do, how its actions are validated, how failures are contained and when human authority is required. In a governed intelligent system, safety is achieved through the combination of model behaviour, architectural constraints, runtime controls, evaluation, observability and appropriate human oversight.

This places safety architecture within the broader control model developed throughout this paper. Models provide capabilities; data and context provide information; guardrails constrain permissible behaviour; orchestration controls execution; tools create external effects; observability records what occurred; evaluation tests whether controls remain effective; and governance establishes responsibility and acceptable risk. Safety is therefore not a property that can be added to an AI system after its principal architecture has been established. It is an emergent property of the boundaries and controls designed into the complete system.

13. Evaluation as an Architectural Capability

Evaluation is a fundamental architectural capability in AI systems because the behaviour of generative and agentic systems is probabilistic, context-dependent and influenced by interactions among multiple components. Conventional software can often be tested against deterministic specifications in which the same input is expected to produce a defined output. AI systems may instead produce different outputs for similar inputs, while an output that is linguistically convincing may nevertheless be factually incorrect, poorly grounded, unsafe or unsuitable for the intended task. Consequently, testing a limited set of examples cannot, by itself, establish reliable system behaviour. The broader evaluation literature similarly demonstrates the limitations of reducing AI performance to a single benchmark or metric (Liang et al., 2022).

Evaluation should therefore be treated as a continuous system capability rather than as a final testing stage. It should operate across development, deployment and runtime operation, with evaluation criteria aligned to the level of the architecture being assessed. A model can be evaluated for its intrinsic capabilities, while retrieval, tools, memory, orchestration, workflows and human interaction require their own forms of evaluation. The architectural objective is not simply to determine whether a model performs well, but whether the complete AI-enabled capability performs acceptably within its intended operating environment.

A useful evaluation architecture can be organised into four complementary levels: model, component, system and workflow.

At the model level, evaluation may consider task accuracy, reasoning performance, language quality, robustness, latency and cost. These measurements help determine whether a model is suitable for a particular architectural role. Foundation-model evaluation provides evidence about general capabilities and limitations, while approaches such as HELM demonstrate the value of assessing models across multiple dimensions rather than relying on a single benchmark (Bommasani et al., 2021; Liang et al., 2022). Model-level results do not, however, establish that the model will perform reliably within a particular application or operational environment.

At the component level, evaluation addresses the mechanisms surrounding the model. Retrieval systems can be evaluated for relevance, coverage, grounding and source quality. Retrieval-augmented generation illustrates why these dimensions matter: the quality of generated output can depend substantially on the information retrieved and supplied as context rather than on the model alone (Lewis et al., 2020). Tool selection can be assessed according to whether the appropriate tool is selected under defined conditions and whether parameters are correctly generated. Memory mechanisms can be evaluated for accuracy, relevance, persistence and the handling of outdated information. Classification systems and guardrails can be evaluated for detection performance, false positives, false negatives and their ability to enforce defined policies. Component evaluation therefore provides evidence about whether individual architectural mechanisms perform their intended functions.

At the system level, evaluation examines the behaviour that emerges from the interaction of these components. Relevant measures may include task completion, factual grounding, failure rates, security performance, latency, resource consumption and cost. System-level evaluation is particularly important because errors can emerge from interactions between components even when each component performs adequately when evaluated in isolation. This reflects the broader systems problem identified in machine-learning engineering, where interactions among data, models, infrastructure and surrounding software can create failure modes that are not visible from the model in isolation (Sculley et al., 2015).

At the workflow level, evaluation moves beyond technical behaviour towards operational and organisational outcomes. Relevant measures may include business-process effectiveness, human override rates, error consequences, control effectiveness, compliance outcomes and the effect of AI on the overall process. A system that produces accurate text but increases downstream operational errors would not constitute a successful deployment merely because its model-level benchmark results are strong. The appropriate unit of evaluation is therefore ultimately linked to the business capability the AI system is intended to support.

These levels form a hierarchy of evaluation, but not a hierarchy of importance. Model benchmarks establish evidence about model capabilities; component evaluations establish evidence about individual architectural mechanisms; system evaluations assess integrated behaviour; and workflow evaluations determine whether the resulting capability produces the intended operational outcomes. The levels are complementary rather than interchangeable. Strong performance at one level cannot be assumed to imply strong performance at the levels above it.

The importance of this distinction is reinforced by the broader evaluation literature. The Holistic Evaluation of Language Models (HELM) framework evaluates language models across multiple dimensions rather than reducing performance to a single benchmark score (Liang et al., 2022). This reflects a general principle: AI evaluation should be multidimensional because performance, robustness, efficiency, safety and other properties can vary independently. A model may perform strongly on task accuracy while exhibiting undesirable behaviour under adversarial conditions, high latency under production workloads or poor performance when supplied with incomplete context.

For production systems, evaluation should also be scenario-based and risk-based. Representative scenarios should reflect the actual conditions under which the system will operate, including ambiguous requests, incomplete information, unusual inputs, adversarial behaviour, unavailable tools, conflicting information and degraded dependencies. Evaluation scenarios should therefore extend beyond normal or idealised inputs. High-impact workflows require greater evaluation depth because the consequences of failure are more significant. This is consistent with the risk-based approach to AI management advocated by NIST and ISO, where evaluation and controls should reflect the context and potential impact of system behaviour (NIST, 2023; ISO, 2023).

Scenario design should also include negative and boundary cases. A system should not only be evaluated on whether it can successfully perform a task, but also on whether it appropriately refuses, escalates or terminates when the task falls outside its authority or operating conditions. This is particularly important for agentic systems because successful execution is only one dimension of acceptable behaviour. Safe non-execution can itself be an intended system outcome.

Agentic systems introduce an additional evaluation requirement because their behaviour unfolds across sequences of actions rather than a single response. Evaluation should therefore consider trajectories: which decisions were made, which tools were selected, what information was retrieved, how the system responded to intermediate observations, whether it recovered from errors and whether it terminated appropriately. Agent architectures such as ReAct and Reflexion explicitly involve iterative reasoning and action, making trajectory-level behaviour an important object of evaluation (Yao et al., 2023; Shinn et al., 2023).

A system may produce an acceptable final answer while reaching it through an unnecessarily risky, inefficient or unauthorised sequence of actions. Conversely, a system may encounter an intermediate error but recover safely without affecting the final outcome. End-to-end evaluation should therefore examine both outcomes and execution paths. This is an important distinction for agentic architectures: the final result does not necessarily provide sufficient evidence about the quality or safety of the process used to obtain it.

Trajectory evaluation should consider properties such as tool-selection accuracy, unnecessary action loops, policy violations, inappropriate escalation, recovery behaviour, resource consumption and termination conditions. For example, an agent that eventually completes a task after repeatedly invoking an inappropriate tool may achieve an acceptable outcome while nevertheless exhibiting undesirable operational behaviour. Evaluation must therefore capture the path through which the system reaches an outcome, not only the outcome itself.

Evaluation must also account for human interaction. Where people review, approve, correct or override AI-generated outputs, the effectiveness of those interactions becomes part of system behaviour. Excessive override rates may indicate poor system performance, inappropriate confidence, inadequate workflow design or a mismatch between system outputs and operational requirements. Conversely, low override rates do not necessarily demonstrate that the system is performing well if users lack sufficient information or authority to challenge its outputs. Human oversight should therefore be evaluated as an architectural control rather than assumed automatically to compensate for system weaknesses.

This creates an important connection between evaluation and the architecture of human authority. Where a human approval gate exists, evaluation should establish whether the information provided to the approver is sufficient, whether approval occurs before consequential execution, whether inappropriate actions can actually be rejected and whether the intervention mechanism functions under realistic operating conditions. In this sense, the effectiveness of human oversight is itself an evaluatable system property.

Evaluation results should also influence architectural decisions. If retrieval quality is insufficient, the appropriate response may be to improve the data or retrieval architecture rather than replace the underlying model. If tool selection is unreliable, additional constraints or a different orchestration strategy may be required. If errors occur primarily during human hand-offs, workflow design may need to change. If failures arise from excessive context or stale information, memory or context-management mechanisms may need to be redesigned. Evaluation should therefore provide feedback into architecture, configuration, deployment and governance, rather than simply produce a pass-or-fail assessment.

This establishes a distinction between evaluation as measurement and evaluation as architectural feedback. The former determines how a system is performing; the latter uses those findings to determine whether the architecture itself requires modification. The second function is particularly important in complex AI systems because the observed failure may originate from a component other than the model. Treating every failure as a model-quality problem can lead to inappropriate remediation and obscure the actual architectural cause.

Evaluation should also be continuous. Models may change, prompts may be modified, retrieval indexes may evolve, tools may be replaced, policies may change and enterprise data may drift. A system that performs adequately at deployment may behave differently after one or more of these changes. Regression testing, ongoing monitoring, controlled experiments and periodic reassessment are therefore necessary to maintain confidence in system behaviour over time (NIST, 2023; ISO, 2023).

This makes evaluation closely connected to change management. Changes to models, prompts, retrieval sources, tool definitions, permissions or workflows can alter system behaviour even when the business process itself appears unchanged. Evaluation should therefore form part of controlled deployment and change processes, with appropriate testing and approval before consequential changes enter production. Model reporting and dataset documentation provide useful foundations for tracking characteristics and limitations, while system-level evaluation extends this principle to the behaviour of the deployed capability (Mitchell et al., 2019; Gebru et al., 2021).

Observability provides the evidence required for this continuous evaluation. Execution traces can record model invocations, retrieved context, tool calls, outputs, errors, state transitions and human interventions. Such evidence makes it possible to investigate failures and identify whether the underlying cause originated in the model, data, context, orchestration, tools, workflow or human interaction. Evaluation and observability should therefore be designed together rather than treated as independent capabilities.

The relationship can be expressed as:

Evaluation measures behaviour; observability provides evidence about that behaviour; architecture provides the mechanisms through which behaviour can be changed.

This creates a continuous feedback loop in which observed system behaviour is evaluated against defined requirements, deviations are investigated using execution evidence, architectural changes are introduced where necessary, and the resulting system is evaluated again. Evaluation consequently becomes part of the system's control loop rather than simply a testing activity performed before deployment.

The resulting architectural view is that AI performance cannot be reduced to model quality. A more useful representation is the interaction among model, data, context, tools, workflow and human interaction. System behaviour emerges from these dependencies, and evaluation must therefore examine the architecture as a whole. This is consistent with the broader systems perspective on machine learning, where model performance alone does not capture the operational behaviour of the complete system (Sculley et al., 2015; Bommasani et al., 2021).

The central principle is that evaluation is not a testing phase added after architecture; it is an architectural capability embedded within the system. A governed AI system should be designed so that its behaviour can be measured, its failures can be investigated, its controls can be tested and changes can be assessed before they create unacceptable operational consequences. The ability to evaluate an AI system is therefore closely connected to the ability to govern it.

In this model, evaluation forms part of the broader enterprise AI control architecture. Models provide capabilities; data and context provide information; orchestration and tools create execution paths; workflows establish operational outcomes; observability provides evidence; evaluation determines whether behaviour remains acceptable; and governance determines what actions should follow from that evidence. Evaluation is consequently not merely evidence of system quality. It is one of the mechanisms through which the organisation continuously determines whether its AI architecture remains fit for purpose.

14. Observability and Explainability

Production AI systems require continuous visibility into their behaviour. In conventional software engineering, observability typically relies on logs, metrics, traces, alerts and distributed tracing to establish what a system is doing, how it is performing and where failures occur. AI systems require these capabilities as well, but their probabilistic behaviour, dependence on external context and increasingly dynamic execution paths create additional requirements for understanding how outputs and actions were produced. In agentic systems, the execution path may involve multiple model calls, retrieval operations, tool invocations, policy checks and human interventions rather than a single deterministic transaction (Yao et al., 2023; Shinn et al., 2023).

AI observability should therefore capture information about both the conventional runtime environment and the AI-specific execution context. Depending on the system and use case, relevant telemetry may include the model and model version, prompt or instruction version, retrieved information, context supplied to the model, tool calls, intermediate outputs, policy decisions, latency, token consumption, evaluation results, human interventions, errors and final outcomes. The appropriate level of capture will depend on the risk and operational requirements of the system, but consequential AI processes require sufficient evidence to understand the relationship between inputs, system components, controls and resulting actions (NIST, 2023; ISO, 2023).

The purpose is not to record every internal computational detail of a model. Rather, observability should provide sufficient evidence to reconstruct and understand the system's externally relevant behaviour. In an agentic workflow, for example, this may require establishing which objective was assigned, which information was retrieved, which tools were invoked, what actions were proposed or executed, which policies were applied and how the system responded to intermediate results. Agent architectures make this particularly important because the final output may be the result of a sequence of interactions rather than a single model invocation (Yao et al., 2023; Wang et al., 2023).

This creates what can be described as an AI execution record or execution audit trail. Such a record can support debugging, performance analysis, security investigation, compliance and governance. It allows organisations to investigate not only whether an outcome was correct, but also the conditions under which it was produced. This distinction is important for enterprise systems because an acceptable output does not necessarily demonstrate that the process used to produce it was authorised, reliable or compliant. Conversely, an incorrect outcome may require examination of the execution chain to determine whether the cause was the model, retrieved information, a tool, a policy decision, an integration failure or a human intervention.

The distinction between observability and explainability is therefore important. Observability concerns the evidence available about system execution: what happened, when it happened, which components were involved, and what inputs, outputs, actions and controls were associated with the event. Explainability concerns the ability to provide an understandable account of why a system produced a particular output or recommendation. These capabilities overlap, but they are not equivalent. A detailed execution trace may establish which information and tools were used without providing a complete explanation of the internal computational processes through which a foundation model generated its output.

This distinction is particularly relevant for foundation models. Their internal representations and computations are not generally expressed as explicit human-readable rules. Foundation models are broad, general-purpose systems whose behaviour emerges from large-scale training and subsequent adaptation rather than from a conventional set of enterprise business rules (Bommasani et al., 2021). Consequently, organisations should avoid assuming that an internal model explanation can always be generated in the same form as an explanation of a deterministic business rule.

For many enterprise applications, a more practical objective is therefore to provide evidence about the system context surrounding the model. This may include the inputs supplied to the model, retrieved sources, instructions, model and configuration versions, tool interactions, applicable policies, permissions, human interventions and resulting actions. Where appropriate, uncertainty or confidence information may also form part of the evidence. Model reporting approaches such as model cards provide an established precedent for documenting model characteristics, limitations and intended uses, although execution-level observability extends beyond static model documentation (Mitchell et al., 2019).

Observability should consequently be designed around the questions that users, engineers, auditors, security teams and governance functions may need to answer. For a consequential recommendation, these questions might include:

What information was available to the system?
Which model and configuration were used?
Which retrieved sources contributed to the response?
Which tools or external systems were invoked?
What permissions were available at the time?
Which policies, guardrails or approval requirements were applied?
Did the system encounter an error, exception or failed control?
Was the output reviewed or modified by a person?
What action was ultimately taken?
What was the resulting business or system outcome?

The ability to answer these questions depends on architectural design. If model calls, retrieval operations, tool invocations, policy decisions and human approvals are not recorded in a coherent execution history, reconstructing an event after the fact may be difficult or impossible. Observability should therefore be treated as a design requirement, rather than as telemetry added after deployment. This reflects the broader principle that machine-learning systems accumulate system-level dependencies and operational complexity that cannot be adequately understood by examining the model alone (Sculley et al., 2015).

The same principle applies to multi-agent systems. When responsibility is distributed across several agents, observability must capture the interactions between them. Relevant evidence may include agent identity or role, messages exchanged, hand-offs, shared-state changes, tool calls, decisions requiring approval and termination conditions. Without such information, failures may be difficult to attribute because the final output may not reveal which component introduced the error. Multi-agent architectures therefore require observability that follows the execution graph rather than treating the overall system as a single opaque component (Wooldridge, 2009; Wang et al., 2023).

Observability also supports the continuous evaluation described elsewhere in the architecture. Evaluation establishes whether a system is performing adequately; observability provides much of the evidence required to understand why performance changed and where failures originated. Holistic evaluation approaches emphasise that AI systems should be assessed across multiple dimensions rather than through a single performance measure (Liang et al., 2022). Execution telemetry complements this by allowing changes in observed performance to be associated with particular system conditions.

A deterioration in task completion, for example, may result from a model update, changes to retrieval data, a modified prompt, tool latency, a policy change or a workflow modification. Without execution-level evidence, these possibilities may be difficult to distinguish. With appropriate telemetry, the organisation can compare executions across versions and identify which component or configuration changed. Observability therefore provides an important bridge between evaluation results and causal investigation.

Observability is also closely connected to security. AI applications may involve prompt injection, inappropriate tool use, unauthorised access, data leakage or other failures whose significance becomes apparent only when execution is considered as a sequence of events. OWASP's treatment of risks in LLM applications reinforces the importance of securing the broader application and interaction environment rather than considering the model in isolation (OWASP, 2025). Observability can support detection and investigation by recording relevant tool interactions, access decisions, policy violations and anomalous execution patterns.

There are, however, important governance considerations associated with observability itself. AI telemetry may contain sensitive prompts, retrieved documents, personal information, business-confidential data, tool parameters or records of human decisions. Logging more information does not automatically produce better governance. Excessive collection can itself create privacy, security and data-management risks. Observability must therefore be subject to appropriate access controls, retention policies, data-minimisation principles, security protections and lifecycle management (NIST, 2023; ISO, 2023).

The audit trail is therefore itself a governed data asset. Organisations must determine what information needs to be retained, who may access it, how long it should be retained, how sensitive information is protected and under what circumstances it may be used for debugging, evaluation, security investigation or audit. The objective is not maximal logging, but sufficient evidence for the risks and accountability requirements of the system.

Observability should also extend to changes over time. Model versions, prompts, retrieval configurations, tool definitions, policies and system components can all change independently. Maintaining version information makes it possible to distinguish between an isolated execution event and a change introduced by a subsequent deployment. This supports reproducibility, regression analysis, incident investigation and controlled change management. Versioned evidence is particularly important where the same workflow may behave differently following changes to a model, retrieval index or external tool (Sculley et al., 2015; NIST, 2023).

This creates an important relationship between observability and reproducibility. Exact reproduction of a foundation-model output may not always be possible because of model updates, stochastic generation, changing external data or dynamic tool responses. Nevertheless, an organisation can often preserve sufficient execution context to establish the conditions under which an event occurred. This distinction is important: the objective of enterprise observability is not necessarily to reproduce every internal computation, but to preserve enough evidence to reconstruct the relevant execution context and investigate consequential behaviour.

For consequential systems, observability can therefore be understood as an evidence architecture connecting technical execution with organisational accountability. It provides the factual record needed to evaluate system behaviour, investigate incidents, demonstrate control operation and support appropriate human oversight. It also provides the evidentiary foundation through which governance can determine whether an AI system operated within its approved boundaries.

This leads to a useful distinction between three related concepts:

Observability asks: What happened, and what evidence do we have about the execution?

Explainability asks: Can the system's output or decision be presented in an understandable way?

Auditability asks: Is there sufficient reliable evidence to establish whether the system operated according to applicable requirements?

These concepts reinforce one another but should not be collapsed into a single requirement. A system may be highly observable without providing a complete explanation of model internals. It may also provide explanations without retaining sufficient execution evidence to establish exactly what happened during a particular event. Enterprise architecture should therefore design for all three where the risk and accountability requirements justify them.

The central principle is that an AI system cannot be effectively governed if its consequential behaviour cannot be observed and reconstructed. Observability does not guarantee that a system is correct or safe, and it does not by itself provide a complete explanation of model reasoning. Its role is to provide reliable evidence about system execution. When designed appropriately, that evidence connects engineering, evaluation, security, governance and accountability into a continuous control system.

The broader architectural implication is that observability should be treated as a cross-cutting capability rather than a monitoring subsystem. It spans models, retrieval, memory, tools, orchestration, policy enforcement, infrastructure and human interaction. The resulting execution record becomes a common evidentiary layer across these components, allowing the organisation to connect what the system was configured to do, what it actually did, what controls were applied and what outcome resulted.

In this sense, observability completes an important part of the enterprise AI control architecture: reasoning generates proposals; orchestration coordinates execution; controls constrain behaviour; tools produce external effects; observability records what occurred; and governance uses that evidence to establish accountability and drive improvement.

15. Governance by Design

Governance is most effective when it is incorporated into the architecture of an AI system rather than treated as documentation added after deployment. Policies, standards, responsibilities and approval requirements establish organisational expectations, but those expectations become operational only when the underlying architecture can enforce them, generate evidence of their application and support corrective action when requirements are not met. AI governance should therefore be understood not simply as a set of policies surrounding a technical system, but as a mechanism through which organisational requirements are translated into system behaviour, control processes and accountable decisions.

The NIST AI Risk Management Framework (AI RMF) provides an important foundation for this approach by organising AI risk management around four functions: Govern, Map, Measure and Manage (NIST, 2023). Governance is explicitly treated as a cross-cutting function rather than as a discrete activity performed at a single point in the AI lifecycle. This reflects an important architectural reality: responsibilities, policies, risk tolerances and accountability mechanisms influence how AI systems are designed, deployed, operated, monitored and changed. Governance is therefore not an activity that occurs around the system; it influences the conditions under which the system is allowed to operate.

ISO/IEC 42001 similarly establishes requirements for an organisational artificial intelligence management system, including its establishment, implementation, maintenance and continual improvement (ISO, 2023). The significance of this management-system approach is that AI governance extends beyond individual model outputs or isolated technical decisions. It encompasses organisational processes for defining responsibilities, managing risks, controlling changes, monitoring performance, maintaining records and continually improving the system of controls. Governance consequently has both a technical dimension, concerned with how requirements are implemented, and an organisational dimension, concerned with who defines, owns, operates and assures those requirements.

These frameworks have an important architectural implication: governance requires technical mechanisms. An organisation may establish a policy requiring human oversight, but that policy has limited operational value if the system cannot identify when human approval is required or prevent execution until approval has been obtained. A policy requiring model-version tracking similarly requires architecture capable of recording which model, configuration and relevant system components were used for a particular execution. Governance requirements therefore need corresponding architectural control points.

The same principle applies to other governance requirements. Access-control policies require identity, authentication and authorisation mechanisms. Incident-management requirements require logging, detection, escalation and response capabilities. Performance-management requirements require evaluation and monitoring. Approval policies require workflow mechanisms capable of stopping or conditioning execution. Audit requirements require reliable evidence of system activity, decisions, interventions and changes. Data-governance requirements require mechanisms for controlling access, provenance, retention and permitted use. Governance becomes operational when these organisational requirements are represented through concrete system capabilities (NIST, 2023; ISO, 2023).

Governance therefore operates through the capabilities established across the AI architecture. Security architecture provides identity, permissions and execution boundaries. Safety mechanisms provide guardrails and runtime policy enforcement. Evaluation provides evidence about system performance, limitations and risk. Observability provides the execution evidence required to reconstruct events. Data architecture provides information provenance and authority. Workflow and orchestration architecture provides points at which actions can be constrained, approved or escalated. Governance connects these capabilities to organisational policies, responsibilities, risk tolerances and assurance processes. This reflects the broader principle that AI risks arise from interactions among models, data, software, infrastructure and organisational processes rather than from model behaviour in isolation (Sculley et al., 2015; Bommasani et al., 2021; NIST, 2023).

This creates a progression from policy to operational control:

Policy → Architecture → Control → Evidence → Assurance → Improvement

A policy establishes what the organisation requires; architecture determines where that requirement is implemented; controls enforce or constrain behaviour; evidence demonstrates whether the controls operated as intended; assurance assesses whether they remain effective; and improvement feeds the resulting findings back into policy and architecture. This cycle represents the central architectural interpretation of governance developed in this paper.

The distinction is important because documentation alone cannot guarantee compliance with a requirement. A document may state that certain actions require approval, but the system must be capable of enforcing the approval boundary. A policy may require that only authorised personnel access sensitive information, but the technical architecture must implement the corresponding identity and access controls. A governance standard may require periodic evaluation, but the system must generate the information necessary to perform that evaluation. Documentation establishes organisational intent; architecture determines whether that intent can be translated into repeatable operational behaviour.

Governance by design therefore requires explicit control points. These may include identity and authorisation checks, policy engines, approval gates, transaction limits, model and prompt version controls, data-access restrictions, evaluation gates, deployment approvals, monitoring thresholds, incident triggers and automated or human escalation mechanisms. The specific mechanisms will vary according to the system and risk context, but the architectural principle remains consistent: governance requirements should be connected to identifiable points at which system behaviour can be constrained, observed or changed (NIST, 2023; ISO, 2023; OWASP, 2025).

Governance by design also requires clear allocation of responsibility. Enterprise AI systems may involve model providers, application owners, data owners, platform teams, security functions, process owners, users and human approvers. The architecture should make the relevant responsibilities visible rather than allowing accountability to become ambiguous as tasks move between models, agents, tools and people. Model cards and datasheets demonstrate the broader importance of documenting characteristics, limitations and provenance, while organisational governance frameworks extend this principle into questions of ownership, risk management and accountability (Mitchell et al., 2019; Gebru et al., 2021; NIST, 2023).

This becomes particularly important in agentic systems. An AI agent may retrieve information, reason over that information, delegate a task, invoke a tool, modify system state and request human approval as part of a single execution chain. Governance must therefore apply across the complete chain rather than stopping at the model boundary. Agent architectures make the distinction between reasoning and execution particularly important because an apparently simple model interaction can result in actions with consequences in external systems (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

The architecture should consequently preserve sufficient evidence to establish which component performed an operation, under whose authority, using which information, subject to which controls and with what resulting effect. This extends observability from technical monitoring into governance evidence. Logs and traces are not merely operational diagnostics; where appropriately designed, they can provide evidence for accountability, incident investigation, audit and assurance. This is particularly important when multiple agents, tools and human decision-makers participate in a single business process.

Governance must also extend to change management. AI systems are not static assets. Models may be updated, prompts modified, retrieval sources changed, tools replaced, permissions altered and workflows redesigned. Each change can affect system behaviour and therefore potentially alter the associated risk profile. Sculley et al. (2015) highlight the broader technical debt and dependency problems that can arise as machine-learning systems evolve, while NIST and ISO emphasise lifecycle-oriented risk management and continual improvement (Sculley et al., 2015; NIST, 2023; ISO, 2023).

Consequently, governance requires mechanisms for version control, impact assessment, testing, approval, deployment, monitoring and, where necessary, rollback. The objective is not to prevent change, but to ensure that consequential changes occur within defined control boundaries. A model update, retrieval-source modification or permission change should therefore be treated as an architectural change that may require evaluation and authorisation rather than as an isolated technical maintenance activity.

An important consequence is that governance should be continuous rather than episodic. Periodic reviews, audits and formal assessments remain valuable, but many governance controls must operate during normal system execution. Access decisions, policy checks, approvals, monitoring, logging and exception handling occur at runtime. Other controls operate across the lifecycle through architecture reviews, evaluations, deployment gates, audits, incident analysis and improvement processes. Governance therefore spans both runtime architecture and lifecycle architecture.

This distinction also clarifies the relationship between governance and assurance. Governance establishes the requirements and accountability structure; controls implement those requirements; observability and evaluation generate evidence; assurance examines whether the controls remain appropriate and effective. Evaluation is therefore not merely a model-development activity. It can form part of the evidence base through which an organisation determines whether an AI system remains within its approved operating conditions (Liang et al., 2022; NIST, 2023).

Governance by design does not, however, mean that every governance requirement should be automated. Some decisions appropriately remain with people, particularly where consequences are significant, requirements are ambiguous or accountability cannot reasonably be delegated. Human oversight remains an architectural control in its own right. The purpose of governance architecture is therefore not to eliminate human judgement, but to make clear where human judgement is required, what information supports it, what authority is delegated, and what controls remain in force before and after the decision.

This leads to an important distinction between governance automation and governance enforcement. Some requirements can be enforced automatically, such as access restrictions, transaction thresholds, version checks or mandatory approval gates. Others require human interpretation, such as determining whether a particular use case is ethically or organisationally acceptable. Effective governance architecture should therefore combine automated controls with explicit human decision points rather than assuming that governance can be reduced to technical policy enforcement.

The resulting model can be represented as a continuous control cycle:

Policy → Architecture → Control → Evidence → Assurance → Improvement → Policy

The cycle is deliberately continuous. Changes in technology, organisational requirements, risk exposure, operating conditions or regulatory expectations can feed new requirements back into architecture and control design. Governance therefore becomes part of the system's lifecycle rather than a final approval step preceding deployment.

The central principle is that governance is an architectural capability as well as an organisational function. Effective AI governance connects policy to enforceable controls, controls to observable evidence, evidence to assurance, and assurance to continuous improvement. In this model, governance is not an external layer surrounding the AI system. It is embedded in the way the system is designed, operated, monitored, evaluated and changed.

This perspective also reinforces the broader architecture developed throughout this paper. Policy defines what is permitted; architecture establishes where those requirements apply; controls constrain behaviour; orchestration determines how actions proceed; observability records what occurred; evaluation tests whether the system remains within acceptable conditions; and governance assigns responsibility for the resulting outcomes. The result is an AI architecture in which organisational intent can be translated into enforceable operational behaviour and subsequently verified through evidence.

Governance by design can therefore be understood as the architectural transformation of organisational requirements into decision rights, control points, technical constraints, evidence mechanisms and accountable processes. Rather than treating governance as documentation surrounding an AI system, the architecture makes governance part of the system itself.

16. Human Oversight and Decision Architecture

The increasing capabilities of AI agents create a fundamental architectural question: how should decision-making authority be distributed between humans and machines? As AI systems move beyond information generation towards recommendation, planning and execution, human involvement can no longer be treated simply as a user-interface or interaction-design choice. It becomes a question of authority, control, accountability and risk. Research on language-model agents demonstrates that contemporary systems can combine reasoning with external actions, while work on reinforcement learning and multi-agent systems provides a broader basis for understanding delegated action within defined environments (Yao et al., 2023; Shinn et al., 2023; Sutton and Barto, 2018; Wooldridge, 2009). Consequently, human oversight needs to be considered as part of the architecture of decision-making rather than as an additional safeguard applied after the system has been designed.

Not every AI-supported task requires the same degree of human participation. Some activities can appropriately remain informational, whereas others may require explicit human approval before an action is executed. The appropriate arrangement depends on factors including the nature of the task, potential consequences of failure, uncertainty, reversibility, data sensitivity, regulatory requirements and the extent to which system behaviour can be constrained, observed and corrected. This is consistent with the broader risk-management principle that AI controls should be proportionate to the context and potential impact of system behaviour (NIST, 2023; ISO, 2023).

A useful architectural spectrum can therefore distinguish several forms of human–AI decision authority.

Information represents the lowest level of delegated authority. The AI system provides information, analysis or synthesis for a human to interpret and act upon. The human retains decision authority, while the AI primarily functions as an information-processing capability. Retrieval-augmented architectures are particularly relevant to this pattern because they can provide external information to support model-generated analysis without transferring decision authority to the model itself (Lewis et al., 2020).

The next level is recommendation, where the AI moves beyond presenting information and proposes a specific action or decision. The human remains responsible for determining whether the recommendation should be accepted. This pattern is particularly relevant where AI can process large quantities of information efficiently but the final decision depends on contextual knowledge, professional judgement or organisational responsibility. Human-feedback approaches demonstrate the importance of shaping model behaviour towards human-defined objectives, but alignment of model outputs does not itself establish organisational authority to act (Ouyang et al., 2022; Bai et al., 2022).

The third level is assisted execution, where the AI prepares or initiates an action but execution remains subject to human approval. The approval therefore represents an explicit control point in the workflow rather than merely an opportunity to review an action after it has effectively been completed. For consequential operations, this distinction is important: human oversight is most meaningful when intervention occurs before an irreversible external effect rather than after the system has already acted.

The fourth level is bounded automation, where the system is permitted to perform predefined actions automatically within established limits. Such automation is appropriate where the permitted actions are sufficiently well specified, the consequences are limited or reversible, and appropriate monitoring, exception handling and recovery mechanisms are available. Agent architectures such as ReAct and Reflexion illustrate how language models can be incorporated into iterative reasoning-and-action processes; however, the ability to perform such actions does not in itself determine the appropriate authority boundary for an enterprise system (Yao et al., 2023; Shinn et al., 2023).

The highest level considered here is conditional autonomy, where an AI system operates with limited direct human intervention within an explicitly defined operating envelope. Autonomy in this context should not be interpreted as unrestricted authority. It represents authority delegated within controlled boundaries, supported by defined permissions, termination conditions, monitoring, escalation mechanisms and intervention capabilities. This interpretation is consistent with the broader conception of agents as systems that operate within environments while pursuing specified objectives and interacting with other agents or systems (Sutton and Barto, 2018; Wooldridge, 2009).

These levels should not, however, be interpreted as a universal maturity ladder. A higher level of autonomy is not inherently a more advanced or desirable architecture. Different activities within the same organisation may appropriately operate at different levels depending on their risk, reversibility, regulatory context and operational characteristics. The objective of architecture is therefore not to maximise autonomy, but to establish an appropriate relationship between capability, authority and control.

The allocation of authority should consider factors including potential impact, reversibility, data sensitivity, regulatory requirements, uncertainty, operational criticality and the consequences of failure. The required level of human involvement should generally increase where errors could produce significant, irreversible, difficult-to-detect or difficult-to-remediate consequences. This reflects the risk-based orientation of contemporary AI governance frameworks, in which controls and oversight are expected to reflect the characteristics and potential impact of the AI system rather than being applied uniformly (NIST, 2023; ISO, 2023).

Controllability is particularly important within this model. A system may be highly capable while remaining difficult to constrain, predict, interrupt or audit. Conversely, a less capable system operating within a narrow and well-defined environment may be substantially easier to control. The architectural question is therefore not simply how capable the underlying model is, but whether the surrounding system can reliably constrain and govern the behaviour that the model enables. This is consistent with research highlighting the importance of system-level dependencies, operational controls and technical debt beyond the behaviour of an individual machine-learning model (Sculley et al., 2015).

This leads to a fundamental distinction between capability and authority. A model may be capable of generating a sophisticated recommendation, but this does not imply that it should have permission to implement that recommendation. Similarly, an agent may technically be capable of accessing an enterprise system or executing a transaction without being authorised to do so. Capability describes what the system can potentially accomplish; authority describes what the system is permitted to accomplish. This distinction is particularly important in agentic architectures because reasoning and action can otherwise become tightly coupled. The architectural separation of reasoning from execution provides an opportunity to place policy, authorisation and human decision rights between what an AI system proposes and what an enterprise system permits it to do (Yao et al., 2023; NIST, 2023; OWASP, 2025).

Human oversight should therefore be implemented through explicit architectural mechanisms rather than through general expectations that users will intervene when necessary. These mechanisms may include approval gates, escalation paths, transaction thresholds, dual-control requirements, exception handling, timeouts, intervention mechanisms and the ability to suspend or terminate execution. Such controls align with the broader requirement for AI systems to remain subject to organisational governance, monitoring and risk management throughout their lifecycle (NIST, 2023; ISO, 2023). The objective is to ensure that human involvement is meaningful and occurs at a point where it can still influence the outcome.

The design of approval mechanisms is equally important. Human oversight can become ineffective if individuals are expected to approve large volumes of AI-generated actions without sufficient information, context or time to assess them. Meaningful oversight requires relevant evidence, clear responsibility and a genuine opportunity to intervene. This makes observability a prerequisite for effective human control. If the system cannot expose what information it used, what reasoning or processing occurred, what tools were invoked, what policies were applied and what action is proposed, the human decision-maker may have insufficient basis for meaningful intervention. Holistic evaluation and model-reporting approaches reinforce the broader requirement for transparency and evidence about system behaviour and limitations (Liang et al., 2022; Mitchell et al., 2019).

Human oversight should also be adaptive. A system operating under normal conditions may be permitted to execute within predefined boundaries, while unusual circumstances can trigger escalation to a human decision-maker. Changes in risk, uncertainty, data quality, system behaviour or external conditions can therefore alter the required level of intervention. This creates a dynamic control model rather than a fixed allocation of human and machine responsibility. Such an approach is consistent with risk-management frameworks that treat AI governance as an ongoing lifecycle activity rather than a one-time design decision (NIST, 2023; ISO, 2023).

The same principle applies to multi-agent systems. Different agents may perform different functions and therefore require different levels of authority. A research agent may operate with considerable independence when retrieving or synthesising information, whereas a transaction agent may require explicit approval before producing an external effect. Human oversight should consequently be associated with the consequences of particular actions, rather than being applied uniformly to every model interaction. Agent architectures make this distinction especially important because a single system may contain multiple reasoning and execution loops with different permissions and operational effects (Wooldridge, 2009; Yao et al., 2023; Wang et al., 2023).

The architecture should also preserve accountability when humans and AI jointly produce an outcome. Where a human approves an AI-generated recommendation, the system should retain sufficient evidence to establish what was presented, what information supported it, which controls were applied and what decision was made. Model cards, datasheets and broader AI governance frameworks provide established precedents for documenting model characteristics, data provenance, limitations and system risks (Mitchell et al., 2019; Gebru et al., 2021; NIST, 2023). In an enterprise setting, these principles can be extended to the decision and execution trail so that responsibility remains traceable across the human–AI workflow.

This does not mean that responsibility can simply be assigned to either the human or the AI. Rather, the architecture should make decision rights explicit and auditable. The system should distinguish what the AI was permitted to recommend, what it was permitted to execute, what required human approval and which organisational actor or process was accountable for the resulting decision. Governance therefore becomes an architectural property rather than merely a policy statement (ISO, 2023; NIST, 2023).

The resulting principle can be expressed as follows:

Autonomy should be proportional to controllability.

An AI system should not receive greater operational authority merely because the underlying model has become more capable. Greater capability may make a broader range of tasks technically feasible, but authority should be delegated only where the surrounding architecture can constrain, monitor, evaluate, interrupt and govern the resulting behaviour. This principle places the emphasis on the control environment surrounding the model, rather than on model capability as a proxy for organisational readiness.

Human oversight is therefore best understood not as a fallback mechanism for unreliable AI, but as a deliberate component of decision architecture. The architectural objective is to determine which decisions can be delegated, which require human participation, what evidence humans require to exercise meaningful judgement, what permissions are granted to AI systems, and what controls remain in force when decisions are delegated to automated processes. In this sense, enterprise AI architecture is not simply concerned with determining what an AI system can do. It must also determine what it may do, under whose authority, within which boundaries, with what evidence, and subject to which mechanisms of intervention and accountability.

This establishes a direct connection between the broader principles of enterprise AI architecture: reasoning can generate proposals, policy can constrain them, orchestration can coordinate execution, tools can produce external effects, observability can make behaviour visible, and governance can assign accountability. The resulting architecture treats autonomy not as an end in itself, but as a controlled delegation of authority within an enterprise operating environment.

17. Data Architecture and AI Readiness

AI architecture ultimately depends on the quality, accessibility, structure, and governance of the data environment in which the system operates. Model capability cannot compensate indefinitely for incomplete, inconsistent, inaccessible, or poorly governed information. When the information supplied to an AI system is unreliable, the resulting system behaviour may also be unreliable, regardless of the sophistication of the underlying model. This reflects a broader characteristic of foundation-model systems: model capability is only one component of a larger system whose behaviour depends on the data, context, interfaces, and operational environment surrounding the model (Bommasani et al., 2021; Sculley et al., 2015).

This makes data architecture a foundational component of AI architecture rather than a supporting technical function. The context and retrieval mechanisms described earlier in this paper depend on an underlying information environment in which relevant information can be identified, accessed, interpreted, and governed. Retrieval-augmented generation provides a clear example: the ability of a model to generate useful responses can be extended through external information retrieval, but the effectiveness of that architecture depends on the quality, relevance, and accessibility of the information being retrieved (Lewis et al., 2020). AI readiness is therefore substantially a property of the organisation's information architecture.

An AI-ready data environment requires authoritative sources from which information can be obtained with appropriate confidence. It also requires common data definitions and semantic consistency so that information has sufficiently stable meaning across applications and organisational boundaries. Metadata provides additional context about the origin, meaning, ownership, status, and appropriate use of data, while lineage allows information to be traced through transformations and downstream uses. These characteristics are consistent with the broader principle that data used in AI systems should be documented in ways that make its provenance, characteristics, limitations, and appropriate use understandable to downstream users and systems (Gebru et al., 2021).

Data quality is equally important. Accuracy, completeness, consistency, timeliness, and validity influence the reliability of retrieval, analysis, recommendations, and automated actions. Data-quality management should therefore be treated as an ongoing architectural capability rather than a one-time preparation exercise. AI systems may expose existing weaknesses in enterprise data because they can consume and combine information at a scale and speed that makes inconsistencies more visible and operationally consequential. The resulting problem is not simply that poor data can produce poor model outputs; poorly managed data can also propagate errors through retrieval, memory, analytics, and downstream automated processes (Sculley et al., 2015; Gebru et al., 2021).

Access control is another fundamental requirement. Making data available to AI does not imply making all enterprise information available to every model or agent. Data access should remain subject to identity, authorisation, purpose, and policy constraints. The architecture should be capable of determining which information a particular AI component is permitted to access and should preserve sufficient evidence to establish how that information was subsequently used. This becomes particularly important for agentic systems that can retrieve information and subsequently use it to inform actions or tool calls (Yao et al., 2023; NIST, 2023).

The distinction between data access and data authority is therefore fundamental. An AI component may technically be able to retrieve information without being authorised to use that information for every purpose. Similarly, information that is accessible may not represent an authoritative organisational record. Architectural controls should consequently separate the questions of Can the system access this information? and Is this information authoritative and appropriate for the intended decision or action? This distinction supports the broader governance principle that AI systems should operate within explicitly defined organisational and risk boundaries (NIST, 2023; ISO, 2023).

Retention and lifecycle management must also extend to AI-relevant data. Documents, embeddings, retrieved content, interaction histories, memories, and derived datasets may all persist beyond the original interaction. Their retention periods, deletion requirements, versioning, and ownership should therefore be governed consistently with the underlying information and applicable organisational requirements. The problem is particularly relevant to AI systems because information can be copied or transformed into multiple representations, creating additional lifecycle dependencies that may not exist in the original source system. Data governance should therefore account not only for source records but also for significant derived representations used by AI systems (Gebru et al., 2021; NIST, 2023).

Interoperability is increasingly important because contemporary AI systems rarely operate against a single data source. An AI-enabled workflow may need to combine structured databases, enterprise documents, event streams, knowledge graphs, vector indexes, application interfaces, and transactional systems. These sources may have different schemas, update frequencies, access models, and levels of authority. AI architecture must therefore provide controlled mechanisms for integrating heterogeneous information rather than assuming that all relevant knowledge exists within a single repository. Retrieval-augmented architectures demonstrate this principle by connecting model generation with external information sources rather than requiring all relevant knowledge to reside within model parameters (Lewis et al., 2020).

This also reinforces the importance of explicit information interfaces. AI systems should not depend on undocumented or uncontrolled access to enterprise data. Interfaces should establish how information is discovered, retrieved, transformed, authorised, and returned to the AI workflow. Emerging protocols for connecting models and applications to external context and tools illustrate the architectural importance of making these interactions explicit rather than treating them as hidden implementation details (Anthropic, 2024). Explicit interfaces can also improve auditability, portability, and the ability to change underlying data services without redesigning the complete AI application.

The distinction between operational data platforms, analytical platforms, and AI platforms is consequently becoming less absolute. Operational systems remain responsible for executing business processes and maintaining authoritative transactional records. Analytical environments support aggregation, analysis, and decision support. AI systems may consume information from both while also creating new derived artefacts, embeddings, summaries, classifications, and other forms of machine-generated information. The architectural challenge is to connect these capabilities without confusing their different roles and authority. In particular, machine-generated information should not automatically acquire the same authority as the source records from which it was derived.

This distinction becomes particularly important for AI-generated data. Summaries, classifications, embeddings, extracted entities, inferred relationships, and stored memories can become inputs to subsequent AI operations. If these derived artefacts are treated as authoritative without appropriate provenance and validation, errors can propagate across successive system interactions. Architecture should therefore preserve relationships between source information and derived representations where that relationship is material to the reliability or governance of the system. Documentation and provenance practices provide an important foundation for maintaining this distinction (Gebru et al., 2021; Mitchell et al., 2019).

Real-time and event-driven access may also be necessary for particular use cases. An AI system supporting fraud detection, cybersecurity, operational monitoring, or dynamic decision-making may require information that changes continuously. In such environments, architectural decisions concerning event streams, latency, freshness, consistency, and failure handling become part of AI system design. The relevant question is not simply whether data is available, but whether it is available at the required time and with sufficient reliability for the decision being made. This connects data architecture directly with the broader resilience and workflow requirements of the AI-enabled business capability.

This also reinforces the distinction between data availability and data authority. Information can be technically accessible without being suitable as an authoritative basis for a decision. An AI system may retrieve a document, message, or generated summary, but its availability does not establish its validity or organisational authority. Systems should therefore preserve provenance and distinguish authoritative records from derived, inferred, or user-generated content. Model documentation and dataset documentation provide complementary mechanisms for making such characteristics visible to downstream users and system designers (Mitchell et al., 2019; Gebru et al., 2021).

AI-ready data architecture must also accommodate change. Data schemas evolve, source systems are replaced, organisational definitions change, and new information sources emerge. Retrieval indexes and other derived representations may become stale as underlying data changes. The architecture should therefore provide mechanisms for synchronisation, versioning, re-indexing, quality monitoring, and controlled change. This is particularly important for retrieval-augmented systems, where the effectiveness of the AI workflow depends on the continued alignment between source information and the representations through which that information is retrieved (Lewis et al., 2020).

Change management should also account for changes in the meaning of information, not merely its technical structure. A schema can remain syntactically compatible while the underlying business definition changes. Similarly, an organisational policy may alter the authoritative status of a data source without changing its interface. AI systems therefore require mechanisms for managing semantic as well as technical change. Common definitions, metadata, ownership, provenance, and explicit authority relationships become important controls for maintaining the reliability of AI-enabled processes as the enterprise evolves.

These requirements suggest that AI readiness should not be defined primarily by whether an organisation has deployed a particular AI platform or accumulated large volumes of data. Readiness is better understood as the ability to provide relevant, authoritative, governed, accessible, and sufficiently current information to AI systems within controlled architectural boundaries. The quantity of available data is therefore not equivalent to AI readiness. An organisation may possess extensive information while still lacking the metadata, quality controls, access mechanisms, semantic consistency, provenance, or integration capabilities required to use that information reliably.

The implication is that AI architecture should be treated as an extension of enterprise information architecture rather than as an isolated innovation platform. The model may provide the reasoning capability, but the information environment determines much of the context in which that capability operates. Retrieval, memory, analytics, automation, and agentic execution all depend on the availability of appropriately structured and governed information. The architecture must therefore connect AI components to enterprise information without weakening the authority, security, or lifecycle controls that apply to the underlying data.

This perspective also changes the way organisations should approach AI investment. Increasing model capability without addressing information architecture may produce diminishing returns when the limiting factor is the quality, accessibility, or authority of the information available to the system. Conversely, improvements to data quality, metadata, provenance, interoperability, and access controls can increase the usefulness of existing models and AI applications. Data architecture is therefore not merely an enabling foundation for AI; it can become a direct determinant of the value and reliability of AI-enabled business capabilities.

The central principle is therefore that AI readiness is fundamentally an information-architecture capability. Organisations do not become AI-ready simply by acquiring more capable models. They become AI-ready when their data can be discovered, understood, authorised, governed, integrated, evaluated, and used reliably within the execution architecture of AI-enabled systems.

In this sense, the progression from data architecture to AI architecture is not a transition from one technical domain to another. It is an extension of the same enterprise information system into a new mode of information processing. The model supplies computational intelligence; retrieval supplies relevant context; data architecture establishes information authority and provenance; governance establishes permissible use; and the wider execution architecture determines how that information is ultimately converted into decisions and actions.

18. Infrastructure and Deployment Architecture

AI workloads impose distinctive infrastructure requirements because contemporary AI systems combine computationally intensive model inference with retrieval, memory, tool invocation, orchestration, monitoring, and other supporting services. Infrastructure therefore provides more than a hosting environment for a model. It forms the runtime foundation through which the complete AI architecture is deployed, scaled, secured, monitored, and operated. This distinction becomes increasingly important as foundation models are embedded within larger systems whose behaviour depends on interactions between models, data, infrastructure, software components, and external services (Bommasani et al., 2021; Sculley et al., 2015).

Relevant infrastructure capabilities may include high-performance compute and accelerator capacity, model serving, inference optimisation, storage, networking, containerisation, workload orchestration, security controls, monitoring, resilience mechanisms, and cost management. The appropriate combination depends on the workload, deployment model, data requirements, latency objectives, and operational risk. The infrastructure should therefore be designed as part of the overall system architecture rather than selected independently as a generic platform for AI workloads.

The infrastructure architecture must also distinguish between different computational requirements. Model training, fine-tuning, batch processing, retrieval, real-time inference, evaluation, and agentic execution may have substantially different resource profiles. Treating all AI workloads as a single infrastructure category can therefore lead to inefficient resource allocation and makes it more difficult to establish appropriate performance and cost controls. The computational requirements of large models also do not scale uniformly across all activities: training and inference create different resource demands, while inference requirements can vary significantly according to model size, context length, concurrency, and workload characteristics (Kaplan et al., 2020; Bommasani et al., 2021).

Inference economics becomes particularly important as AI systems move from simple generation towards iterative agentic execution. A conventional conversational application may invoke a model once or a small number of times in response to a user request. An agentic workflow may invoke one or more models repeatedly while planning, retrieving information, selecting tools, interpreting tool results, revising its approach, and producing a final outcome. Each additional inference introduces potential latency, resource consumption, and cost. Agentic architectures therefore transform inference from a discrete application operation into a potentially iterative workload (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

The effective cost and performance of an AI system must therefore be assessed at the workflow level, not merely at the individual model-call level. A model that is relatively inexpensive per invocation may become costly when an agent repeatedly invokes it. Conversely, a more capable model may reduce overall execution cost if it completes a task in fewer iterations or requires less corrective processing. Architecture should therefore consider the relationship between model capability, number of invocations, context size, tool interactions, latency, and task completion. This reinforces the broader architectural principle that model capability is only one contributor to system-level value and performance (Bommasani et al., 2021; Sculley et al., 2015).

This creates a broader optimisation problem involving latency, reliability, capability, and cost. Improving one dimension may affect another. A larger model may improve task performance while increasing latency and resource consumption. Additional validation steps may improve reliability while increasing execution time. Aggressive caching may reduce cost and latency while introducing freshness considerations. Architectural decisions should therefore be evaluated against the requirements of the complete workload rather than optimising a single metric in isolation. This is consistent with the broader move towards holistic evaluation of AI systems, where performance is understood as multidimensional rather than reducible to a single benchmark score (Liang et al., 2022).

Several techniques can be used to manage these trade-offs. Model routing can direct different tasks to models with different capabilities and resource profiles. Simple or low-risk tasks may use smaller models, while complex reasoning tasks may be directed to more capable models. Caching can avoid unnecessary repeated computation where the relevant inputs and outputs remain valid. Batching can improve resource utilisation for workloads that do not require immediate responses. Selective use of more expensive models can reserve significant computational resources for tasks where their additional capability provides a meaningful system-level benefit.

These techniques should be treated as architectural decisions rather than merely low-level performance optimisations. Model routing affects system behaviour, capability, and potentially resilience. Caching affects data freshness and consistency. Batching affects latency and responsiveness. Model selection affects capability, cost, and potentially data-handling requirements. Infrastructure architecture is therefore closely connected to the functional and governance characteristics of the AI system. Changes to infrastructure behaviour can alter the effective behaviour of the AI capability even when the underlying model remains unchanged.

The relationship between model capability and infrastructure requirements is also influenced by scaling behaviour. Larger models and more demanding workloads can require substantial computational resources, while increases in demand can create additional requirements for concurrency, scheduling, and capacity management. Scaling research demonstrates that model performance and computational requirements are closely related to model and training scale, while enterprise deployment introduces additional dimensions associated with serving and workload demand (Kaplan et al., 2020). Infrastructure architecture must therefore consider not only the computational requirements of an individual model but also the expected workload and growth profile of the complete service.

Deployment architecture must also account for scalability and resilience. Demand may vary substantially across AI workloads, and model inference can place significant pressure on computational resources. Systems may therefore require mechanisms for workload scheduling, capacity management, horizontal scaling, queueing, prioritisation, and graceful degradation. Where AI services become unavailable or capacity is constrained, the architecture should define appropriate fallback behaviour rather than allowing infrastructure failure to propagate unpredictably into business processes. This connects directly with the preceding principle that resilience requires controlled degradation and explicit failure boundaries rather than an assumption of continuous successful execution.

The location and mode of deployment can also affect architectural decisions. AI workloads may be deployed through public cloud services, private infrastructure, hybrid environments, or specialised platforms. Relevant considerations include data sensitivity, latency, regulatory requirements, operational control, model availability, integration requirements, and resilience. The infrastructure choice should therefore follow system requirements rather than being treated as an independent technology decision. This also connects with the earlier discussion of AI sovereignty: deployment location is one dimension of control, but meaningful architectural control also depends on dependencies, interfaces, operational capabilities, and the ability to change or replace components.

Security remains an integral part of deployment architecture. Model-serving infrastructure, containers, orchestration platforms, storage, networks, credentials, secrets, and deployment pipelines all form part of the system's attack surface. The security principles established elsewhere in the architecture therefore extend into the runtime environment. Infrastructure should enforce appropriate identity, access control, isolation, monitoring, vulnerability management, secret protection, and change control. AI-specific security concerns, including prompt injection and unsafe interaction with external tools or services, further demonstrate that the runtime environment cannot be separated from the security architecture of the AI application (OWASP, 2025; NIST, 2023).

Identity and access controls are particularly important where models or agents can invoke enterprise systems. The infrastructure should distinguish between the identity of the human or application initiating a request, the identity under which an AI component operates, and the permissions required for a particular action. Scoped credentials and least-privilege execution can reduce the consequences of erroneous or compromised behaviour. This reinforces the earlier architectural distinction between intelligence and authority: the model may determine what it proposes, while infrastructure and enterprise controls determine what the system is technically permitted to execute (NIST, 2023; ISO, 2023).

Observability is similarly required at the infrastructure level. AI-specific telemetry should be combined with conventional infrastructure measurements such as resource utilisation, service availability, network performance, queue depth, error rates, and capacity. This allows operational teams to distinguish between model-level problems and infrastructure-level problems and to understand how resource conditions affect system behaviour. For agentic workloads, observability should also account for model calls, tool invocations, retrieval operations, execution duration, and termination conditions, thereby connecting infrastructure telemetry to the actual execution behaviour of the AI system (Yao et al., 2023; Shinn et al., 2023).

Infrastructure architecture must also support controlled change. Models, serving frameworks, hardware configurations, dependencies, containers, orchestration components, and deployment configurations can all change over time. Versioning, testing, deployment controls, rollback mechanisms, and environment separation are therefore important components of reliable AI operations. This is particularly important where changes to an upstream model or serving framework can alter downstream system behaviour without corresponding changes to the application itself. Model reporting and evaluation practices provide a complementary basis for understanding changes in model characteristics and limitations (Mitchell et al., 2019; Liang et al., 2022).

The infrastructure layer should also support cost governance. AI consumption can vary according to user demand, model selection, context size, number of model invocations, retrieval operations, tool calls, concurrency, and execution duration. Consequently, infrastructure cost should be associated with business workloads and capabilities rather than treated simply as a fixed platform expense. This supports a bottom-up approach to AI economics in which expected usage patterns and execution architecture are translated into computational requirements and operating costs.

For agentic systems, this becomes particularly significant because the number of model and tool interactions may not be known in advance. A workflow with a nominally simple user interaction can produce substantial computational activity if the agent performs multiple reasoning, retrieval, validation, and tool-execution cycles. Execution limits, workload quotas, concurrency controls, and cost budgets can therefore function as both economic and safety mechanisms. They constrain not only expenditure but also the extent to which an unexpected execution loop can consume organisational resources.

The infrastructure architecture should consequently be capable of supporting policy-driven resource allocation. Different workloads may have different priorities, service-level requirements, cost limits, or risk classifications. High-criticality processes may require dedicated capacity and stronger resilience, while lower-risk workloads may tolerate queueing, reduced model capability, or delayed processing. Infrastructure can therefore become an enforcement point for business and governance policies rather than merely a technical substrate.

This also reinforces the importance of separating platform capability from workload requirements. A technically sophisticated AI platform is not necessarily appropriate for every business capability. Infrastructure should be selected according to the characteristics of the workload, including required model capability, data sensitivity, expected demand, latency, availability, resilience, security, and cost. In some cases, a managed external service may provide an appropriate balance of operational efficiency and control; in others, stronger requirements for isolation, portability, or sovereignty may justify dedicated infrastructure. The architectural decision should follow the requirements of the AI-enabled business capability rather than the availability of a particular technology platform.

The central principle is that AI infrastructure is part of system architecture, not merely a hosting layer. The infrastructure determines the computational environment in which models and agents operate and therefore influences latency, reliability, scalability, security, resilience, and cost. As agentic systems increase the number and complexity of model and tool interactions, these infrastructure characteristics become increasingly important to the overall behaviour and economics of the system.

AI deployment should consequently be designed around the requirements of the complete execution architecture. The objective is not simply to provide sufficient compute for a model, but to establish a runtime environment in which models, data, retrieval, memory, orchestration, tools, governance, and human interaction can operate reliably and economically at the required scale. In this sense, infrastructure becomes an active architectural component: it does not merely host AI capabilities, but helps determine the boundaries within which those capabilities can operate, the economics under which they can scale, and the resilience with which they can be delivered.

19. Resilience and Failure Engineering

AI systems introduce failure modes that differ from those typically encountered in deterministic software. Conventional systems can fail because a service becomes unavailable, a component crashes, or an input violates a defined constraint. AI systems can experience these failures as well, but they can also continue operating while producing incorrect, unsafe, or misleading results. A system may therefore remain technically available while behaving incorrectly. This distinction is important because the operational characteristics of machine learning systems emerge from interactions between models, data, infrastructure, software components, and surrounding processes rather than from a single deterministic execution path (Sculley et al., 2015; NIST, 2023).

Examples include hallucinated or unsupported information, incorrect planning, inappropriate tool selection or use, retrieval failure, context contamination, prompt injection, model degradation, provider outages, excessive latency, runaway agent loops, and outputs that are incorrect but sufficiently plausible to escape immediate detection. These failure modes are particularly significant in systems that combine language models with retrieval, external tools, and autonomous or semi-autonomous execution. Retrieval-augmented systems introduce dependencies on external information retrieval, while agentic systems introduce additional failure modes associated with iterative reasoning and action selection (Lewis et al., 2020; Yao et al., 2023; Shinn et al., 2023). Application-level threats such as prompt injection and inappropriate tool interaction further demonstrate why AI security and resilience cannot be reduced to infrastructure availability alone (OWASP, 2025).

Resilient AI architecture should therefore begin with the assumption that components, dependencies, and AI decisions will sometimes fail. The objective is not to eliminate every failure, which is generally unrealistic, but to ensure that failures are detected where possible, contained within appropriate boundaries, and prevented from producing disproportionate consequences. This reflects a broader systems-engineering principle in which reliability depends not only on preventing individual failures but also on controlling the effects of dependencies and unexpected system behaviour (Sculley et al., 2015).

Conventional resilience mechanisms remain important. Timeouts can prevent stalled dependencies from blocking an entire workflow. Retries can recover from transient failures, although they should be bounded and used carefully where repeated execution could create duplicate or consequential actions. Circuit breakers can prevent repeated calls to degraded services. Rate limits can constrain resource consumption and reduce the impact of runaway behaviour. Fallbacks can provide alternative models, services, or workflows when a primary dependency becomes unavailable. These mechanisms remain necessary because AI systems continue to depend on conventional distributed infrastructure even when their internal behaviour is probabilistic.

AI systems also require failure mechanisms that account for their probabilistic behaviour. Human escalation can provide an alternative path when uncertainty, ambiguity, or policy exceptions exceed defined thresholds. Sandboxing can isolate generated code or other potentially consequential operations. Bounded execution can prevent agents from repeatedly invoking models or tools without making meaningful progress. Transaction controls and rollback mechanisms can limit the consequences of erroneous actions. These controls are particularly relevant to agentic architectures in which models can iteratively reason, select actions, interact with tools, and modify their execution state (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

Termination conditions are particularly important for agentic systems. An agent operating in an iterative reasoning and action loop should not be permitted to continue indefinitely merely because the model continues to generate plausible next steps. The architecture should define explicit conditions under which execution stops, succeeds, fails, or escalates. This follows directly from the distinction between model capability and system authority: the ability of an agent to generate another action does not, by itself, establish that the action should continue to be executed.

These conditions may include a maximum number of iterations, execution timeouts, resource limits, confidence or evidence thresholds, policy violations, repeated failures, unavailable dependencies, or explicit human escalation. The exact conditions should depend on the task and its risk. A research workflow may tolerate more iterations than a transaction-processing workflow, while a high-impact operation may require human intervention after a relatively small number of uncertain steps. Risk-management frameworks similarly emphasise that controls should be proportionate to the context and potential impact of an AI system rather than applied uniformly across all use cases (NIST, 2023; ISO, 2023).

Failure handling should also distinguish between technical failure and semantic failure. A tool returning an error is a technical failure that may be detected directly. An agent selecting the wrong tool or generating a plausible but incorrect conclusion may produce no technical error at all. Resilience therefore requires evaluation and validation mechanisms capable of detecting failures in the meaning and consequences of system behaviour, not merely failures in infrastructure availability. This reinforces the importance of holistic evaluation, in which AI systems are assessed across multiple dimensions rather than through a single measure of model performance (Liang et al., 2022).

This is particularly important for generated outputs that cross system boundaries. An incorrect response presented to a user may be recoverable through human review, whereas an incorrect instruction sent to a transactional system may create an external consequence. Failure engineering should therefore consider the point at which an error becomes consequential and place appropriate validation, approval, or containment mechanisms before that boundary. The architectural distinction between generating an output and authorising an action is consequently important: systems can allow AI to propose actions while retaining explicit control over whether and how those actions are executed.

Graceful degradation is another important principle. When a dependency becomes unavailable or system confidence falls below an acceptable threshold, the appropriate response may not be complete system failure. The architecture may instead reduce functionality, switch to a more constrained workflow, provide an informational response rather than execute an action, or transfer the task to a human. Degradation should be deliberate and defined rather than emerging unpredictably from component failure. This approach is consistent with risk-based AI management, in which organisations are expected to identify and manage potential impacts across the lifecycle of an AI system (NIST, 2023; ISO, 2023).

Fallbacks require similar care. Switching automatically to another model or provider may preserve availability, but the alternative may differ in capability, context capacity, behaviour, security characteristics, cost, or data-handling requirements. A fallback should therefore be treated as an architectural dependency with its own validation and governance requirements rather than assumed to be equivalent to the primary component. Model reporting and evaluation practices reinforce the importance of understanding model characteristics and limitations before relying on a model as a substitute within a production workflow (Mitchell et al., 2019; Liang et al., 2022).

Resilience also depends on observability. If failures cannot be detected or reconstructed, recovery mechanisms may be ineffective. Execution traces, error records, tool results, policy decisions, and system metrics provide the evidence needed to determine whether a failure was isolated, systemic, repeated, or associated with a particular model, data source, tool, or workflow. Observability should therefore extend beyond infrastructure metrics to capture the behaviour and decisions of the AI system itself. For agentic architectures, this may include the sequence of model calls, retrieved information, tool invocations, policy decisions, execution outcomes, and termination conditions (Yao et al., 2023; Shinn et al., 2023).

The architecture should also support failure containment. An error in one agent or workflow should not automatically compromise unrelated processes. Isolation boundaries, scoped credentials, transaction limits, sandboxing, and separate execution contexts can reduce the propagation of failures. This is particularly important in multi-agent systems, where a failure or incorrect assumption by one agent may otherwise be transmitted to other components. Multi-agent architectures increase the number of interactions through which information and decisions can propagate, making explicit boundaries and coordination mechanisms important architectural considerations (Wooldridge, 2009; Wang et al., 2023).

Resilience must extend beyond individual components to external dependencies. AI systems may depend on model providers, retrieval services, data sources, APIs, cloud infrastructure, identity services, and other third-party components. Provider outages or changes can therefore become architectural failure modes. Dependency mapping, alternative execution paths, monitoring, and appropriate contingency planning are necessary where the consequences of dependency failure are significant. This follows directly from the broader observation that machine learning systems accumulate dependencies whose failure characteristics may only become visible during operation or change (Sculley et al., 2015).

The resulting architecture treats failure as part of the normal operating model. Instead of assuming that successful execution is the default and failure is an exceptional event, the system defines how it should behave when information is incomplete, models are uncertain, tools are unavailable, dependencies fail, policies are violated, or an agent stops making useful progress. This changes the architectural question from How do we make the AI execute successfully? to How should the system behave when execution becomes uncertain, invalid, or unsafe?

A useful execution boundary can therefore be understood as a sequence of controlled conditions:

maximum iterations → timeout or resource limit → evidence or confidence threshold → policy check → human escalation or safe termination

The precise sequence will vary by application, and these conditions need not always be strictly sequential. Some controls may operate continuously or be evaluated before each consequential action. The architectural principle nevertheless remains the same: every autonomous process should have explicit boundaries within which it is permitted to operate and defined conditions under which it must stop.

This principle also connects resilience with the earlier discussion of bounded autonomy. An autonomous system is not resilient merely because it can continue operating during disruption. In some circumstances, resilience requires the system to stop, reduce its functionality, or transfer control to a human. The ability to terminate safely is therefore itself an architectural capability. A system that cannot reliably stop when its assumptions become invalid may remain operational while becoming increasingly unsafe.

Security must likewise be treated as part of failure engineering rather than as a separate concern. Prompt injection, unsafe tool use, excessive agency, insecure external connections, and other application-level risks can cause an AI system to behave incorrectly even when underlying infrastructure remains available (OWASP, 2025). Security controls, therefore, contribute not only to confidentiality and integrity but also to the containment of erroneous or adversarial system behaviour.

The central principle is that resilience in AI systems means engineering for failure, not merely engineering for availability. A resilient system anticipates both technical failures and incorrect AI behaviour, detects problems where possible, contains their effects, provides controlled recovery paths, and escalates when automated execution can no longer be trusted to remain within acceptable boundaries.

In this model, failure is not an exception to the architecture. It is one of the conditions for which the architecture must be designed. The mature AI architecture therefore does not ask whether failure can be eliminated; it asks where failure can occur, how it will be detected, how far it can propagate, what the system should do next, and who or what is authorised to take control when automated execution reaches its limits.

20. Sovereignty and Dependency Management

Enterprise AI introduces dependencies across a broad technology and information supply chain. These dependencies may include cloud infrastructure providers, model providers, accelerator manufacturers, software frameworks, data providers, identity services, external APIs, observability platforms, and other third-party services. As AI systems become increasingly interconnected, the ability to understand and manage these dependencies becomes an important architectural concern. Machine learning systems can accumulate substantial dependencies across data, infrastructure, configuration, software components, and external services, creating forms of technical debt and operational coupling that may not be apparent from the model itself (Sculley et al., 2015). In agentic architectures, these dependencies can become even broader because systems may dynamically interact with tools, APIs, external information sources, and execution environments (Yao et al., 2023; Wang et al., 2023).

Sovereignty should therefore not be interpreted simply as the geographical location in which an AI workload is hosted. Physical or contractual hosting location is relevant, particularly where data residency, regulatory requirements, or jurisdictional considerations apply, but it represents only one dimension of control. A system may be hosted within a particular jurisdiction while remaining highly dependent on external models, proprietary software, specialised hardware, or foreign technology suppliers. Conversely, the use of external infrastructure does not necessarily imply an absence of meaningful organisational control if appropriate technical, contractual, governance, and operational mechanisms are in place. The relevant architectural question is therefore broader than physical location.

A more useful architectural conception is control over the AI processing chain. This encompasses the organisation's ability to understand, govern, secure, change, monitor, and, where necessary, replace the components on which its AI capabilities depend. This interpretation is consistent with risk-management approaches that emphasise understanding AI systems within their broader organisational and operational context rather than treating individual models as isolated artefacts (NIST, 2023; ISO, 2023).

The processing chain may include data sources, models, compute infrastructure, identity and access services, interfaces, software dependencies, logging and observability systems, processing locations, model updates, and external services. Each dependency can introduce its own operational, security, commercial, technical, or regulatory constraints. Sovereignty therefore requires visibility into these dependencies rather than assuming that control over a single layer provides control over the complete system. This is particularly important where an AI capability combines multiple externally provided services, since the resulting system may inherit risks and constraints from each component and from the interactions between them (Sculley et al., 2015; NIST, 2023).

This distinction is particularly important for model infrastructure. An organisation may operate a model within its own data centre and therefore retain direct control over the physical execution environment, while remaining dependent on proprietary accelerators, specialised software stacks, model components, or external maintenance arrangements. Physical ownership of infrastructure does not necessarily imply independence from the broader technology supply chain. The distinction reflects a general systems principle: ownership of one component does not automatically establish control over the dependencies required for the component to function.

The converse is also possible. A system deployed through a public cloud provider may provide contractual commitments, security controls, data-management mechanisms, audit capabilities, geographic configuration options, and operational assurances. Whether such a deployment provides an appropriate level of control depends on the specific architecture, contractual arrangements, technical controls, and organisational requirements rather than on the deployment model alone. From an AI risk-management perspective, the relevant question is therefore whether the overall control environment is appropriate to the system's intended use and risk profile (NIST, 2023; ISO, 2023).

Sovereignty should therefore be considered across several dimensions. Data sovereignty concerns control over information, including where it is stored and processed, who can access it, and under which legal and contractual conditions. Model sovereignty concerns dependence on external model providers, model updates, licensing arrangements, and the ability to change or substitute models. Infrastructure sovereignty concerns dependence on compute, networking, storage, and specialised hardware. Operational sovereignty concerns the ability to monitor, manage, secure, and recover the system. Architectural sovereignty concerns the ability to change individual components without losing control of the overall system.

These dimensions should not be interpreted as requiring complete independence. Rather, they provide a framework for identifying where control is retained, where it is delegated, and where dependency creates material exposure. This is particularly important because AI systems frequently cross organisational boundaries, making questions of responsibility, access, security, and control relevant across the entire lifecycle of the system (NIST, 2023; ISO, 2023).

The last dimension, architectural sovereignty, is particularly important because architectural coupling can create dependency even when individual components appear technically replaceable. If an application is tightly coupled to a particular model API, tool protocol, vector store, cloud service, or accelerator stack, changing that dependency may require substantial redesign. Explicit interfaces and abstraction boundaries can therefore contribute to sovereignty by making dependencies visible and replacement technically feasible. This connects directly to the interoperability principles established earlier in the paper. Explicit interfaces between AI applications, models, tools, and external context can reduce unnecessary coupling and improve the ability to modify system components over time (Anthropic, 2024).

Controlled interfaces, standardised protocols, portable data representations, and modular architecture can therefore reduce unnecessary dependency concentration. They do not eliminate dependencies, but they can increase the organisation's ability to substitute components when commercial, operational, regulatory, or security requirements change. Modularity is consequently not merely a software-engineering preference; it can become a mechanism for maintaining strategic flexibility. The principle is consistent with the broader concern that tightly coupled machine learning systems can accumulate dependencies whose costs become apparent only when components need to be changed or the system evolves (Sculley et al., 2015).

Dependency management should also consider concentration risk. An AI system may depend on multiple services that ultimately originate from the same provider or technology ecosystem. Apparent architectural diversity can therefore conceal common dependencies. For example, infrastructure, identity, data services, model hosting, monitoring, and application services may appear as separate architectural components while remaining dependent on a common underlying provider. Resilience analysis should identify such shared dependencies because a single provider outage, supply-chain disruption, licensing change, or technology discontinuation may affect several layers simultaneously. Risk-management frameworks emphasise the importance of understanding dependencies and potential impacts across the broader system rather than evaluating individual components independently (NIST, 2023).

Model updates represent another important dependency. External model providers may introduce new versions, modify behaviour, retire models, change pricing, or alter service characteristics. Such changes can affect application behaviour even when the application itself has not been modified. This creates a distinctive form of dependency because the behaviour of a downstream application may change as a consequence of an upstream model modification. Model versioning, evaluation, change management, and fallback planning should therefore form part of dependency governance. Model reporting practices such as model cards further demonstrate the importance of documenting model characteristics, intended uses, limitations, and changes that may affect downstream users (Mitchell et al., 2019).

The same principle applies to external data and APIs. Changes to an external information source, API contract, authentication mechanism, rate limit, or data policy can alter the behaviour of an AI workflow. This is particularly significant for retrieval-augmented and agentic systems, where external information and tools can form part of the system's effective operating environment (Lewis et al., 2020; Yao et al., 2023). Dependencies should consequently be documented, monitored, evaluated, and associated with appropriate contingency arrangements according to their criticality.

Security considerations further reinforce the need for dependency visibility. External services, plugins, APIs, model interfaces, and connected tools can expand the attack surface of an AI system and introduce risks that are not present within the model itself. OWASP's treatment of risks in large-language-model applications reflects this broader concern with application-level interactions, external components, and system-level security rather than model capability alone (OWASP, 2025). Dependency governance should therefore form part of the system's wider security and risk-management architecture.

Sovereignty also has an important relationship with exit capability. An organisation does not necessarily need to operate every component independently to maintain meaningful control. What matters is whether it understands its dependencies and retains credible options when a critical dependency becomes unacceptable, unavailable, or strategically unsuitable. Exit may involve migration to another provider, substitution of a model, relocation of processing, adoption of an alternative interface, or redesign of a particular workflow. The feasibility of these options depends substantially on the degree of architectural coupling created during initial system design.

This leads to a more practical architectural objective: managed dependency. The goal is not to eliminate all external dependencies, which would often be impractical and potentially inefficient. Instead, organisations should identify which dependencies are strategically or operationally significant, understand their associated risks, establish appropriate controls, and maintain sufficient architectural flexibility to respond when circumstances change. This approach is consistent with the broader systems-engineering principle that dependencies should be recognised and managed as part of the system rather than treated as incidental implementation details (Sculley et al., 2015).

The appropriate level of dependency management will vary according to the criticality of the AI system. A low-impact internal assistant may reasonably depend heavily on external services, whereas a system supporting critical infrastructure, regulated decision-making, sensitive information, or consequential transactions may require stronger control, greater transparency, additional fallback capabilities, or more stringent contractual and technical arrangements. NIST's risk-based approach supports this principle by linking the management of AI risks to the context, potential impacts, and characteristics of the system rather than applying identical controls to every AI application (NIST, 2023).

The central principle is therefore that sovereignty is a property of control across the AI processing chain, not simply a property of where infrastructure is located. Organisations should understand their dependencies across data, models, infrastructure, software, identity, interfaces, and external services, and should design those dependencies so that they remain visible, governed, replaceable where necessary, and proportionate to the importance of the system.

Sovereignty in AI architecture is consequently best understood not as technological isolation, but as the ability to make informed choices about dependencies and retain meaningful control over the systems that depend upon them. The architectural objective is not to eliminate external providers, but to ensure that external dependencies remain deliberate, understood, governed, and proportionate to the organisation's risk and strategic requirements. In this sense, sovereignty becomes an architectural property created through visibility, modularity, interoperability, governance, resilience, and credible exit capability.

21. AI Architecture as Enterprise Architecture

AI architecture cannot remain isolated from enterprise architecture because AI systems increasingly participate directly in business processes, organisational decisions, and operational activities. An enterprise AI system does not operate solely within the boundary of a model or application. It interacts with business processes, applications, data platforms, identity infrastructure, cybersecurity controls, risk management, compliance functions, financial systems, human resources, and organisational decision-making. This reflects the broader systems perspective required for AI deployment, in which model capability is embedded within technical, organisational, and governance environments rather than treated as an isolated technological component (Bommasani et al., 2021; Sculley et al., 2015; NIST, 2023).

This makes enterprise AI fundamentally socio-technical. Technical components determine how AI capabilities are implemented, while organisational structures, processes, responsibilities, policies, and human decisions determine how those capabilities are used and controlled. Consequently, the behaviour and outcomes of an AI system are shaped not only by the underlying model but also by the environment in which that model operates. This is consistent with the broader observation that machine learning systems accumulate dependencies across data, infrastructure, configuration, human processes, and operational environments, making system-level design and management essential (Sculley et al., 2015). Governance frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 similarly treat AI risk and accountability as organisational concerns rather than solely as properties of the model itself (NIST, 2023; ISO, 2023).

The implications extend beyond technical integration. When AI is introduced into an existing enterprise process, it can alter the allocation of work, decision rights, control points, information flows, and accountability. A process that previously depended on human analysis may become partly automated. A workflow may acquire new approval requirements because AI-generated outputs require validation. New dependencies may emerge between business functions, model providers, retrieval services, and external APIs. The introduction of AI can therefore change the structure of the business capability in which it is embedded. This is particularly relevant for agentic systems, where models can participate in iterative reasoning, tool use, and action selection rather than merely generating a static response (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

This is why workflow integration is central to AI value. A highly capable model that operates outside the organisation's actual processes may provide impressive demonstrations without producing substantial operational benefit. Conversely, a comparatively modest model can create meaningful value when it is integrated into a well-designed workflow in which it reduces repetitive work, improves information retrieval, supports decision-making, or automates appropriately bounded activities. Retrieval-augmented generation illustrates this principle: the value of the model is extended by connecting it to external organisational knowledge rather than relying solely on information encoded within model parameters (Lewis et al., 2020). More generally, the effectiveness of an AI capability depends on the interaction between the model and the surrounding system rather than on model performance in isolation (Bommasani et al., 2021; Sculley et al., 2015).

The relevant question is therefore not simply whether a model performs well in isolation. It is whether the AI-enabled process performs better against the objectives for which it exists. Those objectives may include reducing processing time, improving consistency, increasing information accessibility, supporting better decisions, reducing operational effort, improving control effectiveness, or enabling services that were previously impractical. This requires evaluation to extend beyond model-level benchmarks towards the broader context in which the system is deployed. Holistic evaluation research highlights the importance of assessing language models across multiple dimensions rather than reducing their performance to a single benchmark measure (Liang et al., 2022). In an enterprise context, those dimensions must ultimately be connected to business outcomes, operational requirements, and risk objectives.

This shifts the appropriate unit of architectural analysis from the model to the AI-enabled business capability. The model remains an important component, but it is one element within a broader architecture that includes data, context, memory, orchestration, tools, applications, infrastructure, controls, people, and business processes. This systems perspective is consistent with the development of retrieval-augmented systems and agent architectures, in which models operate as components within larger computational structures involving external information, tools, feedback, and environmental interaction (Lewis et al., 2020; Yao et al., 2023; Shinn et al., 2023).

An AI-enabled business capability can therefore be understood in terms of the outcome it is intended to provide and the organisational system through which that outcome is achieved. The architecture must establish which activities are performed by AI, which remain with people, which systems provide authoritative information, which actions can be automated, where approvals are required, and how performance and risk are monitored. This separation between capability and authority is particularly important because a system may possess the technical ability to generate or execute an action without being organisationally authorised to do so. Enterprise governance therefore needs to establish the boundaries within which AI capabilities can operate (NIST, 2023; ISO, 2023).

This perspective also helps distinguish AI adoption from AI transformation. Adding a model to an existing application may improve a particular interaction without fundamentally changing the underlying process. Deeper transformation occurs when AI capabilities are integrated into the workflow itself and the process is redesigned around a new distribution of information processing, decision support, automation, and human responsibility. Agentic AI makes this distinction particularly visible because systems can move beyond generating outputs towards planning, tool use, iterative reasoning, and interaction with external environments (Yao et al., 2023; Wang et al., 2023). However, the architectural significance of such capabilities lies not simply in increasing autonomy, but in determining how that autonomy is bounded and integrated into an accountable organisational process.

Such transformation should not be assumed to be beneficial merely because more activities become automated. Enterprise architecture must consider the resulting effects on control, resilience, security, accountability, workforce responsibilities, customer experience, and operational risk. AI-enabled processes should therefore be evaluated as complete socio-technical systems. This follows from the broader observation that machine learning introduces system-level dependencies and technical debt that may not be visible when attention is restricted to model performance (Sculley et al., 2015). Similarly, AI risk management requires consideration of governance, organisational context, system impacts, and lifecycle processes rather than only technical model characteristics (NIST, 2023; ISO, 2023).

The enterprise context also determines which architectural constraints matter most. A customer-facing assistant may prioritise availability, response quality, privacy, and brand considerations. A financial-control workflow may place greater emphasis on auditability, segregation of duties, authorisation, and evidence. A cybersecurity capability may require rapid response while maintaining strict controls over production systems. The architecture should therefore be driven by the requirements and risk profile of the business capability rather than by the capabilities of a particular model. This aligns with risk-based approaches to AI governance, in which controls are expected to reflect the context, potential impacts, and risks associated with a particular AI system (NIST, 2023; ISO, 2023).

Integration with existing enterprise architecture is also important for governance. AI systems should participate in established identity, security, data-management, risk, compliance, change-management, and operational processes wherever appropriate. Creating separate AI governance mechanisms that are disconnected from enterprise control structures can increase duplication and make accountability less clear. ISO/IEC 42001 explicitly frames AI management as an organisational management-system concern, while the NIST AI RMF emphasises the integration of risk management into organisational processes and decision-making (ISO, 2023; NIST, 2023). AI governance should therefore complement established enterprise governance rather than create an entirely parallel control environment.

The same principle applies to architecture management. AI components should be represented within the organisation's broader understanding of applications, data flows, technology dependencies, business capabilities, and control environments. Model providers, retrieval services, agent workflows, tools, and external APIs may all become material enterprise dependencies and should be considered accordingly. The growing use of explicit interfaces between models, applications, tools, and external sources of context further reinforces the importance of understanding these dependencies as architectural components rather than implementation details (Anthropic, 2024). This is also consistent with the broader systems-engineering concern that machine learning applications accumulate dependencies across data, infrastructure, configuration, and external services (Sculley et al., 2015).

This enterprise perspective also clarifies why AI architecture cannot be evaluated solely through technical metrics. Model accuracy, latency, token consumption, and benchmark performance remain useful, but they must ultimately be connected to business outcomes and control objectives. A model may perform strongly according to a benchmark while failing to improve the target business capability, or it may produce useful outputs while introducing unacceptable operational, security, or governance dependencies. Holistic evaluation therefore provides a more appropriate conceptual basis than reliance on isolated performance measures (Liang et al., 2022). Model cards and datasheets similarly illustrate the importance of documenting characteristics, limitations, intended uses, and relevant contextual information around AI systems and their underlying data (Mitchell et al., 2019; Gebru et al., 2021).

The central principle is therefore that AI architecture is an extension of enterprise architecture. The model is not the primary unit of value or analysis. The relevant architectural unit is the AI-enabled business capability: the combination of people, processes, information, technology, AI components, controls, and organisational responsibilities through which a business outcome is produced. The model provides a source of computational intelligence, but the enterprise architecture determines how that intelligence is contextualised, constrained, integrated, executed, monitored, and governed.

AI architecture becomes strategically meaningful when it moves beyond deploying models and begins to redesign how the organisation processes information, makes decisions, performs work, and exercises control. At that point, AI is no longer an isolated technology capability; it becomes part of the architecture of the enterprise itself. The architectural objective is therefore not maximum model capability or maximum automation, but the creation of trustworthy AI-enabled business capabilities in which intelligence, information, authority, execution, and accountability are deliberately integrated within the wider enterprise system.

22. Seven Fundamental Principles of AI Architecture

The preceding analysis establishes that contemporary AI architecture is not primarily concerned with selecting and deploying increasingly capable models. Rather, it is concerned with designing systems in which model capability is combined with data, context, memory, tools, orchestration, infrastructure, security, governance, evaluation, human oversight and enterprise processes. Research on foundation models and machine-learning systems demonstrates that model capabilities are embedded within wider technical and organisational dependencies, while research on retrieval-augmented generation, agentic systems and AI governance illustrates the increasing importance of the architecture surrounding the model (Bommasani et al., 2021; Sculley et al., 2015; Lewis et al., 2020; Yao et al., 2023; NIST, 2023). The following seven principles provide a concise synthesis of this architectural perspective.

22.1 Principle 1: Separate Intelligence from Control

The model should provide intelligence, while the surrounding architecture determines what that intelligence is permitted to access and do. A model may generate a recommendation, formulate a plan, select a tool or propose an action without possessing authority to execute that action. This distinction becomes particularly important as language models are integrated with tools and external environments, allowing generated outputs to influence real-world actions (Yao et al., 2023; Shinn et al., 2023).

This separation establishes a boundary between capability and authority. Identity, permissions, policy enforcement, validation, approval mechanisms and execution controls should determine whether a proposed action can occur. Such separation is consistent with established principles of access control and risk management, in which permissions should be explicitly defined and proportionate to the required activity rather than inferred from the capability of the system performing it (NIST, 2023; ISO, 2023; OWASP, 2025).

The principle therefore supports least privilege, separation of duties and bounded autonomy while reducing the risk that increased model capability automatically translates into increased operational authority. This is particularly relevant to agentic systems, where the ability to reason about and select actions can otherwise become closely coupled with the ability to execute them (Yao et al., 2023; Wooldridge, 2009).

22.2 Principle 2: Externalise Dynamic Knowledge

Models provide powerful representations of general knowledge, but their parameters should not be treated as a guaranteed source of current, authoritative, organisation-specific or sensitive information. Foundation-model research identifies important limitations associated with the relationship between pretrained model knowledge and the data and applications in which models are subsequently deployed (Bommasani et al., 2021). Knowledge that changes over time or requires explicit provenance should therefore generally remain within controlled external information systems.

Retrieval, enterprise databases, document repositories, knowledge graphs, APIs and other information services allow systems to provide models with current and appropriately governed context. Retrieval-augmented generation provides a concrete architectural pattern for this separation by retrieving relevant external information and incorporating it into the model's input at inference time (Lewis et al., 2020). Documentation approaches for datasets and models further demonstrate the importance of making information about sources, limitations and intended uses explicit rather than leaving such characteristics implicit within a model or dataset (Gebru et al., 2021; Mitchell et al., 2019).

This separation also supports data access controls, lineage, versioning, updating and source-level governance. Emerging interoperability mechanisms for connecting models with external sources of context and capability provide a further illustration of this architectural direction (Anthropic, 2024).

The principle does not imply that all knowledge must be externalised. Rather, it establishes an architectural distinction between model capability and authoritative enterprise information, allowing each to be managed according to its different lifecycle, provenance and control requirements (Bommasani et al., 2021; Lewis et al., 2020).

22.3 Principle 3: Treat Autonomy as a Bounded Capability

Agentic systems should operate within explicit objectives, permissions, resource limits, policies and termination conditions. Autonomy should therefore be understood as delegated authority within a defined operating envelope rather than as unrestricted independence. Research on language-model agents demonstrates that models can be combined with iterative reasoning, external tools, environmental interaction and feedback to perform multi-step tasks (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

An agent may be permitted to retrieve information, invoke selected tools or execute low-risk actions automatically while requiring approval for higher-impact activities. Time limits, iteration limits, transaction thresholds, escalation mechanisms and policy checks can therefore be used to constrain autonomous execution. Such mechanisms are consistent with broader approaches to agent control and reinforcement learning, where an agent's actions are defined within objectives and an environment rather than being treated as unrestricted behaviour (Sutton and Barto, 2018; Wooldridge, 2009).

The objective is not maximum autonomy. It is appropriate autonomy relative to the task, risk and controllability of the system. This follows the broader risk-management principle that the design and operation of an AI system should reflect its intended context and potential consequences (NIST, 2023).

22.4 Principle 4: Design for Modularity and Interoperability

Models, retrieval systems, memory, orchestration components, tools, applications and external services should communicate through explicit interfaces wherever practical. Clear boundaries make dependencies visible and allow individual components to evolve without requiring unnecessary redesign of the complete system. The importance of managing such dependencies is illustrated by research on machine-learning technical debt, which identifies the substantial complexity created by interactions between models, data, configuration, infrastructure and other system components (Sculley et al., 2015).

Modularity also supports evaluation, security, resilience and dependency management. A retrieval component can be evaluated independently of the model; a tool can enforce its own authorisation boundary; a model can be replaced without redesigning every business workflow; and alternative services can be introduced where dependency concentration creates unacceptable risk. The separation of model capabilities from external knowledge and tools in retrieval-augmented and agentic architectures provides practical examples of such componentisation (Lewis et al., 2020; Yao et al., 2023).

Interoperability should therefore be treated as an architectural property that supports both technical flexibility and organisational control. Explicit interfaces can reduce unnecessary coupling and make it easier to change individual components while preserving the broader system architecture. Emerging protocols for connecting AI systems with external context and capabilities further illustrate the importance of standardised or explicit interfaces as AI architectures develop (Anthropic, 2024).

22.5 Principle 5: Build Security and Governance into the Architecture

Security, privacy, accountability, auditability and policy enforcement should be implemented through architectural mechanisms rather than relying primarily on documentation or post-deployment controls. AI risk-management frameworks emphasise the integration of risk identification, assessment and management throughout the AI lifecycle, while ISO/IEC 42001 provides a management-system approach for establishing organisational processes and controls for AI governance (NIST, 2023; ISO, 2023).

Identity and access management, least privilege, input and output validation, policy enforcement, audit trails, human approval mechanisms, data controls, incident management and change management should therefore be incorporated into the system lifecycle. Security guidance for large language model applications similarly identifies vulnerabilities that arise from the interaction between models, applications, inputs, outputs and external capabilities, reinforcing the need for controls at the application and system level rather than solely within the model (OWASP, 2025).

Governance requirements should be translated into enforceable controls wherever appropriate, with sufficient evidence to demonstrate that those controls operated as intended. Documentation remains important, but documentation alone cannot guarantee that a system will behave according to organisational policy. This supports the broader concept of governance by design, in which organisational requirements are translated into technical and operational mechanisms that constrain behaviour, generate evidence and support assurance and improvement (NIST, 2023; ISO, 2023).

22.6 Principle 6: Make Evaluation and Observability Continuous

AI systems require continuous measurement because their behaviour depends on models, data, context, tools, workflows and changing operating environments. Evaluation should therefore extend beyond initial model benchmarking to include components, complete system behaviour, execution trajectories and relevant business outcomes. Research on holistic evaluation demonstrates the limitations of relying on individual benchmarks to characterise the behaviour and risks of language models (Liang et al., 2022).

Observability provides the evidence needed to perform this evaluation and investigate unexpected behaviour. Depending on the system's risk and governance requirements, relevant evidence may include model versions, prompts, retrieved information, tool interactions, policy decisions, errors, human interventions and outcomes. Agentic systems make this particularly important because a single outcome may depend on a sequence of model decisions, retrieval operations, tool calls and environmental interactions (Yao et al., 2023; Shinn et al., 2023).

Continuous evaluation and observability allow organisations to identify degradation, understand failures, assess changes and maintain confidence in system behaviour over time. This is consistent with lifecycle-based approaches to AI risk management, which treat monitoring, evaluation and risk management as continuing organisational activities rather than one-time deployment decisions (NIST, 2023; ISO, 2023). Model cards and datasheets provide complementary mechanisms for documenting the characteristics and limitations of models and datasets, thereby supporting more systematic evaluation and accountability (Mitchell et al., 2019; Gebru et al., 2021).

22.7 Principle 7: Keep Consequential Authority Accountable

AI systems may increasingly perform tasks with limited human intervention, but consequential authority should remain associated with identifiable and accountable organisational structures. Where automation is appropriate, the authority delegated to the system should be explicitly defined, bounded, monitored and proportionate to the consequences of failure. This principle follows the broader governance requirement that organisations remain responsible for the systems they deploy and the risks associated with their operation (NIST, 2023; ISO, 2023).

Human oversight should therefore be designed according to the nature of the decision rather than applied uniformly. Informational tasks may require limited intervention, while consequential or difficult-to-reverse actions may require explicit human approval, dual control or other forms of oversight. The appropriate level of intervention should reflect the potential impact and reversibility of the action, as well as the reliability and controllability of the system performing it (NIST, 2023).

The principle is not that humans must approve every AI action. Rather, it is that decision authority must remain deliberate, attributable and governed, particularly where AI actions can materially affect people, assets, regulated activities or critical operations. This distinction is important because increased autonomy can extend system capability without necessarily transferring ultimate organisational accountability to the system itself (Yao et al., 2023; Wooldridge, 2009; NIST, 2023).

22.8 Synthesis

Taken together, these principles define a coherent architectural philosophy. Intelligence is provided by models; knowledge is supplied through governed information systems; autonomy is bounded by explicit authority; components interact through controlled interfaces; security and governance are embedded into the architecture; behaviour is continuously evaluated and observed; and consequential authority remains accountable. These principles combine insights from foundation-model research, retrieval-augmented generation, agentic architectures, machine-learning systems engineering and AI governance into a single enterprise-oriented architectural perspective (Bommasani et al., 2021; Lewis et al., 2020; Sculley et al., 2015; Yao et al., 2023; NIST, 2023).

The resulting architecture is therefore neither purely model-centric nor purely automation-centric. It is a governed intelligent system in which probabilistic AI capabilities operate within explicit technical, organisational and human boundaries. The model remains an important source of computational intelligence, but the surrounding architecture determines how that intelligence is supplied with information, connected to tools, constrained by policy, evaluated, observed and incorporated into organisational processes (Bommasani et al., 2021; Sculley et al., 2015).

The fundamental architectural objective is consequently not to maximise model capability in isolation. It is to create systems in which capability can be used, constrained, observed, evaluated, governed and held accountable. This principle follows from the combined evidence that model capability alone does not determine system behaviour or organisational value; those outcomes emerge from the interaction of models with data, infrastructure, tools, workflows, human actors and governance mechanisms (Bommasani et al., 2021; Sculley et al., 2015; NIST, 2023).

On this basis, AI can be understood not as an isolated technology component but as an integrated enterprise capability. The architectural task is to connect intelligence to authoritative information, bounded authority, controlled execution and accountable organisational processes while maintaining the mechanisms required to evaluate and govern the resulting system throughout its lifecycle (ISO, 2023; NIST, 2023).

23. A Reference Architecture for Enterprise AI

The seven principles established in the preceding chapter can be brought together into a conceptual reference architecture for enterprise AI. The purpose of such an architecture is not to prescribe a single technology stack or implementation pattern. Rather, it provides a structured way of understanding the principal capabilities that must interact within a governed enterprise AI system and the boundaries that should exist between them. This systems perspective is consistent with research showing that the performance, reliability and risk of AI applications depend on components and dependencies surrounding the underlying model rather than on model capability alone (Bommasani et al., 2021; Sculley et al., 2015).

At the highest level, enterprise AI connects human and business actors with AI-enabled capabilities through an interaction and experience layer. This layer may include conversational interfaces, applications, workflow interfaces, dashboards, decision-support environments, or other mechanisms through which people initiate tasks, provide information, review results, approve actions and receive outcomes. The importance of human interaction and feedback is reflected in research on instruction-following and human-feedback-based alignment, which demonstrates that useful AI behaviour depends not only on model capability but also on how systems are directed and evaluated by people (Ouyang et al., 2022; Bai et al., 2022).

The interaction layer should not itself determine the full authority of the AI system. Requests originating from users should be subject to appropriate identity, policy and authorisation controls before they result in access to information or execution of actions. The identity of the user, the role they hold, the purpose of the interaction, and the sensitivity and potential consequences of the requested activity may all affect what the system is permitted to do. This principle follows broader AI risk-management and information-security approaches in which access, accountability and risk controls must be incorporated into the operation of AI systems rather than treated solely as user-interface concerns (NIST, 2023; ISO, 2023; OWASP, 2025).

Beneath the interaction layer sits a policy and guardrail layer. This provides controls that constrain system behaviour before, during and after AI processing. Depending on the application, these controls may include access policies, content and data controls, input validation, output validation, tool allow-lists, transaction limits, approval requirements, safety policies, escalation rules and other forms of runtime control. Contemporary AI risk-management frameworks emphasise the need to identify and manage risks throughout the AI lifecycle, while application-security guidance for large language models identifies risks arising from the interaction between models, inputs, outputs and surrounding application components (NIST, 2023; OWASP, 2025).

The policy and guardrail layer is particularly important because enterprise AI should not equate a valid user request or a model-generated action with authorised execution. Policies establish the conditions under which an action may proceed, while guardrails provide mechanisms for enforcing those conditions. This distinction is consistent with the separation between model behaviour and organisational governance found in AI risk-management and management-system approaches (NIST, 2023; ISO, 2023). It also follows the broader architectural principle that model outputs should be treated as components of a system rather than as inherently authoritative decisions (Bommasani et al., 2021).

The next major capability is the agent and orchestration layer. This layer coordinates the execution of tasks and determines how the system moves between models, context, memory, tools, applications and human intervention. It may manage planning, task decomposition, workflow state, model selection, tool selection, execution sequencing, error handling, escalation and termination. Agentic research demonstrates that language models can be combined with external actions, iterative reasoning and feedback mechanisms to create systems that perform multi-step tasks rather than simply generating a single response (Yao et al., 2023; Shinn et al., 2023; Wang et al., 2023).

Orchestration may be deterministic, agentic or hybrid. A deterministic workflow can define fixed execution paths where predictability and repeatability are important, while an agentic component can make bounded decisions within a defined portion of that workflow. The reference architecture therefore does not assume that every enterprise AI process should operate as a fully autonomous agent. This is consistent with established work on multi-agent and cognitive architectures, in which different forms of control and interaction can be combined according to the requirements of the system (Laird, 2012; Wooldridge, 2009). It also reflects the emerging distinction between systems that use language models for reasoning and systems that additionally allow models to determine or initiate actions (Yao et al., 2023; Shinn et al., 2023).

The orchestration layer interacts with two closely related capabilities: the model and reasoning layer and the memory and context layer.

The model and reasoning layer provides the system's principal capabilities for language understanding, reasoning, generation, classification, planning and other forms of inference. Transformer architectures established a major foundation for modern language models, while subsequent work on scaling and foundation models demonstrated the increasing breadth of capabilities that can be obtained from large pretrained models (Vaswani et al., 2017; Kaplan et al., 2020; Brown et al., 2020; Bommasani et al., 2021). However, the architecture should treat models as components within a larger system rather than as the complete system. This is particularly important because models have defined capabilities, limitations and dependencies that need to be considered in the context of their intended applications (Bommasani et al., 2021; Mitchell et al., 2019).

The memory and context layer provides the information and state required for the model to operate effectively within a particular task or workflow. This can include retrieved enterprise information, working state, previous interactions, relevant historical information, task context, tool outputs and other information made available during execution. Retrieval-augmented generation provides a clear architectural example of this principle by separating external knowledge retrieval from the model's parametric knowledge and supplying retrieved information as context during generation (Lewis et al., 2020).

The relationship between reasoning and context is therefore bidirectional. The model may determine what additional information is required, while retrieval and memory mechanisms determine what information is available to support subsequent reasoning. In agentic systems, this relationship can become iterative as models reason about available information, select tools or retrieval mechanisms, receive additional information and continue processing (Yao et al., 2023; Shinn et al., 2023). This relationship should remain governed by information-access policies, provenance requirements and appropriate data-management controls. The documentation of datasets and model characteristics provides established mechanisms for making such dependencies and limitations more explicit (Gebru et al., 2021; Mitchell et al., 2019).

Beneath these capabilities is the tool, API and integration layer. This layer provides controlled interfaces through which AI systems interact with enterprise applications and external services. Tools may expose capabilities such as searching a database, retrieving a document, creating a service ticket, querying a financial system, executing software or initiating a business transaction. Recent agentic architectures explicitly demonstrate the use of external tools and environments as extensions of language-model capabilities (Yao et al., 2023; Wang et al., 2023).

The tool layer establishes an important boundary between probabilistic reasoning and system execution. A model can propose an action, but the tool interface should determine whether the request is valid, authorised, correctly formed and within the permitted operating scope. Consequential actions may additionally require policy checks, approval, transaction limits or other forms of control before execution. This separation is consistent with the broader principle that AI-generated outputs should not automatically acquire operational authority and with security frameworks that emphasise controlled access and action boundaries for AI applications (NIST, 2023; OWASP, 2025).

Below the integration layer are the organisation's enterprise data and knowledge systems. These include structured databases, document repositories, analytical platforms, knowledge graphs, event streams, transactional systems, search infrastructure and other authoritative information sources. Foundation-model research highlights the distinction between knowledge and capabilities embedded within models and the broader systems and data on which AI applications depend (Bommasani et al., 2021).

The reference architecture therefore treats enterprise information systems as distinct from the model. Enterprise knowledge should not automatically be assumed to reside reliably within model parameters, particularly where information is dynamic, sensitive, organisation-specific or subject to formal provenance and governance requirements. Retrieval-augmented generation provides an established architectural mechanism for connecting models to external knowledge at inference time (Lewis et al., 2020). More generally, explicit interfaces between models and external sources of context and capability support the separation of model intelligence from enterprise information and execution systems (Anthropic, 2024).

At the foundation is the infrastructure and runtime layer. This provides the computational and operational environment in which the AI system executes. It may include model-serving infrastructure, accelerators, storage, networking, containers, orchestration platforms, security infrastructure, runtime monitoring and resilience mechanisms. The importance of the surrounding infrastructure is reflected in research on machine-learning technical debt, which demonstrates that production machine-learning systems contain substantial dependencies outside the model itself, including infrastructure, configuration, data and interfaces (Sculley et al., 2015).

Infrastructure is therefore not merely a hosting concern. Runtime architecture influences latency, scalability, availability, cost, security, resilience and the ability to operate AI workloads reliably at enterprise scale. Model scaling research demonstrates the relationship between model capability, computation and resource requirements (Kaplan et al., 2020), while systems research highlights the broader engineering dependencies created when machine-learning components are incorporated into production environments (Sculley et al., 2015). Agentic systems can be particularly sensitive to these characteristics because a single business task may involve multiple model calls, retrieval operations, tool invocations and state transitions (Yao et al., 2023; Shinn et al., 2023).

Across all of these layers sits a set of cross-cutting enterprise controls encompassing security, governance, evaluation, observability, identity and auditability. These should not be interpreted as a separate layer through which every interaction simply passes. Rather, they are capabilities that must operate across the architecture.

Security governs identities, access, information flows, interfaces, execution privileges and external effects. Governance establishes accountability, policies, responsibilities, lifecycle requirements and change controls. Evaluation determines whether components and complete systems perform according to their intended objectives. Observability provides evidence of system behaviour, including model interactions, retrieved information, tool calls, policy decisions, errors and human interventions. Identity establishes relationships between people, services, agents and permissions. Auditability provides the evidence necessary to reconstruct relevant decisions and actions. Together, these functions correspond to the broader lifecycle and risk-management principles established in AI governance frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 (NIST, 2023; ISO, 2023).

Evaluation should operate at both component and system levels. Model cards and datasheets provide mechanisms for documenting the characteristics, intended uses and limitations of models and datasets (Mitchell et al., 2019; Gebru et al., 2021), while holistic evaluation research demonstrates the limitations of relying on a single benchmark to characterise model behaviour (Liang et al., 2022). For AI systems incorporating retrieval, tools and agentic execution, evaluation should consequently extend to retrieval quality, action selection, tool use, task completion, robustness, safety and relevant system-level outcomes (Lewis et al., 2020; Yao et al., 2023; Shinn et al., 2023).

The architecture deliberately separates reasoning from authority.

A model may reason that an action would achieve a particular objective, but that reasoning does not by itself establish that the action is permissible. The orchestration layer determines how the task should proceed within its defined workflow. Policy and guardrail mechanisms determine whether proposed activities satisfy applicable constraints. The tool and integration layer provides the controlled mechanism through which an authorised action can be executed. Observability records what occurred, while governance establishes responsibility for the system and its outcomes. This separation reflects the broader principle that AI capability, organisational authority and operational execution should not be treated as interchangeable concepts (NIST, 2023; ISO, 2023; OWASP, 2025).

This can be expressed as a conceptual chain of controlled responsibility:

reasoning proposes; policy constrains; orchestration coordinates; tools execute; observability records; governance assigns accountability.

The chain is conceptual rather than necessarily implemented as six physically separate components. In some systems, several functions may be implemented within the same platform or service. What matters is that their responsibilities remain sufficiently distinct to prevent model capability from being confused with operational authority. This is consistent with the systems perspective in which AI models form components within broader technical and organisational systems rather than functioning as autonomous sources of authority (Bommasani et al., 2021; Sculley et al., 2015).

The reference architecture should therefore be understood as a conceptual boundary model rather than a mandatory implementation blueprint. Different enterprise systems will combine or distribute these capabilities according to their requirements, risk profiles, existing architecture and technology choices. A low-risk internal assistant may require relatively simple orchestration and tool access, whereas a system supporting consequential financial, operational, security or regulated activities may require substantially stronger controls, validation, monitoring and evidence. The appropriate degree of control should consequently be determined by the system's intended use and risk profile rather than by a universal architectural template (NIST, 2023; ISO, 2023).

The central architectural objective is to ensure that intelligence operates within a controlled enterprise environment. Models provide computational intelligence; enterprise systems provide authoritative information and operational capabilities; orchestration coordinates activity; policies constrain behaviour; infrastructure provides reliable execution; and cross-cutting controls provide security, evidence, evaluation and accountability. This composition reflects the broader evolution from standalone machine-learning models towards complex AI systems whose behaviour depends on interactions among models, data, infrastructure, applications and organisational controls (Bommasani et al., 2021; Sculley et al., 2015).

This architecture therefore expresses the fundamental idea established throughout this paper:

Enterprise AI is not a model connected to a collection of tools. It is a governed system in which intelligence, information, authority, execution and accountability are deliberately separated and then integrated through controlled architectural boundaries.

The objective of the architecture is consequently not maximum autonomy or maximum model capability in isolation. It is the controlled conversion of computational intelligence into an operational enterprise capability: one that can access appropriate information, reason within defined boundaries, interact with authorised systems, operate within organisational processes, generate evidence of its behaviour and remain subject to appropriate human and organisational accountability (NIST, 2023; ISO, 2023; Bommasani et al., 2021).

24. Discussion

The evolution of AI architecture represents a transition from models that generate outputs towards systems that participate in organisational processes. The development of transformer architectures and increasingly capable foundation models has substantially expanded the capabilities available to AI applications (Vaswani et al., 2017; Brown et al., 2020; Bommasani et al., 2021). However, the practical consequences of these capabilities depend increasingly on the architecture surrounding the model, including the data, context, retrieval mechanisms, tools, workflows, evaluation processes and governance arrangements through which model capabilities are operationalised (Bommasani et al., 2021; Sculley et al., 2015).

This transition changes the architectural object of concern. In a traditional machine-learning application, the principal engineering challenge may be the performance of a model against a defined task. Contemporary AI systems introduce a broader set of dependencies because system behaviour is influenced not only by model parameters, but also by the information supplied to the model, the context made available at inference time, the tools it can access, the processes through which it interacts with those tools, and the controls governing its operation (Bommasani et al., 2021; Lewis et al., 2020; Yao et al., 2023; Shinn et al., 2023). This is consistent with earlier work on cognitive and multi-agent architectures, in which intelligent behaviour emerges from the interaction of multiple components rather than from an isolated computational model (Laird, 2012; Wooldridge, 2009).

AI system performance is therefore multidimensional. Model capability remains important, but it is only one contributor to system-level value. The preceding analysis suggests the following conceptual formulation:

AI System Value ≈ Model Capability × Data Quality × Context Quality × Workflow Integration × Control × Adoption

This formulation is not proposed as a quantitative measurement model. Rather, it expresses an architectural dependency: weakness in any critical dimension can materially reduce the value of the complete system. The need to consider such dimensions is reflected in research on foundation models, machine-learning technical debt, retrieval-augmented generation, and holistic AI evaluation (Bommasani et al., 2021; Sculley et al., 2015; Lewis et al., 2020; Liang et al., 2022).

A highly capable model operating on poor-quality or poorly governed enterprise data can produce unreliable or inappropriate results. Data quality and provenance are therefore architectural concerns rather than secondary implementation details. The importance of explicitly documenting dataset characteristics and limitations is illustrated by the datasheet approach proposed by Gebru et al. (2021). Similarly, model documentation mechanisms such as model cards recognise that understanding the characteristics, limitations and intended uses of models is necessary for responsible deployment (Mitchell et al., 2019). Strong retrieval mechanisms cannot fully compensate for authoritative sources that are incomplete, outdated, inaccessible or poorly governed; retrieval-augmented generation improves access to external knowledge but remains dependent on the quality and relevance of the retrieved information (Lewis et al., 2020).

The same principle applies to control. Increasing autonomy can expand the range of tasks an AI system can perform, particularly where models are combined with reasoning, planning and external actions (Yao et al., 2023; Shinn et al., 2023). However, greater autonomy can also increase the potential consequences of failure. Agentic systems therefore require explicit boundaries around the actions they are permitted to perform, together with mechanisms for validation, monitoring, feedback and escalation (Wooldridge, 2009; Shinn et al., 2023). Reinforcement-learning research similarly demonstrates that systems capable of selecting actions require mechanisms for defining objectives, evaluating behaviour and managing the consequences of those actions (Sutton and Barto, 2018).

Security and governance consequently become architectural properties rather than controls added after implementation. AI risk management requires organisations to identify, assess and manage risks across the AI lifecycle (NIST, 2023), while ISO/IEC 42001 establishes a management-system framework for organisations governing AI-related activities (ISO, 2023). For generative and large-language-model applications specifically, security risks include application-level vulnerabilities associated with model inputs, outputs, tools and surrounding application components (OWASP, 2025). The architecture must therefore make authority, permissions, data access and system interactions sufficiently explicit to enable effective control.

Adoption represents another important dimension of system value. A technically capable system may generate limited organisational benefit if users cannot integrate it into established ways of working or if its outputs cannot be appropriately evaluated and trusted. Human feedback has been incorporated into the development of language models precisely because model behaviour and usefulness cannot be defined solely through raw predictive capability (Ouyang et al., 2022). Constitutional approaches similarly demonstrate the importance of explicit behavioural principles and feedback mechanisms in shaping model behaviour (Bai et al., 2022). At the organisational level, this implies that enterprise AI requires integration not only with technical workflows but also with human responsibilities, decision processes and accountability structures (NIST, 2023; ISO, 2023).

These dependencies explain why contemporary AI architecture increasingly resembles enterprise systems engineering. The central engineering problem is no longer simply to maximise the intelligence of an individual computational component. It is to compose multiple capabilities into a system that performs a defined organisational function while remaining reliable, secure, controllable, observable and economically sustainable. The concept of machine-learning technical debt is particularly relevant here: system-level complexity can arise from data dependencies, configuration, infrastructure, interfaces and other components surrounding the model, meaning that model quality alone does not determine the maintainability or reliability of the complete system (Sculley et al., 2015).

This perspective also changes how AI systems should be evaluated. Model benchmarks can establish useful evidence about particular capabilities, but they cannot by themselves demonstrate that an enterprise AI system will improve a business process or operate safely in its intended environment. Liang et al. (2022) argue for more holistic evaluation of language models across multiple dimensions rather than reliance on a single benchmark. For systems incorporating retrieval, tools and agentic behaviour, evaluation must consequently extend beyond the model to include retrieval quality, action selection, interaction with external tools, reasoning and feedback mechanisms, workflow performance, safety and relevant organisational outcomes (Lewis et al., 2020; Yao et al., 2023; Shinn et al., 2023; NIST, 2023).

The architecture must consequently balance competing objectives. Greater model capability may increase performance while also affecting computational requirements and cost, reflecting the relationship between model scale, data, computation and performance identified in scaling-law research (Kaplan et al., 2020). Greater autonomy may reduce human intervention while increasing the potential impact of erroneous actions (Yao et al., 2023; Shinn et al., 2023). Stronger controls may reduce certain risks while introducing additional operational constraints, while increased modularity and componentisation can introduce additional system dependencies and integration complexity (Sculley et al., 2015). The appropriate architecture therefore depends on the requirements, risk profile and operating environment of the business capability being supported rather than on maximising any single technical attribute.

This perspective reinforces the importance of architectural boundaries. The model should not become the implicit repository of all enterprise knowledge, the sole decision-maker or the uncontrolled interface to enterprise systems. Retrieval-augmented generation demonstrates how knowledge can instead be externalised and supplied to a model at inference time (Lewis et al., 2020). Agentic architectures similarly demonstrate how reasoning and action can be separated into interactions between models, tools and environments (Yao et al., 2023; Shinn et al., 2023). Emerging interoperability approaches such as the Model Context Protocol further illustrate the architectural trend towards explicit interfaces between models and external sources of context and capability (Anthropic, 2024). These patterns support the principle that data, context, authority and execution should remain sufficiently explicit that each can be governed according to its own requirements.

The resulting conception of AI architecture is therefore fundamentally socio-technical. Technical components provide computation, information access, orchestration, execution, security and observability, while people and organisational structures determine objectives, responsibilities, approvals, escalation and accountability. This perspective is consistent with the broader systems view of foundation models, which recognises that AI applications depend on interactions between models and the wider systems in which they are deployed (Bommasani et al., 2021), as well as with established work on cognitive and multi-agent architectures in which intelligent behaviour emerges through interactions between multiple components and agents (Laird, 2012; Wooldridge, 2009).

The effectiveness of the complete system therefore emerges from the interaction between technical and organisational elements. Evaluation, governance and documentation must extend across the lifecycle rather than being treated as isolated activities. Model cards and datasheets provide mechanisms for documenting model and data characteristics (Mitchell et al., 2019; Gebru et al., 2021), while holistic evaluation approaches seek to assess model behaviour across multiple dimensions (Liang et al., 2022). At the organisational level, AI risk-management and management-system frameworks provide structures for embedding these activities within broader governance processes (NIST, 2023; ISO, 2023).

The central challenge is consequently not simply to make AI more intelligent. It is to make intelligent systems useful, reliable, secure, controllable, observable and economically sustainable within their intended organisational context. This requires architectural decisions that connect model capabilities with data, context, tools, workflows, human authority, evaluation and governance rather than treating the model as the complete AI system (Bommasani et al., 2021; Sculley et al., 2015; NIST, 2023).

This represents the fundamental shift developed throughout this paper: from designing models that produce outputs to engineering governed systems that can participate responsibly in real-world processes. Model capability remains a foundation of this transformation, as demonstrated by advances in transformers, scaling and foundation models (Vaswani et al., 2017; Kaplan et al., 2020; Bommasani et al., 2021). However, architecture determines how that capability is connected to knowledge, context, action, workflows and organisational controls, and therefore how model capability becomes an operational enterprise capability.

25. Conclusion

The fundamentals of AI architecture extend well beyond the foundation model. The development of contemporary AI has progressively shifted the central engineering challenge from building increasingly capable models to designing systems in which those capabilities can be reliably and responsibly applied.

The Transformer established a scalable architecture for modern neural language modelling, while subsequent work on scaling demonstrated systematic relationships between model performance, model size, data, and computation (Vaswani et al., 2017; Kaplan et al., 2020). Few-shot learning further demonstrated that sufficiently capable language models could perform a broad range of tasks through prompting rather than task-specific retraining alone (Brown et al., 2020). Foundation-model research subsequently established the broader concept of models serving as reusable bases for multiple downstream applications and identified the opportunities and risks associated with this paradigm (Bommasani et al., 2021).

Subsequent architectural developments extended the role of models beyond isolated generation. Retrieval-augmented generation introduced mechanisms for combining model inference with externally retrieved knowledge (Lewis et al., 2020). ReAct demonstrated the integration of reasoning and action within iterative interaction with external environments, while Reflexion explored the use of feedback and verbal reflection to influence subsequent behaviour (Yao et al., 2023; Shinn et al., 2023). Voyager further illustrated how language models could participate in persistent, open-ended interaction with an environment through planning, tool use, and accumulated experience (Wang et al., 2023).

Taken together, these developments provide important foundations for contemporary AI systems engineering. They also demonstrate why the model can no longer be treated as the complete architectural object. Once an AI system retrieves external information, maintains state, invokes tools, coordinates multiple steps, interacts with enterprise applications, or operates under explicit policies, its behaviour emerges from the interaction of multiple architectural components.

The resulting architecture is therefore necessarily layered. Models provide reasoning, representation, and generation capabilities; retrieval provides access to external knowledge; memory provides persistent or task-relevant state; tools and APIs provide controlled mechanisms for action; orchestration coordinates execution; infrastructure provides the computational runtime; security controls information and execution boundaries; evaluation provides evidence about system behaviour; observability provides operational visibility; and governance establishes accountability and lifecycle control.

These components should not be understood as independent technical layers alone. They form a socio-technical system in which technical mechanisms interact with people, business processes, organisational responsibilities, policies, and decision rights. The quality of the resulting AI capability therefore depends not only on the performance of individual components but also on the interfaces and controls connecting them.

The central architectural insight is consequently that AI capability is an emergent property of the system, not simply a property of the model. A model may provide substantial underlying capability, but whether that capability becomes useful, reliable, secure, controllable, and economically sustainable depends on the architecture in which it operates.

This distinction is particularly important in enterprise environments. The relevant unit of analysis is not simply the model or AI application, but the AI-enabled business capability. Such a capability combines models with enterprise data, context, workflows, applications, tools, infrastructure, people, controls, and governance to produce a defined organisational outcome.

This leads to a fundamental design principle:

The objective of AI architecture should not be maximum autonomy, but trustworthy capability within an appropriately controlled system.

This principle does not imply that autonomous operation should be avoided. Rather, autonomy should be treated as a capability whose scope is determined by the nature of the task, the consequences of failure, the reversibility of actions, the quality of available evidence, and the effectiveness of the controls surrounding execution. Where autonomous operation is appropriate, it should remain bounded, observable, evaluable, and accountable.

The same principle applies to future developments. AI architectures are likely to become increasingly agentic, distributed, multimodal, and interoperable. As systems acquire broader capabilities and interact with a growing range of information sources and operational environments, architectural boundaries will become increasingly important rather than less so.

The distinction between AI architecture, software architecture, data architecture, cybersecurity architecture, infrastructure architecture, and enterprise architecture will consequently become increasingly difficult to maintain as separate domains. AI systems depend upon all of these architectural disciplines, while increasingly influencing the design of each in return.

This convergence reinforces the central argument of this paper. AI should not be treated as an isolated technology that can be added to an existing enterprise without changing the surrounding architecture. It is becoming an architectural capability embedded within the broader socio-technical enterprise.

Organisations seeking sustainable value from AI must therefore design not only for model capability, but for the complete system through which that capability is accessed, contextualised, executed, controlled, evaluated, observed, and governed.

The fundamental challenge of enterprise AI is ultimately one of system design: creating architectures in which increasingly capable intelligence can operate within clearly defined boundaries, interact with authoritative information and enterprise systems, support human and organisational objectives, and remain subject to appropriate control and accountability.

In this sense, the future of AI architecture is not simply the construction of more powerful models. It is the engineering of governed intelligent systems in which intelligence becomes an integrated, observable, controllable, and accountable component of the enterprise.

References

Anthropic (2024) Model Context Protocol.

Bai, Y., Kadavath, S., Kundu, S. et al. (2022) ‘Constitutional AI: Harmlessness from AI Feedback’, arXiv preprint, arXiv:2212.08073.

Bommasani, R., Hudson, D.A., Adeli, E. et al. (2021) ‘On the Opportunities and Risks of Foundation Models’, arXiv preprint, arXiv:2108.07258.

Brown, T.B., Mann, B., Ryder, N. et al. (2020) ‘Language Models are Few-Shot Learners’, Advances in Neural Information Processing Systems, 33, pp. 1877–1901.

Gebru, T., Morgenstern, J., Vecchione, B. et al. (2021) ‘Datasheets for Datasets’, Communications of the ACM, 64(12), pp. 86–92.

ISO (2023) ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system. Geneva: International Organization for Standardization.

Kaplan, J., McCandlish, S., Henighan, T. et al. (2020) ‘Scaling Laws for Neural Language Models’, arXiv preprint, arXiv:2001.08361.

Laird, J.E. (2012) The Soar Cognitive Architecture. Cambridge, MA: MIT Press.

Lewis, P., Perez, E., Piktus, A. et al. (2020) ‘Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks’, Advances in Neural Information Processing Systems, 33, pp. 9459–9474.

Liang, P., Bommasani, R., Lee, T. et al. (2022) ‘Holistic Evaluation of Language Models’, Transactions on Machine Learning Research.

Mitchell, M., Wu, S., Zaldivar, A. et al. (2019) ‘Model Cards for Model Reporting’, in Proceedings of the Conference on Fairness, Accountability, and Transparency. New York: ACM, pp. 220–229.

National Institute of Standards and Technology (NIST) (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. Gaithersburg, MD: NIST.

Open Worldwide Application Security Project (OWASP) (2025) OWASP Top 10 for LLM Applications 2025. Available at: https://genai.owasp.org/ (Accessed: 22 September 2026).

Ouyang, L., Wu, J., Jiang, X. et al. (2022) ‘Training language models to follow instructions with human feedback’, Advances in Neural Information Processing Systems, 35, pp. 27730–27744.

Sculley, D., Holt, G., Golovin, D. et al. (2015) ‘Hidden Technical Debt in Machine Learning Systems’, in Advances in Neural Information Processing Systems, 28.

Shinn, N., Cassano, F., Berman, E. et al. (2023) ‘Reflexion: Language Agents with Verbal Reinforcement Learning’, Advances in Neural Information Processing Systems, 36.

Sutton, R.S. and Barto, A.G. (2018) Reinforcement Learning: An Introduction. 2nd edn. Cambridge, MA: MIT Press.

Vaswani, A., Shazeer, N., Parmar, N. et al. (2017) ‘Attention Is All You Need’, in Advances in Neural Information Processing Systems, 30.

Wang, G., Xie, Y., Jiang, Z. et al. (2023) ‘Voyager: An Open-Ended Embodied Agent with Large Language Models’, arXiv preprint, arXiv:2305.16291.

Wooldridge, M. (2009) An Introduction to MultiAgent Systems. 2nd edn. Chichester: Wiley.

Yao, S., Zhao, J., Yu, D. et al. (2023) ‘ReAct: Synergizing Reasoning and Acting in Language Models’, in International Conference on Learning Representations (ICLR).

Contact

Reach out via email for inquiries.

Email

Subscribe to newsletter

info@grcadvisory.ch

Fundamentals of AI Architecture

Abstract

Related Papers

1. Introduction

2. From Model-Centric AI to AI Systems Engineering

3. The Layered AI Architecture

4. Foundation Models and the Intelligence Layer

5. Context Engineering and Retrieval

6. Memory and State

7. Agentic Architecture: From Inference to Action

7. Agentic Architecture: From Inference to Action

8. Orchestration

9. Tools, APIs and Interoperability

10. Multi-Agent Systems

11. Security Architecture

12. AI Safety and Guardrails

13. Evaluation as an Architectural Capability

14. Observability and Explainability

15. Governance by Design

16. Human Oversight and Decision Architecture

17. Data Architecture and AI Readiness

18. Infrastructure and Deployment Architecture

19. Resilience and Failure Engineering

20. Sovereignty and Dependency Management

21. AI Architecture as Enterprise Architecture

22. Seven Fundamental Principles of AI Architecture

22.1 Principle 1: Separate Intelligence from Control

22.2 Principle 2: Externalise Dynamic Knowledge

22.3 Principle 3: Treat Autonomy as a Bounded Capability

22.4 Principle 4: Design for Modularity and Interoperability

22.5 Principle 5: Build Security and Governance into the Architecture

22.6 Principle 6: Make Evaluation and Observability Continuous

22.7 Principle 7: Keep Consequential Authority Accountable

22.8 Synthesis

23. A Reference Architecture for Enterprise AI

24. Discussion

25. Conclusion

References

Contact