AI Systems Architecture
AI Systems Architecture addresses the structure of software systems whose intelligent functionality is provided by machine learning models, foundation models, autonomous agents, or embodied components. Its subject is the system around the model: the components, connectors, views, quality attributes, tactics, patterns, and reference architectures by which a system containing one or more models is structured, evaluated, and sustained.
The resources on this page cover four classes of AI systems, task-specific model systems, foundation model systems, agentic AI systems, and physical AI systems, and are intended for software architects, senior developers, educators, and researchers who design, teach, or study systems in which learned components carry significant responsibility.
What are AI Systems?
An AI system is a software system whose intelligent functionality is provided by models, whose behavior is learned from data rather than specified by developers. An AI system consists of model components, which perform inference, and non-model components, such as data pipelines, feature and knowledge stores, serving infrastructure, orchestration logic, monitoring, and user-facing applications. These components interact through pipelines, request–response exchanges, and control loops.
Four Classes of AI Systems
Classifications by model technology, task, or input modality do not predict the architecture of a system: two image classifiers may differ entirely because one serves a batch pipeline and the other an autonomous vehicle. AI systems are therefore classified by the position of the model in the system, characterized by three dimensions. Model provenance distinguishes system-owned models from externally provided ones; control locus distinguishes control flow fixed at design time from control determined by a model at runtime; and action domain distinguishes informational effects from physical effects exerted through actuators.
| Class | Model Provenance | Control Locus | Action Domain | Representative Systems |
|---|---|---|---|---|
| Task-Specific Model Systems | System-owned | Designer-defined | Informational | Forecasting, anomaly detection, recommendation |
| Foundation Model Systems | Externally provided | Designer-defined | Informational | Enterprise assistants, retrieval-augmented generation |
| Agentic AI Systems | Externally provided (in practice) | Model-determined | Informational | Multi-step task automation, multi-agent workflows |
| Physical AI Systems | Either | Either | Physical | Robots, autonomous vehicles, embodied assistants |
The four classes are also four levels of architectural scope. Each class retains the concerns of the preceding one and adds its own: the foundation model class adds dependence on a model the system does not own, the agentic class adds control decided at runtime, and the physical class adds a closed loop with the physical world. Systems that combine the characteristics of several classes are treated as hybrid systems under a dominant class.
Why Architecture for AI Systems?
Engineering Challenges
AI systems present seven engineering challenges that conventional software systems do not. Four arise from the nature of the model itself, and three from the position the model occupies in the system.
- Learned rather than fully specified behavior: the training procedure produces behavior but no specification of it
- Dependence on data quality and representativeness: defects in data become defects in behavior
- Uncertain and variable outputs: results are accepted on statistical rather than exact criteria
- Limited interpretability: the reasons for a model output are not directly available
- Decisions delegated to models: models are given authority over actions with real consequences
- Error propagation through composed models and agents: a faulty output travels along paths formed at runtime
- Continuous evolution of models and data: behavior changes after release as models, data, prompts, and indexes change
Architectural Remedies
None of these challenges can be eliminated by improving the model alone. Each is addressed by an architectural remedy, a design solution realized through the selection, organization, and configuration of components, connectors, and the policies that govern them.
Model-level challenges are remedied by allocating to components at the model boundary the responsibilities the model cannot discharge: an explicit model contract, data treated as a versioned architectural element, output verification placed on the interaction path, and explanation and decision recording assigned outside the model. System-level challenges are remedied by structure between components and over time: authority bounded per decision with points of human oversight, trust boundaries on every connector that carries a model output, and a configuration that spans models, data, prompts, indexes, and policies, with every change governed by evaluation before release.
Architectural remedies can be specified, tested, and evaluated independently of model internals; they confine the consequences of model errors; and they remain valid when a model is replaced by a more capable one. They also expose the two trade-offs that recur in every class of AI system: autonomy versus safety and evolution versus stability.
Architectural Views for AI Systems
Conventional architectural views were designed for systems whose behavior is fixed at design time by code. For AI systems, four conventional views are revised, three views that are optional for many conventional systems become essential, and two new views are introduced, yielding nine views.
- Revised views: the functional view distributes responsibilities across models and agents; the information view covers datasets, context, memory, and embeddings; the behavior view captures inference flows and runtime composition; and the deployment view includes model serving, accelerators, and external model providers
- Essential views: the context view identifies model providers, data sources, and regulators; the development view manages model and data artifacts; and the operation view addresses monitoring, evaluation, and release
- Intelligence View: how the system realizes its intelligent functions, through intelligent functions, realization mechanisms, reasoning schemes, context and knowledge sources, validation, and selection policies
- Learnability View: how the system learns and releases improved behavior, through learning targets, data and feedback sources, learning mechanisms, evaluation gates, release mechanisms, and learning triggers
Quality Attributes of AI Systems
Quality attributes are the criterion by which architectural decisions for AI systems are made. Some attributes are retained from conventional systems, some are redefined because model behavior is statistical, and some are added, such as fairness, explainability, groundedness, and controllability. Requirements on AI components are expressed through bounded, statistical, and policy-based criteria rather than exact equality with expected outputs, and quality attribute scenarios are formulated to hold under non-determinism. Trustworthiness is treated as a composite attribute that combines several of these properties.
Reference Architectures
Each class of AI system is supported by a reference architecture, and each is presented as an extension of the one preceding it. A reference architecture serves both as a starting point for deriving a system-specific architecture and as a completeness check for an existing one.
- Task-Specific Model Systems: data, training, model management, serving, and application layers, with monitoring and feedback elements
- Foundation Model Systems: application, model interaction, retrieval, and resource layers, with foundation models treated as external elements governed by model contracts
- Agentic AI Systems: orchestration, agent, control and governance, capability, and memory layers added to the foundation model layers
- Physical AI Systems: a perception-to-action loop of sensing, world modeling, deliberative planning, reactive control, and actuation, with independent safety supervision and edge and cloud services for fleet learning
For each class, the reference architecture is refined into sub-architectures, such as data and feature, model interaction, multi-agent coordination, and perception and world modeling architectures, and complemented by quality attribute design tactics and a catalog of architecture patterns.
About the Book
These resources are based on the forthcoming book AI Systems Architecture: Predictive, Generative, Agentic, and Physical AI by Soo Dong Kim. The book is organized in seven parts:
- Foundations: AI systems and their classes, the rationale for architecting them, software architecture fundamentals, and architectural views for AI systems
- Architecture of Task-Specific Model Systems
- Architecture of Foundation Model Systems
- Architecture of Agentic AI Systems
- Architecture of Physical AI Systems
- Cross-Class Practices and Integration: architecture evaluation, evolution and operations, security, governance and compliance, and hybrid AI systems
- Catalog of Architecture Patterns for AI Systems
Each class part covers the class definition, its reference architecture, its sub-architectures, quality attribute design, and a complete case study.
Resources
Supplementary resources, including templates for the nine views, checklists, pattern descriptions, and case-study architecture descriptions, will be provided here to support courses, self-study, and professional practice.