ChatGPT
Perplexity
Building agentic AI solutions on Azure starts with defining how agents will access enterprise context, maintain workflow state, invoke tools, and operate within established security controls. These decisions shape whether an agent can complete a real business process reliably.
The requirements become more demanding when an agent creates tickets, routes approvals, updates records, sends customer communications, or triggers remediation. Each action depends on accurate retrieval, caller identity, scoped permissions, traceability, and a clear recovery path when a workflow fails.
On the Microsoft stack, these responsibilities span three connected layers. Microsoft Fabric and Azure AI Search provide governed context. Microsoft Foundry and Copilot Studio coordinate agent workflows. Microsoft Entra ID and Azure’s governance and monitoring services control and observe execution.
This article explains how these layers connect to build and support autonomous Azure agents in production.
Why autonomous agents fail when data, agents, and governance live as separate initiatives
Data, AI, and security teams often design their parts of an agentic system on separate timelines. The dependencies come together only at runtime, when the agent retrieves enterprise data, selects a tool, and executes an action under a specific identity.
This creates gaps between how the agent was designed and how the surrounding systems are governed.
The agent may retrieve stale or overshared information, receive broader tool access than the workflow requires, or reach production before approval paths, evaluation criteria, and monitoring are in place.
Existing access risks also become easier to exploit at scale. Concentric AI’s 2025 Data Risk Report found that Microsoft 365 Copilot accessed nearly three million sensitive records per organization. It also found sensitive information in 57% of organization-wide shared data, rising to more than 70% in financial services and healthcare.
Each production workflow therefore needs its data sources, agent runtime, permissions, and operational controls designed together.
On Azure, Microsoft Fabric and Azure AI Search support governed context and retrieval. Microsoft Foundry and Copilot Studio coordinate agent execution, while Microsoft Purview and Entra ID enforce policy, identity, and access controls.
Simform applied this approach while building an LLM-powered research platform for a global consultancy. The Azure architecture combined grounded retrieval, role-based data isolation, secure identity, and output guardrails, reducing manual analysis by 80% and accelerating insight generation by 5x.
The agent layer: Three services, one architecture decision
The agent layer determines where an agent is built, how it coordinates tools and workflows, and where users interact with it.
Microsoft 365 Copilot provides the workplace experience, Copilot Studio supports low-code agent and workflow development, and Microsoft Foundry provides the managed platform for custom agent applications. The services can work together, but each needs a defined role in the architecture.
The key decision is where to place the user experience, workflow orchestration, and custom agent logic.
An employee-facing agent may be delivered through Microsoft 365 Copilot or Teams, configured in Copilot Studio, and call specialized agent services hosted in Foundry. Defining these boundaries early prevents teams from duplicating orchestration, identity, and monitoring across platforms.
When do you build AI agents in Microsoft Foundry vs. Copilot Studio?
Copilot Studio is suited to agents delivered through Microsoft 365, Teams, Dynamics 365, Power Platform, or other business-facing channels. It provides managed connectors, triggers, approvals, agent flows, and autonomous execution, making it useful when rapid delivery and business-team participation matter.
Microsoft Foundry is suited to agent applications that require custom code, model and framework flexibility, deeper integration with APIs and enterprise systems, or greater control over hosting, scaling, identity, and observability. It supports both portal-authored prompt agents and custom hosted agents, so the decision depends on the required level of engineering control rather than a simple low-code versus pro-code split.
| Use case | Copilot Studio | Microsoft Foundry |
| Best suited for | Business-facing agents and managed workflows | Custom agent applications and platforms |
| Primary ecosystem | Microsoft 365, Teams, Dynamics 365, Dataverse, Power Platform | Azure services, custom applications, APIs, databases, and external systems |
| Development approach | Visual authoring, connectors, triggers, and agent flows | Pro-code development, SDKs, custom orchestration, and agent frameworks |
| Typical ownership | Business and fusion teams with central governance | AI, application, and platform engineering teams |
| Choose it when | Channel integration and managed delivery matter most | Custom logic, framework choice, complex state, and runtime control are required |
Many architectures use both. Copilot Studio can manage the employee-facing interaction or business workflow, while Foundry hosts specialized agent logic behind it.
Where does Semantic Kernel fit in an Azure agent stack
Semantic Kernel remains relevant to existing implementations, but Microsoft Agent Framework is now its direct successor and Microsoft’s forward-looking code-first framework for agents and workflows.
Agent Framework supports tool and MCP integration, session-based state, memory, graph-based workflows, checkpointing, observability, and human intervention. It can be used to build agent logic that runs with Microsoft Foundry, including custom hosted agents.
Teams maintaining Semantic Kernel solutions should assess the migration path before extending the framework across new agent workloads.
The data layer: How to ground Azure AI agents with governed enterprise data
Agent performance depends on the context available at the point of decision. Fragmented sources, stale copies, missing lineage, and inconsistent business definitions can lead an agent to retrieve the wrong information or apply it incorrectly. Greater autonomy increases the impact because the output may inform a tool call or business action before a person reviews it.
On Azure, Microsoft Fabric can provide the governed data foundation for this context.
OneLake as the single governed estate
OneLake creates a unified view of enterprise data across Fabric and external sources. Shortcuts allow selected data to remain in its source system, while mirroring or pipelines can replicate it when the workload requires a local copy.
Agents can then retrieve from governed data products without every team building separate source integrations.
Semantic models as agent vocabulary
Fabric semantic models define shared measures, relationships, and business logic. Grounding agents on these models gives them a consistent interpretation of terms such as quarterly revenue or active customer.
The results still depend on data freshness, filters, and the caller’s permissions, but the underlying definitions remain consistent across use cases.
Fabric IQ and Fabric data agents
Fabric IQ extends this foundation with semantic intelligence. Its ontology capabilities can represent business entities, relationships, and rules over governed data.
Fabric data agents make this context available through natural-language interactions and can be connected to other agent experiences. Some ontology and agent integrations remain in preview and should be assessed accordingly before production adoption.
Establishing this foundation usually requires data modernization. Simform’s TrueMorph accelerator supports legacy discovery, dependency mapping, migration validation, and sensitive-data controls, helping teams complete modernization workloads up to 60% faster.
Azure AI Search as the retrieval engine
Microsoft Fabric organizes and governs enterprise data. Azure AI Search prepares relevant structured and unstructured information for retrieval by agents.
Hybrid search combines full-text and vector queries, with semantic ranking available to improve relevance. Integrated vectorization can automate document chunking and embedding generation as indexers process source content.
Access control must also remain part of the retrieval design. Azure AI Search can enforce document-level permissions through supported ACL integration or security filters configured around user and group identities. Foundry IQ can build permission-aware knowledge bases on top of Azure AI Search, although its agentic retrieval capabilities currently carry preview considerations.
Simform’s ThoughtMesh accelerator packages these retrieval, access-control, and orchestration patterns into a reusable foundation for enterprise RAG and agent applications, reducing the need to rebuild the context layer for every use case.
The governance layer: Responsible AI for Azure AI agents
Governance bolted on after the build is the third disconnected initiative, and the one that blocks release, because it is the only one with veto power.
Designed in, it looks like this:
The Responsible AI Standard as engineering requirements
Microsoft’s Responsible AI principles need to translate into verifiable controls. Reliability requires fallback paths. Transparency shapes how sources and limitations are communicated. Accountability requires a named owner, review criteria, and an escalation path. The controls should reflect the potential impact of the workflow.
Azure AI Content Safety at the orchestration layer
Azure AI Content Safety can inspect prompts, retrieved content, and responses using harm filters, Prompt Shields, and task-adherence checks. Teams should configure policies and thresholds according to each agent’s users, data, and actions.
Evaluations in Microsoft Foundry as the release gate
Evaluation datasets should test retrieval, response quality, safety, and task completion before release. Teams can connect agreed thresholds to CI/CD pipelines so versions that fall below the baseline do not progress.
Purview and Entra ID as the boundary system
Microsoft Purview helps identify and protect sensitive data used or exposed by agents. Microsoft Entra Agent ID gives each agent a managed identity with scoped roles, restricted tools, and revocation controls. Together, they define the data and authorization boundaries within which the agent operates.
One stack end to end: How to build an Azure-native agentic AI architecture
An Azure-native agent architecture is not a sequence where data comes first, agents come second, and governance is added at the end. The three layers intersect throughout the runtime.
Every component either supplies context, coordinates action, or constrains what the system is allowed to do. In most cases, it does at least two of those at once.
1. Entry and identity
A workflow may begin in Microsoft Teams, Microsoft 365 Copilot, a business application, an API, or an event from an operational system. Copilot Studio can provide the business-facing experience, while custom applications can invoke agents hosted in Microsoft Foundry.
The user and agent identities should be established at this point and propagated into downstream retrieval and tool calls. Microsoft Entra Agent ID provides distinct identities for agents so their permissions, owners, and activity can be governed throughout the lifecycle.
2. Agent orchestration
Microsoft Foundry Agent Service provides the runtime for interpreting the objective,
Foundry Agent Service hosts and scales agents while managing conversations, tool calls, identity, and lifecycle operations. Microsoft Agent Framework can add explicit workflow paths, checkpoints, tool approvals, and human intervention for long-running or multi-agent processes.
Each agent should have a defined responsibility, toolset, and permission boundary. Predictable process steps can remain inside conventional workflows, leaving agent reasoning for decisions that require interpretation or context.
3. Governed retrieval
The agent retrieves the context required for the current decision from Microsoft Fabric, Fabric data agents, or content indexed in Azure AI Search. Retrieval should apply the calling identity and document-level permissions so the agent receives only the information available to that user or workflow.
This keeps business definitions, source access, and retrieval controls connected to the reasoning process.
4. Tools, actions, and approvals
Approved actions pass through controlled interfaces such as Azure Functions, Logic Apps, Power Automate, API Management, or scoped application APIs.
Before execution, the workflow validates the request schema, agent identity, business rules, approval threshold, and transaction limit. High-impact actions can pause for human review. Idempotency and recovery controls also prevent retries or partial failures from creating duplicate or inconsistent transactions.
5. State and recovery
Interaction history and workflow state serve different purposes. Foundry conversations and sessions can maintain the context of an interaction. Microsoft Agent Framework checkpoints or application-owned stores such as Azure Cosmos DB can preserve task progress across approvals, failures, and long-running processes.
Long-term agent memory should be introduced only when the workflow requires reusable user or task context. Foundry memory stores currently carry preview considerations, and retained information still requires explicit access, isolation, and deletion policies.
6. Evaluation and observability
Every retrieval, model response, agent handoff, tool invocation, approval, retry, and failure should produce a trace. Foundry Agent Service provides tracing and Application Insights integration, while Agent Framework exposes workflow events and execution state.
Teams can use these traces to evaluate task completion, retrieval quality, tool selection, safety, latency, and cost. Production failures can then become evaluation cases for subsequent releases.
How this architecture works
Simform used this pattern to build a multi-agent field operations platform for a US HVAC enterprise with 23 subsidiaries. The field operations platform deploys Azure AI Foundry-powered agents behind a mobile-first experience for technicians.
Orchestration in Foundry routes multimodal inputs including what a technician types, photographs, or dictates from a job site to the right agent workflows for field operations, billing, and reporting.
Data flows through an integration with Procore, so agents act on the operational system of record rather than a copy of it, and governance controls are built into the agent layer rather than appended to it.
Every layer in the architecture above is present, and none of them was retrofitted. The resulting architecture reduced report creation effort by up to 80% and expense logging effort by up to 60%.
More importantly, it created a reusable agentic operating model that could be extended across the group’s subsidiaries without rebuilding data access, orchestration, and controls for every deployment.
What Simform has learned from taking Azure agents to production
Across Simform’s Azure agent implementations, the most reliable path to production starts with a bounded workflow. Teams define where agent reasoning adds value, keep predictable steps deterministic, carry identity through retrieval and tool calls, and establish approval and recovery paths before increasing autonomy.
This approach shaped the secure research platform that reduced analysis effort by 80% and the multi-agent field operations platform that cut reporting effort by up to 80%. Simform combines these production patterns with Microsoft cloud, data, and AI engineering expertise, supported by accelerators such as ThoughtMesh and TrueMorph for faster time to value.
As a Microsoft Solutions Partner and Azure Expert MSP, we help enterprises turn high-value workflows into secure, observable agentic systems that can be extended across the organization.
