Foundry Agent Service, agents with and without code

Foundry Agent Service, agents with and without code

Foundry Agent Service is a managed platform for building and running agents in Microsoft's cloud. It offers two routes worth separating at the start, since they lead to entirely different kinds of work.

The first is an agent defined by configuration: you choose a model, write instructions, attach tools, and the vendor runs it. There is no code to maintain and no containers to scale. The second is an agent you write yourself, package as a container image, and hand over to be run, while the vendor supplies an endpoint, scaling, identity, and observability.

A naming change worth noting

The product previously carried a name tying it to the Azure AI Foundry platform, and the documentation today speaks of Microsoft Foundry. The change is cosmetic technically and confusing when searching, since older and newer material use different terms for the same thing.

Practically it means one thing: when looking for documentation, check the date and land on pages under the current name. Material describing this product from before the change remains mostly accurate on principles, while resource names and paths sometimes differ.

Worth noting something else that matters when comparing vendors. This service moves in the opposite direction from Bedrock agents, where the equivalent entered maintenance mode and closed to new customers. Here the code free variant remains supported alongside the code based one.

The prompt defined agent

This route suits cases where an agent should check something, summarise something, and call two tools, with all the logic fitting inside instructions.

The definition consists of three things: a model from the catalogue, instructions describing the goal and constraints, and a tool set. You can do that in a graphical interface or programmatically, and the second route is better, since the definition lands in a repository and passes review like any other code.

The advantage is clear and deserves naming. You maintain no process, pay for no compute while waiting, and have no container to patch. Scaling happens by itself, and the bill follows model calls and tools used.

The boundary is equally clear. The flow's shape is fixed, so anything requiring branching driven by your business logic, or calling your own code mid run, falls outside this variant. Then the second route is right.

Practical advice: start here even when you suspect you will end up elsewhere. A prompt defined agent appears in an hour and shows whether the idea works at all, before you invest in infrastructure.

The agent with your own code

The second variant accepts code written with any toolkit: the vendor's own framework, LangChain agents, the OpenAI toolkit, the Anthropic toolkit, or your own loop.

That deserves emphasis, since it separates this service from closed solutions. The platform imposes no way of writing an agent and instead supplies what you would have built anyway: an authenticated endpoint, automatic scaling, a separate identity per agent, session state durability, and run tracing.

You package the code as a container image or as a source archive from which the platform builds the image itself. That second option shortens the path in simpler cases.

Underneath, your code calls the shared responses API on the project endpoint to obtain a model and the platform's tools. That is the same endpoint the code free variant uses, so both share the tool set and the model billing model.

One more possibility deserves knowing, described least often. If your agent already runs elsewhere and you do not want to move it, you can call the same responses API from your own process. You get the platform's models and tools without handing over your code to be run, which is often the right compromise with an existing application.

Tools and the toolbox

The built in set covers web search, file search, memory, code execution, and connecting to servers speaking the protocol described in the MCP registry. Custom tools and integrations with this vendor's services join them.

Some of them, memory and web search included, are in public preview. That is a fact to check before basing a production process on them, since availability is sometimes regionally limited too.

The more interesting element is the toolbox, a tool set defined once and exposed through a single endpoint speaking that protocol. The practical consequence is that any agent understanding the protocol can use it, regardless of what it was written in and where it runs.

The toolbox carries versions, so changing the tool set does not reach everybody immediately. You create a new version, test it, and set it as default. That mechanism's absence in a self built deployment hurts on the first tool schema change in a running system.

Tool authentication supports several modes, acting on a user's behalf among them. That last one solves a real problem: an agent fetching data for a particular person should hold that person's permissions rather than the application's.

The shared responses API

This deserves its own section, since it binds the whole thing together and gets skipped on a first read.

Both agent types, and code running outside the platform too, use the same endpoint. Through it you call the model, and through it you receive the platform's tools.

The practical consequence is that choosing a variant stops being an irreversible decision. An agent written today as code you run yourself, calling this API, can tomorrow be packaged into a container and handed over to the platform, with no logic rewritten. What changes is where it runs, not the code.

That separates this arrangement from solutions where the code free and code based variants are two separate worlds with separate concepts. Here the difference concerns hosting rather than how you talk to a model.

The second consequence concerns migration. If you hold a running application with your own loop, the first step need not be moving it anywhere. Pointing model calls at this endpoint suffices to gain the platform's tools and tracing, while the hosting decision waits.

Remember, though, that this is vendor binding at the level of model calls. A thin layer of your own around those calls costs a few hours and leaves an exit open, which on a decision spanning years is often worth the work.

When this is the right choice

Three conditions deserve checking before starting, since outside them simpler solutions are better.

The first: you are already in this cloud. The value lies in things intertwined with the rest of the infrastructure: identity, permissions, virtual network, billing. A team working elsewhere gets mainly another vendor to handle.

The second: the agent is meant to be a company tool rather than a feature in a product. The publishing layer, distribution through collaboration tools, and the identity registry are designed for internal agents, and for a feature in an application serving outside customers they usually do not apply.

The third: you need infrastructure you do not want to build. Per agent identity, session isolation, and state durability are several weeks of work, and if your case does not require them, a simpler route suffices.

Where any condition fails, consider calling models directly with your own loop. An agent with three tools and no identity requirements does not need a platform of this class, and introducing one adds pieces to configure before a first answer.

Foundry Agent Service against the alternatives

The distinction between the first two rows matters most here and reduces to one question: does your agent need code in the middle of a run. If not, the first row saves an entire infrastructure layer. If so, the second gives control without giving up the platform.

The third row is worth knowing for comparison, since it shows the large vendors' paths diverged. One is winding down its code free variant; the other maintains both.

The last row remains for teams where cloud lock in is a problem. Compute then how much of what the service provides you would genuinely build: per agent identity, session isolation, state durability, and tracing are usually several weeks of work.

Lifecycle, versions, and publishing

This part gets undervalued and decides whether agents inside a company can be maintained.

Versioning works automatically: successive agent changes get recorded as versions you can return to and compare. That is the equivalent of a change history you have to build yourself in a self built deployment.

Publishing promotes an agent to a resource with a stable endpoint, inheriting the project's permissions. Only a published agent suits calling from an application, while the test one stays a test one.

Distribution covers sharing an agent through this vendor's collaboration tools and through the identity registry. That route makes sense for internal agents, where the audience is an employee rather than an outside customer.

An evaluation and tracing layer joins that. Traces showing every model call and every tool invocation are built in rather than added separately, which when diagnosing bad answers is the difference between an hour and a day.

Safeguards and identity

Three elements deserve knowing, since they solve problems hard to get right yourself.

A separate identity per agent means an agent acts under its own account rather than a shared application key. You grant permissions precisely, and on abuse you disable its account rather than the whole integration.

Network isolation lets an agent run inside your virtual network. In the code based variant each session runs in an isolated environment connected to that network, which matters under data location requirements.

Content filters sit in the flow and cover injection detection, including injections arriving in content fetched externally. That is the same layer I described at length under this cloud's content filters, and with agents it matters more than in ordinary chat, since an agent reads documents and tool results.

Common mistakes

The first is choosing the code based variant for a case the prompt defined variant handles. You then add a container to maintain and compute cost with no gain.

The second is defining an agent solely in the graphical interface. The definition then lives outside the repository, with no change history and no review, and the question of who changed an instruction has no answer.

The third is basing a production process on tools in public preview without checking their status and regional availability.

The fourth is skipping acting on a user's behalf where an agent fetches personal data. An agent holding application permissions sees more than it should in the context of a particular conversation.

The fifth is calling the test agent from an application rather than the published one. A change made mid work then reaches production immediately.

The sixth is computing cost from model calls alone. The code based variant adds container compute, and tools bill separately.

FAQ

How do the two agent types differ?

In who writes the flow logic. A prompt defined agent comes from configuration and the vendor runs it, with no code and no containers. An agent with your own code is your program in a container, to which the platform gives an endpoint, scaling, identity, and observability.

Can I use LangGraph or the OpenAI toolkit?

Yes, the code based variant accepts agents written with those libraries, the Anthropic toolkit, and your own loop. The code calls the shared responses API for models and platform tools, so you are not bound to a way of writing agents.

Do I have to move my code into this cloud?

No. If an agent already runs elsewhere, you can call the same responses API from your own process and get the platform's models and tools without handing over your code to be run.

What is the toolbox?

A tool set defined once and exposed through a single endpoint speaking the protocol described in the MCP registry. Any agent understanding that protocol can use it, and versioning means a change to the set does not reach everybody immediately.

How do I compute costs?

The prompt defined variant bills model calls and tool usage. The code based variant adds container compute. Price one real run, since an agent's step count usually dominates the bill more than the rate for a single call.

Documentation sits on the service page, and the tool catalogue in a separate document.