Build a Switchable Multi-Provider AI Gateway with Microsoft.Extensions.AI

What You Will Build

The finished sample is a small ASP.NET Core Minimal API that accepts a provider name, a message, an optional conversation ID, and an optional tool flag. The HTTP layer never constructs an arbitrary model client from user input. It resolves a provider that the server already knows about, then sends the conversation through IChatClient.

HTTP caller
    │
    │ provider name only
    ▼
ASP.NET Core Minimal API
    │
    ├── GET  /api/providers
    ├── POST /api/chat
    └── POST /api/chat/stream
    │
    ▼
ChatGateway
    │
    ├── ConversationStore
    ├── convert_temperature tool
    │
    ▼
ProviderRegistry
    │
    ▼
ProviderDescriptor
(name + capabilities + IChatClient)
    │
    ├── Ollama
    ├── OpenAI
    ├── OpenRouter / OpenAI-compatible
    └── Azure OpenAI

Why Provider Abstraction Matters

A provider-specific SDK is useful at the edge of an application. The expensive part is allowing provider-specific types, endpoint rules, credential handling, and model configuration to leak into every endpoint and service that needs AI.

Provider-coupled endpoint                 Provider-neutral gateway
─────────────────────────                 ────────────────────────
HTTP request                              HTTP request
    │                                         │
    ▼                                         ▼
construct provider SDK                    resolve registered name
    │                                         │
endpoint + key + model                       ▼
    │                                      IChatClient
    ▼                                         │
provider-specific request                     ▼
    │                                      common gateway logic
    ▼
provider-specific response

Change provider? Rewrite application logic.   Change provider? Change trusted registration.

The abstraction does not mean providers become identical. Models and services can differ in tool support, streaming behavior, limits, authentication, and operational characteristics. The sample therefore keeps explicit capability metadata beside each registered client.

Prerequisites and Verified Baseline

This tutorial is pinned to the exact implementation that was restored, compiled, tested, and published. It does not use floating 10.x.y placeholders.

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Azure.AI.OpenAI" Version="2.1.0" />
    <PackageReference Include="Azure.Identity" Version="1.21.0" />
    <PackageReference Include="Microsoft.Extensions.AI" Version="10.10.0" />
    <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="10.10.1" />
    <PackageReference Include="OllamaSharp" Version="5.4.30" />
  </ItemGroup>
</Project>

You need the .NET 10 SDK. Ollama is required only if you want to reproduce the default local live path. OpenAI, OpenRouter, and Azure OpenAI are optional and disabled in the checked-in configuration.

Understand IChatClient

IChatClient is the central Microsoft.Extensions.AI chat abstraction. The interface exposes complete-response and streaming-response operations, while provider adapters can wrap concrete SDK clients behind the same contract.

Microsoft's documentation also makes an important responsibility clear: the abstraction does not automatically solve prompt injection, message-size limits, authorization, or provider-specific safeguards. Those controls remain application concerns.

Project Architecture

The verified project intentionally uses one web application and one test project. The boundaries are logical rather than spread across extra class libraries.

ProviderAgnosticChatGateway/
│
├── src/ProviderAgnosticChatGateway/
│   ├── Program.cs
│   ├── appsettings.json
│   ├── Providers/
│   │   ├── ProviderCapabilities.cs
│   │   ├── ProviderDescriptor.cs
│   │   ├── ProviderRegistry.cs
│   │   └── ProviderClientFactory.cs
│   ├── Conversations/
│   │   ├── ConversationState.cs
│   │   └── ConversationStore.cs
│   ├── Gateway/
│   │   ├── ChatGateway.cs
│   │   ├── ChatRequest.cs
│   │   ├── ChatResult.cs
│   │   └── ChatStreamEvent.cs
│   └── Tools/
│       └── TemperatureTool.cs
│
└── tests/ProviderAgnosticChatGateway.Tests/
    ├── FakeChatClient.cs
    ├── ProviderRegistryTests.cs
    ├── ConversationStoreTests.cs
    ├── ProviderClientFactoryTests.cs
    ├── GatewayTests.cs
    └── ToolTests.cs

If you are following from scratch, create a web project and a separate xUnit project. The companion repository already contains these files, so cloning it is faster if your goal is only to run the verified sample.

mkdir ProviderAgnosticChatGateway
cd ProviderAgnosticChatGateway

dotnet new globaljson --sdk-version 10.0.401 --roll-forward latestFeature
dotnet new sln -n ProviderAgnosticChatGateway

dotnet new web `
  -n ProviderAgnosticChatGateway `
  -f net10.0 `
  -o .\src\ProviderAgnosticChatGateway

dotnet new xunit `
  -n ProviderAgnosticChatGateway.Tests `
  -f net10.0 `
  -o .\tests\ProviderAgnosticChatGateway.Tests

dotnet sln .\ProviderAgnosticChatGateway.slnx add `
  .\src\ProviderAgnosticChatGateway\ProviderAgnosticChatGateway.csproj

dotnet sln .\ProviderAgnosticChatGateway.slnx add `
  .\tests\ProviderAgnosticChatGateway.Tests\ProviderAgnosticChatGateway.Tests.csproj

dotnet add `
  .\tests\ProviderAgnosticChatGateway.Tests\ProviderAgnosticChatGateway.Tests.csproj `
  reference `
  .\src\ProviderAgnosticChatGateway\ProviderAgnosticChatGateway.csproj

Then add the five direct package references shown in the verified project file above. Keep package versions in source control so a future reader can reproduce the same API surface.

The server configuration separates provider identity from HTTP input. The checked-in default enables only Ollama, while the cloud providers remain disabled until you intentionally configure them.

{
  "Gateway": {
    "DefaultProvider": "ollama",
    "AllowProviderSelection": true,
    "MaxMessagesPerConversation": 20,
    "MaxInputCharacters": 8000
  },
  "Providers": {
    "Ollama": {
      "Enabled": true,
      "Endpoint": "http://localhost:11434",
      "Model": "qwen3.8:27b",
      "Capabilities": {
        "Streaming": true,
        "Tools": true,
        "ProviderManagedConversation": false
      }
    },
    "OpenAI": {
      "Enabled": false,
      "Model": "gpt-5-mini",
      "ApiKeyEnvironmentVariable": "OPENAI_API_KEY"
    },
    "OpenRouter": {
      "Enabled": false,
      "Endpoint": "https://openrouter.ai/api/v1",
      "Model": "openai/gpt-5-mini",
      "ApiKeyEnvironmentVariable": "OPENROUTER_API_KEY"
    }
  }
}

ProviderClientFactory is where concrete SDK construction belongs. Every factory method returns IChatClient, so provider-specific types stop at this boundary.

public static IChatClient CreateOllama(string endpoint, string model)
{
    IChatClient client =
        new OllamaApiClient(new Uri(endpoint), model);

    return AddCommonPipeline(client);
}

public static IChatClient CreateOpenAI(string apiKey, string model)
{
    IChatClient client =
        new OpenAI.Chat.ChatClient(model, apiKey)
            .AsIChatClient();

    return AddCommonPipeline(client);
}

private static IChatClient AddCommonPipeline(IChatClient client) =>
    new ChatClientBuilder(client)
        .UseFunctionInvocation()
        .Build();

public static IChatClient CreateOpenAICompatible(
    string endpoint,
    string apiKey,
    string model)
{
    OpenAIClientOptions options =
        new() { Endpoint = new Uri(endpoint) };

    OpenAIClient client =
        new(
            new System.ClientModel.ApiKeyCredential(apiKey),
            options);

    return AddCommonPipeline(
        client.GetChatClient(model).AsIChatClient());
}

public static IChatClient CreateAzureOpenAIWithDefaultCredential(
    string endpoint,
    string deployment)
{
    AzureOpenAIClient client =
        new(new Uri(endpoint), new DefaultAzureCredential());

    return AddCommonPipeline(
        client.GetChatClient(deployment).AsIChatClient());
}

The OpenRouter path uses the OpenAI SDK with a configurable compatible endpoint, then converts its chat client with AsIChatClient(). Azure OpenAI follows the same abstraction boundary after creating its provider-specific client. The common function-invocation pipeline is added once for every provider.

OllamaApiClient ───────────────────────────────┐
                                               │
OpenAI ChatClient ── AsIChatClient() ─────────┤
                                               ├──► IChatClient
OpenAI-compatible client ─ AsIChatClient() ───┤       │
                                               │       ▼
AzureOpenAIClient ─ ChatClient ─ AsIChatClient()┘  UseFunctionInvocation()
                                                       │
                                                       ▼
                                                  gateway client

The registry is both a routing mechanism and a trust boundary. A caller asks for ollama, openai, openrouter, or azure-openai; the request never supplies a raw endpoint or credential.

public sealed record ProviderCapabilities(
    bool Streaming,
    bool Tools,
    bool ProviderManagedConversation);

public sealed record ProviderDescriptor(
    string Name,
    IChatClient Client,
    ProviderCapabilities Capabilities);

public ProviderDescriptor Resolve(string? requestedProvider = null)
{
    string selectedProvider =
        string.IsNullOrWhiteSpace(requestedProvider)
            ? DefaultProvider
            : requestedProvider.Trim();

    if (!_allowProviderSelection &&
        !selectedProvider.Equals(
            DefaultProvider,
            StringComparison.OrdinalIgnoreCase))
    {
        throw new InvalidOperationException(
            $"Provider selection is disabled. The gateway uses '{DefaultProvider}'.");
    }

    if (!_providers.TryGetValue(selectedProvider, out ProviderDescriptor? provider))
    {
        throw new KeyNotFoundException(
            $"Provider '{selectedProvider}' is not registered.");
    }

    return provider;
}

The sample manages conversation history itself instead of relying on provider-managed threads. A new conversation receives a random ID and records the provider name that created it. Reusing the ID with another provider is rejected.

if (!existing.ProviderName.Equals(
        providerName.Trim(),
        StringComparison.OrdinalIgnoreCase))
{
    throw new InvalidOperationException(
        $"Conversation '{normalizedId}' is bound to provider " +
        $"'{existing.ProviderName}' and cannot be continued with " +
        $"'{providerName}'.");
}

return existing;

internal void AppendMessages(
    IEnumerable<ChatMessage> messages,
    int maxMessages)
{
    lock (_sync)
    {
        _messages.AddRange(messages);

        int removeCount = _messages.Count - maxMessages;
        if (removeCount > 0)
        {
            _messages.RemoveRange(0, removeCount);
        }
    }
}

Each conversation also owns a SemaphoreSlim. The gateway waits on that gate before mutating history, preventing two simultaneous turns from interleaving state in the same conversation.

ChatGateway validates input, resolves the registered provider, obtains the provider-bound conversation, builds a common message list, and invokes the selected IChatClient. The orchestration code does not contain OpenAI, Azure OpenAI, or Ollama request types.

public sealed record ChatRequest(
    string Message,
    string? Provider = null,
    string? ConversationId = null,
    bool UseTools = false);

ProviderDescriptor provider =
    ResolveProviderForRequest(request);

ConversationState conversation =
    _conversations.GetOrCreate(
        request.ConversationId,
        provider.Name);

ChatMessage userMessage =
    new(ChatRole.User, request.Message.Trim());

List<ChatMessage> messages =
    [.. conversation.SnapshotMessages(), userMessage];

ChatResponse response =
    await provider.Client.GetResponseAsync(
        messages,
        CreateOptions(request, provider),
        cancellationToken);

conversation.AppendMessages(
    [userMessage, .. response.Messages],
    _conversations.MaxMessages);

The gateway also limits message length using Gateway:MaxInputCharacters. That is intentionally simple, but it demonstrates that input controls belong in the application rather than being delegated to an AI provider.

The sample exposes one deterministic, side-effect-free function: convert_temperature. It is intentionally boring. The goal is to verify the Microsoft.Extensions.AI function pipeline without mixing the tutorial with a write-capable business action.

_temperatureTool = AIFunctionFactory.Create(
    (Func<double, string, string, double>)ConvertTemperatureWithLogging,
    name: "convert_temperature",
    description:
        "Convert a temperature value between Celsius, Fahrenheit and Kelvin.");

private ChatOptions? CreateOptions(
    ChatRequest request,
    ProviderDescriptor provider)
{
    if (!request.UseTools)
    {
        return null;
    }

    if (!provider.Capabilities.Tools)
    {
        throw new InvalidOperationException(
            $"Provider '{provider.Name}' is not configured for tools.");
    }

    return new ChatOptions
    {
        Tools = [_temperatureTool]
    };
}

The provider clients were already wrapped with UseFunctionInvocation(). When a capable model returns a structured function request, the function-invoking client can execute the corresponding AIFunction, send the result back to the model, and continue until a normal response is produced.

$toolBody = @{
    message = "Use the convert_temperature tool to convert 100 Celsius to Fahrenheit. Tell me the numeric result."
    provider = "ollama"
    useTools = $true
} | ConvertTo-Json

Invoke-RestMethod `
    -Uri "http://localhost:5073/api/chat" `
    -Method Post `
    -ContentType "application/json" `
    -Body $toolBody |
    ConvertTo-Json -Depth 5

The HTTP surface is intentionally small. The provider list lets clients discover what is enabled; chat and streaming use the same ChatRequest contract.

app.MapGet("/healthz", () =>
    Results.Ok(new { status = "ok" }));

app.MapGet(
    "/api/providers",
    (ProviderRegistry providers) =>
        Results.Ok(
            providers.Providers
                .OrderBy(p => p.Name)
                .Select(p => new
                {
                    name = p.Name,
                    isDefault = p.Name.Equals(
                        providers.DefaultProvider,
                        StringComparison.OrdinalIgnoreCase),
                    p.Capabilities
                })));

app.MapPost("/api/chat", HandleChatAsync);
app.MapPost("/api/chat/stream", HandleStreamAsync);

{
  "provider": "ollama",
  "conversationId": null,
  "message": "Explain IChatClient in one paragraph.",
  "useTools": false
}

Known client-side errors such as an unknown provider, invalid input, or an invalid conversation/provider combination are translated into HTTP 400 Bad Request. Provider/service failures are not silently rewritten as client errors.

For streaming, the gateway first checks the registered capability metadata, then calls GetStreamingResponseAsync. The HTTP endpoint forwards typed Server-Sent Events and flushes after every event.

context.Response.ContentType = "text/event-stream";
context.Response.Headers["Cache-Control"] = "no-cache";
context.Response.Headers["X-Accel-Buffering"] = "no";

await foreach (
    ChatStreamEvent item
    in gateway.StreamAsync(request, cancellationToken))
{
    string payload = JsonSerializer.Serialize(item);

    await context.Response.WriteAsync(
        $"event: {item.Type}\ndata: {payload}\n\n",
        cancellationToken);

    await context.Response.Body.FlushAsync(cancellationToken);
}

event: meta
data: { conversationId, provider }

event: delta
data: { text: "STREAM" }

event: delta
data: { text: "_OK" }

event: done
data: { conversationId, provider }

The meta event identifies the conversation and provider, each delta carries incremental text, and done marks completion. The gateway also collects the updates and converts them back to a ChatResponse before appending the assistant turn to conversation history, so a streamed response can participate in the next turn.

$streamBody = @{
    message = "Reply with exactly STREAM_OK"
    provider = "ollama"
    useTools = $false
} | ConvertTo-Json

Invoke-WebRequest `
    -Uri "http://localhost:5073/api/chat/stream" `
    -Method Post `
    -ContentType "application/json" `
    -Body $streamBody `
    -UseBasicParsing |
    Select-Object -ExpandProperty Content

If a client error happens after the stream has started, the endpoint emits an error event instead of trying to replace the response with ordinary JSON.

The companion sample treats live-model behavior and deterministic application behavior as separate verification layers. CI can therefore prove the gateway's routing, state, validation, factory construction, streaming mechanics, and tool code without paying for or depending on a cloud model.

dotnet restore ProviderAgnosticChatGateway.slnx

dotnet build   ProviderAgnosticChatGateway.slnx   --configuration Release   --no-restore

dotnet test   ProviderAgnosticChatGateway.slnx   --configuration Release   --no-build

The test suite covers provider selection and rejection, capability metadata, provider factories, bounded conversation state, provider binding, request validation, fake IChatClient behavior, streaming, and the deterministic temperature function.

The checked-in default configuration expects Ollama at http://localhost:11434 and uses qwen3.8:27b. If you use another locally installed model, change the trusted server configuration before starting the gateway.

dotnet run --project `
  .\src\ProviderAgnosticChatGateway\ProviderAgnosticChatGateway.csproj

The verified launch listened on http://localhost:5073. Your port can differ if your launch settings or environment differ.

Invoke-RestMethod http://localhost:5073/api/providers |
  ConvertTo-Json -Depth 5

$body = @{
    message = "Reply with exactly GATEWAY_OK"
    provider = "ollama"
    useTools = $false
} | ConvertTo-Json

Invoke-RestMethod `
    -Uri "http://localhost:5073/api/chat" `
    -Method Post `
    -ContentType "application/json" `
    -Body $body |
    ConvertTo-Json -Depth 5

What the Verified Run Proved

On September 29, 2026, the published sample was run locally against Ollama with qwen3.8:27b. The verification recorded observable gateway behavior rather than assuming that a successful build implied live-model behavior.

Provider Verification Matrix

Do not turn “the factory compiles” into “every provider was tested live.” The companion sample records three different evidence levels.

Security and Trust Boundary

The provider registry is not just a convenience abstraction. It prevents the public request from becoming a generic model-proxy configuration surface.

HTTP request may choose                 Trusted server configuration owns
──────────────────────                  ─────────────────────────────────
provider = "ollama"                     enabled providers
conversationId                          endpoint URLs
message                                 model / deployment names
useTools                                API-key environment variable names
                                        credentials
                                        capability declarations

• Never accept arbitrary provider endpoints from the request body. Otherwise a gateway can become an unintended outbound proxy.
• Never accept raw provider credentials from ordinary chat requests. Secrets belong in server-side secret stores or environment configuration.
• Validate input size before model invocation. This sample enforces a maximum input-character count.
• Keep tools opt-in and capability-aware. A request with useTools=true is rejected if the selected provider is not configured for tools.
• Treat model output as untrusted application data. Encode it before rendering into HTML and do not treat generated text as authorization.

Microsoft's IChatClient documentation similarly places prompt-injection, message-size, and related safeguards on the application unless a concrete implementation explicitly documents otherwise.

Production Considerations

The sample is intentionally a reference gateway, not a finished multi-tenant AI platform. Its boundaries make later production work easier to add without rewriting the basic provider-neutral path.

tutorial gateway
    │
    ├── in-memory conversation state
    ├── configuration-based provider registry
    ├── one deterministic tool
    └── basic application logging
    │
    ▼
production evolution
    │
    ├── authentication + authorization
    ├── rate limits + quotas
    ├── durable / distributed conversation store
    ├── provider health + retry policy
    ├── telemetry + redaction
    ├── tool authorization + audit
    └── model/provider evaluation

• Authentication and authorization: protect provider choice, expensive models, conversation access, and tool access per user or workload.
• Conversation lifecycle: add expiration, ownership, quotas, and a distributed store if multiple application instances must share state.
• Resilience: provider retries, timeouts, circuit breakers, and failover deserve explicit policy. This sample deliberately does not fail over automatically.
• Observability: Microsoft.Extensions.AI supports chat-client pipelines that can host telemetry and custom middleware. Add redaction rules before logging prompts, responses, or tool arguments.
• Capability evaluation: validate the exact provider/model pair instead of assuming a provider-wide capability.
• Tool governance: the tutorial uses a deterministic read-only calculation. Real tools with external side effects need stronger authorization, input validation, audit, and often human approval.

Common Problems and What They Mean

The app starts with “No AI providers are enabled”

At least one provider must have Enabled=true. The checked-in sample enables Ollama. If you disable it, enable and fully configure another provider before starting the application.

The default provider is configured but not registered

The application intentionally resolves the default provider during startup. A misspelled or disabled default therefore fails fast instead of waiting for the first request.

A cloud provider says its environment variable is missing

That provider was enabled, but the configured API-key environment variable was not present. Either provide the secret through the process environment/secret store or disable the provider. Do not paste the key into appsettings.json.

An existing conversation cannot continue with another provider

That is deliberate. The conversation is bound to the provider that created it. Start a new conversation when switching providers rather than silently replaying history against a different backend.

useTools=true is rejected

The selected provider's registered capability metadata does not allow tools. If you change that metadata, separately verify that the exact model actually supports the tool-calling path.

The streaming request produces several small chunks

That is expected. SSE forwards incremental model output. Consumers should concatenate delta text in order and use the done event as the completion signal.

The Ollama model is not installed

Either pull the configured model or change Providers:Ollama:Model to one already installed. The deterministic tests do not require Ollama at all.

A live response does not match the tutorial word for word

Natural-language model output is not deterministic. Verify transport behavior, provider selection, conversation IDs, SSE framing, function logs, and HTTP status codes rather than expecting identical prose.

Frequently Asked Questions

Official Microsoft References

The Microsoft.Extensions.AI concepts used on this page are cross-checked against Microsoft Learn and the public API documentation. The package versions used by the companion sample are pinned separately in the repository.

1. Microsoft Learn: Microsoft.Extensions.AI libraries
2. Microsoft Learn: Use the IChatClient interface
3. Microsoft Learn: AI tool calling in .NET
4. API reference: IChatClient
5. API reference: AIFunctionFactory.Create
6. API reference: FunctionInvokingChatClient
7. API reference: OpenAI ChatClient to IChatClient adapter

Verified dotnet-guide.com Companion Sample

The complete implementation is maintained in the dotnet-guide.com GitHub tutorial repository. It includes the application source, deterministic tests, architecture notes, provider verification matrix, captured live-run evidence, and machine-readable environment record.