In a distributed system, failures are inevitable. Microservices, which rely on network calls to communicate, are especially susceptible to transient faults, latency spikes, and service unavailability. The key to building reliable systems isn’t avoiding failure—it’s designing for resilience.

In this tutorial, we’ll explore how to build resilient microservices in .NET 8 using Polly, a powerful .NET library that provides resilience and transient-fault-handling capabilities.

🚧 Why Resilience Matters

Microservices operate over unreliable networks. A temporary DNS resolution failure, a delayed database call, or a remote service restart can result in cascading failures if not handled gracefully.

Building resilience into services helps ensure:

• Higher system uptime
• Better user experience during failures
• More graceful degradation
• Lower operational risk

🧰 Introducing Polly

Polly is a .NET resilience library that allows you to easily implement:

• Retry
• Circuit Breaker
• Timeout
• Fallback
• Bulkhead Isolation

With Polly, you can define policies that automatically handle these failure modes without polluting your business logic.

📦 Project Setup

Step 1: Create the Service

dotnet new webapi -n ResilientMicroservice
cd ResilientMicroservice

Step 2: Install Required Packages

dotnet add package Microsoft.Extensions.Http.Polly

This adds Polly support to HttpClientFactory.

🔁 Retry Policy

Retry is the most basic and useful policy. It retries a failed request after a specified delay.

builder.Services.AddHttpClient("ResilientClient")
    .AddPolicyHandler(Policy
        .Handle<HttpRequestException>()
        .WaitAndRetryAsync(3, retryAttempt =>
            TimeSpan.FromSeconds(Math.Pow(2, retryAttempt))));

This sets up an exponential backoff retry strategy (1s, 2s, 4s).

🔌 Using the Client

app.MapGet("/api/data", async (IHttpClientFactory factory) =>
{
    var client = factory.CreateClient("ResilientClient");
    var response = await client.GetAsync("https://some-external-api.com/data");
    return response.IsSuccessStatusCode
        ? Results.Ok(await response.Content.ReadAsStringAsync())
        : Results.Problem("Failed to fetch data");
});

⚡ Timeout Policy

Timeout prevents a call from hanging indefinitely.

.AddPolicyHandler(Policy.TimeoutAsync<HttpResponseMessage>(3));

This ensures the HTTP call fails if it takes longer than 3 seconds.

🧯 Fallback Policy

Fallback allows your app to respond with a default value when everything else fails.

.AddPolicyHandler(Policy<HttpResponseMessage>
    .Handle<HttpRequestException>()
    .FallbackAsync(new HttpResponseMessage(HttpStatusCode.OK)
    {
        Content = new StringContent("Fallback content")
    }));

This helps avoid crashing or confusing users with generic error messages.

💡 Circuit Breaker Policy

Circuit Breaker stops making requests when failures reach a threshold, giving time for the remote service to recover.

.AddPolicyHandler(Policy
    .Handle<HttpRequestException>()
    .CircuitBreakerAsync(2, TimeSpan.FromSeconds(30)));

This opens the circuit after 2 failures and blocks further calls for 30 seconds.

🧱 Bulkhead Isolation

Bulkhead prevents one part of your system from exhausting all resources.

.AddPolicyHandler(Policy.BulkheadAsync<HttpResponseMessage>(
    maxParallelization: 4,
    maxQueuingActions: 8));

This allows only 4 concurrent executions and 8 queued requests, rejecting anything beyond.

🔀 Combining Policies

Use PolicyWrap to compose multiple policies.

var retryPolicy = Policy
    .Handle<HttpRequestException>()
    .WaitAndRetryAsync(3, _ => TimeSpan.FromSeconds(2));

var timeoutPolicy = Policy
    .TimeoutAsync<HttpResponseMessage>(5);

builder.Services.AddHttpClient("AdvancedClient")
    .AddPolicyHandler(Policy.WrapAsync(retryPolicy, timeoutPolicy));

📈 Observability and Logging

Instrument your Polly policies using events:

.WaitAndRetryAsync(3, retryAttempt =>
{
    Console.WriteLine($"Retrying... attempt {retryAttempt}");
    return TimeSpan.FromSeconds(1);
});

Or integrate with OpenTelemetry and Serilog to monitor success/failure rates.

🧪 Testing Resilience Locally

Simulate service failures by:

• Calling a test endpoint that randomly fails
• Throttling bandwidth with tc or Docker
• Killing the external service mid-request

Check logs and circuit breaker states to validate behavior.

🚀 Best Practices

• Use different retry policies per service type
• Combine timeout with fallback
• Avoid retries for non-transient errors (like 404)
• Monitor and tune policies based on metrics
• Avoid retry storms with jitter/backoff strategies

🧠 Final Thoughts

By combining .NET 8’s modern hosting model and Polly’s powerful policies, you can make your microservices far more resilient and fault-tolerant.

This means fewer outages, faster recovery, and happier users.

Want to go further?

• Integrate circuit metrics with dashboards
• Use policy registries for cleaner configuration
• Deploy to Kubernetes with probes for health and readiness

👉 Start building smarter services today.