IndianTechnoEra: Chapter 6: Cancellation, Timeouts & Cooperative Cancellation in Async APIs

Introduction

Modern APIs must assume:

• Clients will disconnect
• Mobile networks will drop
• Load balancers will timeout
• Users will cancel requests
• External services will hang

If your async code ignores cancellation:

• Your server will keep executing useless work
• DB connections remain occupied
• Threads stay busy
• Memory grows
• System becomes unstable under load

✅ Proper cancellation is not optional in real systems.

6.1 What Is Cooperative Cancellation?

Cancellation in .NET is:

• Not forced
• Not a thread abort
• Not a kill signal

It is cooperative:

• A CancellationToken is passed down the call chain
• Each layer checks the token
• When cancellation is requested:
  • The operation stops itself
  • Resources are cleaned up safely

This design:

• Prevents corrupted state
• Avoids half-written database rows
• Keeps system stable

6.2 Where Does CancellationToken Come From in Web APIs?

ASP.NET Core automatically creates a token for every request:

HttpContext.RequestAborted

This token is:

Triggered when:

• Client disconnects
• Reverse proxy times out
• Connection drops

Automatically passed into controller parameters if you add it:

Controller Example

[HttpGet("report")] 
public async Task<IActionResult> Generate( 
    CancellationToken cancellationToken) 
{ 
    var data = await _service.GenerateAsync(cancellationToken); 
    return Ok(data); 
}

✅ No manual wiring required.

6.3 Cancellation Flow (End-to-End Pipeline)

Proper cancellation MUST flow through every layer:

Client
  ↓
Controller (CancellationToken)
  ↓
Service (CancellationToken)
  ↓
Repository (CancellationToken)
  ↓
EF Core / SQL / HttpClient

If any layer ignores the token → cancellation becomes useless.

6.4 Cancellation with EF Core (Critical)

EF Core supports cancellation on all async operations.

✅ Correct

public async Task<List<Order>> GetOrdersAsync( 
    CancellationToken token) 
{ 
    return await _db.Orders 
        .AsNoTracking() 
        .ToListAsync(token); 
}

❌ Wrong (Cancellation is Ignored)

public async Task<List<Order>> GetOrdersAsync( 
    CancellationToken token) 
{ 
    return await _db.Orders.ToListAsync(); // token not passed ❌ 
}

⚠️ Even if the request is cancelled:

• DB query continues
• Connection stays open
• Your server wastes resources

6.5 Cancellation with HttpClient (External APIs)

✅ Correct

public async Task<PaymentStatus> GetStatusAsync( 
    string txnId, CancellationToken token) 
{ 
    var response = await _client.GetAsync( 
        $"payments/{txnId}", token); 
    response.EnsureSuccessStatusCode(); 
    return await response.Content 
        .ReadFromJsonAsync<PaymentStatus>(token); 
}

What This Gives You

If the client:

• Closes browser
• Loses network
• Times out at gateway

→ The outbound HTTP call is aborted immediately.

6.6 Internal Cancellation in Long-Running CPU Loops

For CPU-bound workloads, cancellation must be manually checked.

public async Task ProcessAsync( 
    CancellationToken token) 
{ 
    for (int i = 0; i < 10_000_000; i++) 
    { 
        token.ThrowIfCancellationRequested(); 
        DoCpuWork(i); 
        if (i % 1000 == 0) 
            await Task.Yield(); // Keeps thread responsive 
    } 
}

✅ This guarantees:

• Immediate exit on cancellation
• Stable thread pool behavior

6.7 Timeouts vs Cancellation (Very Important Difference)

Concept	Who initiates it	Purpose
Cancellation	Client / system	Stop unnecessary work
Timeout	Your API	Enforce SLA & latency bounds

You must use both together in production.

6.8 API-Level Timeouts Using CancellationTokenSource

Example: Enforcing 5s Service Limit

public async Task<IActionResult> GetReport() 
{ 
    using var cts = new CancellationTokenSource( 
        TimeSpan.FromSeconds(5)); 
    var data = await _service.GenerateAsync(cts.Token); 
    return Ok(data); 
}

✅ If operation crosses 5 seconds → automatically cancelled.

6.9 Combining Client Cancellation + Server Timeout

This is the correct production pattern:

public async Task<IActionResult> GetData( 
    CancellationToken clientToken) 
{ 
    using var timeoutCts = new CancellationTokenSource( 
        TimeSpan.FromSeconds(10)); 
    using var linkedCts = CancellationTokenSource 
        .CreateLinkedTokenSource( 
            clientToken, timeoutCts.Token); 
    var data = await _service 
        .GetAsync(linkedCts.Token); 
    return Ok(data); 
}

✅ Operation stops if:

• Client disconnects OR
• Timeout is hit

6.10 Real-Life Example: Cancelable Report Generation API

Controller

[HttpGet("large-report")] 
public async Task<IActionResult> Generate( 
    CancellationToken token) 
{ 
    var report = await _reportService 
        .GenerateAsync(token); 
    return Ok(report); 
}

Service

public async Task<ReportDto> GenerateAsync( 
    CancellationToken token) 
{ 
    var data = await _repo 
        .GetLargeDatasetAsync(token); 
    
    foreach (var row in data) 
    { 
        token.ThrowIfCancellationRequested(); 
        ProcessRow(row); 
    } 
    
    return BuildReport(data); 
}

Repository

public async Task<List<ReportRow>> GetLargeDatasetAsync( 
    CancellationToken token) 
{ 
    return await _db.ReportRows 
        .AsNoTracking() 
        .ToListAsync(token); 
}

✅ If user closes browser at 30%:

• DB query is cancelled
• Processing stops
• Memory is released
• CPU is freed

6.11 Cancellation in Background Tasks (Very Dangerous Area)

❌ Wrong (Fire & Forget Without Token)

_taskService.StartAsync(); // No token ❌ 
return Ok();

Problems:

• No way to cancel
• Task may run for hours
• App shutdown may corrupt state

✅ Correct Pattern (With HostedService)

public class EmailWorker : BackgroundService 
{ 
    protected override async Task ExecuteAsync( 
        CancellationToken stoppingToken) 
    { 
        while (!stoppingToken.IsCancellationRequested) 
        { 
            await ProcessQueueAsync(stoppingToken); 
        } 
    } 
}

✅ When application shuts down:

• stoppingToken is triggered
• Worker exits gracefully

6.12 What Happens Internally on Cancellation?

When cancellation is requested:

1. The CancellationTokenSource switches to cancelled state
2. All linked tokens are signaled
3. Awaiting async operations:
   • Throw OperationCanceledException
4. ASP.NET Core:
   • Aborts the HTTP pipeline
   • Stops writing the response
5. Logging pipeline records a canceled request

✅ This is normal, expected behavior—not a crash.

6.13 Handling OperationCanceledException Properly

❌ Wrong (Treating as error)

catch (Exception ex) 
{ 
    _logger.LogError(ex, "Failed"); 
}

This pollutes your logs.

✅ Correct Handling

catch (OperationCanceledException) 
{ 
    _logger.LogInformation("Request was cancelled by the client."); 
}

✅ Cancellation is not a failure.

6.14 Cancellation in Parallel Operations

When using Task.WhenAll, cancellation automatically propagates:

var t1 = _api1.GetAsync(token); 
var t2 = _api2.GetAsync(token); 
var t3 = _api3.GetAsync(token); 

await Task.WhenAll(t1, t2, t3);

If token is cancelled:

• All unfinished calls are aborted
• Task.WhenAll throws OperationCanceledException

6.15 Common Cancellation Mistakes (Production Killers)

• ❌ Not passing token to DB calls
• ❌ Not passing token to HttpClient
• ❌ Swallowing OperationCanceledException
• ❌ Fire-and-forget without shutdown token
• ❌ Infinite loops without IsCancellationRequested
• ❌ Cancelling by throwing random exceptions
• ❌ Blocking threads so cancellation is delayed

6.16 Performance & Stability Benefits of Proper Cancellation

• ✅ Frees DB connections immediately
• ✅ Frees thread pool threads
• ✅ Prevents runaway background jobs
• ✅ Avoids memory pressure
• ✅ Prevents cascading failures
• ✅ Improves tail latency (P95, P99)
• ✅ Makes autoscaling accurate
• ✅ Greatly reduces server cost

6.17 Best-Practices Checklist (Save This)

• ✅ Always accept CancellationToken at controller level
• ✅ Always propagate it to:
  • EF Core
  • Dapper
  • HttpClient
  • File I/O
• ✅ Always enforce server-side timeouts
• ✅ Never block a thread in cancelable code
• ✅ Always log cancellation separately from errors
• ✅ Always use BackgroundService for long jobs
• ✅ Always respect stoppingToken on shutdown

6.18 CancellationTokenSource vs CancellationToken - Deep Dive

The Critical Distinction

// CancellationTokenSource - The "controller"
// Creates and manages cancellation
var cts = new CancellationTokenSource();

// CancellationToken - The "signal"
// Read-only token passed to async operations
CancellationToken token = cts.Token;

Relationship: 1-to-many - One source can create many tokens.

// One source, multiple consumers
var cts = new CancellationTokenSource();
var token1 = cts.Token;
var token2 = cts.Token;

// Cancelling the source cancels ALL tokens
cts.Cancel();
// token1.IsCancellationRequested == true
// token2.IsCancellationRequested == true

Memory Management and Disposal

// ALWAYS dispose CancellationTokenSource
// Memory leak risk if not disposed
public async Task ProcessAsync()
{
    using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
    {
        await DoWorkAsync(cts.Token);
    } // cts is disposed automatically here
    
    // Without using: register callback for cleanup
    var cts2 = new CancellationTokenSource();
    cts2.Token.Register(() =>
    {
        // Cleanup resources when cancelled
        _logger.LogInformation("Operation cancelled");
        cts2.Dispose(); // Manual disposal
    });
}

Why disposal matters: CancellationTokenSource holds:

• Callback lists (memory leak if not cleaned)
• Timer resources (for timeouts)
• Linked token references

6.19 Linked Cancellation Tokens - Complex Orchestration

Scenario: Multiple Cancellation Conditions

// Multiple cancellation sources
public async Task ProcessWithMultipleCancellationsAsync(
    CancellationToken userToken,
    TimeSpan maxDuration,
    int maxRetries)
{
    // Timeout-based cancellation
    using var timeoutCts = new CancellationTokenSource(maxDuration);
    
    // Retry-count based cancellation
    using var retryCts = new CancellationTokenSource();
    
    // Link ALL tokens together
    using var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(
        userToken,
        timeoutCts.Token,
        retryCts.Token);
    
    try
    {
        await ProcessWithRetriesAsync(linkedCts.Token, maxRetries);
    }
    catch (OperationCanceledException) when (timeoutCts.IsCancellationRequested)
    {
        _logger.LogWarning("Operation timed out after {Duration}", maxDuration);
    }
    catch (OperationCanceledException) when (userToken.IsCancellationRequested)
    {
        _logger.LogInformation("User cancelled the operation");
    }
}

Detecting Which Source Cancelled

public async Task DetectCancellationSourceAsync()
{
    var userCts = new CancellationTokenSource();
    var timeoutCts = new CancellationTokenSource(TimeSpan.FromSeconds(30));
    
    using var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(
        userCts.Token,
        timeoutCts.Token);
    
    try
    {
        await LongRunningOperationAsync(linkedCts.Token);
    }
    catch (OperationCanceledException)
    {
        // Check which source triggered cancellation
        if (userCts.IsCancellationRequested)
            _logger.LogInformation("Cancelled by user");
        else if (timeoutCts.IsCancellationRequested)
            _logger.LogInformation("Timed out");
        else
            _logger.LogInformation("Unknown cancellation source");
    }
}

6.20 Cancellation Callbacks for Resource Cleanup

Registering Cleanup Actions

public async Task ProcessWithCleanupAsync(CancellationToken token)
{
    // Resources that need cleanup
    var fileStream = File.OpenWrite("temp.txt");
    var dbConnection = new SqlConnection(_connectionString);
    
    // Register cleanup callbacks BEFORE starting operation
    using var cleanupRegistration = token.Register(() =>
    {
        // This runs when cancellation is requested
        try
        {
            fileStream?.Dispose();
            dbConnection?.Close();
            File.Delete("temp.txt");
            _logger.LogInformation("Resources cleaned up due to cancellation");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error during cancellation cleanup");
        }
    });
    
    try
    {
        await dbConnection.OpenAsync(token);
        await ProcessDataAsync(fileStream, dbConnection, token);
    }
    finally
    {
        // Ensure cleanup even if no cancellation
        cleanupRegistration.Dispose();
        fileStream?.Dispose();
        dbConnection?.Dispose();
    }
}

Callback Execution Guarantees

// Callbacks execute:
// 1. Synchronously on the thread that calls Cancel()
// 2. In LIFO (Last-In-First-Out) order
// 3. Exception in callback does NOT stop other callbacks

var cts = new CancellationTokenSource();

cts.Token.Register(() => Console.WriteLine("Callback 1"));
cts.Token.Register(() => 
{
    Console.WriteLine("Callback 2");
    throw new InvalidOperationException("Callback failed");
});
cts.Token.Register(() => Console.WriteLine("Callback 3"));

// Output: Callback 3, Callback 2 (throws), Callback 1
// All callbacks execute despite exception in Callback 2

6.21 Cancellation in Distributed Systems

Propagating Cancellation Across Service Boundaries

// Service A (Initiator)
[HttpPost("process")]
public async Task<IActionResult> ProcessOrder(
    [FromBody] OrderRequest request,
    CancellationToken token)
{
    // Pass cancellation token to downstream service via header
    var httpRequest = new HttpRequestMessage(HttpMethod.Post, 
        "https://service-b/api/orders")
    {
        Content = JsonContent.Create(request)
    };
    
    // Custom header to propagate cancellation timeout
    httpRequest.Headers.Add("X-Cancellation-Timeout", 
        "PT30S"); // ISO 8601 duration
    
    var response = await _httpClient.SendAsync(httpRequest, token);
    return Ok(await response.Content.ReadFromJsonAsync<OrderResponse>());
}

// Service B (Receiver)
[HttpPost("orders")]
public async Task<IActionResult> CreateOrder(
    [FromBody] OrderRequest request)
{
    // Extract timeout from header
    var timeoutHeader = Request.Headers["X-Cancellation-Timeout"].FirstOrDefault();
    var timeout = timeoutHeader != null ? 
        XmlConvert.ToTimeSpan(timeoutHeader) : 
        TimeSpan.FromSeconds(30);
    
    using var cts = new CancellationTokenSource(timeout);
    await _orderService.ProcessAsync(request, cts.Token);
    
    return Ok();
}

Distributed Tracing with Cancellation

public async Task DistributedOperationAsync(CancellationToken token)
{
    using var activity = _activitySource.StartActivity("DistributedProcess");
    
    // Add cancellation info to trace
    activity?.AddTag("cancellation.requested", 
        token.IsCancellationRequested.ToString());
    
    // Link cancellation to trace
    token.Register(() =>
    {
        activity?.AddTag("cancellation.triggered", DateTime.UtcNow.ToString("o"));
        activity?.SetStatus(ActivityStatusCode.Error, "Cancelled");
    });
    
    try
    {
        await CallServiceAAsync(token);
        await CallServiceBAsync(token);
    }
    catch (OperationCanceledException)
    {
        activity?.SetStatus(ActivityStatusCode.Error, "Operation cancelled");
        throw;
    }
}

6.22 Cancellation in EF Core Transactions and SaveChanges

The Problem: SaveChangesAsync and Partial Commits

// DANGER: Cancellation during SaveChangesAsync
public async Task UpdateOrderAsync(
    OrderUpdateDto update, 
    CancellationToken token)
{
    using var transaction = await _db.Database.BeginTransactionAsync(token);
    
    try
    {
        var order = await _db.Orders.FindAsync(update.OrderId, token);
        order.Status = update.NewStatus;
        
        // What if cancellation happens HERE?
        await _db.SaveChangesAsync(token); // ❌ Partial save risk
        
        await transaction.CommitAsync(token);
    }
    catch (OperationCanceledException)
    {
        await transaction.RollbackAsync(CancellationToken.None);
        // Must use CancellationToken.None for rollback!
        throw;
    }
}

Safe Pattern: Atomic Operations

public async Task UpdateOrderSafelyAsync(
    OrderUpdateDto update, 
    CancellationToken token)
{
    // Use separate token for transaction commit/rollback
    using var transaction = await _db.Database.BeginTransactionAsync(token);
    
    try
    {
        // Check cancellation BEFORE starting critical section
        token.ThrowIfCancellationRequested();
        
        var order = await _db.Orders.FindAsync(update.OrderId, token);
        order.Status = update.NewStatus;
        
        // Perform save WITHOUT cancellation token
        // This ensures all-or-nothing behavior
        await _db.SaveChangesAsync(CancellationToken.None);
        
        // Commit also without cancellation
        await transaction.CommitAsync(CancellationToken.None);
    }
    catch (Exception)
    {
        // Rollback without cancellation
        await transaction.RollbackAsync(CancellationToken.None);
        throw;
    }
}

Rule of thumb:

• ✅ Use cancellation token for: Queries, reads, FindAsync
• ❌ Avoid cancellation token for: SaveChanges, Commit, Rollback
• Transactions should be all-or-nothing
• Partial commits can corrupt data

6.23 Monitoring Cancellation Performance

Metrics to Track

public class CancellationMetrics
{
    private readonly Counter<int> _cancellationCounter;
    private readonly Histogram<double> _cancellationLatency;
    
    public CancellationMetrics(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create("MyApp.Cancellation");
        
        _cancellationCounter = meter.CreateCounter<int>(
            "cancellation.count",
            description: "Number of cancelled operations");
        
        _cancellationLatency = meter.CreateHistogram<double>(
            "cancellation.latency",
            unit: "ms",
            description: "Time between start and cancellation");
    }
    
    public async Task TrackOperationAsync(
        Func<CancellationToken, Task> operation,
        CancellationToken token)
    {
        var stopwatch = Stopwatch.StartNew();
        
        try
        {
            await operation(token);
        }
        catch (OperationCanceledException)
        {
            _cancellationCounter.Add(1);
            _cancellationLatency.Record(stopwatch.ElapsedMilliseconds);
            
            // Tag with reason
            var tags = new TagList
            {
                { "reason", token.IsCancellationRequested ? "client" : "timeout" }
            };
            _cancellationCounter.Add(1, tags);
            
            throw;
        }
    }
}

Alerts for Excessive Cancellation

// Monitor these key ratios:
1. **Cancellation Rate** = Cancelled requests / Total requests
   - Alert threshold: > 5% (indicates unhappy clients or timeouts too low)

2. **Cancellation Latency P95** = 95th percentile of time to cancellation
   - If high: Clients waiting too long before cancelling
   - If low: Cancelling too quickly (impatient clients)

3. **Resource Waste Ratio** = Time wasted after cancellation / Total CPU time
   - Measures how well cancellation is implemented
   - Goal: < 1% (minimal waste after cancellation)

4. **Cascading Cancellation Rate** = Downstream cancellations / Upstream cancellations
   - Measures propagation efficiency
   - Goal: ~100% (all cancellations propagate downstream)

6.24 Testing Cancellation Scenarios

Unit Testing Cancellation

[Fact]
public async Task ProcessAsync_ThrowsOperationCanceled_WhenTokenCancelled()
{
    // Arrange
    var cts = new CancellationTokenSource();
    var service = new OrderProcessingService();
    
    // Start operation
    var task = service.ProcessAsync(cts.Token);
    
    // Cancel immediately
    cts.Cancel();
    
    // Act & Assert
    await Assert.ThrowsAsync<OperationCanceledException>(() => task);
}

[Fact]
public async Task ProcessAsync_Completes_WhenTokenNotCancelled()
{
    // Arrange
    using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
    var service = new OrderProcessingService();
    
    // Act
    await service.ProcessAsync(cts.Token);
    
    // Assert
    // If we get here, operation completed before cancellation
    Assert.True(true);
}

Integration Testing with Timeouts

[Fact]
public async Task ApiEndpoint_Returns408_WhenTimeoutExceeded()
{
    // Arrange
    var factory = new WebApplicationFactory<Program>();
    var client = factory.CreateClient();
    
    // Configure short timeout
    client.Timeout = TimeSpan.FromMilliseconds(100);
    
    // Endpoint that takes 1 second (should timeout)
    
    // Act & Assert
    var exception = await Assert.ThrowsAsync<TaskCanceledException>(
        () => client.GetAsync("/api/slow-operation"));
    
    // Verify timeout occurred
    Assert.Contains("timeout", exception.Message, StringComparison.OrdinalIgnoreCase);
}

Chapter 6 Summary

• Cancellation in .NET is cooperative, not forced
• ASP.NET Core automatically provides request cancellation tokens
• Cancellation must propagate through:
  • Controller → Service → Repository → External APIs
• EF Core, HttpClient, Task APIs fully support cancellation
• Timeouts must be enforced using CancellationTokenSource
• Proper cancellation:
  • Saves CPU
  • Saves memory
  • Saves DB connections
  • Stabilizes production systems

✅ Chapter 6 (Deep Dive) is now complete.

Next chapter is the backbone of production-grade reliability:

👉 Chapter 7: Error Handling & Global Exception Handling in Async APIs

• try/catch in async
• Exception flow across await boundaries
• Custom exception types
• Global async exception middleware
• ProblemDetails responses
• Logging & correlation IDs

Would you like me to proceed with Chapter 7 (Deep Dive) now?