Optimizing Performance with Yield and Deferred Execution in C#

Why Deferred Execution Matters

The yield keyword transforms how you work with sequences in C#. Instead of building entire collections in memory, you generate elements one at a time as they're needed. This technique, called deferred execution or lazy evaluation, can dramatically reduce memory usage and improve performance.

You'll encounter situations where loading all data upfront wastes resources. Maybe you're processing a million database records but only need the first 100. Perhaps you're filtering log files that don't fit in memory. The yield keyword handles these scenarios elegantly by producing values on demand.

Understanding yield helps you write more efficient code, implement custom LINQ operators, and handle large datasets without running out of memory.

Getting Started with Yield Return

The yield return statement lets you return elements one at a time from a method that returns IEnumerable or IEnumerator. The method maintains its state between calls, resuming where it left off each time the caller requests another element.

Basic yield return example
public static IEnumerable<int> GetNumbers()
{
    Console.WriteLine("Generating 1");
    yield return 1;

    Console.WriteLine("Generating 2");
    yield return 2;

    Console.WriteLine("Generating 3");
    yield return 3;
}

// Using the iterator
foreach (var number in GetNumbers())
{
    Console.WriteLine($"Received: {number}");
}

// Output:
// Generating 1
// Received: 1
// Generating 2
// Received: 2
// Generating 3
// Received: 3

Notice how the output interleaves. The method doesn't generate all numbers upfront. It produces one, pauses, waits for the caller to request the next one, then continues. This is deferred execution in action.

Memory Benefits Over Regular Collections

Compare yield with returning a complete list. The difference becomes obvious when working with large datasets or expensive operations.

Traditional approach using List
public static List<int> GetSquaresWithList(int count)
{
    var results = new List<int>();

    for (int i = 1; i <= count; i++)
    {
        results.Add(i * i);
    }

    return results; // All values computed and stored
}

// Caller receives complete list
var squares = GetSquaresWithList(1000000);
Console.WriteLine(squares.First()); // List holds 1 million integers
Memory-efficient approach with yield
public static IEnumerable<int> GetSquaresWithYield(int count)
{
    for (int i = 1; i <= count; i++)
    {
        yield return i * i;
    }
}

// Only computes values as needed
var squares = GetSquaresWithYield(1000000);
Console.WriteLine(squares.First()); // Computes only first square

// Take first 10 without computing remaining 999,990
var firstTen = squares.Take(10).ToList();

The yield version computes values on demand. When you call First(), it generates only the first element. Take(10) processes just ten squares instead of a million. This saves both time and memory.

Controlling Iteration with Yield Break

The yield break statement terminates iteration early. It's useful when you've met a condition that means no more elements should be produced.

Using yield break for early termination
public static IEnumerable<string> ReadLinesUntilEmpty(string filePath)
{
    foreach (var line in File.ReadLines(filePath))
    {
        if (string.IsNullOrWhiteSpace(line))
        {
            yield break; // Stop when we hit empty line
        }

        yield return line;
    }
}

// Usage
foreach (var line in ReadLinesUntilEmpty("data.txt"))
{
    Console.WriteLine(line); // Processes only until first empty line
}

Once yield break executes, the iteration stops completely. No more elements will be generated, even if the method contains code that would produce additional values.

Practical Patterns and Use Cases

Yield shines in real-world scenarios where you work with sequences that might be large, infinite, or expensive to compute.

Filtering large datasets
public static IEnumerable<Order> GetLargeOrders(string databaseConnection)
{
    using var connection = new SqlConnection(databaseConnection);
    connection.Open();

    using var command = new SqlCommand("SELECT * FROM Orders", connection);
    using var reader = command.ExecuteReader();

    while (reader.Read())
    {
        var order = new Order
        {
            Id = reader.GetInt32(0),
            Total = reader.GetDecimal(1),
            CustomerName = reader.GetString(2)
        };

        if (order.Total > 1000)
        {
            yield return order; // Stream large orders without loading all
        }
    }
}

// Caller can process orders one by one
foreach (var order in GetLargeOrders(connString).Take(20))
{
    ProcessOrder(order); // Handles first 20 without loading entire table
}
Generating infinite sequences
public static IEnumerable<int> GetFibonacciNumbers()
{
    int current = 0, next = 1;

    while (true) // Infinite loop is safe with yield
    {
        yield return current;

        int temp = current + next;
        current = next;
        next = temp;
    }
}

// Take only what you need from infinite sequence
var firstTenFibonacci = GetFibonacciNumbers().Take(10).ToList();

foreach (var num in firstTenFibonacci)
{
    Console.WriteLine(num);
}

Infinite sequences work because yield generates values lazily. You'll never hit the end of the while loop since you control how many elements to take from the sequence.

Building Custom LINQ-Like Operators

Yield makes it easy to create your own extension methods that work like LINQ operators. These custom operators can chain together and maintain deferred execution.

Custom filtering operator
public static class CustomEnumerableExtensions
{
    public static IEnumerable<T> WhereNotNull<T>(this IEnumerable<T> source)
        where T : class
    {
        foreach (var item in source)
        {
            if (item != null)
            {
                yield return item;
            }
        }
    }

    public static IEnumerable<T> TakeBetween<T>(
        this IEnumerable<T> source,
        int skip,
        int take)
    {
        int count = 0;

        foreach (var item in source)
        {
            if (count >= skip && count < skip + take)
            {
                yield return item;
            }

            count++;

            if (count >= skip + take)
            {
                yield break;
            }
        }
    }
}

// Chain custom operators with LINQ
var results = GetAllOrders()
    .WhereNotNull()
    .Where(o => o.Total > 100)
    .TakeBetween(10, 5)
    .Select(o => o.CustomerName);

Your custom operators participate in deferred execution just like built-in LINQ methods. Nothing executes until you enumerate the sequence with foreach or ToList().

When to Use Yield and When to Avoid It

Yield isn't always the right choice. Understanding when to use it helps you make smart performance decisions.

Use yield when: You're working with large datasets that don't fit in memory. The caller might not need all elements. You're implementing streaming operations. The sequence could be infinite. Computing all values upfront would waste time and resources.

Avoid yield when: The sequence is small and accessed multiple times. You need random access to elements. The collection already exists in memory. You're implementing methods that modify the source sequence. The overhead of the state machine exceeds the memory savings.

Multiple enumerations of yield iterators recompute values each time. If you'll iterate the same sequence repeatedly, materialize it with ToList() after the first iteration.

Frequently Asked Questions (FAQ)

What's the difference between yield return and regular return?

Yield return produces one element at a time and maintains the method's state between calls. Regular return exits the method immediately and returns all elements at once. With yield, you generate values lazily as they're requested, which saves memory when working with large datasets.

When should I use yield instead of returning a collection?

Use yield when you're working with large sequences, performing expensive computations, or when the caller might not need all elements. It's perfect for streaming data, reading large files line by line, or implementing custom LINQ-like operators where deferred execution provides performance benefits.

Can you use yield return inside a try-catch block?

You can use yield return inside try-finally blocks, but not inside catch blocks. The compiler prevents yield return in catch because the state machine generated by yield can't properly handle exception filters. You can still use try-catch around code that doesn't contain yield statements.

Does yield break stop all iterations permanently?

Yield break stops the current iteration sequence immediately, signaling that no more elements will be produced. It's similar to return in a regular method but specifically for iterator methods. The iteration ends cleanly, and the foreach loop or enumerator sees this as reaching the end of the sequence.

How does yield affect performance compared to materializing collections?

Yield improves memory efficiency by generating elements on demand rather than creating entire collections upfront. This reduces peak memory usage significantly with large datasets. However, each element access has slight overhead from the state machine. For small collections accessed multiple times, materializing might be faster.

Back to Articles