Advanced C# A 04 8 topics

LINQ and Iterators

A LINQ query is a recipe, not a result. It runs when you pull items out of it, and it runs again every time you pull. Most LINQ bugs and most LINQ interview questions come from that one fact.

This page assumes you can write Where, Select and OrderBy. It covers what sits under them. You will see how yield return becomes a state machine. You will write your own operator, and learn when a lambda becomes data instead of code. It ends with the operators added in .NET 6 to .NET 10 and the traps that make LINQ slow.

Contents

  1. Deferred vs immediate execution
  2. Multiple enumeration
  3. yield return, yield break and the state machine
  4. Writing a custom LINQ operator
  5. IEnumerable vs IQueryable and expression trees
  6. Query syntax vs method syntax
  7. Newer operators (.NET 6 to .NET 10)
  8. LINQ performance traps
  9. Recap

Every sample below shares one small record. It is defined once here and reused by later blocks.

/// <summary>A product with a name, a category and a price in whole dollars.</summary>
/// <example>new Product("pen", "office", 3)</example>
public sealed record Product(string Name, string Category, int Price);

/// <summary>A mutable counter that iterator samples bump so a test can count calls.</summary>
/// <example>var c = new CallCounter(); c.Count++; // c.Count is 1</example>
public sealed class CallCounter
{
    /// <summary>How many times the watched code ran. Starts at 0 (nothing ran yet).</summary>
    public int Count;
}

1. Deferred vs immediate execution

C# 3.NET 3.5

What it is

Most LINQ operators are deferred. Where, Select, Take and OrderBy return an object that remembers the source and the lambda. No item is read until someone calls foreach, ToList, Count or First. Those calls are immediate. They run the whole pipeline at once.

Deferred operators build a chain of enumerators. The consumer at the end calls MoveNext(). That call pulls one item through every stage, then stops. Nothing moves until the consumer asks.
Each MoveNext() pulls one item through the whole chain source [1, 2, 3] Where x > 1 Select x * 10 consumer foreach / ToList pull pull pull 2 2 20 The item 1 fails Where, so Where pulls again before it returns. Nothing runs at all until the consumer on the right starts pulling.
Figure A4.1 — A deferred query is a chain of enumerators driven from the consumer end.

Reading the figure. Red arrows are MoveNext() calls going upstream. Green arrows are single items coming back. Blue boxes are deferred operators. The amber box is the immediate call that drives everything. Notice that each item makes the full trip before the next one starts.

Why and when

Sample

public static class DeferredDemo
{
    /// <summary>Shows that a deferred query sees items added after it was built.</summary>
    /// <returns>The deferred count and the snapshot count after one more item is added.</returns>
    /// <example>DeferredDemo.SeesLaterItems() returns "deferred=3 snapshot=2"</example>
    public static string SeesLaterItems()
    {
        var source = new List<int> { 1, 2, 3 };
        // Build a deferred query. Nothing is read yet. "> 1" keeps 2 and 3.
        IEnumerable<int> deferred = source.Where(x => x > 1);
        // ToList runs the same filter now and copies the two matches.
        List<int> snapshot = source.Where(x => x > 1).ToList();
        // Change the source after both were built. 4 also passes "> 1".
        source.Add(4);
        return $"deferred={deferred.Count()} snapshot={snapshot.Count}";
    }

    /// <summary>Counts how often a Select lambda runs before and after enumeration.</summary>
    /// <returns>Lambda calls after building, then after two full ToList calls.</returns>
    /// <example>DeferredDemo.LambdaCalls() returns "built=0 after=6"</example>
    public static string LambdaCalls()
    {
        var counter = new CallCounter();
        // The lambda bumps the counter each time it runs on one item.
        var query = new[] { 1, 2, 3 }.Select(x => { counter.Count++; return x; });
        int built = counter.Count;
        // Two ToList calls run the pipeline twice: 3 items times 2 runs is 6 calls.
        query.ToList();
        query.ToList();
        return $"built={built} after={counter.Count}";
    }

    /// <summary>Shows that an exception inside a deferred lambda fires at enumeration.</summary>
    /// <returns>"built" if building did not throw, followed by the caught exception type.</returns>
    /// <example>DeferredDemo.LateException() returns "built,DivideByZeroException"</example>
    public static string LateException()
    {
        var log = new List<string>();
        // Dividing by x fails on the 0 item, but only when that item is pulled.
        var query = new[] { 1, 0 }.Select(x => 10 / x);
        log.Add("built");
        try { query.ToList(); }
        catch (DivideByZeroException ex) { log.Add(ex.GetType().Name); }
        return string.Join(",", log);
    }
}

Pitfalls

Interview questions

Q. Which LINQ operators run immediately?
A. Anything that returns a single value or a concrete collection. That means ToList, ToArray, ToDictionary, Count, Sum, First, Any, Max and friends. Operators that return IEnumerable<T> are deferred.
Q. Is OrderBy deferred even though it must see all items?
A. Yes. Nothing happens when you call it. On the first MoveNext() it reads and sorts the whole source, then yields items one at a time.
Say this out loud: “A LINQ query is a description. It runs when I enumerate it, once per enumeration. If I need the result twice or the source might change, I call ToList to take a snapshot.”

2. Multiple enumeration

C# 3TryGetNonEnumeratedCount: .NET 6

What it is

Each foreach over an IEnumerable<T> calls GetEnumerator() again. For a list that is cheap. For a query over a file, a database or an iterator method, the whole source runs again. Calling Count() and then foreach on the same lazy sequence does the work twice.

Why and when

Watch for it whenever a method takes an IEnumerable<T> parameter and touches it more than once. Fix it with one ToList() at the top. Or take IReadOnlyList<T> or IReadOnlyCollection<T> so callers pass something already in memory. TryGetNonEnumeratedCount (.NET 6) gives you a count only when it is free.

Sample

public static class MultiEnum
{
    /// <summary>An iterator that bumps a counter for every item it produces.</summary>
    /// <param name="n">How many items to produce: 0..n-1.</param>
    /// <param name="counter">Counts produced items across all enumerations.</param>
    /// <returns>The numbers 0 to n-1, lazily.</returns>
    /// <example>Produce(3, c) yields 0, 1, 2 and leaves c.Count at 3</example>
    public static IEnumerable<int> Produce(int n, CallCounter counter)
    {
        // i is the next item. Start at 0 so the range is 0..n-1.
        for (int i = 0; i < n; i++)
        {
            counter.Count++;
            yield return i;
        }
    }

    /// <summary>Counts then sums the same lazy sequence. The source runs twice.</summary>
    /// <param name="n">Size of the source.</param>
    /// <returns>How many items the source produced in total.</returns>
    /// <example>MultiEnum.Twice(5) returns 10</example>
    public static int Twice(int n)
    {
        var counter = new CallCounter();
        IEnumerable<int> items = Produce(n, counter);
        // Count walks every item. Sum walks them all again.
        _ = items.Count() + items.Sum();
        return counter.Count;
    }

    /// <summary>Materialises once, then counts and sums the list. The source runs once.</summary>
    /// <param name="n">Size of the source.</param>
    /// <returns>How many items the source produced in total.</returns>
    /// <example>MultiEnum.Once(5) returns 5</example>
    public static int Once(int n)
    {
        var counter = new CallCounter();
        // One ToList reads the source once. Later calls read the list.
        List<int> items = Produce(n, counter).ToList();
        _ = items.Count + items.Sum();
        return counter.Count;
    }

    /// <summary>Asks for a count without enumerating, for a list and for an iterator.</summary>
    /// <returns>Whether each source had a free count.</returns>
    /// <example>MultiEnum.FreeCount() returns "list=True iterator=False"</example>
    public static string FreeCount()
    {
        var list = new List<int> { 1, 2, 3 };
        // 3 items, but this iterator cannot know its length without running.
        IEnumerable<int> lazy = Produce(3, new CallCounter());
        bool listOk = list.TryGetNonEnumeratedCount(out _);
        bool lazyOk = lazy.TryGetNonEnumeratedCount(out _);
        return $"list={listOk} iterator={lazyOk}";
    }
}

Pitfalls

Interview questions

Q. Your method takes IEnumerable<Order>, checks Any(), then loops. What can go wrong?
A. If the caller passed a lazy query, it runs twice. That costs time, may hit the database twice, and may return different data. Materialise once, or take IReadOnlyCollection<Order>.

3. yield return, yield break and the state machine

C# 2local iterator functions: C# 7

What it is

A method that returns IEnumerable<T> or IEnumerator<T> and contains yield return is an iterator. The compiler rewrites it into a hidden class. Your locals become fields. Each yield return becomes a numbered resume point. MoveNext() jumps to the right point with a switch on a state field. yield break ends the sequence early.

state 0 not started running your code runs state 1 paused after yield return state -1 finished MoveNext() yield return x: MoveNext returns true next MoveNext() end or yield break: false Dispose() runs finally Locals such as loop counters live as fields, so they survive between calls.
Figure A4.2 — The compiler turns an iterator method into a small state machine driven by MoveNext().

Reading the figure. Amber is the only state where your code runs. Blue is a pause at a yield return, with Current holding the value. Green is the end. The dashed arrow is an early Dispose(), which a foreach calls when you break. Real iterators with several yield lines get one paused state per line: 1, 2, 3 and so on.

Why and when

Sample

First, iterators in normal use. The trace method shows when each line of the body runs.

public static class Iterators
{
    /// <summary>An infinite Fibonacci sequence. Safe because callers pull lazily.</summary>
    /// <returns>0, 1, 1, 2, 3, 5 and so on, forever.</returns>
    /// <example>Iterators.Fibonacci().Take(6) gives [0, 1, 1, 2, 3, 5]</example>
    public static IEnumerable<long> Fibonacci()
    {
        // F(0) = 0 and F(1) = 1 are the two seeds of the sequence.
        long a = 0, b = 1;
        // Invariant at the top of each pass: a is the next value to yield, b the one after.
        while (true)
        {
            yield return a;
            (a, b) = (b, a + b);
        }
    }

    /// <summary>Yields items until the first negative one, then stops.</summary>
    /// <param name="xs">Input numbers.</param>
    /// <returns>The prefix of xs before the first negative number.</returns>
    /// <example>UntilNegative([3, 1, -2, 5]) gives [3, 1]</example>
    public static IEnumerable<int> UntilNegative(IEnumerable<int> xs)
    {
        // x is the current input. Every earlier x was 0 or more.
        foreach (int x in xs)
        {
            // Below 0 is the stop signal. yield break ends the sequence for good.
            if (x < 0) yield break;
            yield return x;
        }
    }

    /// <summary>Records the order in which iterator code and consumer code run.</summary>
    /// <returns>A log that interleaves the two.</returns>
    /// <example>Iterators.Trace() returns [start, got 1, resumed, got 2, end]</example>
    public static List<string> Trace()
    {
        var log = new List<string>();
        // A local iterator function. Its body runs only while the foreach pulls.
        IEnumerable<int> Gen()
        {
            log.Add("start");
            yield return 1;   // 1 and 2 are just two sample values
            log.Add("resumed");
            yield return 2;
            log.Add("end");
        }
        foreach (int x in Gen()) log.Add($"got {x}");
        return log;
    }

    /// <summary>Shows that breaking out of a foreach runs the iterator's finally block.</summary>
    /// <returns>A log with the first item and the cleanup line.</returns>
    /// <example>Iterators.FinallyOnEarlyExit() returns [cleanup, take 1]</example>
    public static List<string> FinallyOnEarlyExit()
    {
        var log = new List<string>();
        IEnumerable<int> Gen()
        {
            try
            {
                yield return 1;   // the only value the caller will take
                yield return 2;   // never reached: the caller stops after one item
                log.Add("unreachable");
            }
            finally { log.Add("cleanup"); }
        }
        // First() takes one item, then disposes the enumerator, which runs finally.
        // So "cleanup" is logged inside First(), before First() even returns.
        log.Add($"take {Gen().First()}");
        return log;
    }
}

Next, a hand-written version of what the compiler generates for a simple countdown. You will never write this in production. Writing it once makes yield stop feeling like magic, and interviewers like to ask for it.

/// <summary>
/// Hand-written equivalent of an iterator with the body:
/// for (int i = from; i > 0; i--) yield return i;
/// </summary>
/// <example>new Countdown(3) enumerates as [3, 2, 1]</example>
public sealed class Countdown(int from) : IEnumerable<int>
{
    /// <summary>Each foreach gets a fresh enumerator, so the sequence can restart.</summary>
    /// <returns>A new enumerator positioned before the first item.</returns>
    /// <example>new Countdown(2).GetEnumerator().MoveNext() returns true</example>
    public IEnumerator<int> GetEnumerator() => new Enumerator(from);

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() =>
        GetEnumerator();

    private sealed class Enumerator(int start) : IEnumerator<int>
    {
        // State codes match the compiler's: 0 = not started, 1 = paused, -1 = done.
        private int _state;
        // The loop variable i, hoisted from a local into a field so it survives calls.
        private int _i;

        public int Current { get; private set; }
        object System.Collections.IEnumerator.Current => Current;

        public bool MoveNext()
        {
            switch (_state)
            {
                case 0:            // first call: run the loop initialiser
                    _i = start;
                    break;
                case 1:            // resumed after a yield: run the loop step
                    _i--;          // - 1 is the i-- of the original for loop
                    break;
                default:           // -1: finished, stay finished
                    return false;
            }
            // The loop test "i > 0": 0 is where the countdown stops.
            if (_i > 0)
            {
                Current = _i;
                _state = 1;        // 1 = paused at the only yield return
                return true;
            }
            _state = -1;           // -1 = done
            return false;
        }

        public void Reset() => throw new NotSupportedException();
        public void Dispose() => _state = -1;   // -1 so a later MoveNext returns false
    }
}

Pitfalls

Interview questions

Q. What does the compiler generate for yield return?
A. A private class that implements both IEnumerable<T> and IEnumerator<T>. It has a state field, a Current field, and one field per local. MoveNext() switches on the state to resume after the last yield.
Q. When does the finally in an iterator run if the caller stops early?
A. When the enumerator is disposed. foreach disposes it on break, and First() disposes it after one item.

4. Writing a custom LINQ operator

extension methods: C# 3extension blocks: C# 14

What it is

A LINQ operator is just an extension method on IEnumerable<T>. Deferred operators are iterators. The one rule that separates good operators from buggy ones: check arguments eagerly and iterate lazily. Split the method into a plain public wrapper and a private iterator, usually a static local function.

Why and when

Write one when the same loop shape keeps showing up: pairwise windows, “take every k-th item”, “split on a separator”. It keeps call sites short and composable. Check first whether .NET already has it. Chunk, DistinctBy and Index used to be custom operators everywhere.

Sample

public static class MyLinq
{
    /// <summary>Yields each item paired with the item after it.</summary>
    /// <param name="source">The input sequence. Must not be null.</param>
    /// <returns>Pairs (a0, a1), (a1, a2) and so on. Empty if source has under 2 items.</returns>
    /// <example>new[] { 1, 2, 4 }.Pairwise() gives [(1, 2), (2, 4)]</example>
    public static IEnumerable<(T Prev, T Next)> Pairwise<T>(this IEnumerable<T> source)
    {
        // Eager part: runs at the call, so a null throws where the bug is.
        ArgumentNullException.ThrowIfNull(source);
        return Iterate(source);

        // Lazy part: a static local iterator. It cannot capture anything by accident.
        static IEnumerable<(T, T)> Iterate(IEnumerable<T> src)
        {
            using IEnumerator<T> e = src.GetEnumerator();
            // No first item means no pairs at all.
            if (!e.MoveNext()) yield break;
            T prev = e.Current;
            // Invariant: prev is the item just before e.Current.
            while (e.MoveNext())
            {
                yield return (prev, e.Current);
                prev = e.Current;
            }
        }
    }

    /// <summary>Yields every step-th item, starting with the first.</summary>
    /// <param name="source">The input sequence.</param>
    /// <param name="step">Distance between kept items. Must be 1 or more.</param>
    /// <returns>Items at index 0, step, 2 * step and so on.</returns>
    /// <example>Enumerable.Range(0, 7).TakeEvery(3) gives [0, 3, 6]</example>
    public static IEnumerable<T> TakeEvery<T>(this IEnumerable<T> source, int step)
    {
        ArgumentNullException.ThrowIfNull(source);
        // A step below 1 would never advance, so reject it at the call.
        ArgumentOutOfRangeException.ThrowIfLessThan(step, 1);
        return Iterate(source, step);

        static IEnumerable<T> Iterate(IEnumerable<T> src, int k)
        {
            // i is the index of item. Keep it when i is a multiple of k.
            int i = 0;
            foreach (T item in src)
            {
                // % k == 0 marks indexes 0, k, 2k and so on.
                if (i % k == 0) yield return item;
                i++;
            }
        }
    }

    /// <summary>The buggy version: the null check sits inside the iterator.</summary>
    /// <param name="source">The input sequence.</param>
    /// <returns>The same items. A null source throws only when enumerated.</returns>
    /// <example>MyLinq.LazyCheck<int>(null!) does not throw until ToList()</example>
    public static IEnumerable<T> LazyCheck<T>(IEnumerable<T> source)
    {
        // This line is part of the state machine. It runs on the first MoveNext().
        ArgumentNullException.ThrowIfNull(source);
        foreach (T item in source) yield return item;
    }
}

C# 14 adds extension blocks. They let you declare extension properties as well as methods. The method form above still works and is what most code uses.

public static class SequenceExtensions
{
    // C# 14 extension block: every member inside extends IEnumerable<T>.
    extension<T>(IEnumerable<T> source)
    {
        /// <summary>True when the sequence has no items. An extension property.</summary>
        /// <example>new int[0].IsEmpty is true</example>
        public bool IsEmpty => !source.Any();
    }
}

Pitfalls

Interview questions

Q. Why split a LINQ operator into a wrapper and a local iterator?
A. The wrapper is a normal method, so argument checks throw at the call site. The iterator stays lazy. Without the split the checks run only when someone enumerates.
Q. Why make the local iterator static?
A. A static local function cannot capture outer locals. That stops it from closing over the unchecked parameter by mistake, and avoids a closure allocation.

5. IEnumerable vs IQueryable and expression trees

C# 3.NET 3.5

What it is

The same lambda can compile two ways. Passed to a parameter of type Func<T, bool>, it becomes a delegate: code. Passed to Expression<Func<T, bool>>, it becomes an expression tree: an object graph that describes the code. Enumerable.Where takes the first. Queryable.Where takes the second. A provider such as Entity Framework walks the tree and turns it into SQL.

IEnumerable<T>.Where(x => x.Price > 10) Func<T, bool> compiled IL runs in your process one call per item every row in memory then filtered IQueryable<T>.Where(x => x.Price > 10) Expression tree data: (x.Price > 10) provider translates walks the tree WHERE Price > 10 only matches come back Same lambda text. The parameter type decides whether it becomes code or data.
Figure A4.3 — IEnumerable runs your lambda as code, while IQueryable ships it as data to a provider.

Reading the figure. The blue lane is LINQ to Objects. The violet lane is a query provider. Red marks the cost: all rows travel before filtering. Green marks the win: the database filters and returns only matches.

Why and when

Sample

using System.Linq.Expressions;

public static class ExprDemo
{
    /// <summary>Shows that an expression lambda keeps its structure as data.</summary>
    /// <returns>The body of the tree printed as text.</returns>
    /// <example>ExprDemo.BodyText() returns "(x > 2)"</example>
    public static string BodyText()
    {
        // Same lambda text as a delegate would use. 2 is just a sample threshold.
        Expression<Func<int, bool>> isBig = x => x > 2;
        return isBig.Body.ToString();
    }

    /// <summary>Compiles an expression tree into a delegate and calls it.</summary>
    /// <returns>The result of the compiled add for 2 and 3.</returns>
    /// <example>ExprDemo.CompileAndRun() returns 5</example>
    public static int CompileAndRun()
    {
        Expression<Func<int, int, int>> add = (a, b) => a + b;
        Func<int, int, int> compiled = add.Compile();
        // 2 and 3 are sample inputs. Compile once and reuse: it is slow to build.
        return compiled(2, 3);
    }

    /// <summary>Builds "item => item.Property > threshold" at run time from a name.</summary>
    /// <param name="property">Name of an int property on TItem, for example "Price".</param>
    /// <param name="threshold">Items above this value pass.</param>
    /// <returns>The tree, ready for IQueryable.Where or for Compile().</returns>
    /// <example>GreaterThan<Product>("Price", 10) prints as item => (item.Price > 10)</example>
    public static Expression<Func<TItem, bool>> GreaterThan<TItem>(
        string property, int threshold)
    {
        // The lambda parameter. "item" is only the printed name.
        ParameterExpression item = Expression.Parameter(typeof(TItem), "item");
        // item.Property, looked up by name. Throws if the property does not exist.
        MemberExpression member = Expression.Property(item, property);
        // item.Property > threshold
        BinaryExpression body = Expression.GreaterThan(member, Expression.Constant(threshold));
        return Expression.Lambda<Func<TItem, bool>>(body, item);
    }

    /// <summary>Uses the built tree on an in-memory IQueryable, as a provider would.</summary>
    /// <param name="items">Products to filter.</param>
    /// <returns>Names of products priced above 10.</returns>
    /// <example>ExprDemo.Expensive([pen 3, lamp 40]) returns ["lamp"]</example>
    public static List<string> Expensive(IEnumerable<Product> items) =>
        items.AsQueryable()
             .Where(GreaterThan<Product>("Price", 10))   // 10 dollars is the sample cutoff
             .Select(p => p.Name)
             .ToList();

    /// <summary>Shows the expression tree an IQueryable builds as you chain operators.</summary>
    /// <returns>True when the tree text mentions both Where and Select.</returns>
    /// <example>ExprDemo.QueryIsData() returns true</example>
    public static bool QueryIsData()
    {
        // % 2 == 0 keeps even numbers. * 10 is a sample projection.
        IQueryable<int> q = new[] { 1, 2, 3, 4 }.AsQueryable()
                                                .Where(x => x % 2 == 0)
                                                .Select(x => x * 10);
        string text = q.Expression.ToString();
        return text.Contains("Where") && text.Contains("Select");
    }
}

Pitfalls

Interview questions

Q. What is the difference between IEnumerable<T> and IQueryable<T>?
A. IEnumerable operators take delegates and run in memory. IQueryable operators take expression trees and let a provider translate the whole query, usually to SQL. Same syntax, very different place where work happens.
Q. What is an expression tree used for besides databases?
A. Dynamic predicates built from user input, mocking libraries reading x => x.Method(), and compiling fast accessors once instead of using reflection on every call.

6. Query syntax vs method syntax

C# 3

What it is

Query syntax (from ... where ... select) is sugar. The compiler rewrites it into method calls by name: where becomes .Where(...), orderby becomes .OrderBy(...), and so on. Both forms produce the same code and the same speed.

Why and when

Sample

/// <summary>An order line that refers to a product by name.</summary>
/// <example>new OrderLine(1, "pen", 4)</example>
public sealed record OrderLine(int Id, string ProductName, int Qty);

public static class SyntaxForms
{
    /// <summary>Products over 10 dollars, priciest first, as "name:price". Query syntax.</summary>
    /// <param name="items">Products to search.</param>
    /// <returns>Labels in descending price order.</returns>
    /// <example>QueryForm([pen 3, lamp 40, desk 90]) returns ["desk:90", "lamp:40"]</example>
    public static List<string> QueryForm(IEnumerable<Product> items) =>
        (from p in items
         where p.Price > 10            // 10 dollars is the sample cutoff
         let label = $"{p.Name}:{p.Price}"
         orderby p.Price descending
         select label).ToList();

    /// <summary>The same query in method syntax.</summary>
    /// <param name="items">Products to search.</param>
    /// <returns>Labels in descending price order.</returns>
    /// <example>MethodForm([pen 3, lamp 40, desk 90]) returns ["desk:90", "lamp:40"]</example>
    public static List<string> MethodForm(IEnumerable<Product> items) =>
        items.Where(p => p.Price > 10)      // same 10-dollar cutoff
             .OrderByDescending(p => p.Price)
             .Select(p => $"{p.Name}:{p.Price}")
             .ToList();

    /// <summary>Joins order lines to products and totals the cost of each line.</summary>
    /// <param name="items">The product catalogue.</param>
    /// <param name="lines">Order lines. Lines with unknown products are dropped.</param>
    /// <returns>"id:total" per matched line, in line order.</returns>
    /// <example>JoinTotals([pen 3], [(1, pen, 4)]) returns ["1:12"]</example>
    public static List<string> JoinTotals(IEnumerable<Product> items,
                                          IEnumerable<OrderLine> lines) =>
        (from line in lines
         join p in items on line.ProductName equals p.Name
         select $"{line.Id}:{line.Qty * p.Price}").ToList();

    /// <summary>Counts products per category with group ... into.</summary>
    /// <param name="items">Products to group.</param>
    /// <returns>"category=count", sorted by category name.</returns>
    /// <example>CountPerCategory([pen office, lamp home, desk office]) returns ["home=1", "office=2"]</example>
    public static List<string> CountPerCategory(IEnumerable<Product> items) =>
        (from p in items
         group p by p.Category into g
         orderby g.Key
         select $"{g.Key}={g.Count()}").ToList();
}

Pitfalls

Interview questions

Q. Is query syntax slower than method syntax?
A. No. The compiler translates query syntax into the same method calls before it emits IL. Pick whichever reads better.

7. Newer operators (.NET 6 to .NET 10)

.NET 6.NET 7.NET 9.NET 10

What it is

Each release since .NET 6 has added operators that people used to write by hand.

Why and when

They are shorter, and some are faster. MinBy is one O(n) pass where OrderBy(...).First() sorts in O(n log n). CountBy skips the group objects that GroupBy builds. In an interview, using them shows you are current. Say what they replace in case the interviewer runs an older version.

Sample

public static class NewOperators
{
    /// <summary>Splits 1..7 into chunks of 3. The last chunk is short. (.NET 6)</summary>
    /// <returns>Each chunk as "a b c".</returns>
    /// <example>NewOperators.Chunks() returns ["1 2 3", "4 5 6", "7"]</example>
    public static List<string> Chunks() =>
        Enumerable.Range(1, 7)          // 1 is the start, 7 is the count
                  .Chunk(3)             // 3 items per chunk
                  .Select(c => string.Join(" ", c))
                  .ToList();

    /// <summary>Cheapest and priciest product in one pass each. (.NET 6)</summary>
    /// <param name="items">A non-empty product list.</param>
    /// <returns>"cheapest/priciest" names.</returns>
    /// <example>MinMax([pen 3, desk 90]) returns "pen/desk"</example>
    public static string MinMax(IEnumerable<Product> items) =>
        $"{items.MinBy(p => p.Price)!.Name}/{items.MaxBy(p => p.Price)!.Name}";

    /// <summary>Keeps the first product of each category. (.NET 6)</summary>
    /// <param name="items">Products in any order.</param>
    /// <returns>Names of the first product seen per category.</returns>
    /// <example>FirstPerCategory([pen office, lamp home, desk office]) returns ["pen", "lamp"]</example>
    public static List<string> FirstPerCategory(IEnumerable<Product> items) =>
        items.DistinctBy(p => p.Category).Select(p => p.Name).ToList();

    /// <summary>Sorts numbers by their own value without a key lambda. (.NET 7)</summary>
    /// <param name="xs">Numbers to sort.</param>
    /// <returns>xs in ascending order.</returns>
    /// <example>Sorted([3, 1, 2]) returns [1, 2, 3]</example>
    public static List<int> Sorted(IEnumerable<int> xs) => xs.Order().ToList();

    /// <summary>Counts words by first letter without building groups. (.NET 9)</summary>
    /// <param name="words">Non-empty words.</param>
    /// <returns>"letter=count" in first-seen order.</returns>
    /// <example>ByFirstLetter(["ant", "bee", "asp"]) returns ["a=2", "b=1"]</example>
    public static List<string> ByFirstLetter(IEnumerable<string> words) =>
        words.CountBy(w => w[0])        // [0] is the first character
             .Select(kv => $"{kv.Key}={kv.Value}")
             .ToList();

    /// <summary>Sums price per category in one pass. (.NET 9)</summary>
    /// <param name="items">Products to total.</param>
    /// <returns>"category=total" in first-seen order.</returns>
    /// <example>TotalPerCategory([pen office 3, desk office 90]) returns ["office=93"]</example>
    public static List<string> TotalPerCategory(IEnumerable<Product> items) =>
        items.AggregateBy(p => p.Category,
                          seed: 0,                         // 0: an empty category totals 0
                          (sum, p) => sum + p.Price)
             .Select(kv => $"{kv.Key}={kv.Value}")
             .ToList();

    /// <summary>Pairs each item with its position. Replaces Select((x, i) => ...). (.NET 9)</summary>
    /// <param name="xs">Items to number.</param>
    /// <returns>"index:item" for each item, starting at index 0.</returns>
    /// <example>Numbered(["a", "b"]) returns ["0:a", "1:b"]</example>
    public static List<string> Numbered(IEnumerable<string> xs) =>
        xs.Index().Select(t => $"{t.Index}:{t.Item}").ToList();

    /// <summary>Every line with its product price, or 0 if unknown. (.NET 10)</summary>
    /// <param name="lines">Order lines, kept even with no match.</param>
    /// <param name="items">The catalogue.</param>
    /// <returns>"id:price" per line.</returns>
    /// <example>PricesOrZero([(1, pen, 1), (2, ghost, 1)], [pen 3]) returns ["1:3", "2:0"]</example>
    public static List<string> PricesOrZero(IEnumerable<OrderLine> lines,
                                            IEnumerable<Product> items) =>
        lines.LeftJoin(items, l => l.ProductName, p => p.Name,
                       // p is null when no product matched. 0 is the "unknown" price.
                       (l, p) => $"{l.Id}:{p?.Price ?? 0}")
             .ToList();

    /// <summary>The last two items using Take with a range from the end. (.NET 6)</summary>
    /// <param name="xs">Items.</param>
    /// <returns>Up to the last two items.</returns>
    /// <example>LastTwo([1, 2, 3, 4]) returns [3, 4]</example>
    public static List<int> LastTwo(IEnumerable<int> xs) =>
        xs.Take(^2..).ToList();         // ^2.. means "from 2 before the end to the end"
}

Pitfalls

Interview questions

Q. How do you find the cheapest product without sorting?
A. items.MinBy(p => p.Price). It is one O(n) pass. Before .NET 6 you would use Aggregate or a loop. OrderBy(...).First() works but costs O(n log n).
Q. What does CountBy save over GroupBy(...).Select(g => g.Count())?
A. GroupBy stores every item in a list per key. CountBy keeps only one counter per key, so it uses far less memory.

8. LINQ performance traps

all versionsmany fast paths: .NET 7 to .NET 9

What it is

LINQ is fast enough for most code. It costs a delegate call per item and an enumerator allocation per operator. Recent .NET versions vectorise Sum, Min and Max on arrays and spans, and special-case List and arrays in many operators. The real traps are algorithmic: hidden O(n²) and repeated work.

Why and when

Reach for a plain loop in a hot path, inside a tight inner loop, or when a profiler says so. Otherwise prefer readable LINQ and fix the shape of the query.

The common traps

Sample

public static class LinqTraps
{
    /// <summary>Common ids with List.Contains inside Where: O(n * m) work.</summary>
    /// <param name="a">First id list.</param>
    /// <param name="b">Second id list.</param>
    /// <returns>Ids of a that also appear in b, in a's order.</returns>
    /// <example>SlowCommon([1, 2, 3], [3, 1]) returns [1, 3]</example>
    public static List<int> SlowCommon(List<int> a, List<int> b) =>
        a.Where(x => b.Contains(x)).ToList();     // linear scan of b per item of a

    /// <summary>Same result with a HashSet: O(n + m) work.</summary>
    /// <param name="a">First id list.</param>
    /// <param name="b">Second id list.</param>
    /// <returns>Ids of a that also appear in b, in a's order.</returns>
    /// <example>FastCommon([1, 2, 3], [3, 1]) returns [1, 3]</example>
    public static List<int> FastCommon(List<int> a, List<int> b)
    {
        // Build the set once. Each lookup is then O(1) on average.
        var inB = new HashSet<int>(b);
        return a.Where(inB.Contains).ToList();
    }

    /// <summary>Counts items produced when ElementAt is called in a loop on a lazy source.</summary>
    /// <param name="n">Length of the source.</param>
    /// <returns>Total items produced. Grows as n * (n + 1) / 2.</returns>
    /// <example>LinqTraps.ElementAtCost(4) returns 10</example>
    public static int ElementAtCost(int n)
    {
        var counter = new CallCounter();
        IEnumerable<int> lazy = MultiEnum.Produce(n, counter);
        // i is the position we ask for. Each ElementAt(i) restarts and walks i + 1 items.
        for (int i = 0; i < n; i++) _ = lazy.ElementAt(i);
        return counter.Count;
    }

    /// <summary>Shows that Any stops early while Count walks the whole lazy source.</summary>
    /// <param name="n">Length of the source.</param>
    /// <returns>"any=items count=items" produced by each check.</returns>
    /// <example>LinqTraps.AnyVsCount(1000) returns "any=1 count=1000"</example>
    public static string AnyVsCount(int n)
    {
        var forAny = new CallCounter();
        var forCount = new CallCounter();
        _ = MultiEnum.Produce(n, forAny).Any();
        // "> 0" asks the same question as Any, but only after counting every item.
        _ = MultiEnum.Produce(n, forCount).Count() > 0;
        return $"any={forAny.Count} count={forCount.Count}";
    }
}

Pitfalls

Interview questions

Q. You see orders.Where(o => vipIds.Contains(o.CustomerId)) and vipIds is a List<int>. What do you say?
A. That is O(n × m). Turn vipIds into a HashSet<int> once. Each lookup becomes O(1). For an IQueryable the list version is fine, because it becomes a SQL IN clause.
Q. When would you replace LINQ with a loop?
A. In a measured hot path. A loop avoids delegate calls, enumerator objects and closures. It also lets you exit early with custom logic or work on a Span<T>.
Say this out loud: “I keep LINQ for clarity, but I watch the shape. No Contains on a list inside Where, no ElementAt in a loop, Any instead of Count, and MinBy instead of sorting to find one item.”

Recap

The things to carry forward

Where this goes next

Iterators pause and resume on MoveNext(). Async methods use the same trick, but resume when a task completes. A 05, Async and Await, builds on the state machine from this page.


← A 03 — Delegates and Lambdas A 05 — Async and Await →