Why Parallel.ForEach Didn't Fix My Performance Problem

The numbers that nearly killed the feature

I had a background service that ingested record files. Each file was text — a header identifying which US state it came from, and a block of free-form content underneath. The service worked out which state a file belonged to, pulled a set of regex patterns for that state out of the database, ran them to extract the fields, and saved the result.

It was correct. Every file that went in came out as the right record, nothing was lost, and it recovered cleanly from crashes. Getting that right is a separate article.

Then we ran it against a realistic batch.

Perfectly linear, about six seconds per file. And the partner system could drop a thousand files at once. That’s an hour and a half for one batch, on a background service that was supposed to keep up with a continuous feed. This wasn’t slow, it was disqualifying.

The loop looked innocent enough:

foreach (var path in Directory.EnumerateFiles(inboxFolder, "*.xml"))
{
    var content  = File.ReadAllText(path);
    var source   = _resolver.Resolve(XDocument.Parse(content), path);

    // Which fields to extract, and the regex for each one.
    var patterns = _db.GetFieldPatterns(source.State, source.FormatCode, source.Type);

    var record = new ParsedRecord();
    foreach (var p in patterns)
    {
        var match = Regex.Match(content, p.Pattern, RegexOptions.Compiled);
        if (match.Success)
            record[p.FieldName] = match.Groups["value"].Value;
    }

    _db.Insert(record);
}

Read a file, work out where it came from, fetch its patterns, run them, save. Six seconds of that per file seemed absurd. Nothing in there is an expensive algorithm.

Reaching for threads, and getting nothing

The files are independent. Independent work is the textbook case for parallelism. So I did the obvious thing:

Parallel.ForEach(files, path => ProcessFile(path));

On an eight-core machine I expected something in the neighbourhood of a 6–8× improvement. I got a fraction of that — enough to notice on a graph, nowhere near enough to matter. Thirty minutes became something like twenty.

I did what you do when a hammer doesn’t work. I got a bigger hammer. MaxDegreeOfParallelism turned up. PLINQ. Reworking pieces into async/await. Concurrent collections everywhere so nothing contended on a lock.

Barely moved. In some configurations, more threads made it measurably worse.

That “worse” was the useful signal, and I ignored it for longer than I should have. If adding workers slows a system down, the workers aren’t the constraint. They’re queuing for something, and I hadn’t worked out what.

Actually measuring, instead of guessing

Eventually I stopped tuning and instrumented the loop body — just a stopwatch around each stage, totals dumped at the end. That took twenty minutes and I should have done it first.

For a single file, roughly:

Reading and parsing — the two things I’d have guessed were the problem, since this is a file-processing pipeline — were three percent combined.

Almost everything was in three places, and once I looked at each one, all three turned out to be the same mistake wearing different clothes.

The patterns were being fetched once per file, over and over. The regexes lived in the database rather than in code, keyed by state, so that when a state changed its layout somebody could fix a pattern with a row update instead of a build and a deployment. On a service installed on machines that were awkward to update, that was the right call and I’d make it again. But it meant a database round trip inside the per-file loop, and there were only a few dozen distinct pattern sets in total. Three hundred files, most from a handful of states, meant fetching the same twenty rows hundreds of times. The data hadn’t changed since the service started.

The regexes were being rebuilt for every match. This one hurt. Regex.Match(input, pattern) — the static overload — caches compiled patterns internally, so it looks free. That cache holds fifteen entries by default (Regex.CacheSize). We had more distinct patterns than that in play, so the cache thrashed: every eviction meant re-parsing the pattern from scratch on the next use.

And I’d passed RegexOptions.Compiled, which made it dramatically worse rather than better. Compiled emits IL for the pattern — genuinely faster to execute, but the code generation costs milliseconds up front. It’s an optimisation that pays off across thousands of matches with one Regex instance, and is pure loss if you rebuild the instance every time. I was paying the expensive setup on every single match and throwing the result away.

Every record was its own transaction. _db.Insert(record) with no explicit transaction means SQLite wraps each statement in an implicit one — and committing a transaction means an fsync to guarantee durability. A thousand records was a thousand fsyncs, each one a synchronous wait on physical disk.

Why more threads couldn’t have helped

With that laid out, the failure of Parallel.ForEach stops being mysterious.

Roughly 67% of the time (patterns + insert) was waiting on I/O round trips — the database and the disk. Threads do not make a round trip faster. Sixteen threads waiting on a database is sixteen threads waiting on a database, and they’re now also contending for a connection pool and hammering the same table.

Worse, blocking I/O on thread pool threads triggers a specific pathology. A blocked thread is still occupied. The pool notices work queuing up and injects more threads to compensate — but deliberately slowly, on the order of one or two per second, because rapid injection usually makes things worse. So the system spends its first stretch starved, then oversubscribed, with more threads than cores context-switching against each other and multiplying fsync contention on one disk. That’s the “more threads made it slower” I’d seen and dismissed.

The regex cost, the one genuinely CPU-bound part, was parallelisable — but it was pure waste. The fastest way to run redundant work on eight cores is to not run it.

This is Amdahl’s law with the serifs filed off: parallelism only speeds up the part that’s actually parallel. With two-thirds of the runtime stuck in serialised round trips, perfect parallelism on the rest caps out around a 1.5× improvement. I measured about that. The theory was working correctly; my expectations weren’t.

The mistake wasn’t using Parallel.ForEach. It was using it as a first move — treating parallelism as a way to make slow code fast, when it’s really a way to make already-efficient code use more cores.

The rewrite: separate phases instead of one loop

The redesign came from asking a different question. Not “how do I run this loop faster,” but “why is this work inside the loop at all?”

Almost none of it needed to be. So the single interleaved loop became three phases, each doing one kind of work:

Phase one — read everything. Once. All file contents into memory up front, with bounded concurrency so the disk isn’t thrashed by fifty threads seeking at once:

var fileContents = new ConcurrentDictionary<string, string>();
using var gate = new SemaphoreSlim(8);

await Task.WhenAll(files.Select(async path =>
{
    await gate.WaitAsync();
    try
    {
        fileContents[path] = await File.ReadAllTextAsync(path);
    }
    finally { gate.Release(); }
}));

Phase two — preload every pattern and compile each regex exactly once. All pattern sets in a single query, then one Regex instance per pattern, built once and reused for the entire run:

// One query for every pattern set, instead of one per file.
var patternSets = _db.GetAllFieldPatterns()
    .GroupBy(p => new SourceKey(p.State, p.FormatCode, p.RecordType))
    .ToDictionary(
        g => g.Key,
        g => g.OrderBy(p => p.Order)
              .Select(p => new CompiledField(
                  p.FieldName,
                  // Built once, reused for every file. NOW Compiled pays for itself.
                  new Regex(p.Pattern, RegexOptions.Compiled | RegexOptions.CultureInvariant)))
              .ToArray());

The RegexOptions.Compiled flag didn’t change. Its position did. Constructed once and used across thousands of matches, the IL generation cost is amortised to nothing and every match afterwards runs at full speed. Same flag, opposite outcome, purely because of where it sits relative to the loop.

Phase three — parse in parallel, in memory, touching nothing external. Now, and only now, Parallel.ForEach earns its place:

var parsed = new ConcurrentBag<ParsedRecord>();

Parallel.ForEach(fileContents, kvp =>
{
    var source = _resolver.Resolve(XDocument.Parse(kvp.Value), kvp.Key);
    if (!patternSets.TryGetValue(source, out var fields)) return; // quarantine

    var record = new ParsedRecord { SourceFile = kvp.Key };
    foreach (var field in fields)
    {
        // Regex instances are thread-safe for matching — one instance, all threads.
        var match = field.Pattern.Match(kvp.Value);
        if (match.Success)
            record[field.Name] = match.Groups["value"].Value;
    }

    parsed.Add(record);
});

No file I/O. No database calls. No allocation of regex machinery. Just CPU against strings already in memory — which is exactly the workload multiple cores are good at. That Regex instances are thread-safe for matching (though Match objects are not) is what lets every thread share one compiled instance rather than each building its own.

Phase four — write everything in one batch, in one transaction:

using var tx = connection.BeginTransaction();
foreach (var record in parsed)
    insertCommand.ExecuteNonQuery();   // parameters rebound per record
tx.Commit();                           // one fsync for the entire batch

A thousand fsyncs became one commit.

The result

Batches that had taken tens of minutes finished in seconds — including runs of around a thousand files, which under the old design would have been well over an hour. The scaling curve changed shape too: the old version was punishingly linear in file count, while the new one spends most of its time in a phase that genuinely divides across cores.

What’s worth noticing is that the parallelism wasn’t the win. Almost all of it came from deleting work — hundreds of redundant database round trips, thousands of redundant regex compilations, and a thousand redundant fsyncs. Parallelism only started contributing once it had something CPU-bound left to bite on.

The honest caveat

Reading every file into a dictionary before processing has an obvious ceiling: memory. At a thousand modest text files it’s comfortable. At a hundred thousand, or with much larger files, it isn’t — you’d be trading a time problem for an out-of-memory one.

The fix is straightforward and I’d build it in from the start next time: process in chunks of a few thousand, running all phases per chunk. You keep the batching wins, the preloaded patterns and compiled regexes stay hoisted above everything, and memory stays flat regardless of backlog size. Phase separation doesn’t require holding everything at once — just enough to batch usefully.

What I actually took away

Three things, in the order I wish I’d learned them.

Measure before optimising. I burned days tuning thread counts on a workload where threads were never the constraint. Twenty minutes with a stopwatch would have redirected all of it. Every guess I made about where the time went was wrong — including the ones that felt obvious for a file-processing pipeline.

Parallelism amplifies a good design; it does not rescue a bad one. If the work is round-trip bound, more workers means more waiting in parallel. Get the work per item genuinely small first. Then parallelise it.

Watch for invariant work inside a loop. All three culprits were the same bug: something that only needed to happen once was happening once per file. The patterns don’t change between files. The regexes don’t change between files. Durability doesn’t need guaranteeing between every record. Hoisting invariants out of loops is the most basic optimisation there is, and it’s easy to miss when the invariant is disguised as a service call or a convenient static helper.

The architecture was already right — the routing, the error isolation, the recovery behaviour all stayed exactly as they were. The rewrite didn’t change what the pipeline did. It just stopped doing the same work over and over, and that turned out to be the whole difference between a feature that shipped and one that didn’t.

If you’re building something similar, the companion piece to this one is Processing Thousands of Files Without Losing Any of Them — same pipeline, but the correctness problems rather than the speed ones.