Linux CPU benchmarks for AI: synthetic scores rarely predict pipeline throughput

Training workloads demand sustained data throughput to GPUs—a requirement Geekbench scores fail to predict. Geekbench won’t tell you. Phoronix runs, SPEC CPU numbers, and sysbench loops measure standardised compute kernels under conditions that look nothing like an AI preprocessing pipeline. For systems where the CPU’s job is to decode images, tokenise text, run NumPy transforms, and shovel tensors across PCIe, those benchmarks are at best a sanity check that the silicon isn’t broken.

The CPU’s role in an AI system is best understood as the upstream supplier to a downstream consumer that costs roughly an order of magnitude more per hour. When the supplier can’t keep up, the GPU sits idle and the system-limited pattern we cover in GPUs are part of a larger system takes over. The useful question is not “how fast is this CPU on a generic compute test?” — it’s “how many samples per second can this CPU push through the actual preprocessing graph this workload needs, under the kernel and library configuration that will run in production?”That framing has a consequence worth stating plainly: the result belongs to the machine, not to the part. A number produced on your box is bound to that executor — the device, the backend it ran through, and the driver, framework, and runtime present on the host — which is why the chassis, the CPU, and the DIMM population are inside the measurement rather than controlled away. It also means the reverse is off the table: a whole-machine throughput figure does not decompose into a per-component contribution. You learn what this executor achieved, not how many samples-per-second the CPU “owns” versus the link versus the accelerator.

What the CPU actually does in an AI pipeline

Operation	CPU‑bound	Parallelisable	Benchmark approach
Image decoding (JPEG/PNG)	Yes	Per‑image	Decode throughput test
Text tokenisation	Yes	Per‑document	Tokeniser throughput test
NumPy / SciPy preprocessing	Yes	Per‑batch	Benchmark actual operations
Data augmentation	Yes	Per‑sample	Augmentation throughput test
PCIe data transfer to GPU	I/O‑bound	Limited	Bandwidth test
Python control overhead	Yes	No	Profiling (cProfile, py-spy)

This table breaks down preprocessing stages as independent components, not as a temporal workflow. In a PyTorch DataLoader with num_workers > 0, several of these run concurrently in worker processes while the main process orchestrates batching and host-to-device transfer. The CPU benchmark question becomes: which of these rows is the binding constraint on the specific workload, and which Linux measurement actually exposes that row’s throughput?

Synthetic CPU benchmarks: useful for what they are, misleading for AI

Throttling detection, basic capability validation, and procurement discussions justify synthetic benchmarks despite their limited relevance to training pipelines. They become misleading the moment that number is treated as a predictor of AI pipeline throughput.

Geekbench reports single-core and multi-core scores built from a mix of compression, image filtering, and machine-learning-flavoured kernels. It’s a reasonable proxy for “is this CPU broadly competitive in 2026?” and a poor proxy for “how fast will my JPEG decode plus augmentation pipeline run?”

sysbench is more useful for diagnostics than ranking. Its CPU mode hammers integer and floating-point operations and reliably surfaces thermal throttling under sustained load. Its memory mode gives a first-cut bandwidth number that, while not as rigorous as STREAM, is fast to run and good for spotting misconfigured DIMM populations.

# CPU throughput under sustained load — useful for throttling detection
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) run

# First-cut memory bandwidth
sysbench memory --memory-block-size=1M --memory-total-size=100G run

STREAM is the one synthetic that earns its place in an AI benchmark plan, because memory bandwidth genuinely binds the operations that dominate preprocessing — large tensor moves, contiguous array transforms, and copy-heavy augmentation. Its copy/scale/add/triad pattern approximates the kind of streaming access an image pipeline produces.

# STREAM memory bandwidth — directly relevant to preprocessing throughput
wget https://www.cs.virginia.edu/stream/FTP/Code/stream.c
gcc -O3 -fopenmp stream.c -o stream
./stream

mbw is a lighter alternative that’s easier to run repeatedly during NUMA tuning. In our experience, mbw results correlate more closely with DataLoader throughput on memory-bound preprocessing than any general-purpose CPU score does — this is an observed pattern across several engagements rather than a benchmarked rate published anywhere.

Workload-specific benchmarks: time the pipeline you actually run

Profile your actual preprocessing code on the target system and measure wall-clock time with the GPU idle—no synthetic substitute matches this. A few lines of Python expose more than any synthetic ever will:

import time
from PIL import Image
import numpy as np

# Image decode throughput, isolated from the GPU
images = ['test.jpg'] * 1000
start = time.time()
for img_path in images:
    img = Image.open(img_path)
    arr = np.array(img)
elapsed = time.time() - start
print(f'{len(images)/elapsed:.0f} images/sec')

Wrap the real pipeline with cProfile or py-spy and the breakdown becomes concrete: how much time is in JPEG decode versus resize versus the random crop versus the host-to-device copy. That breakdown is the benchmark. Everything else is an indirect estimate.

The decision rule we apply in practice: if the CPU produces batches faster than the GPU consumes them by at least 2×, CPU performance is not the binding constraint, regardless of what any synthetic score reports. If the margin is below 1.2×, the CPU is the bottleneck and any GPU upgrade will hit a wall. Between those two thresholds, the system is sensitive to load variance and worth tuning.

Why NUMA topology dominates multi-socket Linux servers

Benchmark suites routinely underweight NUMA topology effects that dominate real memory access patterns. On a dual-socket server, a PyTorch DataLoader worker running on CPU 0 that reads from memory attached to CPU 1 pays an inter-socket bandwidth penalty in the 30–60% range — an observed pattern across the multi-socket deployments we’ve measured, not a single-source benchmark. The fix is straightforward once the topology is mapped:

# Map the NUMA layout first
numactl --hardware

# Bind workers to a single NUMA node
numactl --cpunodebind=0 --membind=0 python train.py

In one engagement on a dual-socket EPYC 7763 system, local-node memory bandwidth measured roughly 190 GB/s against approximately 85 GB/s remote — a 2.2× gap. Binding DataLoader workers to the local NUMA node reduced per-batch image preprocessing time from around 4.2 ms to 2.1 ms on ImageNet-scale inputs. Those figures are operational measurements from that specific configuration, not a transferable benchmark — a different CPU generation, BIOS revision, or kernel version will produce different absolute numbers, though the structural pattern is consistent.

The connection to the broader system-balance story is direct: PCIe topology and interconnect choice sit downstream of NUMA. A worker bound to the wrong node not only loses memory bandwidth, it also takes a longer route to the GPU’s PCIe root complex. Both penalties compound, and both register as “GPU is slow” in the dashboards without naming the actual cause.This is also where reading a utilisation dashboard at face value misleads. A box can show CPU at around 40% and the GPU at 96% and still not be CPU-bound: a 40% CPU figure is an average across cores that hides a single saturated DataLoader worker, and 96% GPU only means a kernel was scheduled, not that the GPU ran at full throughput. On an AI workload, the number that decides the answer is samples-per-second on each side, not the percentage gauges — if the CPU is feeding batches faster than the GPU drains them, the system is GPU-bound regardless of what either dial reads. Interconnect bandwidth ties into the same picture: PCIe generation only becomes the binding constraint once host-to-device transfer dominates the per-batch budget, which is why measuring the actual transfer time, not the link’s nameplate spec, is what tells you whether PCIe 3.0, 4.0, or a 5.0 x16 slot would change anything for this pipeline.

Compiler, library, and kernel choices: the configuration that benchmarks hide

Build flags, library versions, and kernel tuning parameters turn a single CPU score into dozens of possible outcomes. Three sources of variation matter enough to swamp small CPU model differences.

Numerical library linkage. PyTorch built against Intel MKL typically runs batch matrix multiplications and convolutions roughly 15–30% faster than the same PyTorch built against generic OpenBLAS on Intel CPUs — a range we’ve seen across multiple builds, framed here as an observed pattern rather than a single benchmark. On AMD CPUs, BLIS or OpenBLAS with AMD-specific tuning is competitive with MKL. The diagnostic is one line:

import torch
print(torch.__config__.show())

If MKL is listed in the build but not installed at runtime, PyTorch falls back silently to a slower internal path. We check this before investigating anything more exotic, because it explains a surprising fraction of “this server is slow” tickets.

SIMD-accelerated preprocessing. Pillow-SIMD, a drop-in fork of Pillow, runs image decode and resize roughly 4–6× faster than stock Pillow on AVX-2 and AVX-512 CPUs (figures consistent with the project’s published benchmarks and with our observed runs). For preprocessing-heavy training, this single substitution often moves the CPU/GPU ratio from “starved GPU” to “comfortable margin” without changing hardware.

Kernel and BIOS tuning. Transparent Huge Pages, CPU governor, I/O scheduler, IRQ affinity, and BIOS power-cap settings each contribute measurable differences that no synthetic benchmark captures. The most reliable single change is the governor:

# Pin the governor to performance to prevent clock fluctuation
cpupower frequency-set -g performance

In our experience, the performance governor adds roughly 3–8% on sustained preprocessing throughput compared with ondemand or powersave, simply by preventing the clock from sagging during steady-state load. The effect is larger on laptops and small-form-factor machines than on tuned server platforms.

A practical Linux CPU benchmark plan for AI

The plan we apply when scoping or auditing a Linux AI box:

1. Run STREAM and mbw to characterise sustained memory bandwidth, both single-node and inter-node where multi-socket.
2. Run sysbench cpu under sustained load to confirm the CPU does not throttle.
3. Map NUMA topology with numactl --hardware and confirm workers are bound to the correct node.
4. Time the actual preprocessing pipeline end-to-end with time plus a cProfile or py-spy breakdown.
5. Verify library linkage with torch.__config__.show() and confirm MKL / BLIS / Pillow-SIMD are present where expected.
6. Compare CPU samples-per-second to GPU samples-per-second and apply the 2× margin rule (a planning heuristic) — and run the CPU-only path on the same scale, so “is a GPU worth buying here at all?” gets an answer rather than an assumption.
7. Raise the workload size until throughput stops improving. The point where it flattens is usually where a system-level constraint — memory bandwidth, a NUMA hop, the host-to-device copy — rather than the accelerator becomes the thing being measured.
8. Record kernel, governor, THP setting, library versions, and BIOS tuning state alongside every number.

Step 8 is the part most published Linux CPU AI benchmarks skip, and it’s the part that determines whether the number transfers. LynxBenchAI treats Linux CPU evaluation for AI as workload-and-tuning-bound — kernel, scheduler, NUMA, IRQ affinity, BIOS settings — because the same CPU under different Linux configurations produces materially different pipeline throughput.Two things follow from that. First, CPU-only is a first-class point on the same scale rather than a footnote, so the upgrade question can be answered against a measured baseline instead of a guess. Second, comparisons stay inside a release: a 26Q3 result is read against other 26Q3 results, because the catalogue and the scoring are fixed per release and do not travel across release names.

Frequently Asked Questions

When is the honest answer that the CPU-only path is good enough for an AI workload?

When the pipeline is preprocessing-heavy, batch sizes are modest, or latency targets are loose, a well-tuned CPU path with MKL or BLIS and Pillow-SIMD can be sufficient — and the only way to know is to measure it. Running CPU-only on the same scale as the accelerator turns “is a GPU upgrade worth buying at all?” into a comparison of two measured numbers rather than an assumption. If the CPU-only samples-per-second already clears the requirement, the upgrade buys headroom, not capability.

If a whole-machine result cannot be split per component, what can a reader still legitimately conclude?

You can conclude what this executor achieved under the conditions you declared — the device, backend, driver, framework, runtime, kernel, governor, and NUMA binding recorded next to the number. That is enough to compare your machine against another machine, and enough to detect a regression after a driver or library change. What it does not license is an attribution: the run does not tell you how much of the throughput the CPU, the PCIe link, or the accelerator each contributed. For that you go back to the timed pipeline breakdown with cProfile or py-spy.

How do you find the point where a system-level constraint, rather than the CPU, starts being measured?

Raise the workload size — batch size, worker count, resolution — until throughput stops improving. The plateau is informative: past it, you are usually measuring memory bandwidth, an inter-socket NUMA hop, or the host-to-device copy rather than raw core speed. Recording where the curve flattens, alongside the STREAM and mbw numbers, is more useful for capacity planning than any single peak figure.

Do Linux CPU-for-AI numbers stay comparable over time?

Only within a fixed catalogue and scoring definition. Results carry a release name for exactly this reason, and a 26Q3 result should be read against other 26Q3 results rather than against numbers from a different release. Across releases the honest move is to re-run, not to interpolate — and to keep the kernel, library, and BIOS state recorded with each number so the comparison has context at all.

Kernel scheduler tuning unlocks 40% latency improvements

Power-saving CPU governors ship as defaults in most distributions, introducing latency spikes that synthetic tests never reveal. So the question to carry forward is this: do you know the executor, the rules, and the release behind the number in front of you — and if not, what would it take to find out?