Mac AI performance is architecturally different from NVIDIA GPU performance A developer ports a transformer inference script from a CUDA workstation to an M3 Max MacBook Pro, runs the same torch.matmul benchmark, and reads off a TFLOPS number. Then the comparison starts: is the Mac “half as fast” as a discrete GPU? “A quarter”? “An eighth”? None of those framings hold up. Mac AI performance is not a discounted version of NVIDIA performance — it is a different point in the hardware-software design space, governed by unified memory, the Metal Performance Shaders (MPS) backend, and a release cadence pinned to macOS rather than CUDA. Apple Silicon (M1, M2, M3, M4 series) uses a unified memory architecture in which CPU and GPU share the same physical memory. This eliminates the PCIe bandwidth bottleneck between CPU and GPU that constrains discrete-GPU systems. It also means GPU memory capacity equals total system RAM — an M3 Max with 96 GB RAM gives the GPU 96 GB. The implication for AI benchmarking is straightforward: the metric that matters depends on the workload, and any single number quoted against a CUDA benchmark is misleading by default. Why a single TFLOPS number tells you almost nothing on Mac Treating peak FP32 TFLOPS as a portable comparison across Apple Silicon and NVIDIA hardware is the kind of layer-by-layer reasoning that breaks down on AI workloads. The hardware spec sheet describes one slice of the stack. The MPS backend in PyTorch — its operator coverage, its precision support, its CPU-fallback behaviour — describes another. The macOS version under which the test runs is a third. None of these factors is constant across the comparison, and none can be ignored. Factor Apple Silicon Discrete GPU (NVIDIA) GPU memory Shared with system RAM Dedicated VRAM (8–80 GB) GPU-CPU transfer bandwidth Very high (no PCIe) PCIe 4.0: ~64 GB/s per direction Memory type LPDDR5X (high efficiency) HBM2e/GDDR6X (high bandwidth) Peak GPU memory bandwidth Roughly 200–400 GB/s Roughly 900–3350 GB/s (A100/H100, per NVIDIA’s published specifications) Power efficiency Very high (laptop-class TDP) Lower (data-centre class) Apple Silicon wins on efficiency and on the amount of memory the GPU can address. Discrete data-centre GPUs win on raw compute throughput and on memory bandwidth. Which axis dominates depends on the workload — and the only way to know is to measure on the specific stack you will deploy on. A minimal MPS benchmark — and what it does not measure The standard way to read raw FP32 throughput on Apple Silicon is via PyTorch’s MPS backend: import torch import time # Check MPS availability print(torch.backends.mps.is_available()) device = torch.device("mps") a = torch.randn(4096, 4096, device=device, dtype=torch.float32) b = torch.randn(4096, 4096, device=device, dtype=torch.float32) # Warmup for _ in range(5): torch.matmul(a, b) torch.mps.synchronize() start = time.time() for _ in range(100): torch.matmul(a, b) torch.mps.synchronize() elapsed = time.time() - start flops = 2 * 4096**3 * 100 print(f'{flops/elapsed/1e12:.2f} TFLOPS FP32') The number this produces is an observed-pattern measurement of one operator at one shape on one machine on one macOS version. It is not portable. The structural limits to keep in mind: MPS is not CUDA. Operator coverage is narrower, and some ops fall back to CPU silently, which can dominate end-to-end latency in ways the matmul number hides. Mixed-precision (FP16 / BF16) support on MPS is less mature than on CUDA. A model that runs cleanly in BF16 on an A100 may need different precision handling on MPS. Not all model architectures run on MPS at all; some still require CPU execution paths. macOS point releases shift MPS and Accelerate behaviour. The same benchmark on the same hardware can move by 5–15% across OS versions in our experience. This is what it means to treat performance as a stack rather than a hardware property: the relevant unit of measurement is the (chip, backend, OS version, workload) tuple, not the chip. When Mac is the right tool, and when it isn’t The honest version of the Mac-for-AI question is contextual. Apple Silicon is a reasonable choice for development and prototyping (silent, energy-efficient, local inference), for LLM inference at moderate context lengths where llama.cpp and Apple’s mlx framework are well-optimised, and for models that fit in unified memory — large RAM configurations let large models load without sharding. Apple Silicon is not appropriate for large-scale training (sustained throughput is typically an order of magnitude or more below an H100, per observed-pattern measurements across published community benchmarks), for multi-GPU training (Apple Silicon is single-GPU per machine), or for production serving at scale. The unified memory advantage is genuine and worth being concrete about. An M2 Ultra with 192 GB of unified memory can load and run a 70B-parameter model at FP16 without model sharding — something that would otherwise require two NVIDIA A100 80GB GPUs connected over NVLink. For developers and researchers experimenting with large models locally, this is a meaningful capability difference, not a marketing point. The tradeoff is throughput. That same M2 Ultra generates tokens at roughly 10–15 tokens per second on a 70B model, while a pair of A100s generates roughly 80–120 tokens per second (observed-pattern, varies with quantisation, context length, and software stack). For interactive single-user experimentation, the M2 Ultra is adequate. For production serving, it is not competitive. We use Apple Silicon machines in two specific roles: local development and validation of models before deploying to GPU infrastructure, and small-scale inference for internal tools where latency tolerance is high and deployment cost sensitivity is extreme. For any workload that requires serving external users, training, or processing data at scale, discrete GPU systems are the correct choice. A testing workflow that matches the architecture Performance testing on Apple Silicon needs different tools than Linux GPU testing. There is no nvidia-smi equivalent on macOS. powermetrics provides power-consumption data, Activity Monitor shows GPU utilisation at a coarse level, and asitop provides real-time monitoring of the Neural Engine, GPU, and CPU. The mlx framework gives Apple Silicon-optimised inference paths, and coremltools covers Core ML conversion. The workflow we use for AI development on Apple Silicon validates functionality and performance in three stages: Correctness first. Test that the model loads and produces the expected output via mlx or coremltools. This catches MPS-backend compatibility issues before they masquerade as performance problems. Single-inference latency. Measure latency at the target sequence length. This determines whether interactive use is viable for the intended workflow. Sustained throughput. Run a 10-minute sustained throughput test. This reveals thermal throttling that a short benchmark hides. For MacBook Pro chassis, thermal throttling typically reduces sustained throughput by 10–20% compared to the first minute (observed-pattern, varies with ambient temperature and chassis). Mac Studio and Mac Pro systems, with better cooling, generally show less than 5% degradation. This difference matters for developers who run multi-hour fine-tuning jobs locally. We re-run these tests after macOS updates because Metal Performance Shaders and the Accelerate framework receive optimisations in each release. Testing after OS updates prevents surprises from both improvements and regressions — and reinforces that the performance number is bound to the stack version, not the chip. The broader argument for this framing — that performance emerges from the hardware-software stack rather than from hardware alone — applies generally, not only to Apple Silicon. The Mac case just makes the stack-dependence visible because the backend (MPS) and the OS (macOS) move on their own cadence, independent of the chip. LynxBench AI treats Apple Silicon AI evaluation as a unified-memory and MPS-backend-specific measurement, because Mac AI performance does not translate from CUDA benchmarks and the operating-system release cadence shifts the MPS performance surface in ways a one-time score cannot capture. For any Mac-for-AI performance claim you intend to act on: are the macOS version, MPS backend version, and unified-memory pressure pinned and re-measured against the workload over time — or is the figure being treated as a static property of the chip?
