Laptop chassis is a benchmark environment variable that desktop benchmarks erase For AI inference the laptop chassis is part of the benchmark environment: TDP cap, thermal design, memory configuration, and sustained-power policy together change the measured throughput, and desktop benchmarks erase all of them. A “laptop RTX 4090” and a “desktop RTX 4090” share a model name and very little else under sustained AI load. The silicon is similar; the power envelope, thermal headroom, and memory subsystem around it are not. Laptop GPU comparisons that quote 3DMark, synthetic compute scores, or short-burst inference numbers borrow a desktop curve the laptop will never reach once it is doing real work for more than about a minute. Disclosing the chassis configuration is therefore not optional context; it is the comparison. The dominant variable is TDP — thermal design power, the watts the part is allowed to draw and dissipate in steady state. Desktop benchmarks run at the desktop TDP. Laptop variants of the same model number run at a fraction of that, often configurable by the manufacturer within a range NVIDIA or AMD defines. For AI workloads that hold the GPU busy for minutes or hours — training, long inference sequences, retrieval-augmented generation against a local model — the TDP gap is the variable that decides whether the workload completes in a reasonable wall-clock time. Everything else is secondary. This article walks through what that gap looks like in practice, which laptop GPU specifications actually matter for AI inference, and how to test a candidate machine before you commit to it. For the broader framing — why a single “best GPU” answer is the wrong question to start with — why GPU performance is not a single number covers the underlying argument; this post is the laptop-specific instance of it. TDP and sustained AI performance Laptop GPUs ship in power configurations that vary by chassis, not just by SKU. The table below shows the spread; the precise numbers come from NVIDIA’s published mobile specifications, and the sustained-performance ratios are observed-pattern figures from our own development laptops rather than a benchmarked rate. GPU variant Desktop TDP Laptop TDP range Typical sustained performance ratio RTX 4090 450 W 80–150 W roughly 30–50% of desktop RTX 4080 320 W 60–150 W roughly 35–55% of desktop RTX 4070 200 W 35–115 W roughly 40–65% of desktop RTX 4060 115 W 35–115 W roughly 45–80% of desktop The wide TDP ranges have an awkward consequence: two laptops marketed as “RTX 4070” can differ by a factor of three in delivered AI throughput, with nothing on the spec sheet to tell you which is which beyond a small footnote about the maximum graphics power. In our experience, the wattage number is more predictive of sustained AI performance than the model name above it. A second-order effect compounds this. Most laptop cooling systems can hold the rated TDP for the first 30 to 60 seconds of load — the thermal mass of the heatsink absorbs the heat — and then the fans and heat pipes have to keep up on their own. That is exactly the window most consumer GPU benchmarks run in. The benchmark sees the burst; the AI workload sees what comes after. Which laptop GPU specifications matter for AI inference? Three numbers determine practical performance for AI inference on a laptop, and the GPU model name is not one of them. Memory capacity. This sets the largest model that fits. A 16 GB laptop GPU can hold approximately an 8B-parameter model at FP16, or up to about 16B at INT8 with aggressive quantisation through tools like bitsandbytes or GPTQ. Below that, the options are CPU offloading via the PyTorch CPU device (slow), or splitting the model across CPU and GPU memory with frameworks like llama.cpp or DeepSpeed (complex and latency-sensitive). For local AI development, capacity is a hard wall before it is anything else. Memory bandwidth. This sets tokens-per-second for LLM inference and images-per-second for vision models, because both are dominated by moving weights from VRAM into the compute units. Laptop GPUs use GDDR6 or GDDR6X, typically delivering 200–500 GB/s depending on bus width and clocks. Desktop GPUs at similar price points reach 500–1000 GB/s; data-centre parts with HBM3 sit well above that. The bandwidth gap translates fairly directly into a roughly 40–60% lower inference throughput on laptops for memory-bound workloads — which is most LLM and diffusion inference, once batch size is small. TDP at the operating point. A 150 W RTX 4090 Mobile and an 80 W RTX 4090 Mobile share the same memory, the same number of CUDA cores, and the same nominal capabilities. They do not share sustained throughput. Under continuous CUDA load through PyTorch or TensorRT, the lower-TDP part clocks down to stay within thermal limits, and the practical inference rate follows. TechnoLynx has tested inference across several laptop GPU tiers in internal development environments. The pattern we see is that 8 GB VRAM with roughly 250 GB/s memory bandwidth functions as a practical floor for useful local AI development; below that, quantised 7B models run in the single-digit tokens-per-second range on the laptops we measured — fast enough to verify a pipeline runs, slow enough to disrupt iteration. Above 16 GB VRAM with roughly 400 GB/s bandwidth, most research and development tasks proceed without VRAM-bound stalls. Training remains impractical on laptop GPUs for models much larger than about 1B parameters, because thermal envelopes limit sustained throughput regardless of how much VRAM is on the part. These ranges are observed-pattern figures from our own development laptops, not a published benchmark. What to test before you buy A short burst tells you nothing useful for AI. A short, structured sustained test tells you a lot. Run all three of the following on the candidate machine before committing. Three-minute inference loop at the model size and batch shape you actually intend to use. Load the model in PyTorch or via your inference runtime of choice (vLLM, llama.cpp, ONNX Runtime), then loop generation or forward passes continuously for three minutes. Record throughput at the 0–1 minute, 1–2 minute, and 2–3 minute marks. Tokens per second for LLMs, images per second for vision models, samples per second for whatever else. If throughput drops more than roughly 15% between the first and third intervals, the system is thermally constrained. That is not a defect — it is information. Decide whether you can live with it for the work you do. Two ancillary checks are worth running once. First, verify the model actually fits: a 7B-parameter model at FP16 needs 14 GB just for weights, before activations, KV cache, and CUDA workspace; an 8 GB card cannot hold it without quantisation. Second, run the sustained test in the configuration you will actually use — on battery if that matters, on a desk without a cooling pad if that is the realistic case. Sustained performance varies materially with ambient temperature and chassis ventilation, and a benchmark on a chilled bench has limited predictive value for a hot conference room. External GPU options for laptop AI development For developers who need the laptop form factor but want more GPU than any mobile part delivers, external GPU (eGPU) enclosures sit in the middle. A Thunderbolt 4 connection provides roughly 40 Gbps to a full-size desktop GPU in an external enclosure — a real desktop RTX 4090 with its full 450 W TDP, attached over a cable. The bandwidth limitation is the catch. PCIe x16 Gen4 inside a desktop provides 256 Gbps; Thunderbolt 4 provides 40 Gbps, roughly a 6.4× reduction. For inference workloads where the model stays resident in GPU memory and only small input and output tensors cross the bus, the penalty is typically 5–15%. For training, where large batches move from host to device every iteration, the penalty is much larger. TechnoLynx has tested eGPU configurations with RTX 4090 cards for development use. The setup works well for iterative model development — code-test cycles where the GPU is used intermittently and the model stays loaded — and poorly for sustained training, where the Thunderbolt link between GPU and host memory becomes the bottleneck and effective training throughput degrades materially relative to the same card in a desktop PCIe Gen4 x16 slot. The exact gap depends on model size, batch shape, and host CPU; the 25–35% degradation we observed is an observed-pattern figure on the configurations we tested, not a published benchmark. Laptop vs cloud for AI workloads For development and prototyping, a capable laptop GPU — an RTX 4080 or 4090 Mobile at the upper end of its TDP range — provides offline capability, low latency to the model, and freedom from cloud billing. For training runs longer than a few hours, cloud GPU instances running at full TDP typically deliver better cost-efficiency per useful FLOP than sustained laptop compute, because the laptop is fighting its thermal envelope the whole time and the cloud part is not. The decision is rarely “laptop or cloud” in the abstract. It is “where does this specific workload sit on the curve of duration, model size, and iteration cadence?” Short interactive work that benefits from immediacy belongs on the laptop. Long unattended jobs belong in the cloud. The middle — multi-hour fine-tuning on small models, batch inference over moderate datasets — is where the choice actually matters, and where the TDP and bandwidth numbers above decide it. LynxBench AI treats laptop GPU evaluation for AI inference as a TDP-bounded, sustained-window measurement under realistic battery and thermal regimes, because the same silicon clocks and throttles differently in a chassis than in a desktop test harness. When evaluating any laptop-GPU AI inference comparison, ask: are the measurement window, TDP envelope, and chassis-level thermal context disclosed at the operating point the user will actually inhabit — or does the comparison borrow a desktop curve the laptop will never reach?
