A team running molecular dynamics on a tuned CUDA kernel spends three weeks shaving 12% off kernel runtime, then discovers that swapping the neighbour-list data layout from array-of-structs to struct-of-arrays cuts total simulation time by another 40% — figures illustrative of the asymmetry in which the biggest GPU speedups come from algorithmic restructuring (data layout, batching strategy, compute decomposition), not from squeezing the last few percent out of a single kernel.
This spoke explores where that boundary sits across molecular dynamics, virtual screening, and deep-learning property prediction — three workloads that dominate pharmaceutical R&D compute budgets. The structural reasoning behind the choice is covered in the parent CCU, when algorithmic restructuring gives bigger GPU speedups than kernel tuning.
Why algorithmic structure dominates kernel tuning in drug discovery
Modern NVIDIA H100 and A100 GPUs expose tens of thousands of CUDA cores optimized for SIMT execution, a stark contrast to the dozens of general-purpose cores found in a CPU socket. The practical consequence is less obvious. A kernel that achieves 60% of peak FLOPs but feeds the SMs the wrong data layout will lose more performance to memory-bandwidth stalls than any further register-allocation tuning can recover. We see this pattern regularly in drug-discovery code: the kernel is not the bottleneck; the surrounding data movement is.
Friedrichs et al. (2009) reported one of the early concrete cases — molecular dynamics on early CUDA hardware ran roughly 100× faster than equivalent CPU code, but most of that speedup came from restructuring the integration loop and neighbour-list representation, not from kernel-level tricks. Stone et al. (2010) made the same observation about NAMD and VMD: porting an algorithm to GPU without reshaping its memory access patterns leaves most of the available speedup on the table.
When you have hit the kernel-tuning ceiling
A few signals usually appear together:
- Nsight Compute or
ncureports memory throughput near peak HBM bandwidth, but compute utilisation stays modest. - Profiler traces show long stalls on
LDG/STGinstructions or on__shfl_syncwaits, not on arithmetic. - Doubling the SM clock changes wall-clock by less than 5%.
- The kernel already uses shared memory, coalesced loads, and reasonable occupancy.
When those four conditions hold simultaneously, further kernel tuning is an observed pattern of diminishing returns. The next-largest speedup lives at the algorithm level.
Three algorithmic levers that move the needle
Data layout
Representing atomic coordinates and neighbour lists efficiently stands out as the highest-leverage change in molecular dynamics and docking codes. Array-of-structs layouts ({x,y,z,charge,type} per atom) feel natural in C++ but defeat memory coalescing — adjacent threads read non-adjacent bytes. Struct-of-arrays ({all x}, {all y}, {all z}) lets a warp of 32 threads issue one coalesced load. The kernel may look almost identical; the throughput is not.
For deep-learning compound property prediction, the equivalent lever is tensor layout — NHWC versus NCHW, or the choice of mixed-precision storage. Goh et al. (2017) noted that practitioners porting cheminformatics models to GPU often gained more from precision and layout decisions than from optimising the convolution kernels themselves.
Batching strategy
Virtual screening — docking millions of compounds against a target protein — is embarrassingly parallel at the compound level but not at the kernel level. A naive implementation launches one kernel per compound and starves the GPU on launch overhead. Batched docking, where hundreds of compound poses share a single kernel launch, often produces order-of-magnitude wall-clock improvements without touching the inner loop. Brown et al. (2020) describe this as a structural property of high-throughput screening pipelines: the right batch size for a given GPU dominates per-kernel optimisation.
For training deep-learning property models, batch size interacts with three things at once: SM occupancy (too small a batch leaves SMs idle), HBM bandwidth (too large a batch spills activation memory), and gradient noise (batch size affects optimiser dynamics). On an A100 80GB training a graph neural network for ADMET prediction, the right batch size is often the difference between a 12-hour and a 3-hour epoch.
Compute decomposition
Some calculations are structurally hostile to the GPU until decomposed differently. Long-range electrostatics in molecular dynamics is the classic case — the naive O(N²) all-pairs formulation maps onto the GPU but scales badly. Particle-Mesh Ewald or Fast Multipole decompositions are algorithmically more complex but match the hardware’s strengths. Compiling MD code that already uses PME and then tuning its kernels yields a different — and much larger — final throughput than tuning kernels on the O(N²) version.
Where this leaves drug discovery teams
| Workload | Kernel-tuning ceiling | Algorithmic lever with bigger payoff |
|---|---|---|
| Molecular dynamics | Coalesced loads, shared-memory tiling done | Neighbour-list layout, PME vs all-pairs |
| Virtual screening / docking | Per-pose kernel optimised | Batched launches across compounds and poses |
| DL property / ADMET models | Mixed-precision GEMM tuned | Batch size, tensor layout, graph compilation (TorchScript, ONNX, TensorRT) |
| Genome alignment | Banded alignment vectorised | Read batching, suffix-array vs hash-index choice |
Across these rows, kernel tuning consistently delivers real engineering value within bounded returns. Algorithmic restructuring is where the next 10× tends to live (as a rule of thumb).
What a structured GPU performance analysis looks like
Auditing this domain requires more than a list of kernel timings. It is a classification: each intervention labelled as algorithmic or micro-level, with an estimated impact and an effort cost. The methodology side of this question is treated in profiling GPU kernels to find the real bottleneck, and the cross-platform implications of these algorithmic choices are covered in what cross-platform GPU performance portability actually requires.
Across our GPU engagements in life sciences and biotech, the audits that produce the largest follow-on speedups are the ones that explicitly separate “the kernel is slow” from “the algorithm is wrong for this hardware”. That separation is the first output of any GPU Performance Audit we run, before a single kernel gets touched. The two findings have very different remediation costs and very different ceilings. Conflating them is how teams end up spending engineering quarters on the wrong layer.
FAQ
When does algorithmic restructuring give a bigger GPU speedup than kernel-level tuning?
Memory bandwidth saturation, launch overhead, and suboptimal data layout push GPU drug discovery pipelines toward algorithmic restructuring even after achieving good occupancy and coalesced access. In drug discovery, this is the common case — most large wins come from changing neighbour-list representation, batching strategy, or compute decomposition (PME, batched docking), not from further kernel optimisation.
How do I tell that my GPU code has hit its kernel-tuning ceiling?
Four signals together: profiler reports memory throughput near peak HBM bandwidth but moderate compute utilisation; long stalls on global memory load/store instructions; raising SM clock changes wall-clock by less than 5% (a rule of thumb diagnostic threshold); the kernel already uses shared memory, coalesced loads, and reasonable occupancy. When those co-occur, the next gain lives at the algorithm level.
Which algorithmic changes typically unlock the biggest speedups?
In molecular dynamics: data layout (struct-of-arrays vs array-of-structs) and switching long-range electrostatics from O(N²) to PME or FMM. In virtual screening: batched docking across compounds and poses to amortise kernel-launch overhead. In deep-learning property prediction: batch size tuning, tensor layout (NHWC vs NCHW), and mixed-precision storage decisions.
How does batch size interact with GPU occupancy and memory bandwidth in deep-learning workloads?
Batch size sets three things at once. Too small and SMs sit idle because there is not enough work to fill warps — occupancy collapses. Too large and activation memory spills, forcing HBM traffic that the compute units cannot hide. The right batch size is the one that fills SMs while keeping activations in cache hierarchies as long as possible — typically found by sweeping, not by formula.
What does a structured GPU performance analysis look like beyond “make the kernel faster”?
It classifies each candidate intervention as algorithmic (data layout, batching, decomposition) or micro-level (occupancy, register pressure, instruction selection), estimates the impact and effort of each, and sequences them so that algorithmic changes happen before micro-level tuning. The micro-level work is then done on the right algorithm, not the wrong one.
References
- Brown, N., Ertl, P. and Lewis, R. (2020) Artificial Intelligence in Drug Discovery. Journal of Medicinal Chemistry, 63(16), pp. 8657–8666.
- Friedrichs, M.S., Eastman, P. and Vaidyanathan, V. (2009) Accelerating Molecular Dynamics Simulations on GPUs. Journal of Computational Chemistry, 30(6), pp. 864–872.
- Goh, G.B., Hodas, N.O. and Vishnu, A. (2017) Deep Learning for Computational Chemistry. Journal of Chemical Information and Modeling, 57(8), pp. 1757–1772.
- Stone, J.E., Hardy, D.J. and Phillips, J.C. (2010) GPU Computing in Molecular Modelling. Journal of Molecular Graphics and Modelling, 29(2), pp. 116–125.
- Zou, J., Huss, M. and Abid, A. (2019) A Primer on Deep Learning in Genomics. Nature Genetics, 51(1), pp. 12–18.
Image credits: Freepik.
