The constraint
The pipeline ran a 3D object detection model over full volumetric CT scans — hundreds of slices per study, processed as a single volume. On our development machine, an RTX 6000 Ada with 48 GB of VRAM, it worked fine. Peak allocation sat around 44 GB.
That was fine until it wasn't. We needed to run on additional machines to scale, and the RTX 6000 Ada costs several times what a 16 GB consumer card does. The ask was blunt: make it run on an RTX 5070 Ti. Not "make it run worse on a 5070 Ti" — the outputs had to be identical, because the model's predictions had already been validated and we weren't about to revalidate them against a new configuration.
So: a 4× memory reduction with zero tolerance for output drift. My first instinct was that this was impossible without quantisation or a smaller model. That instinct was wrong, and finding out why was the interesting part.
Where the memory was actually going
Before touching anything, I profiled. This matters more than it sounds — my assumption going in was that model weights were the problem. They weren't. The weights were a rounding error.
The memory was going almost entirely into intermediate activations during the forward pass over large input patches. A 3D convolutional network holds feature maps for every layer, and those feature maps scale with the cube of the spatial dimensions. Double the patch size in each axis and you're not doubling memory — you're multiplying it by eight.
# rough shape of the problem patch = (512, 512, 192) # ~50M voxels activations ≈ O(voxels × channels × depth) peak allocated = 44.0 GB # reduce Z-depth → ~3x fewer voxels per patch patch = (512, 512, 64) # ~16.7M voxels peak allocated = 10.59 GB
The pipeline was also, in some configurations, falling back to processing the whole volume in one pass rather than tiling it. Sliding-window inference existed in the code, but it wasn't being enforced — under certain input dimensions the window covered the entire volume, which is functionally the same as not having a window at all. That was the quiet bug underneath the loud problem.
The most useful thing I learned here: when a model "needs" a huge GPU, check whether it's the model or the input pipeline. Nine times out of ten it's the input pipeline, and that's the cheap thing to fix.
What I tried
1. Mixed precision — helped, not enough
The obvious first move. Automatic mixed precision cut allocation meaningfully but nowhere near 4×, and it introduced a question I didn't want to answer: does reduced precision change the detection scores near the decision threshold? Sometimes, marginally. "Marginally" is not "identically", so this couldn't be the whole answer.
2. Smaller batch size — already at one
A dead end I checked anyway. Inference was already running one volume at a time. There was no batch dimension left to shrink.
3. Patch size sweep — this was it
I ran a systematic sweep over patch dimensions, measuring peak allocated memory and wall-clock time for each, then compared detection outputs against the 48 GB reference run. The trade-off is straightforward once you see it laid out: smaller patches mean less memory and more windows to process, so you pay in time. I also discovered that shrinking the X and Y axes too much created dangerous edge-effect artifacts.
| Patch size | Peak VRAM | Fits 16GB? | Output parity |
|---|---|---|---|
| 512 × 512 × 192 | 44.0 GB | no (OOM) | reference |
| 512 × 512 × 100 | ~22.0 GB | no (OOM) | match |
| 512 × 512 × 80 | >16.0 GB | no (OOM) | match |
| 192 × 192 × 80 | <16.0 GB | yes | failed (extra false positives) |
| 512 × 512 × 64 | 10.59 GB | yes | match |
512 × 512 × 64 was the sweet spot. It maintained the full axial context (X, Y) to prevent edge false-positives,
while reducing the Z-depth just enough to sit comfortably inside 16 GB with headroom for
the driver and display, without shrinking patches so far that the window count made inference
unacceptably slow.
4. Enforcing the sliding window
The fix that made the patch size actually stick. Rather than letting the window size be derived from input dimensions, I made it explicit and non-negotiable in configuration, with an assertion that fails loudly if a volume would ever be processed in a single pass. A crash at startup is infinitely better than an out-of-memory error forty minutes into a batch run.
Proving nothing changed
This was the part that took longest, and the part that mattered most. A memory optimisation nobody trusts is a memory optimisation nobody uses.
I ran both configurations across a held-out set of studies and compared:
- Detection counts per study — same objects found, no more, no fewer
- Confidence scores — compared per-detection, not just in aggregate, because averages hide individual drift
- Bounding box coordinates — to catch any seam artefacts introduced at window boundaries
Window boundaries were the risk I was most worried about. If an object sits across the edge of two patches, does the model see it twice, or half of it twice? The overlap and blending settings in the sliding-window implementation handle this, but "handle this" needed to be demonstrated, not assumed. The outputs matched exactly.
Results
| Metric | Before | After |
|---|---|---|
| Peak VRAM | ~44 GB | 10.59 GB |
| Minimum viable GPU | RTX 6000 Ada (48GB) | RTX 5070 Ti (16GB) |
| Detection output | reference | identical |
| Hardware cost per node | ~$7,000 | ~$900 |
The practical outcome was that we could add cluster capacity by buying consumer cards, and the configuration became the one used for processing real batches on our newer machines.
What I'd do differently
I'd profile before theorising. I spent the first half day reasoning about model architecture when ten minutes with a memory profiler would have pointed straight at the input patches. I now start every performance problem with measurement, not hypothesis.
I'd make the parity check a permanent test, not a one-off. I verified output equivalence manually for this change. It should have become a regression test that runs on every release — the exact comparison I did by hand, automated. That's a gap I'd close if I were starting again.
I'd document the trade-off curve, not just the answer. The table above is more useful to the next person than the single chosen configuration, because their constraint might be different from mine. I eventually wrote this up as a hardware profiling report, but it should have existed from day one.