↓ Skip to main content
Side Effects
  1. Home projects/

Faster CT reconstruction: GPU performance engineering

··Index: 9·
Cuda C++ Gpu Computed Tomography Performance
Table of Contents

My day job is software for industrial X-ray computed tomography. A scanner takes thousands of X-ray images of an object while it rotates, and a reconstruction engine turns them into a 3D volume. I spent a year trying to make one such GPU engine a lot faster.

The code belongs to my employer, so I don’t name products, customers or data sets, and all of the results are relative: the same data on the same machine, before and after.

The problem
#

Volumes are routinely bigger than GPU memory, which is why the engine builds them in slabs, and for every slab it used to upload the whole input again. Over one run that added up to about 35 times the size of the input sent to the GPU. On top of that the GPU spent much of its time waiting, and at the end of each run the volume writer blocked it.

What I did
#

  • Upload only the detector rows a slab needs, and keep the rest cached on the GPU.
  • Overlap copies with computation, so the GPU doesn’t sit idle.
  • Check memory before the run. A machine that can’t do the job says so at once, not after an hour.
  • Write the volume in parallel.
Timeline of one reconstruction before and after: copies, computation and writing run one after another before, and overlap after.Before: the whole input is re-sent for every slabcopycomputewriteblocks the GPUAfter: only the needed rows, copy overlaps computecopycomputewriteparallel writeSchematic, not to scale. Whole run: more than 2 times faster. Data sent to the GPU: 89% to 99.8% less.
Schematic of one run before and after the changes

Results
#

WhatChange
End-to-end reconstructionmore than 2× faster
Data sent to the GPU89% to 99.8% less
Volume writing2.3× faster
Run start-up110× faster
Largest volumesfrom “does not fit” to a normal run

Not every idea paid off. One CPU vectorisation made single loops 19× faster and changed nothing for the whole run, because CPU work was under 1% of the critical path, so I didn’t ship it.

Same image, bit for bit
#

Speed is worthless if images change, so every change I made had to reproduce reference volumes exactly. A test tool compares them slice by slice, and along the way it found an old bug in the original code, a small drift of detector rows between slabs. Since the fix, a volume built in one slab and the same volume built in eight are identical.

Tools around it
#

I also built a live viewer to see what goes on inside the GPU pipeline. It runs in a separate process, so a crash of the viewer can never stop a reconstruction.

In the same year I moved a scientific charting library from .NET Framework to .NET 9. With 100 charts open, the new version is about 12× faster and needs 8.6× less memory.

Related