Why the tracker kept merging two different objects

The problem, in one sentence

When the same subject is scanned at two different times, the system has to decide which object in scan B is the same object as one in scan A. Get that wrong and you don't just lose an object — you invent a history for it. Two separate things get reported as one thing that changed over time.

That's what was happening. Distinct objects sitting near each other were being linked across timepoints as if they were the same one, and the resulting measurements were meaningless.

Inheriting code that reports success

I took over this part of the pipeline from a colleague who had left. The handover documentation said the registration and tracking evaluation had been run and passed.

The first thing I did was re-run it, because I wanted to understand the code by watching it work. It didn't work. Some runs crashed outright in the registration stage. Others completed and wrote output that, when I actually opened it and looked, was wrong.

Two bugs, stacked

Bug one: registration was crashing

Before you can compare two scans you have to align them. The registration stage ran a chain of transforms — Rigid, then Affine, then BSpline — each refining the previous.

The Affine stage was where Elastix was falling over. On certain input pairs it would fail outright, and on others it would produce a transform so distorted that everything downstream inherited the damage. I spent a while trying to tune it into stability before asking a more useful question: what is the Affine stage actually contributing here?

For our data — the same subject, same scanner, months apart — the answer was: not much. The meaningful correction is rigid (position and orientation) followed by deformable (BSpline). The affine step in the middle was adding scaling and shear degrees of freedom that our data didn't need, and those extra degrees of freedom were exactly what was letting the optimiser wander off into failure.

Removing it made registration stable and, on visual comparison, no worse in quality. Deleting a stage felt like the wrong kind of fix until I could show the outputs were equivalent. Then it just felt correct.

# before — three stages, unstable
Rigid → Affine → BSpline      # crashes, distorted transforms

# after — two stages, stable, same result quality
Rigid → BSpline

Bug two: matching had no physical grounding

With registration fixed, the merging problem was still there. The matching logic was comparing candidate objects using overlap in voxel index space — the raw array coordinates — without properly accounting for the fact that two scans can have different slice spacing and different origins.

Two objects that are centimetres apart in the real world can look adjacent in index space if the two volumes are sampled differently. The matcher was, in effect, comparing apples to a different grid of apples.

I made three changes:

1. Match in physical space, not index space. Convert coordinates using the image's spacing, origin and direction before any comparison. This alone removed a large share of the bad matches.
2. Require Z-axis slice overlap. Two candidates are only allowed to match if their extents along the through-plane axis actually intersect. Objects that don't occupy any common slice range cannot be the same object, regardless of how similar they look in-plane. This was the single highest-impact rule.
3. Tune the Dice threshold deliberately. The overlap threshold that decides "same object" had been set to a default nobody had revisited. I swept it against ground-truth matches and picked a value where false merges dropped without legitimate matches starting to break.

The trade-off nobody can avoid

Every matching threshold sits on a spectrum between two failure modes:

Threshold	Failure mode	Consequence
Too permissive	false merges	Two objects reported as one with a fabricated history
Too strict	false splits	One object reported as new each time, history lost
Tuned	balanced	Errors pushed to genuinely ambiguous cases

You cannot eliminate both. What you can do is decide which is worse in your context and tune accordingly — and be explicit about that decision so the next person doesn't quietly reverse it. In our case a false merge is more damaging than a false split, because a fabricated history is actively misleading while a lost history is merely incomplete.

Making it verifiable

The fix wasn't finished when the code worked. It was finished when someone other than me could confirm it worked.

I ran the old and new pipelines side by side on the same studies and produced visual comparisons — same case, previous logic on the left, new logic on the right — so a reviewer could see the difference rather than take my word for it. I also automated the generation of evaluation sheets so that reviewing a batch became a normal task rather than a research project, and wrote an SOP describing what the tracking logic guarantees and what it doesn't.

What I took from it

Re-run inherited code before trusting it. Not out of distrust of the person who wrote it — out of the recognition that "it ran" and "it was right" are different claims, and handover notes rarely distinguish them.

Removing a stage is a legitimate fix. My instinct was to make the Affine step work. The better question was whether it should exist. Complexity that isn't earning its keep is a liability, and it usually fails first.

Geometry is not a detail. Spacing, origin and direction are metadata that's easy to ignore right up until it silently corrupts your results. Any comparison between two volumes belongs in physical space.