Kris T. Huang, MD, PhD, CTO

On the instabilities of deep learning

There’s no free lunch.

Can you see it? 

Source article here (PNAS).

TL;DR:

• Deep learning (DL) has attracted intense interest in image classification and reconstruction.
• However, DL typically produces unstable methods subject to failure with small, nearly invisible perturbations (including plain signal noise).
• For image classification, this can result in misclassification. For image reconstruction, this can cause severe artifacts and missing image structures (like small tumors).
• The instability phenomenon is not easy to remedy, and is seemingly ubiquitous despite a variety of tested DL network types.

Over the past couple of decades, deep learning (DL) has helped advance the automation of certain problems, such as classification of handwriting, objects, and speech. More recently, it has attracted a lot of attention in certain medical applications, such as image reconstruction, and at least a bit of consideration in image registration. In the case of reconstruction, the prospect of producing attractive, seemingly pristine MR images from a quick, undersampled k-space is pretty enticing. But alas, there is no free lunch.

On top of the fundamental limit of how good a recon can be with restricted scan time (and data),  there is mounting evidence that DL produces unstable models that can unexpectedly fail, even with minor data perturbations like patient movement. For diagnosticians, this means blurred or omitted details, and potentially, missed diagnoses and patient harm.

Image registration presents a particularly interesting puzzle because of the lack of large datasets with known transformations, in addition to the theoretical prediction and empirical observation of deep learning instability. That’s what makes the registration technology in Autofuse so unique: it is AI but not deep learning, based on human vision but requires no training, and it merges computer vision with more classical engineering techniques to create a stable, general solution that rivals DL at its best, with none of the weaknesses.

Pre-flight checklist

The technology in Autofuse has always been developed and tested with the most demanding synthetic and clinical data. Now that software is maturing with a more complete feature set, Pymedix is gearing up to pilot Autofuse at academic medical centers! Stay tuned.