By The B-Team: Jason Phang, Jungkyu (JP) Park, Thibault Fevry, Zhe Huang
Overview
- Data: >200k screening mammograms, >Sk biopsy reports
- To effectively apply deep learning methods to breast cancer detection, many sub-problems need to be solved
- We catalog our significant progress on multiple sub-problems, each contributing improved performance and newfound insight
Contributions
- Patchwise classification model using pretrained convolutional layers attain significant validation AUC-used to generate features for exam-level cancer classification model
- Mammogram+Patchwise features lead to 98% test AUC on cancer classification for both benign and malignant classes
- Mammogram-wise image registration allows for effective incorporation of prior exam information and improved performance on Bl-RADS and cancer classification
- Adapted Mask-RCNN for high-resolution mammograms for lesion detection and localization
Multi-View CNN for Cancer Detection
- ResNet-based architecture, pretrained on Bl-RADS classification
- Four view: [L-CC, R-CC, L-MLO, R-MLO]
- Multi-class prediction:
- [Benign/Not Benign], [Malignant/Not Malignant]
Patch Classification
- Sample 256×256 patches from high-resolution mammograms
- Patch classification allows model to solely focus on local information for cancer detection, while allowing for higher-capacity models
- Models used: DenseNet and ResNet
- Transfer learning: Initializing weights from models pretrained on lmageNet led to better performance, over training from
- scratch
- Ensembling: Based on a 21-model library, we composed an ensemble using forward stepwise selection
Model | Val AUC | Total AUC |
---|---|---|
Model Ensemble | 0.875 | 0.801 |
Best Single Model (DenseNet) | 0.856 | 0.777 |
Exam-level Cancer Prediction
- Using a patch-classification model, we generate a map of predicted probabilities, and use them as an additional input channel to our ResNet
- Combination of local and global information from mammograms
Overall Population

Biopsy Population

Lesion detection using Mask-RCNN
- Goal: Adapt Mask-RCNN for high-resolution, multi-view,
noisy-labeled images - Modifications: Image-level prediction branch, Lowering loU
threshold, raising NMS threshold, prediction over multiple views
Architecture of Mask-RCNN

Results

Sample Predictions

Incorporating Prior Exam Information

- Baseline: Concatenate representation of each exam after pooling layer (BI-RADs, 0=lncomplete, l=Negative/Normal, 2=Benign)
- Image registration:

first image, affine transformation which optimizes mutual information to the second
image is applied. (d) Overlap of the second and third images. Now the downscaled spatial
representation before pooling should roughly be aligned.
- Improved model: Register pair of images, concatenate the representation of each exam before pooling layer, add additional convolutional layer. Test AUC improves as follows.
Model | MAC AUC | 0 Vs. All (Not Cancer) | 1 Vs. All (Cancer) | 2 Vs. All (Unknown) |
---|---|---|---|---|
One Exam Network | 0.743 | 0.708 | 0.798 | 0.725 |
Two Exam Network | 0.763 | 0.726 | 0.821 | 0.745 |
Acknowledgements
We would like to acknowledge Krzysztof Geras, Dr. Linda Moy, Dr. Laura Heacock, Nan Wu, and Yiqi Shen for their invaluable help and mentorship.