Predict Total Knee Replacement Using MRI With Supervised and Semi-Supervised Networks Poster

By Team Glosy: Hong Gao, Mingsi Long, Yulin Shen, and Jie Yang

View Image of Original Poster

Objectives

Osteoarthritis (OA) is a synovial joint tissues disease which lead to pain and long term disabilities. Overtime, patients may develop sufficient progression such that they are eligible for total knee replacement (TKR). Knowing this ahead of time can help clinicians and researchers to start early intervention including clinical trials to the right patients. One way is to use deep learning in Magnetic Resonance Imaging (MRI) to learn representations that signal the TKR. Goal: Use 3D MRI to predict the probability of TKR within eight years.

Example of Data Used

Data example

Data Sources & Cohorts

Data Source: NIH Osteoarthritis Initiative (OAI)

  1. Multicenter, prospective, longitudinal
  2. 4,796 participants ages 45-79 from 2004
  3. Clinical information including 3D MRI
  4. About 10% of the patients received TKR

Supervised Cohort

There are 1,170 patients in the supervised cohort. We define cases as individuals with a positive OA progression outcome in either knee after the baseline enrollment date; individuals who receive a partial knee replacement in either knee or have a knee replacement before baseline are removed. We define controls as individuals who appear at the 96-month visit and do not have a positive OA progression outcome in either knee by that visit.

Unsupervised Cohort

The subjects who are not selected for the supervised cohort will form unsupervised cohort. This cohort includes 3,626 subjects.

Results: Supervised

chart of training and validation loss: train accuracy vs validation accuracy
Figure 1: GlosyNet Train and Validation Loss
chart of training and validation loss: train accuracy vs validation accuracy
Figure 2: GlosyNet Train and Validation Accuracy

Results: Semi-Supervised Learning

Chart of Receiver Operating Characteristic: False Positive Rate vs. True Positive Rate
Figure 3: LadderNet Receiver Operating Characteristic
Chart of Ladder Loss in Training: Epoch vs Cost
Figure 4: LadderNet Learning Curve

Method 1

Our supervised learning model GlosyNet is based on the Very
Deep Convolutional Networks (VGG). It took 1,170 fixed-size
448 × 448 × 160 3 dimensional MRI images, cropped them to size of 348 × 348 × 32 combined with various augmentation mechanisms including randomly crop and flip. Processed images are passed through a stack of convolutional (conv.) layers, where we used 3 × 3 filters (Figure 5).

GlosyNet chart
Figure 5: GlosyNet

Method 2

We used Ladder Network, a semi-supervised network. It make use of both labeled and unlabeled images for training. We used GlosyNet as the encoder and generated the cross entropy error. The unsupervised loss is the mean square error of the original image representation and the reconstructed image representation with noise from the decoder.

LadderNet chart
Figure 6: LadderNet

Recommended Future Steps

  • Incorporate patient demographics and clinical information, such as pain level
  • Tune hyperparameter of both GlosyNet and the LadderNet based on GlosyNet
  • Take advantage of more data collected from the dataset

Sponsor

Sponsor: Cem Deniz, PhD

Department NYU Department of Radiology Web

References

  1. Rasmus A. et al. Semi-supervised learning with ladder networks. MIT Press, 2015.
  2. Simonyan K. Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv, page 1409.1556, 2014.
← Back to Capstone Page