ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate

Wei Shao; Linda Banh; Christian A Kunder; Richard E Fan; Simon J C Soerensen; Jeffrey B Wang; Nikola C Teslovich; Nikhil Madhuripan; Anugayathri Jawahar; Pejman Ghanouni; James D Brooks; Geoffrey A Sonn; Mirabela Rusu

doi:10.1016/j.media.2020.101919

ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate

Med Image Anal. 2021 Feb:68:101919. doi: 10.1016/j.media.2020.101919. Epub 2020 Dec 17.

Authors

Affiliations

¹ Department of Radiology, Stanford University, Stanford, CA 94305, USA. Electronic address: weishao@stanford.edu.
² Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA.
³ Department of Pathology, Stanford University, Stanford, CA 94305, USA.
⁴ Department of Urology, Stanford University, Stanford, CA 94305, USA.
⁵ School of Medicine, Stanford University, Stanford, CA 94305, USA.
⁶ Department of Radiology, University of Colorado, Aurora, CO 80045, USA.
⁷ Loyola University Medical Center, Maywood, IL 60153, USA.
⁸ Department of Radiology, Stanford University, Stanford, CA 94305, USA.
⁹ Department of Radiology, Stanford University, Stanford, CA 94305, USA; Department of Urology, Stanford University, Stanford, CA 94305, USA.
¹⁰ Department of Radiology, Stanford University, Stanford, CA 94305, USA. Electronic address: mirabela.rusu@stanford.edu.

Abstract

Magnetic resonance imaging (MRI) is an increasingly important tool for the diagnosis and treatment of prostate cancer. However, interpretation of MRI suffers from high inter-observer variability across radiologists, thereby contributing to missed clinically significant cancers, overdiagnosed low-risk cancers, and frequent false positives. Interpretation of MRI could be greatly improved by providing radiologists with an answer key that clearly shows cancer locations on MRI. Registration of histopathology images from patients who had radical prostatectomy to pre-operative MRI allows such mapping of ground truth cancer labels onto MRI. However, traditional MRI-histopathology registration approaches are computationally expensive and require careful choices of the cost function and registration hyperparameters. This paper presents ProsRegNet, a deep learning-based pipeline to accelerate and simplify MRI-histopathology image registration in prostate cancer. Our pipeline consists of image preprocessing, estimation of affine and deformable transformations by deep neural networks, and mapping cancer labels from histopathology images onto MRI using estimated transformations. We trained our neural network using MR and histopathology images of 99 patients from our internal cohort (Cohort 1) and evaluated its performance using 53 patients from three different cohorts (an additional 12 from Cohort 1 and 41 from two public cohorts). Results show that our deep learning pipeline has achieved more accurate registration results and is at least 20 times faster than a state-of-the-art registration algorithm. This important advance will provide radiologists with highly accurate prostate MRI answer keys, thereby facilitating improvements in the detection of prostate cancer on MRI. Our code is freely available at https://github.com/pimed//ProsRegNet.

Keywords: Histopathology; Image registration; MRI; deep learning; prostate cancer; radiology-pathology fusion.

Publication types

Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

MeSH terms

Algorithms
Deep Learning*
Humans
Magnetic Resonance Imaging
Male
Prostatic Neoplasms* / diagnostic imaging

Abstract

Publication types

MeSH terms

Grants and funding