Incorporating minimal user input into deep learning based image segmentation

Maysam Shahedi, Martin Halicek, James D. Dormer, Baowei Fei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations


Computer-assisted image segmentation techniques could help clinicians to perform the border delineation task faster with lower inter-observer variability. Recently, convolutional neural networks (CNNs) are widely used for automatic image segmentation. In this study, we used a technique to involve observer inputs for supervising CNNs to improve the accuracy of the segmentation performance. We added a set of sparse surface points as an additional input to supervise the CNNs for more accurate image segmentation. We tested our technique by applying minimal interactions to supervise the networks for segmentation of the prostate on magnetic resonance images. We used U-Net and a new network architecture that was based on U-Net (dual-input path [DIP] U-Net), and showed that our supervising technique could significantly increase the segmentation accuracy of both networks as compared to fully automatic segmentation using U-Net. We also showed DIP U-Net outperformed U-Net for supervised image segmentation. We compared our results to the measured inter-expert observer difference in manual segmentation. This comparison suggests that applying about 15 to 20 selected surface points can achieve a performance comparable to manual segmentation.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2020
Subtitle of host publicationImage Processing
EditorsIvana Isgum, Bennett A. Landman
ISBN (Electronic)9781510633933
StatePublished - 2020
EventMedical Imaging 2020: Image Processing - Houston, United States
Duration: Feb 17 2020Feb 20 2020

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2020: Image Processing
Country/TerritoryUnited States


  • Convolutional neural network (CNN)
  • Deep learning
  • Image segmentation
  • MRI
  • Prostate

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Incorporating minimal user input into deep learning based image segmentation'. Together they form a unique fingerprint.

Cite this