Segmentation of the prostate and organs at risk in male pelvic CT images using deep learning

Samaneh Kazemifar, Anjali Balagopal, Dan Nguyen, Sarah McGuire, Raquibul Hannan, Steve Jiang, Amir Owrangi

Research output: Contribution to journalArticlepeer-review

68 Scopus citations


Inter-and intra-observer variation in delineating regions of interest (ROIs) occurs because of differences in expertise level and preferences of the radiation oncologists. We evaluated the accuracy of a segmentation model using the U-Net structure to delineate the prostate, bladder, and rectum in male pelvic CT images. The dataset used for training and testing the model consisted of raw CT scan images of 85 prostate cancer patients. We designed a 2D U-Net model to directly learn a mapping function that converts a 2D CT grayscale image to its corresponding 2D OAR segmented image. Our network contains blocks of convolution 2D layers with variable kernel sizes, channel number, and activation functions. On the left side of the U-Net model, we used three 3 × 3 convolutions, each followed by a rectified linear unit (ReLu) (activation function), and one max pooling operation. On the right side of the U-Net model, we used a 2 × 2 transposed convolution and two 3 × 3 convolution networks followed by a ReLu activation function. The automatic segmentation using the U-Net generated an average dice similarity coefficient (DC) and standard deviation (SD) of the following: DC ± SD (0.88 ± 0.12), (0.95 ± 0.04), and (0.92 ± 0.06) for the prostate, bladder, and rectum, respectively. Furthermore, the mean of average surface Hausdorff distance (ASHD) and SD were 1.2 ± 0.9 mm, 1.08 ± 0.8 mm, and 0.8 ± 0.6 mm for the prostate, bladder, and rectum, respectively. Our proposed method, which employs the U-Net structure, is highly accurate and reproducible for automated ROI segmentation. This provides a foundation to improve automatic delineation of the boundaries between the target and surrounding normal soft tissues on a standard radiation therapy planning CT scan.

Original languageEnglish (US)
Article number055003
JournalBiomedical Physics and Engineering Express
Issue number5
StatePublished - Jul 23 2018


  • artificial intelligence organ contouring
  • deep machine learning
  • male pelvic region
  • neural network
  • prostate
  • segmentation

ASJC Scopus subject areas

  • Nursing(all)


Dive into the research topics of 'Segmentation of the prostate and organs at risk in male pelvic CT images using deep learning'. Together they form a unique fingerprint.

Cite this