Multi-Modality and Multi-View 2D CNN to Predict Locoregional Recurrence in Head Neck Cancer

Jinkun Guo, Rongfang Wang, Zhiguo Zhou, Kai Wang, Rongbin Xu, Jing Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Locoregional recurrence (LRR) remains one of leading causes in head and neck (HN) cancer treatment failure despite the advancement of multidisciplinary management. Accurately predicting LRR in early stage can help physicians make an optimal personalized treatment strategy. In this study, we propose an end-to-end multi-modality and multi-view convolutional neural network model (mMmV-CNN) for LRR prediction in HN cancer. In mMmV, a dimension reduction operator is designed, projecting the 3D volume onto 2D images in different directions, and a multi-view strategy is used to replace the original 3D method, which reduces the complexity of the algorithm while preserving important 3D information. Meanwhile, multi-modal data is used for the classification by making full use of the complementary information from cross modality data. Furthermore, we design a multi-modality deep neural network which is trained in an end-to-end manner and jointly optimize the deep features of CT, PET and clinical features. A HN dataset which consists of 206 patients was used to evaluate the performance. Experimental results demonstrated that mMm V-CNN can obtain an AUC value of 0.81 and outperform a state of the art CNN-based method.

Original languageEnglish (US)
Title of host publicationIJCNN 2021 - International Joint Conference on Neural Networks, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9780738133669
StatePublished - Jul 18 2021
Event2021 International Joint Conference on Neural Networks, IJCNN 2021 - Virtual, Shenzhen, China
Duration: Jul 18 2021Jul 22 2021

Publication series

NameProceedings of the International Joint Conference on Neural Networks


Conference2021 International Joint Conference on Neural Networks, IJCNN 2021
CityVirtual, Shenzhen


  • head and neck cancer
  • local recurrence
  • multi-view convolutional neural network
  • outcome prediction

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Multi-Modality and Multi-View 2D CNN to Predict Locoregional Recurrence in Head Neck Cancer'. Together they form a unique fingerprint.

Cite this