Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features

Shulong Li, Panpan Xu, Bin Li, Liyuan Chen, Zhiguo Zhou, Hongxia Hao, Yingying Duan, Michael R Folkert, Jianhua Ma, Shiying Huang, Steve Jiang, Jing Wang

Research output: Contribution to journalArticlepeer-review

45 Scopus citations


To predict lung nodule malignancy with a high sensitivity and specificity for low dose CT (LDCT) lung cancer screening, we propose a fusion algorithm that combines handcrafted features (HF) into the features learned at the output layer of a 3D deep convolutional neural network (CNN). First, we extracted twenty-nine HF, including nine intensity features, eight geometric features, and twelve texture features based on grey-level co-occurrence matrix (GLCM). We then trained 3D CNNs modified from three 2D CNN architectures (AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at the output layer. For each 3D CNN, the CNN features combined with the 29 HF were used as the input for the support vector machine (SVM) coupled with the sequential forward feature selection (SFS) method to select the optimal feature subset and construct the classifiers. The fusion algorithm takes full advantage of the HF and the highest level CNN features learned at the output layer. It can overcome the disadvantage of the HF that may not fully reflect the unique characteristics of a particular lesion by combining the intrinsic CNN features. Meanwhile, it also alleviates the requirement of a large scale annotated dataset for the CNNs based on the complementary of HF. The patient cohort includes 431 malignant nodules and 795 benign nodules extracted from the LIDC/IDRI database. For each investigated CNN architecture, the proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and specificity scores among all competitive classification models.

Original languageEnglish (US)
Article number175012
JournalPhysics in medicine and biology
Issue number17
StatePublished - Sep 4 2019


  • convolutional neural network
  • fusion algorithm
  • handcrafted feature
  • lung nodule malignancy
  • radiomics

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features'. Together they form a unique fingerprint.

Cite this