TY - JOUR
T1 - Characterization of indeterminate breast lesions on B-mode ultrasound using automated machine learning models
AU - Wang, Shuo
AU - Niu, Sihua
AU - Qu, Enze
AU - Forsberg, Flemming
AU - Wilkes, Annina
AU - Sevrukov, Alexander
AU - Nam, Kibo
AU - Mattrey, Robert F.
AU - Ojeda-Fournier, Haydee
AU - Eisenbrey, John R.
N1 - Funding Information:
For the original clinical trial from which data were obtained, the ultrasound contrast agent was provided by Lantheus Medical Imaging and the ultrasound scanner was provided by GE Healthcare. The work was funded in part by the National Institutes of Health (R01 CA140338) and the Department of Defense (Grant No. W81XWH-11-1-0630).
Publisher Copyright:
© 2020 Society of Photo-Optical Instrumentation Engineers (SPIE)
PY - 2020/9/1
Y1 - 2020/9/1
N2 - Purpose: While mammography has excellent sensitivity for the detection of breast lesions, its specificity is limited. Adjunct screening with ultrasound may partially alleviate this issue but also increases false positives, resulting in unnecessary biopsies. Our study investigated the use of Google AutoML Vision (Mountain View, California), a commercially available machine learning service, to both identify and characterize indeterminate breast lesions on ultrasound. Approach: B-mode images from 253 independent cases of indeterminate breast lesions scheduled for core biopsy were used for model creation and validation. The performances of two sub-models from AutoML Vision, the image classification model and object detection model, were evaluated, while also investigating training strategies to enhance model performances. Pathology from the patient’s biopsy was used as a reference standard. Results: The image classification models trained under different conditions demonstrated areas under the precision–recall curve (AUC) ranging from 0.85 to 0.96 during internal validation. Once deployed, the model with highest internal performance demonstrated a sensitivity of 100% [95% confidence interval (CI) of 73.5% to 100%], specificity of 83.3% (CI ¼ 51.6% to 97.9%), positive predictive value (PPV) of 85.7% (CI ¼ 62.9% to 95.5%), and negative predictive value (NPV) of 100% (CI non-evaluable) in an independent dataset. The object detection model demonstrated lower performance internally during development (AUC ¼ 0.67) and during prediction in the independent dataset [sensitivity ¼ 75% (CI ¼ 42.8 to 94.5), specificity ¼ 80% (CI ¼ 51.9 to 95.7), PPV ¼ 75% (CI ¼ 50.8 to 90.0), and NPV ¼ 80% (CI ¼ 59.3% to 91.7%)], but was able to demonstrate the location of the lesion within the image. Conclusions: Two models appear to be useful tools for identifying and classifying suspicious areas on B-mode images of indeterminate breast lesions.
AB - Purpose: While mammography has excellent sensitivity for the detection of breast lesions, its specificity is limited. Adjunct screening with ultrasound may partially alleviate this issue but also increases false positives, resulting in unnecessary biopsies. Our study investigated the use of Google AutoML Vision (Mountain View, California), a commercially available machine learning service, to both identify and characterize indeterminate breast lesions on ultrasound. Approach: B-mode images from 253 independent cases of indeterminate breast lesions scheduled for core biopsy were used for model creation and validation. The performances of two sub-models from AutoML Vision, the image classification model and object detection model, were evaluated, while also investigating training strategies to enhance model performances. Pathology from the patient’s biopsy was used as a reference standard. Results: The image classification models trained under different conditions demonstrated areas under the precision–recall curve (AUC) ranging from 0.85 to 0.96 during internal validation. Once deployed, the model with highest internal performance demonstrated a sensitivity of 100% [95% confidence interval (CI) of 73.5% to 100%], specificity of 83.3% (CI ¼ 51.6% to 97.9%), positive predictive value (PPV) of 85.7% (CI ¼ 62.9% to 95.5%), and negative predictive value (NPV) of 100% (CI non-evaluable) in an independent dataset. The object detection model demonstrated lower performance internally during development (AUC ¼ 0.67) and during prediction in the independent dataset [sensitivity ¼ 75% (CI ¼ 42.8 to 94.5), specificity ¼ 80% (CI ¼ 51.9 to 95.7), PPV ¼ 75% (CI ¼ 50.8 to 90.0), and NPV ¼ 80% (CI ¼ 59.3% to 91.7%)], but was able to demonstrate the location of the lesion within the image. Conclusions: Two models appear to be useful tools for identifying and classifying suspicious areas on B-mode images of indeterminate breast lesions.
KW - Artificial intelligence
KW - Breast lesions
KW - Deep learning
KW - Machine learning
KW - Ultrasound imaging
UR - http://www.scopus.com/inward/record.url?scp=85096695629&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096695629&partnerID=8YFLogxK
U2 - 10.1117/1.JMI.7.5.057002
DO - 10.1117/1.JMI.7.5.057002
M3 - Article
AN - SCOPUS:85096695629
SN - 0720-048X
VL - 7
JO - Journal of Medical Imaging
JF - Journal of Medical Imaging
IS - 5
M1 - 057002-1
ER -