Improving performance and reliability of interactive CAD schemes

An interactive computer-aided detection or diagnosis (ICAD) scheme allows observers to query suspicious abnormalities (lesions) depicted on medical images. Once a suspicious region is queried, ICAD segments the abnormal region, computes a set of image features, searches for and identifies the reference regions depicted on the verified lesions that are similar to the queried one. Based on the distribution of the selected similar regions, ICAD generates a detection (or classification) score of the queried region depicting true-positive disease. In this study, we assessed the performance and reliability of an ICAD scheme when using a database including total 1500 positive images depicted verified breast masses and 1500 negative images depicted ICAD-cued false-positive regions as well as the leave-one-out testing method. We conducted two experiments. In the first experiment, we tested the relationship between ICAD performance and the size of reference database by systematically increasing the size of reference database from 200 to 3000 images. In the second experiment, we tested the relationship between ICAD performance and the similarity level between the queried image and the retrieved similar references by applying a set of thresholds to systematically remove the queried images whose similarity level to their most «similar» reference images are lower than threshold. The performance was compared based on the areas under ROC curves (AUC). The results showed that (1) as the increase of reference database, AUC value monotonically increased from 0.636±0.041 to 0.854±0.004 and (2) as the increase of similarity threshold values, AUC value also monotonically increased from 0.854±0.004 to 0.932±0.016. The increase of AUC values and the decrease of their standard deviations indicate the improvement of both CAD performance and reliability. The study suggested that (1) assembling the large and diverse reference databases and (2) assessing and reporting the reliability of ICAD-generated results based on the similarity measurement are important in development and application of the ICAD schemes.