TY - JOUR
T1 - Unsupervised domain adaptation for nuclei segmentation
T2 - Adapting from hematoxylin & eosin stained slides to immunohistochemistry stained slides using a curriculum approach
AU - Wang, Shidan
AU - Rong, Ruichen
AU - Gu, Zifan
AU - Fujimoto, Junya
AU - Zhan, Xiaowei
AU - Xie, Yang
AU - Xiao, Guanghua
N1 - Funding Information:
This work was partially supported by the National Institutes of Health [ P50CA70907 , R35GM136375 , R01DE030656 , R01GM115473 , U01CA249245 and R01GM140012 ], and the Cancer Prevention and Research Institute of Texas [ RP190107, RP230330 and RP180805 ].
Publisher Copyright:
© 2023
PY - 2023/11
Y1 - 2023/11
N2 - Background and objective: Unsupervised domain adaptation (UDA) is a powerful approach in tackling domain discrepancies and reducing the burden of laborious and error-prone pixel-level annotations for instance segmentation. However, the domain adaptation strategies utilized in previous instance segmentation models pool all the labeled/detected instances together to train the instance-level GAN discriminator, which neglects the differences among multiple instance categories. Such pooling prevents UDA instance segmentation models from learning categorical correspondence between source and target domains for accurate instance classification; Methods: To tackle this challenge, we propose an Instance Segmentation CycleGAN (ISC-GAN) algorithm for UDA multiclass-instance segmentation. We conduct extensive experiments on the multiclass nuclei recognition task to transfer knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images. Specifically, we fuse CycleGAN with Mask R-CNN to learn categorical correspondence with image-level domain adaptation and virtual supervision. Moreover, we utilize Curriculum Learning to separate the learning process into two steps: (1) learning segmentation only on labeled source data, and (2) learning target domain segmentation with paired virtual labels generated by ISC-GAN. The performance was further improved through experiments with other strategies, including Shared Weights, Knowledge Distillation, and Expanded Source Data. Results: Comparing to the baseline model or the three UDA instance detection and segmentation models, ISC-GAN illustrates the state-of-the-art performance, with 39.1% average precision and 48.7% average recall. The source codes of ISC-GAN are available at https://github.com/sdw95927/InstanceSegmentation-CycleGAN. Conclusion: ISC-GAN adapted knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images, suggesting the potential for reducing the need for large annotated pathological image datasets in deep learning and computer vision tasks.
AB - Background and objective: Unsupervised domain adaptation (UDA) is a powerful approach in tackling domain discrepancies and reducing the burden of laborious and error-prone pixel-level annotations for instance segmentation. However, the domain adaptation strategies utilized in previous instance segmentation models pool all the labeled/detected instances together to train the instance-level GAN discriminator, which neglects the differences among multiple instance categories. Such pooling prevents UDA instance segmentation models from learning categorical correspondence between source and target domains for accurate instance classification; Methods: To tackle this challenge, we propose an Instance Segmentation CycleGAN (ISC-GAN) algorithm for UDA multiclass-instance segmentation. We conduct extensive experiments on the multiclass nuclei recognition task to transfer knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images. Specifically, we fuse CycleGAN with Mask R-CNN to learn categorical correspondence with image-level domain adaptation and virtual supervision. Moreover, we utilize Curriculum Learning to separate the learning process into two steps: (1) learning segmentation only on labeled source data, and (2) learning target domain segmentation with paired virtual labels generated by ISC-GAN. The performance was further improved through experiments with other strategies, including Shared Weights, Knowledge Distillation, and Expanded Source Data. Results: Comparing to the baseline model or the three UDA instance detection and segmentation models, ISC-GAN illustrates the state-of-the-art performance, with 39.1% average precision and 48.7% average recall. The source codes of ISC-GAN are available at https://github.com/sdw95927/InstanceSegmentation-CycleGAN. Conclusion: ISC-GAN adapted knowledge from hematoxylin and eosin to immunohistochemistry stained pathology images, suggesting the potential for reducing the need for large annotated pathological image datasets in deep learning and computer vision tasks.
KW - Curriculum learning
KW - Instance segmentation
KW - Nuclear segmentation
KW - Pathology image
KW - Unsupervised domain adaptation
UR - http://www.scopus.com/inward/record.url?scp=85170438745&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85170438745&partnerID=8YFLogxK
U2 - 10.1016/j.cmpb.2023.107768
DO - 10.1016/j.cmpb.2023.107768
M3 - Article
C2 - 37619429
AN - SCOPUS:85170438745
SN - 0169-2607
VL - 241
JO - Computer Methods and Programs in Biomedicine
JF - Computer Methods and Programs in Biomedicine
M1 - 107768
ER -