Conditional Generative Adversarial Network (cGAN) for Synthesis of Digital Histologic Images from Hyperspectral Images

Hyperspectral imaging (HSI) has been demonstrated in various digital pathology applications. However, the intrinsic high dimensionality of hyperspectral images makes it difficult for pathologists to visualize the information. The aim of this study is to develop a method to transform hyperspectral images of hemoxylin & eosin (H&E)-stained slides to natural-color RGB histologic images for easy visualization. Hyperspectral images were obtained at 40× magnification with an automated microscopic imaging system and downsampled by various factors to generate data equivalent to different magnifications. High-resolution digital histologic RGB images were cropped and registered to the corresponding hyperspectral images as the ground truth. A conditional generative adversarial network (cGAN) was trained to output natural color RGB images of the histological tissue samples. The generated synthetic RGBs have similar color and sharpness to real RGBs. Image classification was implemented using the real and synthetic RGBs, respectively, with a pretrained network. The classification of tumor and normal tissue using the HSI-synthesized RGBs yielded a comparable but slightly higher accuracy and AUC than the real RGBs. The proposed method can reduce the acquisition time of two imaging modalities while giving pathologists access to the high information density of HSI and the quality visualization of RGBs. This study demonstrated that HSI may provide a potentially better alternative to current RGB-based pathologic imaging and thus make HSI a viable tool for histopathological diagnosis.