TY - JOUR
T1 - Deep Learning of Spatiotemporal Filtering for Fast Super-Resolution Ultrasound Imaging
AU - Brown, Katherine G.
AU - Ghosh, Debabrata
AU - Hoyt, Kenneth
N1 - Funding Information:
Manuscript received February 11, 2020; accepted April 12, 2020. Date of publication April 15, 2020; date of current version August 27, 2020. This work was supported in part by NIH under Grant R01EB025841 and in part by Texas CPRIT under Award RP180670. (Corresponding author: Katherine G. Brown.) Katherine G. Brown and Kenneth Hoyt are with the Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: katherine.brown. . dallas.edu; kenneth.hoyt. . utdallas.edu).
Publisher Copyright:
© 1986-2012 IEEE.
PY - 2020/9
Y1 - 2020/9
N2 - Super-resolution ultrasound (SR-US) imaging is a new technique that breaks the diffraction limit and allows visualization of microvascular structures down to tens of micrometers. The image processing methods for the spatiotemporal filtering needed in SR-US, such as singular value decomposition (SVD), are computationally burdensome and performed offline. Deep learning has been applied to many biomedical imaging problems, and trained neural networks have been shown to process an image in milliseconds. The goal of this study was to evaluate the effectiveness of deep learning to realize a spatiotemporal filter in the context of SR-US processing. A 3-D convolutional neural network (3DCNN) was trained on in vitro and in vivo data sets using SVD as ground truth in tissue clutter reduction. In vitro data were obtained from a tissue-mimicking flow phantom, and in vivo data were collected from murine tumors of breast cancer. Three training techniques were studied: training with in vitro data sets, training with in vivo data sets, and transfer learning with initial training on in vitro data sets followed by fine-tuning with in vivo data sets. The neural network trained with in vitro data sets followed by fine-tuning with in vivo data sets had the highest accuracy at 88.0%. The SR-US images produced with deep learning allowed visualization of vessels as small as 25μ m in diameter, which is below the diffraction limit (wavelength of 110μ m at 14 MHz). The performance of the 3DCNN was encouraging for real-time SR-US imaging with an average processing frame rate for in vivo data of 51 Hz with GPU acceleration.
AB - Super-resolution ultrasound (SR-US) imaging is a new technique that breaks the diffraction limit and allows visualization of microvascular structures down to tens of micrometers. The image processing methods for the spatiotemporal filtering needed in SR-US, such as singular value decomposition (SVD), are computationally burdensome and performed offline. Deep learning has been applied to many biomedical imaging problems, and trained neural networks have been shown to process an image in milliseconds. The goal of this study was to evaluate the effectiveness of deep learning to realize a spatiotemporal filter in the context of SR-US processing. A 3-D convolutional neural network (3DCNN) was trained on in vitro and in vivo data sets using SVD as ground truth in tissue clutter reduction. In vitro data were obtained from a tissue-mimicking flow phantom, and in vivo data were collected from murine tumors of breast cancer. Three training techniques were studied: training with in vitro data sets, training with in vivo data sets, and transfer learning with initial training on in vitro data sets followed by fine-tuning with in vivo data sets. The neural network trained with in vitro data sets followed by fine-tuning with in vivo data sets had the highest accuracy at 88.0%. The SR-US images produced with deep learning allowed visualization of vessels as small as 25μ m in diameter, which is below the diffraction limit (wavelength of 110μ m at 14 MHz). The performance of the 3DCNN was encouraging for real-time SR-US imaging with an average processing frame rate for in vivo data of 51 Hz with GPU acceleration.
KW - Contrast agents
KW - contrast-enhanced ultrasound (CEUS)
KW - deep learning
KW - microbubbles (MBs)
KW - super-resolution ultrasound (SR-US)
UR - http://www.scopus.com/inward/record.url?scp=85090078401&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85090078401&partnerID=8YFLogxK
U2 - 10.1109/TUFFC.2020.2988164
DO - 10.1109/TUFFC.2020.2988164
M3 - Article
C2 - 32305911
AN - SCOPUS:85090078401
SN - 0885-3010
VL - 67
SP - 1820
EP - 1829
JO - IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
JF - IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control
IS - 9
M1 - 9068269
ER -