TY - GEN
T1 - Fast and Robust Compression of Deep Convolutional Neural Networks
AU - Wen, Jia
AU - Yang, Liu
AU - Shen, Chenyang
N1 - Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - Deep convolutional neural networks (CNNs) currently demonstrate the state-of-the-art performance in several domains. However, a large amount of memory and computing resources are required in the commonly used CNN models, posing challenges in training as well as deploying, especially on those devices with limited computational resources. Inspired by the recent advancement of random tensor decomposition, we introduce a Hierarchical Framework for Fast and Robust Compression (HFFRC), which significantly reduces the number of parameters needed to represent a convolution layer via a fast low-rank Tucker decomposition algorithm, while preserving its expressive power. In the merit of randomized algorithm, the proposed compression framework is robust to noises in parameters. In addition, it is a general framework that any tensor decomposition method can be easily adopted. The efficiency and effectiveness of the proposed approach have been demonstrated via comprehensive experiments conducted on the benchmarks CIFAR-10 and CIFAR-100 image classification datasets.
AB - Deep convolutional neural networks (CNNs) currently demonstrate the state-of-the-art performance in several domains. However, a large amount of memory and computing resources are required in the commonly used CNN models, posing challenges in training as well as deploying, especially on those devices with limited computational resources. Inspired by the recent advancement of random tensor decomposition, we introduce a Hierarchical Framework for Fast and Robust Compression (HFFRC), which significantly reduces the number of parameters needed to represent a convolution layer via a fast low-rank Tucker decomposition algorithm, while preserving its expressive power. In the merit of randomized algorithm, the proposed compression framework is robust to noises in parameters. In addition, it is a general framework that any tensor decomposition method can be easily adopted. The efficiency and effectiveness of the proposed approach have been demonstrated via comprehensive experiments conducted on the benchmarks CIFAR-10 and CIFAR-100 image classification datasets.
KW - Deep convolutional neural networks
KW - Model compression
KW - Random Tucker decomposition
UR - http://www.scopus.com/inward/record.url?scp=85094100660&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094100660&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-61616-8_5
DO - 10.1007/978-3-030-61616-8_5
M3 - Conference contribution
AN - SCOPUS:85094100660
SN - 9783030616151
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 52
EP - 63
BT - Artificial Neural Networks and Machine Learning – ICANN 2020 - 29th International Conference on Artificial Neural Networks, Proceedings
A2 - Farkaš, Igor
A2 - Masulli, Paolo
A2 - Wermter, Stefan
PB - Springer Science and Business Media Deutschland GmbH
T2 - 29th International Conference on Artificial Neural Networks, ICANN 2020
Y2 - 15 September 2020 through 18 September 2020
ER -