TY - JOUR
T1 - Model compression optimized neural network controller for nonlinear systems
AU - Li, Li Jiang
AU - Zhou, Sheng Lin
AU - Chao, Fei
AU - Chang, Xiang
AU - Yang, Longzhi
AU - Yu, Xiao
AU - Shang, Changjing
AU - Shen, Qiang
N1 - Funding Information:
This study was supported by the Natural Science Foundation of Fujian Province of China (No. 2021J01002 ).
Publisher Copyright:
© 2023 Elsevier B.V.
PY - 2023/4/8
Y1 - 2023/4/8
N2 - Neural network-based controllers are widely used within the domain of robotic control systems. A network controller with more neurons typically achieves better performance, but an excessive number of neurons may make the model computationally intensive, resulting in slow dynamic responses in real-world environments. This paper reports a network compression method that integrates knowledge distillation technology for the development of concise neural network-based controllers to achieve a balance between the control performance and computational costs. The method first trains a full-size teacher model, which is then pruned, leading to a concise network with a minimum compromise of performance. From in this study, the resulting concise network is considered to be the prototype of a student model, which is further trained by a knowledge distillation process. The proposed compression method was applied to three classical networks, and the resultant compact controllers were tested on a robot manipulator for efficacy and potential demonstration. The experimental results from a comparative study confirm that the student models with fewer neurons resulting from the proposed model compression approach can achieve similar performance to that of the teacher models for intelligent dynamic control but with faster convergence speed.
AB - Neural network-based controllers are widely used within the domain of robotic control systems. A network controller with more neurons typically achieves better performance, but an excessive number of neurons may make the model computationally intensive, resulting in slow dynamic responses in real-world environments. This paper reports a network compression method that integrates knowledge distillation technology for the development of concise neural network-based controllers to achieve a balance between the control performance and computational costs. The method first trains a full-size teacher model, which is then pruned, leading to a concise network with a minimum compromise of performance. From in this study, the resulting concise network is considered to be the prototype of a student model, which is further trained by a knowledge distillation process. The proposed compression method was applied to three classical networks, and the resultant compact controllers were tested on a robot manipulator for efficacy and potential demonstration. The experimental results from a comparative study confirm that the student models with fewer neurons resulting from the proposed model compression approach can achieve similar performance to that of the teacher models for intelligent dynamic control but with faster convergence speed.
KW - Knowledge distillation
KW - Model compression
KW - Neural network pruning
KW - Neural-network-based controller
UR - https://www.sciencedirect.com/science/article/pii/S0950705123000618?via%3Dihub
U2 - 10.1016/j.knosys.2023.110311
DO - 10.1016/j.knosys.2023.110311
M3 - Article
AN - SCOPUS:85148323622
SN - 0950-7051
VL - 265
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 110311
ER -