Journal of Northeastern University(Natural Science) ›› 2021, Vol. 42 ›› Issue (2): 201-207.DOI: 10.12068/j.issn.1005-3026.2021.02.008

• Information & Control • Previous Articles     Next Articles

Relationship Between the Number of Linear Regions and Expressive Power of Piecewise Linear Neural Networks

MA Hai-tao, LU Jia-rui, YU Wen-xin, YU Chang-yong   

  1. School of Computer Technology, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China.
  • Received:2020-01-08 Revised:2020-01-08 Accepted:2020-01-08 Published:2021-03-05
  • Contact: LU Jia-rui
  • About author:-
  • Supported by:
    -

Abstract: The neural network with piecewise linear activation function(PLNN) is extensively applied in machine learning. This paper gives a measure of the expressive power of PLNN model, i.e., the number of linear regions, with the mathematical representation of linear regions presented. The relationship between linear regions is analyzed and the number of combined linear regions is calculated. A method for calculating the number of linear regions based on Z curve is developed. A case is given to calculate the number of linear regions of different network structures and the number of linear regions after merging, and the correlation between the number of linear regions and the accuracy of different network structures is analyzed. The results show that the number of linear regions can reflect the expressive power of PLNN model, which has great research significance for selecting network hyperparameters and explaining model boundaries.

Key words: machine learning; piecewise linear neural network; expressive power; linear region

CLC Number: