线性区域数量与PLNN表达能力的相关性

doi:10.12068/j.issn.1005-3026.2021.02.008

东北大学学报（自然科学版） ›› 2021, Vol. 42 ›› Issue (2): 201-207.DOI: 10.12068/j.issn.1005-3026.2021.02.008

线性区域数量与PLNN表达能力的相关性

马海涛，路家蕊，于文鑫，于长永

(东北大学秦皇岛分校计算机技术学院，河北秦皇岛066004)

收稿日期:2020-01-08 修回日期:2020-01-08 接受日期:2020-01-08 发布日期:2021-03-05
通讯作者: 马海涛
作者简介:马海涛(1977-)，男，黑龙江伊春人，东北大学讲师，博士.
基金资助:
国家自然科学基金资助项目(61772124).

Relationship Between the Number of Linear Regions and Expressive Power of Piecewise Linear Neural Networks

MA Hai-tao， LU Jia-rui， YU Wen-xin， YU Chang-yong

School of Computer Technology， Northeastern University at Qinhuangdao， Qinhuangdao 066004， China.

Received:2020-01-08 Revised:2020-01-08 Accepted:2020-01-08 Published:2021-03-05
Contact: LU Jia-rui
About author:-
Supported by:
-

摘要/Abstract

摘要： 使用分段线性激活函数的神经网络(PLNN)在机器学习中得到广泛应用.本文给出了一种PLNN模型表达能力的度量值——线性区域数量，并给出了线性区域的数学表示.分析了线性区域之间的关系并计算合并后的线性区域数量，同时给出一种基于Z曲线的线性区域数量的计算方法.针对一个任务实例进行分析，计算不同网络结构的线性区域数量及合并后的线性区域数量，分析了线性区域数量与不同网络结构的准确性的关联.结果表明，线性区域数量能够表现PLNN模型的表达能力，对于选择网络超参数及解释模型边界具有研究意义.

关键词: 机器学习;分段线性神经网络;表达能力;线性区域

Abstract: The neural network with piecewise linear activation function(PLNN) is extensively applied in machine learning. This paper gives a measure of the expressive power of PLNN model， i.e.， the number of linear regions， with the mathematical representation of linear regions presented. The relationship between linear regions is analyzed and the number of combined linear regions is calculated. A method for calculating the number of linear regions based on Z curve is developed. A case is given to calculate the number of linear regions of different network structures and the number of linear regions after merging， and the correlation between the number of linear regions and the accuracy of different network structures is analyzed. The results show that the number of linear regions can reflect the expressive power of PLNN model， which has great research significance for selecting network hyperparameters and explaining model boundaries.

Key words: machine learning; piecewise linear neural network; expressive power; linear region

中图分类号:

TP183

马海涛，路家蕊，于文鑫，于长永. 线性区域数量与PLNN表达能力的相关性[J]. 东北大学学报（自然科学版）, 2021, 42(2): 201-207.

MA Hai-tao， LU Jia-rui， YU Wen-xin， YU Chang-yong. Relationship Between the Number of Linear Regions and Expressive Power of Piecewise Linear Neural Networks[J]. Journal of Northeastern University(Natural Science), 2021, 42(2): 201-207.

参考文献

[1]Wang S H，Phillips P， Sui Y X，et al.Classification of Alzheimers disease based on eight-layer convolutional neural network with leaky rectified linear unit and max pooling［J/OL］.Journal of Medical Systems，2018 ［2019-12-05］.https://link.springer.com/article/10.1007/s10916-018-0932-7.DOI:10.1007/s10916-018-0932-7.
[2]Croce F，Hein M.A randomized gradient-free attack on ReLU networks［C/OL］ // Proceedings of 40th German Conference on Pattern Recognition.Stuttgart，2018(2018-11-28)［2019-12-06］.https://arxiv.org/pdf/1811.11493.pdf.DOI:10.1007/978-3-030-12939-2_16.
[3]Pascanu R，Montúfar G，Bengio Y.On the number of response regions of deep feed forward networks with piece-wise linear activations［J/OL］.(2014-02-14) ［2020-01-04］.https://arxiv.org/abs/1312.6098v4.DOI:10.1002/art.23474.
[4]Montúfar G，Pascanu R，Cho K，et al.On the number of linear regions of deep neural networks［C］ // Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge，MA:MIT Press，2014:2924-2932.
[5]Eldan R，Shamir O.The power of depth for feed forward neural networks［C/OL］.(2015-12-12) ［2019-12-05］.https://arxiv.org/abs/1512.03965v1.
[6]Serra T，Tjandraatmadja C，Ramalingam S.Bounding and counting linear regions of deep neural networks［C/OL］.(2017-11-06)［2019-12-08］.https://www.researchgate.net/publication/320920537_Bounding_and_Counting_Linear_Regions_of_Deep_Neural_Networks.
[7]Raghu M，Poole B ，Kleinberg J ，et al.On the expressive power of deep neural networks［C］ // Proceedings of the 34th International Conference on Machine Learning.Sydney，2017:2847-2854.
[8]Zhang L W，Naitzat G，Lim L H.Tropical geometry of deep neural networks［C］ // Proceedings of the 35th International Conference on Machine Learning.Stockholm，2018:5824-5832.
[9]Ribeiro M T，Singh S，Guestrin C.“Why should I trust you?”:explaining the predictions of any classifier［C］ // Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Demonstrations.San Diego，CA，2016:97-101.
[10]Fong R，Vedaldi A.Interpretable explanations of black boxes by meaningful perturbation［C］ // IEEE International Conference on Computer Vision.Venice，2017:3429-3437.
[11]Chu L Y，Hu X，Hu J H，et al.Exact and consistent interpretation for piecewise linear neural networks:a closed form solution［C］ // Processes of the 24th ACM SIGKDD International Reference on Knowledge Discovery & Data Mining.London，2018:1244-1253.

线性区域数量与PLNN表达能力的相关性

Relationship Between the Number of Linear Regions and Expressive Power of Piecewise Linear Neural Networks

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 10

编辑推荐

Metrics

本文评价

[1]	翟莹莹，左丽，张恩德. 基于参数优化的RBF神经网络结构设计算法[J]. 东北大学学报:自然科学版, 2020, 41(2): 176-182.
[2]	张华伟，郑晓涛. 基于遗传算法优化神经网络的拼焊板压边力预测[J]. 东北大学学报:自然科学版, 2020, 41(2): 241-245.
[3]	于长永，何鑫，祁欣，马海涛. GMSDenseNet:基于组多结构卷积的轻量级DenseNet[J]. 东北大学学报:自然科学版, 2020, 41(1): 12-16.
[4]	蒋芳芳，徐敬傲，李任，徐礼胜. 基于CNN的心冲击信号阵发性房颤自动检测方法[J]. 东北大学学报:自然科学版, 2019, 40(11): 1539-1543.
[5]	王占山，康云云，牛海莎. 一类求解优化问题的神经网络及其全局吸引性分析[J]. 东北大学学报:自然科学版, 2017, 38(2): 153-157.
[6]	杨明，罗艳红，王义贺. 模型未知非零和博弈问题的策略迭代算法[J]. 东北大学学报:自然科学版, 2015, 36(3): 318-322.
[7]	韩鹏，郭天，汪晋宽，史泽伟. 基于mRMR-ESN的单变量光伏功率预测[J]. 东北大学学报（自然科学版）, 2021, 42(2): 174-179.
[8]	蒋芳芳，王浩乾，程天庆，洪楚航. 基于相空间重构的心冲击信号房颤检测方法[J]. 东北大学学报（自然科学版）, 2021, 42(11): 1547-1554.
[9]	金瑾，王鹏. 历史数据驱动的多尺度量子谐振子优化算法[J]. 东北大学学报（自然科学版）, 2022, 43(2): 160-167.
[10]	李大鹏，赵琪珲，邢铁军，赵大哲. 基于分层注意力循环神经网络的司法案件刑期预测[J]. 东北大学学报（自然科学版）, 2022, 43(3): 344-349.