基于ROLS算法的递归RBF神经网络结构设计

doi:10.11949/j.issn.0438-1157.20170771

化工学报 ›› 2018, Vol. 69 ›› Issue (3): 1191-1199.DOI: 10.11949/j.issn.0438-1157.20170771

基于ROLS算法的递归RBF神经网络结构设计

乔俊飞^1,2, 马士杰^1,2, 杨翠丽^1,2

1 北京工业大学信息学部, 北京 100124;
2 计算智能与智能系统北京市重点实验室, 北京 100124

收稿日期:2017-06-14 修回日期:2017-08-14 出版日期:2018-03-05 发布日期:2018-03-05
通讯作者: 乔俊飞
基金资助:
国家自然科学基金项目（61533002，61603012）；北京市教育委员会科研计划项目（KM201710005025）。

Structure design for recurrent RBF neural network based on recursive orthogonal least squares

QIAO Junfei^1,2, MA Shijie^1,2, YANG Cuili^1,2

1 Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China;
2 Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China

Received:2017-06-14 Revised:2017-08-14 Online:2018-03-05 Published:2018-03-05
Supported by:
supported by the National Natural Science Foundation of China(61533002, 61603012) and the Beijing Municipal Education Commission Foundation(KM201710005025).

摘要/Abstract

摘要：

针对递归RBF神经网络结构难以自适应问题，提出一种基于递归正交最小二乘（recursive orthogonal least squares，ROLS）算法的结构设计方法。首先，利用ROLS算法来计算隐含层神经元的独立贡献度和损失函数，以此判断增加或归为不活跃组的神经元，同时调整神经网络的拓扑结构，并且利用奇异值分解（singular value decomposition，SVD）决定最佳的隐含层神经元个数，以此来删除不活跃组中相对不活跃的神经元，有效地解决了递归RBF神经网络结构冗余和难以自适应问题。其次，利用梯度下降算法更新递归RBF神经网络的参数来保证神经网络的精度。最后，通过对Mackey-Glass时间序列预测、非线性系统辨识和污水处理过程中关键水质参数动态建模，证明了该结构设计方法的可行性和有效性。

关键词: 神经网络, 结构设计, 算法, 奇异值分解, 动态建模

Abstract:

Aiming at the problem of recurrent radial basis function (RRBF) neural network structure which is difficult to be self-adaptive, this paper proposes a structure design method based on recursive orthogonal least square (ROLS) algorithm. Firstly, ROLS algorithm is used to calculate the contribution and the loss function of hidden layer neurons, which determines to increase or be grouped into inactive neurons, and the topology structure of neural network is adjusted accordingly. At the same time, singular value decomposition (SVD) is applied to determine the best number of hidden layer neurons in order to delete the neurons of the inactive group, which effectively solves the problems of RRBF neural network structure which is redundant and hardly self-adaptive. Secondly, the gradient descent algorithm is utilized to update the parameters of RRBF neural network in order to ensure the accuracy of neural network. Finally, several experiments including the Mackey-Glass time series prediction, nonlinear system identification and key water quality parameters dynamic modeling in wastewater treatment process are conducted, and the simulation results prove the feasibility and effectiveness of the structure design method.

Key words: neural network, structure design, algorithm, singular value decomposition, dynamic modeling

中图分类号:

TP183

乔俊飞, 马士杰, 杨翠丽. 基于ROLS算法的递归RBF神经网络结构设计[J]. 化工学报, 2018, 69(3): 1191-1199.

QIAO Junfei, MA Shijie, YANG Cuili. Structure design for recurrent RBF neural network based on recursive orthogonal least squares[J]. CIESC Journal, 2018, 69(3): 1191-1199.

参考文献 31

[1]	YE J, GE L D, WU Y X. An application of improved RBF neural network in modulation recognition[J]. Acta Automatica Sinica, 2007, 33(6):652-654.
[2]	CHEN S, WANG X X, BROWN D J. Sparse incremental regression modeling using correlation criterion with boosting search[J]. IEEE Signal Processing Letters, 2005, 12(3):198-201.
[3]	CHEN S, WOLFGANG A, HARRIS C J, et al. Symmetric RBF classifier for nonlinear detection in multiple-antenna-aided systems[J]. IEEE Transactions on Neural Networks, 2008, 19(5):737-745.
[4]	CARA A, POMARES H, ROJAS I. A new methodology for the online adaptation of fuzzy self-structuring controllers[J]. IEEE Transactions on Fuzzy Systems, 2011, 19(3):449-464.
[5]	FAN H J, SONG Q. A linear recurrent kernel online learning algorithm with sparse updates[J]. Neural Networks, 2014, 50(3):142-153.
[6]	NIU B, LI L. Adaptive neural network tracking control for a class of switched strict feedback nonlinear systems with input delay[J]. Neurocomputing, 2016, 173(3):2121-2128.
[7]	HAN H G, LI Y, GUO Y N, et al. A soft computing method to predict sludge volume index based on a recurrent self-organizing neural network[J]. Applied Soft Computing, 2016, 38(1):477-486.
[8]	ZHANG H, WANG Z, LIU D. A comprehensive review of stability analysis of continuous-time recurrent neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(7):1229-1262.
[9]	LEE W, LEE S, PARK P. Improved stability criteria for recurrent neural networks with interval time-varying delays via new Lyapunov functionals[J]. Neurocomputing, 2015, 155(1):128-134.
[10]	XIE T, YU H, HEWLETT J, et al. Fast and efficient second-order method for training radial basis function networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(4):609-619.
[11]	WANG X, YU J, LI C, et al. Robust stability of stochastic fuzzy delayed neural networks with impulsive time window[J]. Neural Networks, 2015, 67(1):84-91.
[12]	HAN H G, WU X L, QIAO J F. Nonlinear systems modeling based on self-organizing fuzzy-neural-network with adaptive computation algorithm[J]. IEEE Transactions on Cybernetics, 2014, 44(4):554-564.
[13]	MENG D, PEI Z. Dynamic adaptive learning algorithm based on two-fuzzy neural-networks[J]. Neurocomputing, 2014, 125(1):88-94.
[14]	CRUZ D, MAIA R, SILVA L, et al. A bee-inspired data clustering approach to design RBF neural network classifiers[J]. Neurocomputing, 2016, 172(3):427-437.
[15]	KARAYIANNNIS N B, MI G W. Growing radial basis neural networks:merging supervised and unsupervised learning with network growth techniques[J]. IEEE Transactions on Neural Network, 1997, 8(6):1492-1506.
[16]	SUBRAHMANYA N, SHIN Y C. Constructive training of recurrent neural networks using hybrid optimization[J]. Neurocomputing, 2010, 73(13):2624-2631.
[17]	HSU C F. Adaptive backstepping Elman-based neural control for unknown nonlinear systems[J]. Neurocomputing, 2014, 136(1):170-179.
[18]	ZHENG P S, DIMITRAKAKIS C, TRIESCH J. Network self-organization explains the statistics and dynamics of synaptic connection strengths in cortex[J]. Computational Biology, 2013, 9(1):1-8.
[19]	LEUNG C S, LAM P M. A local training-pruning approach for recurrent neural networks[J]. International Journal of Neural Systems, 2003, 13(1):25-38.
[20]	乔俊飞, 韩红桂. RBF神经网络的结构动态优化设计[J]. 自动化学报, 2010, 36(6):865-872. QIAO J F, HAN H G. Optimal structure design for RBFNN structure[J]. Acta Automatica Sinica, 2010, 36(6):865-872.
[21]	TOK D K, YU D L, MATHEWS C. Adaptive structure radial basis function network model for processes with operating region migration[J]. Neurocomputing, 2015, 155:186-193.
[22]	GOMM J B, YU D L. Selecting radial basis function network centers with recursive orthogonal least squares training[J]. IEEE Transactions on Neural networks, 2000, 11(2):306-314.
[23]	BROOMHEAD D S, LOWE D. Multivariable functional interpolation and adaptive networks[J]. Complex Systems, 1988, 2:321-355.
[24]	KAMINSKI W, STRUMILLO P. Kernel orthonomalization in radial basis function neural networks[J]. IEEE Transactions on Neural Networks, 1997, 8(5):1177-1183.
[25]	AKAIKE H. Fitting autoregressive models for prediction[J]. Annals of the Institute of Statistical Mathematics, 1969, 21:425-439.
[26]	TEOH E J, TAN K C, XIANG C. Estimating the number of hidden neurons in a feedforward network using the singular value decomposition[J]. IEEE Transactions on Neural Networks, 2006, 17(6):1623-1629.
[27]	KONSTANTINIDES K, YAO K. Statistical analysis of effective singular values in matrix rank determination[J]. IEEE Transactions on Acoustics, Speech Signal Process, 1988, 36(5):757-763.
[28]	HAN H G, CHEN Q L, QIAO J F. An efficient self-organizing RBF neural network for water quality prediction[J]. Neural Networks, 2011, 24(7):717-725.
[29]	韩红桂, 乔俊飞, 薄迎春. 基于信息强度的RBF神经网络结构设计研究[J]. 自动化学报, 2012, 38(7):1083-1090. HAN H G, QIAO J F, BO Y C. On structure design for RBF neural network based on information strength[J]. Acta Automatica Sinica, 2012, 38(7):1083-1090.
[30]	乔俊飞, 安茹, 韩红桂. 基于RBF神经网络的出水氨氮预测研究[C]//杨辉. 第26届中国过程控制会议. 南昌:中国自动化学会, 2015:120-127. QIAO J F, AN R, HAN H G. Based on the RBF neural network water ammonia nitrogen prediction research[C]//YANG H. The 26th Process Control Meeting in China. Nanchang:China Society of Automation, 2015:120-127.
[31]	杨琴, 谢淑云. BP神经网络在洞庭湖氨氮浓度预测中的应用[J]. 水资源与水工程学报, 2006, 17(1):65-67. YANG Q, XIE S Y. Application of BP neural network into predicting NH3-N concentration of Dongting Lake[J]. Journal of Water Resources & Water Engineering, 2006, 17(1):65-70.

基于ROLS算法的递归RBF神经网络结构设计

Structure design for recurrent RBF neural network based on recursive orthogonal least squares

PDF (PC)

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献 31

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曹跃, 余冲, 李智, 杨明磊. 工业数据驱动的加氢裂化装置多工况切换过渡状态检测[J]. 化工学报, 2023, 74(9): 3841-3854.
[2]	温凯杰, 郭力, 夏诏杰, 陈建华. 一种耦合CFD与深度学习的气固快速模拟方法[J]. 化工学报, 2023, 74(9): 3775-3785.
[3]	诸程瑛, 王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报, 2023, 74(8): 3429-3437.
[4]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[5]	尹刚, 李伊惠, 何飞, 曹文琦, 王民, 颜非亚, 向禹, 卢剑, 罗斌, 卢润廷. 基于KPCA和SVM的铝电解槽漏槽事故预警方法[J]. 化工学报, 2023, 74(8): 3419-3428.
[6]	徐野, 黄文君, 米俊芃, 申川川, 金建祥. 多源信息融合的离心式压缩机喘振诊断方法[J]. 化工学报, 2023, 74(7): 2979-2987.
[7]	高学金, 姚玉卓, 韩华云, 齐咏生. 基于注意力动态卷积自编码器的发酵过程故障监测[J]. 化工学报, 2023, 74(6): 2503-2521.
[8]	邵伟明, 韩文学, 宋伟, 杨勇, 陈灿, 赵东亚. 基于分布式贝叶斯隐马尔可夫回归的动态软测量建模方法[J]. 化工学报, 2023, 74(6): 2495-2502.
[9]	周继鹏, 何文军, 李涛. 异形催化剂上乙烯催化氧化失活动力学反应工程计算[J]. 化工学报, 2023, 74(6): 2416-2426.
[10]	黄磊, 孔令学, 白进, 李怀柱, 郭振兴, 白宗庆, 李平, 李文. 油页岩添加对准东高钠煤灰熔融行为影响的研究[J]. 化工学报, 2023, 74(5): 2123-2135.
[11]	郑书闽, 郭鹏程, 颜建国, 王帅, 李文博, 周淇. 微小通道内过冷流动沸腾阻力特性实验及预测研究[J]. 化工学报, 2023, 74(4): 1549-1560.
[12]	贠程, 王倩琳, 陈锋, 张鑫, 窦站, 颜廷俊. 基于社团结构的化工过程风险演化路径深度挖掘[J]. 化工学报, 2023, 74(4): 1639-1650.
[13]	吴心远, 刘奇磊, 曹博渊, 张磊, 都健. Group2vec：基于无监督机器学习的基团向量表示及其物性预测应用[J]. 化工学报, 2023, 74(3): 1187-1194.
[14]	张江淮, 赵众. 碳三加氢装置鲁棒最小协方差约束控制及应用[J]. 化工学报, 2023, 74(3): 1216-1227.
[15]	顾学荣, 刘硕士, 杨思宇. 基于并行EGO和代理模型辅助的多参数优化方法研究[J]. 化工学报, 2023, 74(3): 1205-1215.