优化增量型随机权神经网络及应用

doi:10.11949/0438-1157.20190635

摘要/Abstract

摘要：

针对传统增量型随机权神经网络(I-RVFLNs)存在网络参数难以优化确定、模型收敛速度慢和结构复杂的问题，提出一种优化增量型随机权神经网络算法，即O-I-RVFLNs。与传统I-RVFLNs不同，所提O-I-RVFLNs算法首先设定了一个期望的建模残差向量，然后在每次新增隐层节点时，选择可以达到或小于此节点期望残差的输入权值和偏置作为该节点的输入参数，进而提高网络的收敛速度。除此之外，考虑到算法在不断迭代更新过程中建模误差越来越小，下降趋势越来越不明显的问题，将各指标参数相邻两次迭代均方根误差的差值考虑在算法终止条件内，并借鉴统计过程控制中的西电规则制定了相应的算法收敛判定准则。最后，基于UCI能效数据和实际高炉工业数据，对所提O-I-RVFLNs算法进行了验证和应用。结果表明，相对于其他RVFLNs算法，所提算法建立的数据模型能够获得更紧凑的网络结构以及更好的泛化性能和预测精度。

关键词: 神经网络, 过拟合, 高炉炼铁, 优化, 动态建模

Abstract:

Aiming at the problem that network parameters are difficult to be optimally determined, the model convergence speed is slow and the structure is complex in the traditional incremental random vector functional-link networks (I-RVFLNs), an optimized incremental random vector functional-link networks algorithm, namely O-I-RVFLNs, is proposed. Different from the traditional I-RVFLNs, the proposed O-I-RVFLNs algorithm sets a desired residual error vector, and then selects the input weights and biases that can reach or less than the expected residual error as the input parameters of the node each time a hiddennode is added, thereby improving the convergence rate of the network. In addition, considering that the modeling error of the algorithm is smaller and smaller and the downward trend is less obvious in the process of continuous iteration updating, the RMSE difference between adjacent iterations of each index parameter is considered in the termination condition of the algorithm, and the corresponding convergence criteria are formulated by referring to the Western Electricity Rules in statistical process control. Finally, based on UCI energy efficiency data and actual blast furnace industrial data, the proposed O-I-RVFLNs algorithm is verified and applied. The results show that compared with other RVFLNs algorithms, the data model built by the proposed algorithm can obtain more compact network structure, better generalization performance and prediction accuracy.

Key words: neural networks, overfitting, blast furnace ironmaking, optimization, dynamic modeling

中图分类号:

TQ 028.8

姜乐,周平. 优化增量型随机权神经网络及应用[J]. 化工学报, 2019, 70(12): 4710-4721.

Yue JIANG,Ping ZHOU. Optimized incremental random vector functional-link networks and its application[J]. CIESC Journal, 2019, 70(12): 4710-4721.

图/表 15

图1 优化增量型随机权神经网络策略图

Fig.1 Strategic diagram of optimized incremental random vector functional-link networks

图2 各指标RMSE与隐层节点数的关系

Fig.2 Relationship between RMSE of each index and number of hidden nodes

图3 各指标ε与隐层节点数的关系

Fig.3 Relationship between ε of each index and number of hidden nodes

图4 不同方法RMSE随隐层节点数增加的变化曲线

Fig.4 RMSE variation curves of different methods as number of hidden layer nodes increases

表1 相同隐层节点下采用不同算法的建模结果对比

Table 1 Comparison of modeling results using different algorithms under the same hidden layer node

算法	隐层节点数目	RMSE
算法	隐层节点数目	Heating load	Cooling load
RVFLNs	40	3.8066	4.5477
RVFLNs	50	4.5690	5.8099
RVFLNs	60	4.8659	6.3166
I-RVFLNs	40	3.6951	3.8338
I-RVFLNs	50	3.5300	3.5697
I-RVFLNs	60	3.3696	3.4109
O-I-RVFLNs	40	3.0929	3.3198
O-I-RVFLNs	50	2.9963	3.1709
O-I-RVFLNs	60	2.8696	3.0851

图5 基于不同算法的能效预测建模效果比较

Fig.5 Comparison of modeling effects of energy efficiency prediction based on different algorithms

表2 不同增量型随机权神经网络算法的建模结果对比

Table 2 Comparison of modeling results of different incremental random vector functional-link networks algorithms

算法	隐层节点数	迭代次数	训练时间	测试时间	RMSE
					Training		Testing
					Heating load	Cooling load	Heating load	Cooling load
I-RVFLNs	1000	1000	5.7600	0.9840	2.6126	3.0687	3.4773	3.7051
EI-RVFLNs	842	842	8.8090	0.8340	2.5144	3.0250	3.1070	3.4349
O-I-RVFLNs	158	158	3.2600	0.1540	2.3604	2.9174	2.6709	2.9803

图6 高炉炼铁工艺示意图

Fig.6 Schematic diagram of blast furnace ironmaking process

图7 所提O-I-RVFLNs算法的铁水质量建模结果

Fig.7 Modeling results of molten iron quality based on proposed O-I-RVFLNs algorithm

图8 基于不同算法的铁水质量在线预测结果

Fig.8 On-line molten iron quality prediction based on different algorithms

图9 各铁水质量指标的RMSE与隐层节点数的关系

Fig.9 Relationship between RMSE of each molten iron quality index and number of hidden layer nodes

图10 各铁水质量指标的ε与隐层节点数的关系

Fig.10 Relationship between ε of each molten iron quality index and number of hidden layer nodes

图11 逐一增加隐层节点数时训练集和测试集的RMSE变化(O-I-RVFLNs)

Fig.11 RMSE changes of training set and testing set as number of hidden layer nodes increases(O-I-RVFLNs)

图12 逐一增加隐层节点数时训练集和测试集的RMSE变化(RVFLNs)

Fig.12 RMSE changes of training set and testing set as number of hidden layer nodes increases(RVFLNs)

表3 不同增量型随机权神经网络算法的铁水质量建模结果对比

Table 3 Comparison of modeling results of molten iron quality based on different incremental random vector functional-link networks algorithms

算法	隐层节点数目	训练时间	测试时间	RMSE
算法	隐层节点数目	训练时间	测试时间	[Si]	[P]	[S]	MIT
I-RVFLNs	1000	2.6090	0.2630	0.1177	0.0083	0.0057	9.1580
EI-RVFLNs	943	11.1340	0.2410	0.1172	0.0080	0.0055	9.1565
O-I-RVFLNs	173	2.1330	0.0490	0.1156	0.0078	0.0054	8.9221

参考文献 26

1	Igelnik B, Pao Y H. Stochastic choice of basis functions in adaptive function approximation and the functional-link net[J]. IEEE Trans. Neural Networks, 1995, 6(6): 1320-1329.
2	Pao Y H, Takefuji Y. Functional-link net computing: theory, system, architecture, and functionalities[J]. Computer, 1992, 25(2): 76-79.
3	Suganthan P N. On non-iterative learning algorithms with closed-form solution[J]. Applied Soft Computing, 2018, 70: 1078-1082.
4	Ren Y, Suganthan P N, Srikanth N, et al. Random vector functional link network for short-term electricity load demand forecasting[J]. Information Sciences, 2016, 367: 1078-1093.
5	Zhou P, Lv Y B, Wang H, et al. Data-driven robust RVFLNs modeling of blast furnace ironmaking process using Cauchy distribution weighted M-estimation[J]. IEEE Trans. Industrial Electronics, 2017, 64(9): 7141-7151.
6	Dai W, Chen Q X, Chu F. Robust regularized random vector functional link network and its industrial application [J]. IEEE Access, 2017, 5: 16162-16172.
7	Pratama M, Angelov P P, Lughofer E, et al. Parsimonious random vector functional link network for data streams [J]. Information Science, 2018, 430/431: 519-537.
8	Zhang L, Suganthan P N. Visual tracking with convolutional random vector functional link network [J]. IEEE Trans. Cybernetics, 2017, 47(10): 3243-3253.
9	Dai W, Liu Q, Chai T Y. Particle size estimate of grinding processes using random vector functional link networks with improved robustness[J]. Neurocomputing, 2015, 169: 361-372.
10	Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: theory and applications[J]. Neurocomputing, 2006, 70(1): 489-501.
11	Huang G B, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Trans. Syst., Man, Cybern. Syst., 2012, 42(2): 513-529.
12	Oneto L, Fumeo E, Clerico G. Dynamic delay predictions for large-scale railway networks: deep and shallow extreme learning machines tuned via thresholdout[J]. IEEE Trans. Syst., Man, Cybern. Syst., 2017, 47(10): 2754-2767.
13	贺敏, 汤健, 郭旭琦, 等. 基于流形正则化域适应随机权神经网络的湿式球磨机负荷参数软测量[J]. 自动化学报, 2019, 45(2): 398-406.
	He M, Tang J, Guo X Q, et al. Soft sensor for ball mill load using DAMRRWNN model[J]. Acta Automatica Sinica, 2019, 45(2): 398-406.
14	Huang G B, Lei C, Siew C K. Universal approximation using incremental constructive feedforward networks with random hidden node[J]. IEEE Trans. Neural Networks, 2006, 17(4): 879-892.
15	Zhang L, Zhou P, Song H D, et al. Multivariable dynamic modeling for molten iron quality using incremental random vector functional-link networks[J]. Journal of Iron and Steel Research International, 2016, 23(11): 1151-1159.
16	Ying L. Orthogonal incremental extreme learning machine for regression and multiclass classification[J]. Neural Computing and Applications. 2016, 27(1): 111-120.
17	Huang G, Chen L. Enhanced random search based incremental extreme learning machine[J]. Neurocomputing, 2008, 71(16): 3460-3468.
18	Huang G, Chen L. Convex incremental extreme learning machine[J]. Neurocomputing, 2007, 70(16): 16-18.
19	Barron A R. Universal approximation bounds for superpositions of a sigmoidal function[J]. IEEE Trans. Information Theory, 1993, 39(3): 930-945.
20	Jian L, Gao C H, Xia Z H. Constructing multiple kernel learning framework for blast furnace automation[J]. IEEE Trans. Automation Science and Engineering, 2012, 9(4): 763-777.
21	宋贺达, 周平, 王宏, 等. 高炉炼铁过程多元铁水质量非线性子空间建模及应用[J]. 自动化学报, 2016, 42(21): 1664-1679.
	Song H D, Zhou P, Wang H, et al. Nonlinear subspace modeling of multivariate molten iron quality in blast furnace ironmaking and its application[J]. Acta Automatica Sinica, 2016, 42(21): 1664-1679.
22	Zhou P, Guo D, Wang H, et al. Data-driven robust M-LS-SVR-based NARX modeling for estimation and control of molten iron quality indices in blast furnace ironmaking[J]. IEEE Trans. Neural Networks & Learning Systems, 2018, 29(9): 4007-4021.
23	宋菁华, 杨春节, 周哲, 等. 改进型EMD-Elman神经网络在铁水硅含量预测中的应用[J]. 化工学报, 2016, 67(3): 729-735.
	Song J H, Yang C J, Zhou Z, et al. Application of improved EMD-Elman neural network to predict silicon content in hot metal[J]. CIESC Journal, 2016, 67(3): 729-735.
24	李泽龙, 杨春节, 刘文辉, 等. 基于LSTM-RNN模型的铁水硅含量预测[J]. 化工学报, 2018, 69(3): 992-997.
	Li Z L, Yang C J, Liu W H, et al. Research on hot metal Si-content prediction based on LSTM-RNN[J]. CIESC Journal, 2018, 69(3): 992-997.
25	蒋朝辉, 董梦林, 桂卫华, 等. 基于Bootstrap 的高炉铁水硅含量二维预报[J]. 自动化学报, 2016, 42(5): 715-723.
	Jiang Z H, Dong M L, Gui W H, et al. Two-dimensional prediction for silicon content of hot metal of blast furnace based on bootstrap[J]. Acta Automatica Sinica, 2016, 42(5): 715-723.
26	Zhou P, Yuan M, Wang H, et al. Multivariable dynamic modeling for molten iron quality using online sequential random vector functional-link networks with self-feedback connections[J]. Information Sciences, 2015, 325: 237-255.

[1]	杨欣, 王文, 徐凯, 马凡华. 高压氢气加注过程中温度特征仿真分析[J]. 化工学报, 2023, 74(S1): 280-286.
[2]	温凯杰, 郭力, 夏诏杰, 陈建华. 一种耦合CFD与深度学习的气固快速模拟方法[J]. 化工学报, 2023, 74(9): 3775-3785.
[3]	何松, 刘乔迈, 谢广烁, 王斯民, 肖娟. 高浓度水煤浆管道气膜减阻两相流模拟及代理辅助优化[J]. 化工学报, 2023, 74(9): 3766-3774.
[4]	陈哲文, 魏俊杰, 张玉明. 超临界水煤气化耦合SOFC发电系统集成及其能量转化机制[J]. 化工学报, 2023, 74(9): 3888-3902.
[5]	齐聪, 丁子, 余杰, 汤茂清, 梁林. 基于选择吸收纳米薄膜的太阳能温差发电特性研究[J]. 化工学报, 2023, 74(9): 3921-3930.
[6]	邢雷, 苗春雨, 蒋明虎, 赵立新, 李新亚. 井下微型气液旋流分离器优化设计与性能分析[J]. 化工学报, 2023, 74(8): 3394-3406.
[7]	张曼铮, 肖猛, 闫沛伟, 苗政, 徐进良, 纪献兵. 危废焚烧处理耦合有机朗肯循环系统工质筛选与热力学优化[J]. 化工学报, 2023, 74(8): 3502-3512.
[8]	诸程瑛, 王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报, 2023, 74(8): 3429-3437.
[9]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[10]	尹刚, 李伊惠, 何飞, 曹文琦, 王民, 颜非亚, 向禹, 卢剑, 罗斌, 卢润廷. 基于KPCA和SVM的铝电解槽漏槽事故预警方法[J]. 化工学报, 2023, 74(8): 3419-3428.
[11]	陈国泽, 卫东, 郭倩, 向志平. 负载跟踪状态下的铝空气电池堆最优功率点优化方法[J]. 化工学报, 2023, 74(8): 3533-3542.
[12]	刘文竹, 云和明, 王宝雪, 胡明哲, 仲崇龙. 基于场协同和耗散的微通道拓扑优化研究[J]. 化工学报, 2023, 74(8): 3329-3341.
[13]	吴文涛, 褚良永, 张玲洁, 谭伟民, 沈丽明, 暴宁钟. 腰果酚生物基自愈合微胶囊的高效制备工艺研究[J]. 化工学报, 2023, 74(7): 3103-3115.
[14]	汤晓玲, 王嘉瑞, 朱玄烨, 郑仁朝. 基于Pickering乳液的卤醇脱卤酶催化合成手性环氧氯丙烷[J]. 化工学报, 2023, 74(7): 2926-2934.
[15]	徐野, 黄文君, 米俊芃, 申川川, 金建祥. 多源信息融合的离心式压缩机喘振诊断方法[J]. 化工学报, 2023, 74(7): 2979-2987.