基于自编码神经网络特征提取的回声状态网络研究及过程建模应用

doi:10.11949/0438-1157.20191350

摘要/Abstract

摘要：

在复杂化工建模过程中，由于过程数据的时序性、高非线性以及高维数的特点，导致传统的静态神经网络建模无法满足一定的精度。为了解决该问题，提出一种基于自编码神经网络特征提取的回声状态网络模型（features extracted from auto-encoder based echo state network，FEAE-ESN）。传统回声状态网络(echo state network, ESN)方法中，储备池的节点数目很多，输出的维数很高，数据间存在共线性。为解决上述问题，待回声状态网络训练好之后，使用自编码神经网络对其储备池输出进行特征提取。通过自编码网络特征提取，一方面可以有效地降低储备池输出的维数，从而降低数据的复杂度；另一方面提取的特征去除了原有储备池输出的共线性，能够进一步提高广义逆的计算性能；最终提高回声状态网络的建模精度。所提方法FEAE-ESN用于田纳西-伊斯曼复杂过程建模，仿真结果验证了所提方法的有效性。

关键词: 自编码神经网络, 回声状态网络, 特征提取, 软测量, 过程建模

Abstract:

In the process of complex chemical modeling, the traditional static neural network modeling can not meet certain accuracy because of the time sequence, high nonlinearity and high dimension of process data. To solve this problem, a feature extracted from auto-encoder based echo state network (FEAE-ESN) is proposed in this paper. In the traditional echo state network (ESN) method, the number of nodes in the reserve pool of ESN is large, and then the dimension of the reserve pool output is very high. Therefore, to solve this problem, the auto-encoder is used to extract features from the reserve pool output of well-trained ESN. Through the feature extraction of auto-encoder, on one hand, the dimension of the output of the reserve pool can be effectively reduced, thereby reducing the complexity of the data; on the other hand, the collinearity of the outputs of the original reserve pool can be removed through extracting features, which can further improve the calculation performance of generalized inverse. Ultimately, the modeling accuracy of ESN is improved. The proposed FEAE-ESN is applied to modeling the Tennessee-Eastman process. The simulation results verify the effectiveness of the proposed method.

Key words: auto-encoder, echo state network, feature extraction, soft sensor, process modeling

中图分类号:

TP 29

朱宝, 乔俊飞. 基于自编码神经网络特征提取的回声状态网络研究及过程建模应用[J]. 化工学报, 2019, 70(12): 4770-4776.

Bao ZHU, Junfei QIAO. Features extracted from auto-encoder based echo state network and its applications to process modeling[J]. CIESC Journal, 2019, 70(12): 4770-4776.

图/表 10

图1 回声状态网络结构

Fig.1 Structure of ESN

图2 自编码神经网络结构

Fig.2 Structure of auto-encoder

图3 FEAE-ESN模型结构

Fig.3 Structure of the proposed FEAE-ESN

图4 FEAE-ESN模型建立流程

Fig.4 Procedure of establishing the proposed FEAE-ESN

表1 田纳西-伊斯曼过程输入数据信息

Table 1 Information of inputs of Tennessee-Eastman process

No.	Variable description
1	A feed
2	D feed
3	E feed
4	A and C feed
5	recycle flow
6	reactor feed rate
7	reactor temperature
8	purge rate
9	product separator temperature
10	product separator pressure
11	product separator underflow
12	stripper pressure
13	stripper temperature
14	stripper steam flow
15	reactor cooling water outlet temperature
16	separator cooling water outlet temperature

图5 ESN测试输出与真实值的对比

Fig.5 Comparison of expected values and predictions using ESN

图8 FEAE-ESN测试残差分布

Fig.8 Distributions of testing residual errors of FEAE-ESN

图6 ESN测试残差分布

Fig.6 Distributions of testing residual errors of ESN

图7 FEAE-ESN测试输出与真实值的对比

Fig.7 Comparison of expected values and predictions using FEAE-ESN

表2 ESN与FEAE-ESN建模测试精度对比

Table 2 Modeling accuracy comparison between ESN and FEAE-ESN

方法	网络测试RMSE
ESN	0.0615
FEAE-ESN	0.0207

参考文献 30

1	朱群雄, 张晓晗, 顾祥柏, 等. 基于特征提取的函数连接神经网络研究及其化工过程建模应用[J]. 化工学报, 2018, 69(3): 907-912.
	Zhu Q X, Zhang X H, Gu X B, et al. Research and application of feature extraction derived functional link neural network[J]. CIESC Journal, 2018, 69(3): 907-912.
2	Binois M, Gramacy R B, Ludkovski M. Practical heteroscedastic Gaussian process modeling for large simulation experiments[J]. Journal of Computational and Graphical Statistics, 2018, 27(4): 808-821.
3	Li W, Zhao C, Gao F. Linearity evaluation and variable subset partition based hierarchical process modeling and monitoring[J]. IEEE Transactions on Industrial Electronics, 2017, 65(3): 2683-2692.
4	Wu F Y, Asada H H. Implicit and intuitive grasp posture control for wearable robotic fingers: a data-driven method using partial least squares [J]. IEEE Transactions on Robotics, 2016, 32(1): 1-11.
5	Li H D, Xu Q S, Liang Y Z. libPLS: an integrated library for partial least squares regression and linear discriminant analysis[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 176: 34-43.
6	Awad M, Khanna R. Support vector regression [J]. Neural Information Processing Letters & Reviews, 2007, 11(10): 203-224.
7	Bourinet J M. Rare-event probability estimation with adaptive support vector regression surrogates[J]. Reliability Engineering & System Safety, 2016, 150: 210-221.
8	徐圆, 张伟, 张明卿, 等. 基于FEEMD-AE与反馈极限学习机组合模型预测研究与应用[J]. 化工学报, 2018, 69(3): 1064-1070.
	Xu Y, Zhang W, Zhang M Q, et al. Prediction research and application of a combination model based on FEEMD-AE and feedback extreme learning machine[J]. CIESC Journal, 2018, 69(3): 1064-1070.
9	Tang J, Deng C, Huang G B. Extreme learning machine for multilayer perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 27(4): 809-821.
10	Sadeghi B H M. A BP-neural network predictor model for plastic injection molding process[J]. Journal of Materials Processing Technology, 2000, 103(3): 411-416.
11	Ren T, Liu S, Yan G, et al. Temperature prediction of the molten salt collector tube using BP neural network[J]. IET Renewable Power Generation, 2016, 10(2): 212-220.
12	Wu J, Long J, Liu M. Evolving RBF neural networks for rainfall prediction using hybrid particle swarm optimization and genetic algorithm[J]. Neurocomputing, 2015, 148: 136-142.
13	赵忠盖, 刘飞. 动态因子分析模型及其在过程监控中的应用[J]. 化工学报, 2009, 60(1): 183-186.
	Zhao Z G, Liu F. Modeling using dynamic factor analysis and its application in process monitoring[J]. CIESC Journal, 2009, 60(1): 183-186.
14	Ghiassi M, Saidane H, Zimbra D K. A dynamic artificial neural network model for forecasting time series events [J]. International Journal of Forecasting, 2005, 21(2): 341-362.
15	Du X, Vasudevan R, Johnson-Roberson M. Bio-LSTM: a biomechanically inspired recurrent neural network for 3-D pedestrian pose and gait prediction [J]. IEEE Robotics and Automation Letters, 2018, 4(2): 1501-1508.
16	Zen H, Sak H. Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis[C]//2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015: 4470-4474.
17	Feng W, Wu Y, Fan Y. A new method for the prediction of network security situations based on recurrent neural network with gated recurrent unit[J]. International Journal of Intelligent Computing and Cybernetics, 2018, 11(6): 511-525.
18	Jaeger H. The “echo state” approach to analysing and training recurrent neural networks—with an erratum note[R].Bonn: German National Research Center for Information Technology GMD Technical Report, 2001.
19	Aboelmaged M G. Knowledge sharing through enterprise social network (ESN) systems: motivational drivers and their impact on employees productivity[J]. Journal of Knowledge Management, 2018, 22(2): 362-383.
20	Gillum T L, George R H, Leitmeyer J E. An autoencoder for clinical and regulatory data processing [J]. Drug Information Journal, 1995, 29(1): 107-113.
21	Kodirov E, Xiang T, Gong S. Semantic autoencoder for zero-shot learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 3174-3183.
22	Zhang Z, Song Y, Qi H. Age progression/regression by conditional adversarial autoencoder[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 5810-5818.
23	彭荻, 贺彦林, 徐圆, 等. 基于数据特征提取的AANN-ELM研究及化工应用[J]. 化工学报, 2012, 63(9): 2920-2925.
	Peng D, He Y L, Xu Y, et al. Research and chemical application of data feature extraction based AANN-ELM neural network[J]. CIESC Journal, 2012, 63(9): 2920-2925.
24	才轶, 徐圆, 朱群雄, 等. 基于自联想神经网络的数据滤波功能与应用[J]. 计算机与应用化学, 2009, 26(5): 673-676.
	Cai Y, Xu Y, Zhu Q X, et al. Data filtering method and application based on auto-associative neural network[J]. Computers and Applied Chemistry, 2009, 26(5): 673-676.
25	Hou X, Shen L, Sun K, et al. Deep feature consistent variational Autoencoder [C]//2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2017: 1133-1141.
26	Yu J, Hong C, Rui Y, et al. Multitask autoencoder model for recovering human poses [J]. IEEE Transactions on Industrial Electronics, 2017, 65(6): 5060-5068.
27	Fu H, Lei P, Tao H, et al. Improved semi-supervised autoencoder for deception detection [J]. PloS ONE, 2019, 14(10): e0223361.
28	Co-Reyes J D, Liu Y X, Gupta A, et al. Self-consistent trajectory autoencoder: hierarchical reinforcement learning with trajectory embeddings [C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm, Sweden, 2018.
29	Zhu Q X, Meng Q Q, He Y L. Novel multidimensional feature pattern classification method and its application to fault diagnosis [J]. Industrial & Engineering Chemistry Research, 2017, 56(31): 8906-8916.
30	Xu Y, Shen S Q, He Y L, et al. A novel hybrid method integrating ICA-PCA with relevant vector machine for multivariate process monitoring [J]. IEEE Transactions on Control Systems Technology, 2018, 27(4): 1780-1787.

[1]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[2]	邵伟明, 韩文学, 宋伟, 杨勇, 陈灿, 赵东亚. 基于分布式贝叶斯隐马尔可夫回归的动态软测量建模方法[J]. 化工学报, 2023, 74(6): 2495-2502.
[3]	罗顺桦, 王振雷, 王昕. 基于二子空间协同训练算法的半监督软测量建模[J]. 化工学报, 2022, 73(3): 1270-1279.
[4]	刘聪, 谢莉, 杨慧中. 基于改进DPC的青霉素发酵过程多模型软测量建模[J]. 化工学报, 2021, 72(3): 1606-1615.
[5]	陈忠圣, 朱梅玉, 贺彦林, 徐圆, 朱群雄. 基于分位数回归CGAN的虚拟样本生成方法及其过程建模应用[J]. 化工学报, 2021, 72(3): 1529-1538.
[6]	李东, 黄道平, 刘乙奇. 基于协同训练的半监督异构自适应软测量建模方法的研究[J]. 化工学报, 2020, 71(5): 2128-2138.
[7]	贺彦林, 田业, 顾祥柏, 徐圆, 朱群雄. 基于正则化的函数连接神经网络研究及其复杂化工过程建模应用[J]. 化工学报, 2020, 71(3): 1072-1079.
[8]	杜宇浩, 阎高伟, 李荣, 王芳. 基于局部线性嵌入的测地线流式核多工况软测量建模方法[J]. 化工学报, 2020, 71(3): 1278-1287.
[9]	杨逸俊,王振雷,王昕. 基于最近邻与神经网络融合模型的软测量建模方法[J]. 化工学报, 2020, 71(12): 5696-5705.
[10]	王雅欣,徐宝昌,徐朝农,董秀娟,许立伟. 基于工厂数据的注意力LSTM网络辨识方法[J]. 化工学报, 2020, 71(12): 5664-5671.
[11]	代学志,熊伟丽. 基于核极限学习机的快速主动学习方法及其软测量应用[J]. 化工学报, 2020, 71(11): 5226-5236.
[12]	侯延彬,高宪文,李翔宇. 采油过程多尺度状态特征生成的有杆泵动态液面预测[J]. 化工学报, 2019, 70(S2): 311-321.
[13]	廉小亲, 王俐伟, 安飒, 魏伟, 刘载文. 基于SOM-RBF神经网络的COD软测量方法[J]. 化工学报, 2019, 70(9): 3465-3472.
[14]	秦美华, 朱红求, 李勇刚, 陈俊名, 张凤雪, 李文婷. 基于STA-K均值聚类的电化学废水处理过程离子浓度软测量[J]. 化工学报, 2019, 70(9): 3458-3464.
[15]	吴菁, 刘乙奇, 刘坚, 黄道平, 邱禹, 于广平. 基于动态多核相关向量机的软测量建模研究[J]. 化工学报, 2019, 70(4): 1472-1484.