Attention LSTM network identification method based on factory data

doi:10.11949/0438-1157.20201067

Abstract

Abstract:

The control system of chemical enterprises is becoming more and more complex, and identifying the controlled object model is the primary task of automatic control and optimization design. In view of the problem that most chemical process identification experiments need to apply test signals to the process, which may lead to production interruption or safety accidents, a long short-term memory(LSTM) nonlinear dynamic model identification algorithm combined with attention mechanism is proposed to adapt to plant time series data with characteristics of high dimension, strong coupling and nonlinearity. Based on LSTM model, the algorithm considers the importance of the input variables to the target variables, pays more attention to the key features that affect the output results in the input sequence, and improves the generalization ability of the LSTM model. The LSTM network model based on the daily operation data of the plant can be used as the digital virtual device of the identified object, and the local linear model can be identified offline on the virtual device by using the designed test data. The identification experiments on Tennessee-Eastman (TE) process verify the effectiveness of this method.

Key words: chemical process modeling, system identification, nonlinear dynamic model, LSTM, digital virtual device

摘要：

化工企业控制系统日益复杂，辨识被控对象模型是自动控制和优化设计的首要任务。针对化工过程多数辨识实验需要对过程施加测试信号，可能导致生产中断或引发安全事故的问题，利用长短时记忆（long short-term memory，LSTM）网络对含高维度、强耦合、非线性等特点的工厂时序数据具有的强适应性的特点，提出一种结合注意力机制思想的LSTM非线性动态模型辨识算法。该算法在LSTM模型基础上考虑输入变量对目标变量的重要性，为输入序列中影响输出结果的关键特征分配更多注意力，提高了LSTM模型的泛化能力。基于工厂日常运行数据构建LSTM网络模型可作为被辨识对象的数字化虚拟装置，利用人工测试信号在虚拟装置上离线辨识局部线性模型。在Tennessee-Eastman（TE）过程上的辨识实验验证了本文方法的有效性。

关键词: 化工过程建模, 系统辨识, 非线性动态模型, 长短时记忆, 数字化虚拟装置

CLC Number:

TP 273

WANG Yaxin,XU Baochang,XU Chaonong,DONG Xiujuan,XU Liwei. Attention LSTM network identification method based on factory data[J]. CIESC Journal, 2020, 71(12): 5664-5671.

王雅欣,徐宝昌,徐朝农,董秀娟,许立伟. 基于工厂数据的注意力LSTM网络辨识方法[J]. 化工学报, 2020, 71(12): 5664-5671.

Figures/Tables 10

Fig.1 Process model identification framework based on plant data

Fig.2 Architectures of LSTM

Fig.3 Flow chart of LSTM algorithm combined with attention mechanism

Fig.4 LSTM network structure combined with attention mechanism

Table 1 Parameters of network model

参数名称	数值
输入维数	9
数出维数	1
LSTM层数	1
隐藏节点数	18
训练数据	6000组
测试数据	3000组

Fig.5 Training error of DFP algorithm and gradient descent algorithm

Fig.6 Comparison of model prediction

Fig. 7 Comparison of off-line experimental results at 2820 kPa

Table 2 Model test and off-line experimental results

工作点／kPa	常规LSTM		本文所提模型
工作点／kPa	模型测试MAPE／%	离线实验MAPE／%	模型测试MAPE／%	离线实验MAPE／%
2810	1.64	8.56	1.16	6.05
2820	5.72	11.54	3.07	8.46
2830	4.65	9.92	2.89	7.32

Fig.8 Transfer function model and its step response curves

References 31

1	Verma B, Padhy P K. Indirect IMC-PID controller design[J]. IET Control Theory & Applications, 2019, 13(2): 297-305.
2	L􀆳Afflitto A, Blackford T A. Constrained dynamical systems, robust model reference adaptive control, and unreliable reference signals[J]. International Journal of Control, 2020, 93(5): 1039-1052.
3	Capaci R B, Vaccari M, Scali C, et al. Enhancing MPC formulations by identification and estimation of valve stiction[J]. Journal of Process Control, 2019, 81: 31-39.
4	方崇智, 萧德云. 过程辨识[M]. 北京: 清华大学出版社, 1988.
	Fang C Z, Xiao D Y. Processing Identification[M]. Beijing: Tsinghua University Press, 1988.
5	Cerone V, Razza V, Regruto D. Set-membership errors-in-variables identification of MIMO linear systems[J]. Automatica, 2018, 90: 25-37.
6	Cherif I, Fnaiech F. Nonlinear system identification with a real–doded genetic algorithm (RCGA)[J]. International Journal of Applied Mathematics and Computer Science, 2015, 25(4): 863-875.
7	林海龙. 基于深度神经网络的非线性系统辨识研究[D]. 广州: 广州大学, 2019.
	Lin H L. Research in nonlinear system identification based in deep neural network[D]. Guangzhou : Guangzhou University, 2019.
8	Shabnam Y, Morteza M G. A novel gas turbine fault detection and identification strategy based on hybrid dimensionality reduction and uncertain rule-based fuzzy logic[J]. Computers in Industry, 2020, 115: 103131.
9	Heidari Z, Roe D R, Galindo-Murillo R, et al. Using wavelet analysis to assist in identification of significant events in molecular dynamics simulations[J]. Journal of Chemical Information and Modeling, 2016, 56(7): 1282-1291.
10	武东升, 王玉芹, 张亮亮, 等. 先进控制系统在乙烯裂解炉上的应用[J]. 计算机与应用化学, 2017, 34(9): 729-734.
	Wu D S, Wang Y Q, Zhang L L, et al. The application of advanced control system in the ethylene cracking furnace[J]. Computers and Applied Chemistry, 2017, 34(9): 729-734.
11	马国英, 王冲. 丁二烯装置脱重脱轻精馏单元先进控制研究与应用[J]. 石油化工自动化, 2017, 53(2): 60-62.
	Ma G Y, Wang C. Research and application of advanced control in heavy and light distillation unit of butadiene plant[J]. Automation in Petro-Chemical Industry, 2017, 53(2): 60-62.
12	Shardt Y A W, Huanag B. Data quality assessment of routine operating data for process identification[J]. Computers and Chemical Engineering, 2013, 55(8): 19-27.
13	Rubio J J, Pan Y P, Lughofer E, et al. Fast learning of neural networks with application to big data processes[J]. Neurocomputing, 2019, 390: 294-296.
14	Narendra K S, Parthasarathy K. Identification and control of dynamical systems using neural networks[J]. IEEE Transactions on nNeural Networks, 1990, 1(1): 4-27.
15	Nidhil K J, Sreeraj S, Vijay B, et al. System identification using artificial neural network[C]// International Conference on Circuit. IEEE, 2015: 1-4.
16	于吉, 吕剑虹. 基于循环神经网络的动态模型辨识[J]. 工业控制计算机, 2018, 31(3): 151-152+155.
	Yu J, Lyu J H. A dynamic model based on recurrent neural network[J]. Industrial Control Computer, 2018, 31(3): 151-152+155.
17	刘军, 何星, 许晓鸣. 基于神经网络非线性模型的多级工作点阶跃响应扩展DMC预测控制[J]. 控制与决策, 2000, (3): 342-344.
	Liu J, He X, Xu X M. Extension of DMC predictive control using neural network based nonlinear models and multi-step operation point response[J]. Control and Decision, 2000, (3): 342-344.
18	耿志强, 徐猛, 朱群雄, 等. 基于深度学习的复杂化工过程软测量模型研究与应用[J]. 化工学报, 2019, 70(2): 564-571.
	Geng Z Q, Xu M, Zhu Q X, et al. Research and application of soft measurement model for complex chemical processes based on deep learning[J]. CIESC Journal, 2019, 70(2): 564-571.
19	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
20	Cai M, Liu J. Maxout neurons for deep convolutional and LSTM neural networks in speech recognition[J]. Speech Communication, 2016, 77(C): 53-64.
21	Tang Y, Yang J Y, CHEN J. Comparative research on influencing factors of LSTM deep neural network in stock market time series prediction[J]. Research in Economics and Management, 2019, 4(1): 84-99.
22	Zhang X, Chen M H, Qin Y. NLP-QA framework based on LSTM-RNN[C]//2018 2nd International Conference on Data Science and Business Analytics. IEEE Computer Society, 2018, 1: 307-311.
23	Lipton Z C, Berkowitz J, Elkan C. A critical review of recurrent neural networks for sequence learning[J]. Computer Science, 2015, ArXiV ID:1506. 00019.
24	张祥. 基于LSTM和动态模型的化工过程混合故障诊断[D]. 青岛: 青岛科技大学, 2018.
	Zhang X. Hybrid fault diagnosis method of chemical process based on LSTM and dynamic model[D]. Qingdao: Qingdao University of Science and Technology, 2018.
25	Tsinghua W K, Huang D, Yang F, et al. Soft sensor development and applications based on LSTM in deep neural networks[C]//IEEE Symposium Series on Computational Intelligence. IEEE, 2017.
26	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Advances in Neural Information Processing Systems 30 (NIPS 2017). Long Beach, CA, USA, 2017.
27	Feng Q, Gao C Q, Wang L, et al. Spatio-temporal fall event detection in complex scenes using attention guided LSTM[J]. Pattern Recognition Letters, 2020, 130: 242-249.
28	Zhang M X, Yang Y, Ji Y L, et al. Recurrent attention network using spatial-temporal relations for action recognition[J]. Signal Processing, 2018, 145: 137-145.
29	惠振东. TE化工过程优化控制仿真系统研究[D]. 北京: 北方工业大学, 2017.
	Hui Z D. Research on simulation system of TE chemical process optimization control[D]. Beijing: North China University of Technology, 2017.
30	Croux C, Dehon C. Influence functions of the Spearman and Kendall correlation measures[J]. Statal Methods and Applications, 2010, 19(4): 497-515.
31	Kong J, Ji H B. Wireless sensor networks localization based on weighted DFP algorithm[J]. Computer Engineering, 2009, 35(21): 108-110.

[1]	Le ZHOU, Chengkai SHEN, Chao WU, Beiping HOU, Zhihuan SONG. Deep fusion feature extraction network and its application in chemical process soft sensing [J]. CIESC Journal, 2022, 73(7): 3156-3165.
[2]	Huangfeng FANG, Yaoyao LIU, Wenbiao ZHANG. Biomass moisture content prediction in fluidized bed dryer based on LSTM neural network [J]. CIESC Journal, 2020, 71(S1): 307-314.
[3]	Xuejin GAO, Tengfei LIU, Zidong XU, Huihui GAO, Yongchuan YU. Intermittent process fault monitoring based on recurrent autoencoder [J]. CIESC Journal, 2020, 71(7): 3172-3179.
[4]	Shan DOU, Guangyu ZHANG, Zhihua XIONG. Anomaly detection of process unit based on LSTM time series reconstruction [J]. CIESC Journal, 2019, 70(2): 481-486.
[5]	WANG Luyao, WU Bin, DU Zhimin, JIN Xinqiao. Sensor fault detection and diagnosis for data center air conditioning system based on LSTM neural network [J]. CIESC Journal, 2018, 69(S2): 252-259.
[6]	CAO Pengfei, LUO Xionglin. Modeling of soft sensor for chemical process [J]. CIESC Journal, 2013, 64(3): 788-800.
[7]	BAO Zhejing, PI Daoying, SUN Youxian. Nonlinear model predictive control based on support vector machine with multi-kernel [J]. , 2007, 15(5): 691-697.
[8]	HUANG Dexian, JN Yihui, ZHANG Jie, A. J. Morris. Non-linear Chemical Process Modelling and Application in Epichlorhydrine Production Plant Using Wavelet Networks [J]. , 2002, 10(4): 435-443.