Deep fusion feature extraction network and its application in chemical process soft sensing

doi:10.11949/0438-1157.20220349

Abstract

Abstract:

The observation data of complex chemical processes often contain both nonlinear and strong dynamic characteristics, and the traditional soft sensing method of chemical process cannot accurately extract the nonlinear dynamic characteristics of the observation data, so as to affect the accuracy of data modeling and quality prediction. In this paper, a deep fusion features extraction network (DFFEN) is proposed. Under the framework of variational auto encoder, the nonlinear dynamic latent variables are extracted by constructing latent feature information transfer channel. In addition, a self-attention mechanism is used to fuse key hidden layer information and optimize the problem that the potential features are forgotten, which is mainly caused by the excessively long information transmission channel. Then, the regression model between latent variables and key quality variables is constructed in the backend network to achieve the prediction of key quality variables. Finally, the feasibility and effectiveness of the proposed DFFEN model are verified by numerical cases and an actual ammonia synthesis process.

Key words: process control, nonlinear dynamic modeling, neural networks, deep fusion feature extraction, syngas

摘要：

复杂化工过程的观测数据往往同时包含非线性和强动态特性，而传统的化工过程软测量方法无法准确提取观测数据的非线性动态特征，以至影响数据建模和质量预报的准确性。提出了一种基于变分自编码器的深度融合特征提取网络(deep fusion features extraction network， DFFEN)。在变分自编码器框架下，通过构建潜隐特征信息传递通道，提取非线性动态潜隐变量。并利用自注意力机制(self-attention)融合关键的隐层信息，优化因信息传递通道过长而导致的潜在特征被遗忘的问题。此外，在后端网络构建潜隐变量和关键质量变量之间的回归模型，以实现关键质量变量的预报。最后，通过数值案例和实际的合成氨过程验证了所提出的DFFEN模型的可行性和有效性。

关键词: 过程控制, 非线性动态建模, 神经网络, 深度融合特征, 合成气

CLC Number:

TP 183

Le ZHOU, Chengkai SHEN, Chao WU, Beiping HOU, Zhihuan SONG. Deep fusion feature extraction network and its application in chemical process soft sensing[J]. CIESC Journal, 2022, 73(7): 3156-3165.

周乐, 沈程凯, 吴超, 侯北平, 宋执环. 深度融合特征提取网络及其在化工过程软测量中的应用[J]. 化工学报, 2022, 73(7): 3156-3165.

Figures/Tables 11

Fig.1 Model structure of the DFFEN

Fig.2 Flow chart of DFFEN for soft sensing

Fig.3 The evaluation indices R2 and RMSE versusT

Table 1 Prediction results for different models using numerical case

Models	$R 2$	RMSE
PPCR	0.8944	2.0062
SSAE	0.9187	1.7601
SNDS	0.9233	1.7098
DFFEN
T=2	0.9202	1.7472
T=3	0.9343	1.5838
T=4	0.9364	1.5573
T=5	0.9282	1.6561

Table 1 Prediction results for different models using numerical case

Models	$R 2$	RMSE
PPCR	0.8944	2.0062
SSAE	0.9187	1.7601
SNDS	0.9233	1.7098
DFFEN
T=2	0.9202	1.7472
T=3	0.9343	1.5838
T=4	0.9364	1.5573
T=5	0.9282	1.6561

Fig.4 Soft sensing results for different models in the numerical case

Fig.5 Predition error for different models

Fig.6 The flowchart of primary reformer

Table 2 The description of the variables in primary reformer

编号	变量描述
FR03001	流入03B001的燃气流量
FR03002	流入03B001的外置燃气流量
PC03002	03E005出口处燃气外置燃料压力
PC03007	03B001炉膛出口烟气压力
TI03001	03E005出口处的燃料放气温度
TI03009	03B002E06出口燃气的温度
TR03012	03B001入口处的加工气体温度
TI03013	03B001左上角炉膛烟气温度
TI03014	03B001右上角炉膛烟气温度
TR03015	03B001炉顶混炉烟气温度
TR03016	03B001左出口的转化气体温度
TR03017	03B001右出口的转化气体温度
TR03020	03B001出口转化气体的温度
AR03001	炉顶氧浓度

Table 3 Prediction results for different models in the synthetic ammonia process

Models	$R 2$	RMSE
PPCR	-0.3291	1.1737
SSAE	0.5389	0.6913
SNDS	0.1121	0.8833
DFFEN	0.6946	0.5181

Table 3 Prediction results for different models in the synthetic ammonia process

Models	$R 2$	RMSE
PPCR	-0.3291	1.1737
SSAE	0.5389	0.6913
SNDS	0.1121	0.8833
DFFEN	0.6946	0.5181

Fig.7 Prediction results for different models in the synthetic ammonia process

Fig.8 Prediction error for different models in the synthetic ammonia process

References 33

1	陈忠圣, 朱梅玉, 贺彦林, 等. 基于分位数回归CGAN的虚拟样本生成方法及其过程建模应用[J]. 化工学报, 2021, 72(3): 1529-1538.
	Chen Z S, Zhu M Y, He Y L, et al. Quantile regression CGAN based virtual samples generation and its applications to process modeling[J]. CIESC Journal, 2021, 72(3): 1529-1538.
2	李海波, 柴天佑, 岳恒. 浮选工艺指标KPCA-ELM软测量模型及应用[J]. 化工学报, 2012, 63(9): 2892-2898.
	Li H B, Chai T Y, Yue H. Soft sensor of technical indices based on KPCA-ELM and application for flotation process[J]. CIESC Journal, 2012, 63(9): 2892-2898.
3	Kadlec P, Gabrys B, Strandt S. Data-driven soft sensors in the process industry[J]. Computers & Chemical Engineering, 2009, 33(4): 795-814.
4	Yuan X F, Huang B, Wang Y L, et al. Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE[J]. IEEE Transactions on Industrial Informatics, 2018, 14(7): 3235-3243.
5	Ge Z Q, Song Z H, Ding S X, et al. Data mining and analytics in the process industry: the role of machine learning[J]. IEEE Access, 2017, 5: 20590-20616.
6	Massy W F. Principal components regression in exploratory statistical research[J]. Journal of the American Statistical Association, 1965, 60(309): 234-256.
7	Ma M, Khatibisepehr S, Huang B. A Bayesian framework for real-time identification of locally weighted partial least squares[J]. AIChE Journal, 2015, 61(2): 518-529.
8	Fortuna L, Giannone P, Graziani S, et al. Virtual instruments based on stacked neural networks to improve product quality monitoring in a refinery[J]. IEEE Transactions on Instrumentation and Measurement, 2007, 56(1): 95-101.
9	Kaneko H, Funatsu K. Application of online support vector regression for soft sensors[J]. AIChE Journal, 2014, 60(2): 600-612.
10	Ku W F, Storer R H, Georgakis C. Disturbance detection and isolation by dynamic principal component analysis[J]. Chemometrics and Intelligent Laboratory Systems, 1995, 30(1): 179-196.
11	Dong Y N, Qin S J. Regression on dynamic PLS structures for supervised learning of dynamic data[J]. Journal of Process Control, 2018, 68: 64-72.
12	Chen J, Liu K C. On-line batch process monitoring using dynamic PCA and dynamic PLS models[J]. Chemical Engineering Science, 2002, 57(1): 63-75.
13	Ge Z Q, Chen X R. Dynamic probabilistic latent variable model for process data modeling and regression application[J]. IEEE Transactions on Control Systems Technology, 2019, 27(1): 323-331.
14	Everett B. An Introduction to Latent Variable Models[M]. Berlin:Springer Science & Business Media, 2013.
15	Dong Y N, Qin S J. Dynamic latent variable analytics for process operations and control[J]. Computers & Chemical Engineering, 2018, 114: 69-80.
16	Zhou L, Li G, Song Z H, et al. Autoregressive dynamic latent variable models for process monitoring[J]. IEEE Transactions on Control Systems Technology, 2017, 25(1): 366-373.
17	Zhou L, Zheng J Q, Ge Z Q, et al. Multimode process monitoring based on switching autoregressive dynamic latent variable model[J]. IEEE Transactions on Industrial Electronics, 2018, 65(10): 8184-8194.
18	Schölkopf B, Smola A, Müller K R. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Computation, 1998, 10(5): 1299-1319.
19	Rosipal R, Trejo L J. Kernel partial least squares regression in reproducing kernel Hilbert space[J]. Journal of Machine Learning Research, 2001, 2: 97-123.
20	Kingma D P, Welling M. Auto-encoding variational Bayes[J/OL]. [2022-03-07]..
21	McCoy J T, Kroon S, Auret L. Variational autoencoders for missing data imputation with application to a simulated milling circuit[J]. IFAC-PapersOnLine, 2018, 51(21): 141-146.
22	Xie R M, Jan N M, Hao K R, et al. Supervised variational autoencoders for soft sensor modeling with missing data[J]. IEEE Transactions on Industrial Informatics, 2020, 16(4): 2820-2828.
23	Odiowei P P, Cao Y. Nonlinear dynamic process monitoring using canonical variate analysis and kernel density estimations[J]. IEEE Transactions on Industrial Informatics, 2010, 6(1): 36-45.
24	Yuan X F, Li L, Wang Y L. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE Transactions on Industrial Informatics, 2020, 16(5): 3168-3176.
25	Yuan X F, Li L, Shardt Y A W, et al. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development[J]. IEEE Transactions on Industrial Electronics, 2021, 68(5): 4404-4414.
26	Yao L, Jiang X Y, Huang G P, et al. Virtual sensing f-CaO content of cement clinker based on incremental deep dynamic features extracting and transferring model[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-10.
27	Huang Y, Chen C H, Huang C J. Motor fault detection and feature extraction using RNN-based variational autoencoder[J]. IEEE Access, 2019, 7: 139086-139096.
28	Geng Z Q, Chen Z W, Meng Q C, et al. Novel transformer based on gated convolutional neural network for dynamic soft sensor modeling of industrial processes[J]. IEEE Transactions on Industrial Informatics, 2022, 18(3): 1521-1529.
29	Lee Y S, Ooi S K, Tanny D, et al. Developing soft-sensor models using latent dynamic variational autoencoders[J]. IFAC-PapersOnLine, 2021, 54(3): 61-66.
30	Yao L, Ge Z Q. Dynamic features incorporated locally weighted deep learning model for soft sensor development[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-11.
31	Shen B B, Ge Z Q. Supervised nonlinear dynamic system for soft sensor application aided by variational auto-encoder[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(9): 6132-6142.
32	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [C]//Advances in Neural Information Processing Systems 30 (NIPS 2017. LongBeach, CA, USA, 2017: 5998-6008.
33	Zhang M X, Yang Y, Ji Y L, et al. Recurrent attention network using spatial-temporal relations for action recognition[J]. Signal Processing, 2018, 145: 137-145.

[1]	Fei KANG, Weiguang LYU, Feng JU, Zhi SUN. Research on discharge path and evaluation of spent lithium-ion batteries [J]. CIESC Journal, 2023, 74(9): 3903-3911.
[2]	Yue CAO, Chong YU, Zhi LI, Minglei YANG. Industrial data driven transition state detection with multi-mode switching of a hydrocracking unit [J]. CIESC Journal, 2023, 74(9): 3841-3854.
[3]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[4]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[5]	Xuejin GAO, Yuzhuo YAO, Huayun HAN, Yongsheng QI. Fault monitoring of fermentation process based on attention dynamic convolutional autoencoder [J]. CIESC Journal, 2023, 74(6): 2503-2521.
[6]	Zefeng GE, Yuqing WU, Mingxun ZENG, Zhenting ZHA, Yuna MA, Zenghui HOU, Huiyan ZHANG. Effect of ash chemical components on biomass gasification properties [J]. CIESC Journal, 2023, 74(5): 2136-2146.
[7]	Xiaodan SU, Ganyu ZHU, Huiquan LI, Guangming ZHENG, Ziheng MENG, Fang LI, Yunrui YANG, Benjun XI, Yu CUI. Optimization of wet process phosphoric acid hemihydrate process and crystallization of gypsum [J]. CIESC Journal, 2023, 74(4): 1805-1817.
[8]	Cheng YUN, Qianlin WANG, Feng CHEN, Xin ZHANG, Zhan DOU, Tingjun YAN. Deep-mining risk evolution path of chemical processes based on community structure [J]. CIESC Journal, 2023, 74(4): 1639-1650.
[9]	Xinyuan WU, Qilei LIU, Boyuan CAO, Lei ZHANG, Jian DU. Group2vec: group vector representation and its property prediction applications based on unsupervised machine learning [J]. CIESC Journal, 2023, 74(3): 1187-1194.
[10]	Zhongqiu ZHANG, Hongguang LI, Yilin SHI. A multi-task learning approach for complex chemical processes based on manual predictive manipulating strategies [J]. CIESC Journal, 2023, 74(3): 1195-1204.
[11]	Jianghuai ZHANG, Zhong ZHAO. Robust minimum covariance constrained control for C₃ hydrogenation process and application [J]. CIESC Journal, 2023, 74(3): 1216-1227.
[12]	Weiyi SU, Jiahui DING, Chunli LI, Honghai WANG, Yanjun JIANG. Research progress of enzymatic reactive crystallization [J]. CIESC Journal, 2023, 74(2): 617-629.
[13]	Xin LI, Shaojuan ZENG, Kuilin PENG, Lei YUAN, Xiangping ZHANG. Research progress and tendency of CO₂ electrocatalytic reduction to syngas [J]. CIESC Journal, 2023, 74(1): 313-329.
[14]	Xuejin GAO, Kun CHENG, Huayun HAN, Huihui Gao, Yongsheng QI. Fault diagnosis of chillers using central loss conditional generative adversarial network [J]. CIESC Journal, 2022, 73(9): 3950-3962.
[15]	Yuelin WANG, Wei CHAO, Xiaocheng LAN, Zhipeng MO, Shuhuan TONG, Tiefeng WANG. Review of ethanol production via biological syngas fermentation [J]. CIESC Journal, 2022, 73(8): 3448-3460.