基于MHSA-LSTM的软测量建模及其在化工过程中的应用

doi:10.11949/0438-1157.20240613

化工学报

• •

基于MHSA-LSTM的软测量建模及其在化工过程中的应用

李文华¹(), 叶洪涛¹^,²^,³(), 罗文广¹, 刘乙奇⁴^,⁵

^1.广西科技大学自动化学院，广西柳州 545616
^2.广西自动检测技术与仪器重点实验室（桂林电子科技大学），广西桂林 541004
^3.广西汽车零部件与整车技术重点实验室（广西科技大学），广西柳州 545616
^4.华南理工大学自动化科学与工程学院，广东广州 510640
^5.造纸与污染国家工程中心，广东广州 510640

收稿日期:2024-06-03 修回日期:2024-07-10 出版日期:2024-07-16
通讯作者: 叶洪涛
作者简介:李文华（1998—），女，硕士研究生，20220201015@stdmail.gxust.edu.cn
基金资助:
国家自然科学基金项目(62273151);广东省基础与应用基础研究项目(2021B1515420003);2023年广西汽车零部件与整车技术重点实验室自主研究课题(2023GKLACVTZZ07)

Soft sensor modeling based on MHSA-LSTM and its application in chemical process

Wenhua LI¹(), Hongtao YE¹^,²^,³(), Wenguang LUO¹, Yiqi LIU⁴^,⁵

^1.Guangxi University of Science and Technology, School of Automation, Liuzhou 545616, Guangxi, China
^2.Guangxi Key Laboratory of Automatic Detection Technology and Instruments (Guilin University of Electronic Technology), Guilin 541004, Guangxi, China
^3.Guangxi Key Laboratory of Automotive Components and Vehicle Technology (Guangxi University of Science and Technology), Liuzhou 545616, Guangxi, China
^4.South China University of Technology, School of Automation Science and Engineering, Guangzhou 510640, Guangdong, China
^5.National Engineering Center for Paper Making and Pollution, Guangzhou 510640, Guangdong, China

Received:2024-06-03 Revised:2024-07-10 Online:2024-07-16
Contact: Hongtao YE

摘要/Abstract

摘要：

化工过程数据的动态性和非线性等特性常使传统的软测量方法难以准确提取数据的动态和非线性特征，从而影响关键质量变量的预测精度和系统整体的控制优化。因此，文中提出了一种融合多头自注意力机制的长短期记忆网络（Multi-Head Self-Attention Mechanism Long Short-Term Memory Network，MHSA-LSTM）的软测量建模方法。首先，利用LSTM充分挖掘数据的时序特征，以便提取化工过程数据的动态变化信息；其次，使用多头自注意力机制对LSTM 隐藏层的输出特征进行加权，可有效地捕捉不同尺度特征向量的长期相关性，且能提高模型的长期记忆能力；然后将提取的特征向量与其对应的特征权重相乘得出加权结果输入至全连接层，可有效地提高关键质量变量预测的精度。最后，所提出方法在脱丁烷塔过程和硫回收单元进行仿真验证，结果表明所建模型的预测精度优于门控循环单元、LSTM以及融合自注意力机制的LSTM软测量模型。

关键词: 软测量, 化工过程, 神经网络, 多头自注意力机制, 预测, 实验验证

Abstract:

The dynamic and nonlinear characteristics of chemical process in data often make it difficult even impossible for traditional soft sensing methods to accurately extract the dynamics and nonlinearity, which affects the prediction accuracy of key quality variables negatively and the overall control optimization of the system.

Key words: soft sensor, chemical process, neural network, multi-head self-attention mechanism, prediction, experimental verification

中图分类号:

TP273

李文华, 叶洪涛, 罗文广, 刘乙奇. 基于MHSA-LSTM的软测量建模及其在化工过程中的应用[J]. 化工学报, DOI: 10.11949/0438-1157.20240613.

Wenhua LI, Hongtao YE, Wenguang LUO, Yiqi LIU. Soft sensor modeling based on MHSA-LSTM and its application in chemical process[J]. CIESC Journal, DOI: 10.11949/0438-1157.20240613.

图/表 18

图1 LSTM结构图

Fig.1 Structure of the LSTM

图2 多头自注意力机制与LSTM融合示意图

Fig.2 Schematic diagram of the fusion of multi-head self-attention mechanism and LSTM

图3 MHSA-LSTM模型架构

Fig.3 MHSA-LSTM model architecture

图4 基于MHSA-LSTM的建模流程图

Fig.4 Modeling flowchart based on MHSA-LSTM

图5 脱丁烷塔过程流程示意图[33]

Fig.5 The flowchart of debutanizer column process[33]

表1 DC变量说明

Table1 Description of the variables for the DC

变量	描述	单位
x₁	塔顶温度	℃
x₂	塔顶压力	kg/cm²
x₃	回流流量	m³/h
x₄	进入下一个流程的流量	m³/h
x₅	第六托盘温度	℃
x₆	塔底温度A	℃
x₇	塔底温度B	℃
y	C₄浓度	%

图6 各时间步长下MHSA-LSTM模型的RMSE和MAE

Fig.6 RMSE and MAE of MHSA-LSTM model at each time step

表2 四种模型在DC测试集上的评估结果

Table2 Evaluation results of four models on the DC testing set

模型	RMSE	MAE	R²
GRU	0.0155	0.0113	0.9926
LSTM	0.0123	0.0084	0.9953
SA-LSTM	0.0079	0.0062	0.9981
MHSA-LSTM	0.0063	0.0046	0.9988

图7 各模型在脱丁烷塔过程的预测结果

Fig.7 Prediction results for different models in the debutanizer column process

图8 不同方法绝对预测误差的箱型图

Fig.8 Box plot of absolute prediction errors for different methods

表3 MHSA-LSTM在不同h时的预测性能

Table3 The prediction performance of MHSA-LSTM at different h

h	0	1	2	4	8
MAE	0.0084	0.0062	0.0056	0.0055	0.0046
RMSE	0.0123	0.0079	0.0076	0.0071	0.0063
推理时间/s	0.0058	0.0077	0.0080	0.0085	0.0094

图9 SRU的简化工业流程示意图[34]

Fig.9 The simplified industrial flowchart of the SRU[34]

表4 SRU变量说明

Table4 Description of the variables for the SRU

变量	描述	单位
x₁	MEA气体流量	m³/s
x₂	MEA第一空气流量	m³/s
x₃	MEA第二空气流量	m³/s
x₄	SWS气体流量	m³/s
x₅	SWS空气流量	m³/s
y	尾气H₂S浓度	mol/m³

图10 各时间步长下MHSA-LSTM模型的RMSE和MAE

Fig.10 RMSE and MAE of MHSA-LSTM model at each time step

表5 四种模型在SRU测试集上的评估结果

Table5 Evaluation results of four models on the SRU testing set

模型	RMSE	MAE	R²
GRU	0.0139	0.0108	0.8329
LSTM	0.0111	0.0084	0.8930
SA-LSTM	0.0089	0.0057	0.9312
MHSA-LSTM	0.0076	0.0043	0.9491

图11 各模型在SRU的预测结果

Fig.11 Prediction results for different models in the SRU

图12 不同方法绝对预测误差的箱型图

Fig.12 Box plot of absolute prediction errors for different methods

表6 MHSA-LSTM在不同h时的预测性能

Table6 The prediction performance of MHSA-LSTM at different h

h	0	1	2	4	8
MAE	0.0084	0.0057	0.0048	0.0043	0.0071
RMSE	0.0111	0.0089	0.0080	0.0076	0.0095
推理时间/s	0.0063	0.0098	0.0100	0.0091	0.0099

参考文献 34

1	孟庆超. 化工过程软测量建模方法研究及应用[D]. 北京: 北京化工大学, 2021.
	Meng Q C. Research and application of soft sensing modeling method in chemical process[D]. Beijing: Beijing University of Chemical Technology, 2021.
2	Liu Y Q, Xie M. Rebooting data-driven soft-sensors in process industries: a review of kernel methods[J]. Journal of Process Control, 2020, 89: 58-73.
3	黄德先, 江永亨, 金以慧. 炼油工业过程控制的研究现状、问题与展望[J]. 自动化学报, 2017, 43(6): 902-916.
	Huang D X, Jiang Y H, Jin Y H. Present research situation, major bottlenecks, and prospect of refinery industry process control[J]. Acta Automatica Sinica, 2017, 43(6): 902-916.
4	钱锋, 杜文莉, 钟伟民, 等. 石油和化工行业智能优化制造若干问题及挑战[J]. 自动化学报, 2017, 43(6): 893-901.
	Qian F, Du W L, Zhong W M, et al. Problems and challenges of smart optimization manufacturing in petrochemical industries[J]. Acta Automatica Sinica, 2017, 43(6): 893-901.
5	Jia M W, Xu D Y, Yang T, et al. Graph convolutional network soft sensor for process quality prediction[J]. Journal of Process Control, 2023, 123: 12-25.
6	Hasnen S H, Shahid M, Zabiri H, et al. Semi-supervised adaptive PLS soft-sensor with PCA-based drift correction method for online valuation of NOx emission in industrial water-tube boiler[J]. Process Safety and Environmental Protection, 2023, 172: 787-801.
7	Li Z, Jin H P, Dong S L, et al. Semi-supervised ensemble support vector regression based soft sensor for key quality variable estimation of nonlinear industrial processes with limited labeled data[J]. Chemical Engineering Research and Design, 2022, 179: 510-526.
8	顾俊发, 许明阳, 马方圆, 等. 基于MIC的支持向量回归及其在化工过程中的应用[J]. 化工学报, 2021, 72(3): 1480-1486.
	Gu J F, Xu M Y, Ma F Y, et al. Support vector regression based on maximal information coefficient and its application in chemical industrial processes[J]. CIESC Journal, 2021, 72(3): 1480-1486.
9	Shang C, Yang F, Huang D X, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of Process Control, 2014, 24(3): 223-233.
10	Liu Y Q, Yuan J Y, Cai B P, et al. Multi-step and multi-task learning to predict quality-related variables in wastewater treatment processes[J]. Process Safety and Environmental Protection, 2023, 180: 404-416.
11	Sun K, Huang S H, Wong D S H, et al. Design and application of a variable selection method for multilayer perceptron neural network with LASSO[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(6): 1386-1396.
12	Wang Y L, Pan Z F, Yuan X F, et al. A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network[J]. ISA Transactions, 2020, 96: 457-467.
13	Yan X F, Wang J, Jiang Q C. Deep relevant representation learning for soft sensing[J]. Information Sciences, 2020, 514: 263-274.
14	Yuan X F, Zhou J, Huang B, et al. Hierarchical quality-relevant feature representation for soft sensor modeling: a novel deep learning strategy[J]. IEEE Transactions on Industrial Informatics, 2020, 16(6): 3721-3730.
15	周乐, 沈程凯, 吴超, 等. 深度融合特征提取网络及其在化工过程软测量中的应用[J]. 化工学报, 2022, 73(7): 3156-3165.
	Zhou L, Shen C K, Wu C, et al. Deep fusion feature extraction network and its application in chemical process soft sensing [J]. CIESC Journal, 2022, 73(7): 3156-3165.
16	Yuan X F, Li L, Wang Y L. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE Transactions on Industrial Informatics, 2020, 16(5): 3168-3176.
17	孙凯, 隋璘, 张芳芳, 等. 基于非负绞杀与长短期记忆神经网络的动态软测量算法[J]. 控制理论与应用, 2023, 40(1): 83-93.
	Sun K, Sui L, Zhang F F, et al. Dynamic soft sensor algorithm based on nonnegative garrote and long short-term memory neural network[J]. Control Theory & Applications, 2023, 40(1): 83-93.
18	常树超, 赵春晖. 一种时空协同的图卷积长短期记忆网络及其工业软测量应用[J]. 控制与决策, 2022, 37(1): 77-86.
	Chang S C, Zhao C H. A spatio-temporal synergistic graph convolution long short-term memory network and its application for industrial soft sensors[J]. Control and Decision, 2022, 37(1): 77-86.
19	He Y L, Lv S H, Zhu Q X, et al. Novel multiattribute space-based LSTM for industrial soft sensor applications[J]. IEEE Transactions on Industrial Informatics,2024, 20(3): 4745-4752.
20	Zhou J Y, Wang X L, Yang C H, et al. A novel soft sensor modeling approach based on difference-LSTM for complex industrial process[J]. IEEE Transactions on Industrial Informatics, 2022, 18(5): 2955-2964.
21	李祥宇, 隋璘, 马君霞, 等. 基于时序迁移与双流加权的ONLSTM软测量建模[J]. 化工学报, 2023, 74(11): 4622-4633.
	Li X Y, Sui L, Ma J X, et al. ONLSTM soft sensor modeling based on time series transfer and dual stream weighting[J]. CIESC Journal, 2023, 74(11): 4622-4633.
22	罗顺桦, 王振雷, 王昕. 基于注意力机制的Multi-head CNN-LSTM软测量建模[J]. 控制工程, 2022, 29(10): 1821-1828.
	Luo S H, Wang Z L, Wang X. Multi-head CNN-LSTM soft sensing modeling based on attention mechanism[J]. Control Engineering of China, 2022, 29(10): 1821-1828.
23	Ma L W, Zhao Y, Wang B, et al. A multistep sequence-to-sequence model with attention LSTM neural networks for industrial soft sensor application[J]. IEEE Sensors Journal, 2023, 23(10): 10801-10813.
24	Yuan X F, Li L, Shardt Y A W, et al. Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development[J]. IEEE Transactions on Industrial Electronics, 2021, 68(5): 4404-4414.
25	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报,2023, 74(8): 3407-3418.
	Yan L Q, Wang Z L. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model[J]. CIESC Journal, 2023, 74(8): 3407-3418.
26	李祥宇, 隋璘, 熊伟丽. 基于自注意力机制与卷积ONLSTM网络的软测量算法[J]. 智能系统学报, 2023, 18(5): 957-965.
	Li X Y, Sui L, Xiong W L. Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network[J]. CAAI Transactions on Intelligent Systems, 2023, 18(5): 957-965.
27	Yuan X F, Huang L F, Ye L J, et al. Quality prediction modeling for industrial processes using multiscale attention-based convolutional neural network[J]. IEEE Transactions on Cybernetics, 2024, 54(5): 2696-2707.
28	Zhu X L, Hao K R, Xie R M, et al. Soft sensor based on eXtreme gradient boosting and bidirectional converted gates long short-term memory self-attention network[J]. Neurocomputing, 2021, 434: 126-136.
29	Yu X, Zhang D M, Zhu T Q, et al. Novel hybrid multi-head self-attention and multifractal algorithm for non-stationary time series prediction[J]. Information Sciences, 2022, 613: 541-555.
30	田永林, 王雨桐, 王建功, 等. 视觉Transformer研究的关键问题: 现状及展望[J]. 自动化学报, 2022, 48(4): 957-979.
	Tian Y L, Wang Y T, Wang J G, et al. Key problems and progress of vision transformers: the state of the art and prospects[J]. Acta Automatica Sinica, 2022, 48(4): 957-979.
31	Yuan X F, Qi S B, Wang Y L, et al. Quality variable prediction for nonlinear dynamic industrial processes based on temporal convolutional networks[J]. IEEE Sensors Journal, 2021, 21(18): 20493-20503.
32	Fortuna L, Graziani S, Xibilia M G. Soft sensors for product quality monitoring in debutanizer distillation columns[J]. Control Engineering Practice, 2005, 13(4): 499-508.
33	齐帅宾. 基于卷积神经网络局部时空特征提取的工业数据软测量[D]. 长沙: 中南大学, 2022.
	Qi S B. Local spatiotemporal feature representation and soft sensors for industrial data based on convolutional neural network[D]. Changsha: Central South University, 2022.
34	Fortuna L, Rizzo A, Sinatra M, et al. Soft analyzers for a sulfur recovery unit[J]. Control Engineering Practice, 2003, 11(12): 1491-1500.

[1]	黎宏陶, 王振雷, 王昕. 基于即时学习的改进条件高斯回归软测量[J]. 化工学报, 2024, 75(6): 2299-2312.
[2]	张晗, 张淑宁, 刘珂, 邓冠龙. 基于慢特征分析与最小二乘支持向量回归集成的草酸钴合成过程粒度预报[J]. 化工学报, 2024, 75(6): 2313-2321.
[3]	陈思睿, 毕景良, 王雷, 李元媛, 陆规. 气液两相流流型特征无监督提取的卷积自编码器：机理及应用[J]. 化工学报, 2024, 75(3): 847-857.
[4]	张领先, 刘斌, 邓琳, 任宇航. 基于改进TSO优化Xception的PEMFC故障诊断[J]. 化工学报, 2024, 75(3): 945-955.
[5]	董益斌, 熊敬超, 王敬宇, 汪守康, 王亚飞, 黄群星. 融合激光雷达料位测算的锅炉燃烧优化模型预测控制[J]. 化工学报, 2024, 75(3): 924-935.
[6]	李诗浩, 吴振华, 赵展烽, 吴洪, 杨冬, 石家福, 姜忠义. 化工过程中的电子传递、质子传递和分子传递[J]. 化工学报, 2024, 75(3): 1052-1064.
[7]	蒙西, 王岩, 孙子健, 乔俊飞. 基于注意力模块化神经网络的城市固废焚烧过程氮氧化物排放预测[J]. 化工学报, 2024, 75(2): 593-603.
[8]	肖拥君, 时兆翀, 万仁, 宋璠, 彭昌军, 刘洪来. 反向传播神经网络用于预测离子液体的自扩散系数[J]. 化工学报, 2024, 75(2): 429-438.
[9]	王林, 江荣鼎, 张春晓, 李修真, 谈莹莹. 含R1234yf混合工质汽液相平衡的混合规则评估与预测研究[J]. 化工学报, 2024, 75(2): 475-483.
[10]	刘起超, 张世博, 周云龙, 李昱庆, 陈聪, 冉议文. 起伏振动水平管气液两相流型及转变机理[J]. 化工学报, 2024, 75(2): 493-504.
[11]	尹刚, 钱中友, 曹文琦, 全鹏程, 许亨权, 颜非亚, 王民, 向禹, 向冬梅, 卢剑, 左玉海, 何文, 卢润廷. 基于Adaboost-PSO-SVM的铝电解槽健康状态诊断方法研究[J]. 化工学报, 2024, 75(1): 354-365.
[12]	温凯杰, 郭力, 夏诏杰, 陈建华. 一种耦合CFD与深度学习的气固快速模拟方法[J]. 化工学报, 2023, 74(9): 3775-3785.
[13]	尹刚, 李伊惠, 何飞, 曹文琦, 王民, 颜非亚, 向禹, 卢剑, 罗斌, 卢润廷. 基于KPCA和SVM的铝电解槽漏槽事故预警方法[J]. 化工学报, 2023, 74(8): 3419-3428.
[14]	诸程瑛, 王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报, 2023, 74(8): 3429-3437.
[15]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.

基于MHSA-LSTM的软测量建模及其在化工过程中的应用

Soft sensor modeling based on MHSA-LSTM and its application in chemical process

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 34

相关文章 15

编辑推荐

Metrics

本文评价