ONLSTM soft sensor modeling based on time series transfer and dual stream weighting

doi:10.11949/0438-1157.20230893

Abstract

Abstract:

The modeling of actual chemical processes has characteristics such as multi-variability, nonlinearity, and dynamism, which can lead to increased model complexity and the generation of redundant information and temporal distribution shift when extracting features. Therefore, an ordered neurons long short-term memory network (ONLSTM) model based on time series transfer and dual stream weighting is proposed. First, temporal transfer is used to match feature distributions to adaptively represent feature distribution information and training is performed by dividing the time domain with the largest difference in feature distribution to reduce timing distribution mismatch, thereby solving the problem of timing distribution drift. Secondly, a dual stream weighted ONLSTM model is embedded within the time series transfer framework, and the ONLSTM main forgetting gate and main input gate are weighted separately to more accurately control the transmission of information. Further combining the dual flow structure to design the corresponding gating unit for dual information flow control, reducing the coupling effect during parameter adjustment, reducing model complexity, and improving its predictive performance. Finally, the proposed model was applied to soft sensing modeling of SO₂ concentration in the sulfur recovery process and the flue gas emissions from a certain thermal power plant desulfurization process, and compared with other deep learning networks to verify the effectiveness of the model.

Key words: time series transfer, weighted ordered neurons long short-term memory, dual stream, soft sensor, neural networks, process control, dynamic modeling

摘要：

实际化工过程建模具有多变量、非线性和动态性等特点，会导致模型复杂度提高且提取特征时产生冗余信息和时序分布漂移问题，因此提出一种基于时序迁移和双流加权的有序神经元长短时记忆网络（ONLSTM）模型。首先，利用时序迁移对特征分布进行匹配以自适应表征特征分布信息，采用划分特征分布差异最大时间域进行训练，减小时序分布失配，从而解决时序分布漂移问题；其次，在时序迁移框架内嵌入双流加权ONLSTM模型，通过对ONLSTM主遗忘门和主输入门分别加权，更精确控制传递信息；进一步结合双流结构设计双信息流控制相应门控单元，减小参数调节过程中的耦合影响，降低模型复杂度，提高其预测性能；最后，将所提模型应用于硫回收过程以及某火电厂脱硫过程排放烟气SO₂浓度软测量建模，并与其他深度学习网络进行对比，验证了模型有效性。

关键词: 时间序列迁移, 加权有序神经元长短时记忆网络, 双流结构, 软测量, 神经网络, 过程控制, 动态建模

CLC Number:

TP 273

Xiangyu LI, Lin SUI, Junxia MA, Weili XIONG. ONLSTM soft sensor modeling based on time series transfer and dual stream weighting[J]. CIESC Journal, 2023, 74(11): 4622-4633.

李祥宇, 隋璘, 马君霞, 熊伟丽. 基于时序迁移与双流加权的ONLSTM软测量建模[J]. 化工学报, 2023, 74(11): 4622-4633.

Figures/Tables 15

Fig.1 ONLSTM structure diagram

Fig.2 Weighted ONLSTM structure diagram

Fig.3 Dual stream weighted ONLSTM structure diagram

Fig.4 TT-DSW-ONLSTM structure diagram

Fig.5 Process flow diagram of sulfur recovery

Table 1 SRU process sampling variables

变量	变量描述
u₁	GAS气流
u₂	AIR空气流
u₃	AIR二次空气流
u₄	SWS区域GAS气流
u₅	SWS区域AIR空气流
y₁	SO₂浓度

Fig.6 RMSE of TT-DSW-ONLSTM model under different K values in sulfur recovery process

Fig.7 Prediction curves of SRU process under different models

Fig.8 Prediction error of SO2 concentration by various network models in SRU process

Table 2 Prediction results of SO2 concentration by various network models

算法模型	MSE/10^-4	MAE	R²
LSTM	6.9696	0.0257	0.9933
ONLSTM	3.4971	0.0143	0.8925
WONLSTM	2.1054	0.0114	0.9353
DSW-ONLSTM	1.5775	0.0104	0.9589
TT-WONLSTM	0.6773	0.0072	0.9791
TT-DSW-ONLSTM	0.3158	0.0042	0.9919

Fig.9 Process flow diagram of flue gas absorption desulfurization

Fig.10 RMSE of TT-DSW-ONLSTM model under different α and β values in dual flow structure of flue gas desulfurization process

Fig.11 Prediction curves of flue gas desulfurization process under different models

Fig.12 Prediction error of SO2 concentration by various network models in flue gas desulfurization process

Table 3 Prediction results of SO2 concentration by various network models in flue gas desulfurization process

算法模型	MSE	MAE	R²
LSTM	7.7574	2.2530	0.6961
ONLSTM	5.8056	1.9712	0.7630
WONLSTM	4.4563	1.5972	0.8254
DSW-ONLSTM	3.9376	1.5350	0.8457
TT-WONLSTM	3.5981	1.4902	0.8651
TT-DSW-ONLSTM	3.1725	1.5196	0.8757

References 31

1	周乐, 沈程凯, 吴超, 等. 深度融合特征提取网络及其在化工过程软测量中的应用[J]. 化工学报, 2022, 73(7): 3156-3165.
	Zhou L, Shen C K, Wu C, et al. Deep fusion feature extraction network and its application in chemical process soft sensing[J]. CIESC Journal, 2022, 73(7): 3156-3165.
2	Yuan X F, Li L, Wang Y L. Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J]. IEEE Transactions on Industrial Informatics, 2020, 16(5): 3168-3176.
3	Ren J Y, Chen X, Zhao C H. Spatio-temporal view GAIN for data imputation and dynamic soft sensor[C]//2022 IEEE 11th Data Driven Control and Learning Systems Conference (DDCLS). New York: IEEE, 2022: 1446-1451.
4	杜宇浩, 阎高伟, 李荣, 等. 基于局部线性嵌入的测地线流式核多工况软测量建模方法[J]. 化工学报, 2020, 71(3): 1278-1287.
	Du Y H, Yan G W, Li R, et al. Multiple working conditions soft sensor modeling method of geodesic flow kernel based on locally linear embedding[J]. CIESC Journal, 2020, 71(3): 1278-1287.
5	潘贝, 金怀平, 杨彪, 等. 基于多样性加权相似度的集成局部加权偏最小二乘软测量建模[J]. 信息与控制, 2019, 48(2): 217-223, 231.
	Pan B, Jin H P, Yang B, et al. Soft sensor development based on ensemble locally weighted partial least squares using diverse weighted similarity measures[J]. Information and Control, 2019, 48(2): 217-223, 231.
6	Liu J L, Wang Y K, Zhang Y. A novel isomap-SVR soft sensor model and its application in rotary kiln calcination zone temperature prediction[J]. Symmetry, 2020, 12(1): 167.
7	Shi X D, Kang Q, Zhou M C, et al. Soft sensing of nonlinear and multimode processes based on semi-supervised weighted Gaussian regression[J]. IEEE Sensors Journal, 2020, 20(21): 12950-12960.
8	Shang C, Yang F, Huang D X, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of Process Control, 2014, 24(3): 223-233.
9	Yao L, Ge Z Q. Deep learning of semisupervised process data with hierarchical extreme learning machine and soft sensor application[J]. IEEE Transactions on Industrial Electronics, 2018, 65(2): 1490-1498.
10	Sui L, Sun K, Ma J X, et al. Input variable selection and structure optimization for LSTM-based soft sensor with a dual nonnegative garrote approach[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-11.
11	Wang K C, Shang C, Liu L, et al. Dynamic soft sensor development based on convolutional neural networks[J]. Industrial & Engineering Chemistry Research, 2019, 58(26): 11521-11531.
12	Wang Y L, Pan Z F, Yuan X F, et al. A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network[J]. ISA Transactions, 2020, 96: 457-467.
13	Guo R Y, Liu H. A hybrid mechanism- and data-driven soft sensor based on the generative adversarial network and gated recurrent unit[J]. IEEE Sensors Journal, 2021, 21(22): 25901-25911.
14	邵伟明, 葛志强, 李浩, 等. 基于循环神经网络的半监督动态软测量建模方法[J]. 电子测量与仪器学报, 2019, 33(11): 7-13.
	Shao W M, Ge Z Q, Li H, et al. Semisupervised dynamic soft sensing approaches based on recurrent neural network[J]. Journal of Electronic Measurement and Instrumentation, 2019, 33(11): 7-13.
15	Lippi M, Montemurro M A, Degli Esposti M, et al. Natural language statistical features of LSTM-generated texts[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(11): 3326-3337.
16	Zhi Z, Liu L S, Liu D T, et al. Fault detection of the harmonic reducer based on CNN-LSTM with a novel denoising algorithm[J]. IEEE Sensors Journal, 2022, 22(3): 2572-2581.
17	Lui C F, Liu Y Q, Xie M. A supervised bidirectional long short-term memory network for data-driven dynamic soft sensor modeling[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-13.
18	Ren L, Wang T, Laili Y J, et al. A data-driven self-supervised LSTM-DeepFM model for industrial soft sensor[J]. IEEE Transactions on Industrial Informatics, 2021, 18(9): 5859-5869.
19	Xie R M, Hao K R, Huang B, et al. Data-driven modeling based on two-stream λ gated recurrent unit network with soft sensor application[J]. IEEE Transactions on Industrial Electronics, 2020, 67(8): 7034-7043.
20	Shen Y K, Tan S, Sordoni A, et al. Ordered neurons: integrating tree structures into recurrent neural networks[EB/OL]. 2018. .
21	Zhang Q, Chai B, Song B C, et al. A hierarchical fine-tuning based approach for multi-label text classification[C]//2020 IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA). Chengdu, China: IEEE, 2020: 51-54.
22	Dai S Z, Li Z H, Li L, et al. Investigating the dynamic memory effect of human drivers via ON-LSTM[J]. Science China Information Sciences, 2020, 63(9): 1-11.
23	Shi F, Cao H R, Wang Y K, et al. Chatter detection in high-speed milling processes based on ON-LSTM and PBT[J]. The International Journal of Advanced Manufacturing Technology, 2020, 111(11): 3361-3378.
24	Shi X D, Xu J, Bao H Q, et al. Domain adaptation soft sensing with parameter transferring[C]//2022 IEEE International Conference on Networking, Sensing and Control (ICNSC).New York: IEEE, 2023: 1-5.
25	Zhang X R, Song C Y, Zhao J, et al. Domain adaptation mixture of Gaussian processes for online soft sensor modeling of multimode processes when sensor degradation occurs[J]. IEEE Transactions on Industrial Informatics, 2022, 18(7): 4654-4664.
26	Chai Z, Zhao C H, Huang B, et al. A deep probabilistic transfer learning framework for soft sensor modeling with missing data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(12): 7598-7609.
27	Du Y T, Wang J D, Feng W J, et al. AdaRNN: adaptive learning and forecasting of time series[EB/OL]. 2021. .
28	Schäfer P. Scalable time series classification[J]. Data Mining and Knowledge Discovery, 2016, 30(5): 1273-1298.
29	Luo Y, Wang Y L. A statistical time-frequency model for non-stationary time series analysis[J]. IEEE Transactions on Signal Processing, 2020, 68: 4757-4772.
30	di Bella A, Fortuna L, Graziani S, et al. Soft sensor design for a sulfur recovery unit using genetic algorithms[C]//2007 IEEE International Symposium on Intelligent Signal Processing. New York: IEEE, 2008: 1-6.
31	Sun K, Wu X L, Xue J Y, et al. Development of a new multi-layer perceptron based soft sensor for SO₂ emissions in power plant[J]. Journal of Process Control, 2019, 84: 182-191.

[1]	Yue CAO, Chong YU, Zhi LI, Minglei YANG. Industrial data driven transition state detection with multi-mode switching of a hydrocracking unit [J]. CIESC Journal, 2023, 74(9): 3841-3854.
[2]	Fei KANG, Weiguang LYU, Feng JU, Zhi SUN. Research on discharge path and evaluation of spent lithium-ion batteries [J]. CIESC Journal, 2023, 74(9): 3903-3911.
[3]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[4]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[5]	Xuejin GAO, Yuzhuo YAO, Huayun HAN, Yongsheng QI. Fault monitoring of fermentation process based on attention dynamic convolutional autoencoder [J]. CIESC Journal, 2023, 74(6): 2503-2521.
[6]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[7]	Xiaodan SU, Ganyu ZHU, Huiquan LI, Guangming ZHENG, Ziheng MENG, Fang LI, Yunrui YANG, Benjun XI, Yu CUI. Optimization of wet process phosphoric acid hemihydrate process and crystallization of gypsum [J]. CIESC Journal, 2023, 74(4): 1805-1817.
[8]	Cheng YUN, Qianlin WANG, Feng CHEN, Xin ZHANG, Zhan DOU, Tingjun YAN. Deep-mining risk evolution path of chemical processes based on community structure [J]. CIESC Journal, 2023, 74(4): 1639-1650.
[9]	Xinyuan WU, Qilei LIU, Boyuan CAO, Lei ZHANG, Jian DU. Group2vec: group vector representation and its property prediction applications based on unsupervised machine learning [J]. CIESC Journal, 2023, 74(3): 1187-1194.
[10]	Zhongqiu ZHANG, Hongguang LI, Yilin SHI. A multi-task learning approach for complex chemical processes based on manual predictive manipulating strategies [J]. CIESC Journal, 2023, 74(3): 1195-1204.
[11]	Jianghuai ZHANG, Zhong ZHAO. Robust minimum covariance constrained control for C₃ hydrogenation process and application [J]. CIESC Journal, 2023, 74(3): 1216-1227.
[12]	Weiyi SU, Jiahui DING, Chunli LI, Honghai WANG, Yanjun JIANG. Research progress of enzymatic reactive crystallization [J]. CIESC Journal, 2023, 74(2): 617-629.
[13]	Xuejin GAO, Kun CHENG, Huayun HAN, Huihui Gao, Yongsheng QI. Fault diagnosis of chillers using central loss conditional generative adversarial network [J]. CIESC Journal, 2022, 73(9): 3950-3962.
[14]	Taoyan ZHAO, Jiangtao CAO, Ping LI, Lin FENG, Yu SHANG. Application of interval type-2 fuzzy immune PID controller to temperature control system for uncatalysed oxidation of cyclohexane [J]. CIESC Journal, 2022, 73(7): 3166-3173.
[15]	Le ZHOU, Chengkai SHEN, Chao WU, Beiping HOU, Zhihuan SONG. Deep fusion feature extraction network and its application in chemical process soft sensing [J]. CIESC Journal, 2022, 73(7): 3156-3165.