TTPA-LSTM soft sensor modeling for multi-sampling rate data

doi:10.11949/0438-1157.20241121

Abstract

Abstract:

In practical industrial production, the time lags and sampling rate differences among process variables can deteriorate the modeling quality, rendering many soft sensing models inapplicable. Therefore, a soft sensing modeling approach based on time-aware temporal pattern attention (TTPA) mechanism and long short-term memory network is proposed. In this study, we first reconstruct the data corresponding to high and low sampling rates into short-term and long-term information, respectively. A time-aware module is utilized to decompose the input information while considering the characteristics of time intervals. To address the issue of low proportion of quality-related information, a non-increasing heuristic decay function is designed to weight the short-term information. By combining these weighted components, we derive an integrated feature set that encapsulates both short-term and long-term information, thereby mitigating the impact of data loss resulting from multiple sampling rates. Secondly, a feature optimization module is introduced to achieve two-dimensional filtering of features, and the time lag information in the multivariate time series is analyzed across time steps to obtain more effective quality-related features. Finally, a soft sensing model based on TTPA-based long short-term memory network is established. The effectiveness and superiority of the proposed model were verified through the application simulation of IndPensim process and debutanizer process.

Key words: multi-sampling rate, time-aware temporal pattern attention, long short-term memory network, soft-sensor, neural networks, process control, dynamic modeling

摘要：

实际工业生产中，过程变量间存在的时滞和采样率差异会降低建模质量，使得许多软测量模型无法适用。因此，提出一种基于时间感知模式注意力（time-aware temporal pattern attention，TTPA）机制和长短时记忆网络的软测量建模方法。首先，将高、低采样率对应的数据分别重构为短期和长期信息，采用时间感知模块将输入信息分解并考虑时间间隔特性，针对质量相关信息占比低的问题，设计非递增启发式衰减函数对短期信息进行加权，组合后获得长短期信息集成特征，降低因多采样率产生的数据缺失影响。其次，引入特征优化模块实现特征二维滤波，跨时间步解析多元时间序列中的时滞信息，获取更有效的质量相关特征。最后，搭建了基于TTPA的长短期记忆网络软测量模型。通过工业青霉素发酵过程和脱丁烷塔过程的应用仿真，验证了所提模型的有效性和优越性。

关键词: 多采样率, 时间感知模式注意力, 长短时记忆网络, 软测量, 神经网络, 过程控制, 动态建模

CLC Number:

TP 273

Fazheng WANG, Lin SUI, Weili XIONG. TTPA-LSTM soft sensor modeling for multi-sampling rate data[J]. CIESC Journal, 2025, 76(4): 1635-1646.

王法正, 隋璘, 熊伟丽. 面向多采样率数据的TTPA-LSTM软测量建模[J]. 化工学报, 2025, 76(4): 1635-1646.

Figures/Tables 18

Fig.1 Structure of TTPA-LSTM soft sensor model

Fig.2 Illustrations of the multirate sampled data in industrial processes

Fig.3 Multirate sampling feature processing flow

Table 1 Variable description of IndPensim

符号	变量名称	采样间隔/min
R_W	加热/冷却水流速	12
V_V	容器体积	12
V_W	容器质量	12
$E_{C O_{2}}$	废气二氧化碳浓度	24
$E_{O_{2}}$	废气氧浓度	24
$R_{C O_{2}}$	碳吸收速率	12
$R_{O_{2}}$	氧吸收速率	12
P	青霉素浓度	60

Table 1 Variable description of IndPensim

符号	变量名称	采样间隔/min
R_W	加热/冷却水流速	12
V_V	容器体积	12
V_W	容器质量	12
$E_{C O_{2}}$	废气二氧化碳浓度	24
$E_{O_{2}}$	废气氧浓度	24
$R_{C O_{2}}$	碳吸收速率	12
$R_{O_{2}}$	氧吸收速率	12
P	青霉素浓度	60

Table 2 The prediction evaluation metrics for penicillin concentration of various models

所用模型	RMSE	MAE	MAPE	R²
MLP	5.55531	4.48582	4.03364	0.69172
RNN	4.54358	3.77206	5.97488	0.79377
NI^[16]	3.52126	2.63703	1.24222	0.87614
MSDF^[17]	3.32183	2.54239	1.53001	0.88978
SFA-MLP^[19]	2.81958	2.05189	1.21357	0.92058
SA-TLSTM	2.57029	1.87089	0.84318	0.93401
TTPA-LSTM	1.22438	0.92089	0.34144	0.98503

Fig.4 Comparison of penicillin concentration results among various models

Fig.5 Penicillin concentration prediction error among various models

Table 3 The prediction evaluation metrics for penicillin concentration of ablation experiments

模型	RMSE	MAE	MAPE	R²
LSTM	3.80403	3.07299	1.71857	0.85545
TLSTM	2.86516	2.46471	2.02471	0.91799
TPA-LSTM	2.74571	2.02671	0.68686	0.92469
TTPA-LSTM	1.22438	0.92089	0.34144	0.98503

Fig.6 Comparison of penicillin concentration results of ablation experiments

Table 4 The prediction evaluation metrics for penicillin concentration of K-fold cross-validation

子集序号	RMSE	MAE	MAPE	R²
1	1.38802	1.07046	0.82576	0.98066
2	1.26901	0.99769	0.85675	0.98213
3	1.36301	1.04380	0.43557	0.98136
4	1.43175	1.07177	0.55785	0.97943
5	1.22438	0.92089	0.34144	0.98503

Fig.7 Process flow of debutanizer

Table 5 Variable description of debutanizer

符号	变量名称	采样间隔/min
x₁	塔顶端温度	10
x₂	塔顶端压力	10
x₃	塔顶端回流量	20
x₄	塔顶端出料量	20
x₅	塔板6温度	10
x₆	塔底温度1	10
x₇	塔底温度2	10
y	塔底丁烷含量	40

Table 6 Prediction and evaluation indicators of multirate sampling models in the debutanizer dataset

模型	RMSE	MAE	MAPE	R²
NI^[16]	0.04298	0.03221	0.12219	0.93912
MSDF^[17]	0.03623	0.02836	0.11641	0.95685
SFA-MLP^[19]	0.03494	0.02507	0.09652	0.95978
TTPA-LSTM	0.02424	0.01822	0.08607	0.98089

Fig.8 Histogram of prediction error for butane concentration at the bottom of a multirate sampling model tower

Fig.9 Box plots of prediction errors for butane concentration using various multirate sampling models

Fig.10 Comparison of numerical simulation results for multirate models

Table 7 Predictive evaluation indicators for ablation experiments on the debutanizer dataset

模型	RMSE	MAE	MAPE	R²
LSTM	0.04665	0.03765	0.14688	0.92874
TLSTM	0.03296	0.02649	0.10634	0.96469
TPA-LSTM	0.03698	0.02693	0.09838	0.95512
TTPA-LSTM	0.02424	0.01822	0.08607	0.98089

Fig.11 RMSE prediction of butane concentration using TTPA-LSTM model under various parameters

References 33

1	熊伟丽, 孙文心, 马君霞. 带自相关约束的NARX动态软测量模型[J]. 控制与决策, 2020, 35(4): 816-822.
	Xiong W L, Sun W X, Ma J X. Autocorrelation constrained NARX dynamic soft sensing model[J]. Control and Decision, 2020, 35(4): 816-822.
2	Jiang Y C, Yin S, Dong J W, et al. A review on soft sensors for monitoring, control, and optimization of industrial processes[J]. IEEE Sensors Journal, 2021, 21(11): 12868-12881.
3	黎宏陶, 王振雷, 王昕. 基于即时学习的改进条件高斯回归软测量[J]. 化工学报, 2024, 75(6): 2299-2312.
	Li H T, Wang Z L, Wang X. Improved conditional Gaussian regression soft sensor based on just-in-time learning[J]. CIESC Journal, 2024, 75(6): 2299-2312.
4	李祥宇, 隋璘, 马君霞, 等. 基于时序迁移与双流加权的ONLSTM软测量建模[J]. 化工学报, 2023, 74(11): 4622-4633.
	Li X Y, Sui L, Ma J X, et al. ONLSTM soft sensor modeling based on time series transfer and dual stream weighting[J]. CIESC Journal, 2023, 74(11): 4622-4633.
5	Narayanan H, Luna M F, von Stosch M, et al. Bioprocessing in the digital age: the role of process models[J]. Biotechnology Journal, 2020, 15(1): e1900172.
6	Wang K C, Shang C, Liu L, et al. Dynamic soft sensor development based on convolutional neural networks[J]. Industrial & Engineering Chemistry Research, 2019, 58(26): 11521-11531.
7	Yin S L, Li Y G, Sun B, et al. Mixed kernel principal component weighted regression based on just-in-time learning for soft sensor modeling[J]. Measurement Science and Technology, 2022, 33(1): 015102.
8	程康明, 熊伟丽. 一种自训练框架下的三优选半监督回归算法[J]. 智能系统学报, 2020, 15(3): 568-577.
	Cheng K M, Xiong W L. Three-optimal semi-supervised regression algorithm under self-training framework[J]. CAAI Transactions on Intelligent Systems, 2020, 15(3): 568-577.
9	Li X H, Ma J W. Domain adaptation based on semi-supervised cross-domain mean discriminative analysis and kernel transfer extreme learning machine[J]. Sensors, 2023, 23(13): 6102.
10	Ye Z, Cheng X, Ding X, et al. A new unconditionally stable FDTD method based on artificial neural network[J]. IEEE Transactions on Microwave Theory and Techniques, 2024, 72(10): 5759-5767.
11	Wang J Y, Yao L, Xiong W L. Novel semi-supervised deep probabilistic slow feature extraction for online chemical process soft sensing application[J]. IEEE Transactions on Instrumentation and Measurement, 2024, 73: 2517711.
12	Chai Z, Zhao C H, Huang B. Variational progressive-transfer network for soft sensing of multirate industrial processes[J]. IEEE Transactions on Cybernetics, 2022, 52(12): 12882-12892.
13	Liu D J, Wang Y L, Liu C L, et al. Multirate-former: an efficient transformer-based hierarchical network for multistep prediction of multirate industrial processes[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 73: 2502313.
14	Lyu Y T, Zhou L, Cong Y, et al. Multirate mixture probability principal component analysis for process monitoring in multimode processes[J]. IEEE Transactions on Automation Science and Engineering, 2024, 21(2): 2027-2038.
15	Lin B, Recke B, Knudsen J K H, et al. Data-driven soft sensor design with multiple-rate sampled data: a comparative study[C]//2007 European Control Conference (ECC). Kos, Greece: IEEE, 2007: 4740-4745.
16	Grujić D N. Numerical Hilbert transform algorithm for causal interpolation of functions represented by cubic and exponential splines[J]. IEEE Access, 2021, 9: 136702-136709.
17	Zhang S M, Zhao C H. Slow-feature-analysis-based batch process monitoring with comprehensive interpretation of operation condition deviation and dynamic anomaly[J]. IEEE Transactions on Industrial Electronics, 2019, 66(5): 3773-3783.
18	He K J, Yang Q, Ji L, et al. Financial time series forecasting with the deep learning ensemble model[J]. Mathematics, 2023, 11(4): 1054.
19	Zheng J Q, Ma L W, Wu Y, et al. Nonlinear dynamic soft sensor development with a supervised hybrid CNN-LSTM network for industrial processes[J]. ACS Omega, 2022, 7(19): 16653-16664.
20	Baytas I M, Xiao C, Zhang X, et al. Patient subtyping via time-aware LSTM networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax, NS, Canada: ACM, 2017: 65-74.
21	曹鲁慧, 秦丰林, 闫中敏. 基于TLSTM的医疗保险欺诈检测[J]. 计算机工程与应用, 2020, 56(21): 237-241.
	Cao L H, Qin F L, Yan Z M. TLSTM-based medical insurance fraud detection[J]. Computer Engineering and Applications, 2020, 56(21): 237-241.
22	李妍君, 熊伟丽, 徐保国. 一种带过程变量时滞估计的在线软测量建模方法[J]. 信息与控制, 2016, 45(6): 641-646.
	Li Y J, Xiong W L, Xu B G. An online soft-sensor modeling method including process variable time-delay estimation[J]. Information and Control, 2016, 45(6): 641-646.
23	陈为胜, 李俊民. 非线性时滞大系统自适应神经网络分散控制[J]. 控制与决策, 2006, 21(8): 873-878.
	Chen W S, Li J M. Adaptive neural network decentralized control for nonlinear time-delay large-scale systems[J]. Control and Decision, 2006, 21(8): 873-878.
24	张晓迪, 蔡云泽, 何星, 等. 基于神经网络的不稳定时滞对象控制[J]. 上海交通大学学报, 2014, 48(7): 1033-1038.
	Zhang X D, Cai Y Z, He X, et al. Neural network control for unstable processes with time delay[J]. Journal of Shanghai Jiao Tong University, 2014, 48(7): 1033-1038.
25	孙凯, 隋璘, 张芳芳, 等. 基于非负绞杀与长短期记忆神经网络的动态软测量算法[J]. 控制理论与应用, 2023, 40(1): 83-93.
	Sun K, Sui L, Zhang F F, et al. Dynamic soft sensor algorithm based on nonnegative garrote and long short-term memory neural network[J]. Control Theory & Applications, 2023, 40(1): 83-93.
26	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
	Yan L Q, Wang Z L. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model[J]. CIESC Journal, 2023, 74(8): 3407-3418.
27	Xia J, Feng Y W, Lu C, et al. LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems[J]. Engineering Failure Analysis, 2021, 125: 105385.
28	Shih S Y, Sun F K, Lee H Y. Temporal pattern attention for multivariate time series forecasting[J]. Machine Learning, 2019, 108(8): 1421-1441.
29	Duan R, Liu J, Zhou J Z, et al. Study on performance evaluation and prediction of francis turbine units considering low-quality data and variable operating conditions[J]. Applied Sciences, 2022, 12(10): 4866.
30	Ren L, Wang T, Yuanjun L L, et al. A data-driven self-supervised LSTM-DeepFM model for industrial soft sensor[J]. IEEE Transactions on Industrial Informatics, 2022, 18(9): 5859-5869.
31	Zhang X R, Song C Y, Zhao J, et al. Domain adaptation mixture of Gaussian processes for online soft sensor modeling of multimode processes when sensor degradation occurs[J]. IEEE Transactions on Industrial Informatics, 2022, 18(7): 4654-4664.
32	Jia M W, Xu D Y, Yang T, et al. Graph convolutional network soft sensor for process quality prediction[J]. Journal of Process Control, 2023, 123: 12-25.
33	Sun W X, Xiong W L, Chen H T, et al. A novel CVAE-based sequential Monte Carlo framework for dynamic soft sensor applications[J]. IEEE Transactions on Industrial Informatics, 2024, 20(3): 3789-3800.

[1]	Xinmei ZHANG, Ao ZHANG, Dehua QIU, Xiaoshuang LIU, Chen CHEN. Dynamic domino effect assessment method based on thermal response mechanism of pool fire in tank farm [J]. CIESC Journal, 2025, 76(4): 1885-1897.
[2]	Hanxiao ZHANG, Ruiqi WANG, Yating ZHANG. Prediction of scale factor of heat exchangers based on CNN-LSTM neural network [J]. CIESC Journal, 2025, 76(4): 1671-1679.
[3]	Xinyu ZHENG, Zehua REN, Li ZHOU, Shiyang CHAI, Xu JI. Lattice energy regression model based on crystal graph convolutional neural networks [J]. CIESC Journal, 2025, 76(3): 1084-1092.
[4]	Dafen WANG, Lili TANG, Xinyan ZHANG, Chunyu NIE, Mingzhu LI, Jing WU. Multi-output tri-training heterogeneous soft sensor modeling based on time difference [J]. CIESC Journal, 2025, 76(3): 1143-1155.
[5]	Shaoji WANG, Kuangrong HAO, Lei CHEN. Research on temperature forecasting of polyester fiber esterification process based on federated learning [J]. CIESC Journal, 2025, 76(1): 283-295.
[6]	Xin GUO, Wenjing LI, Junfei QIAO. Prediction of effluent parameters in wastewater treatment process using self-organizing modular neural network [J]. CIESC Journal, 2024, 75(9): 3242-3254.
[7]	Qianqian WANG, Bing LI, Weibo ZHENG, Guomin CUI, Bingtao ZHAO, Pingwen MING. Three-dimensional modeling of local dynamic characteristics in hydrogen fuel cells [J]. CIESC Journal, 2024, 75(8): 2812-2820.
[8]	Junxia MA, Lintao LI, Weili XIONG. A semi-supervised soft sensor modeling method based on the Tri-training GPR [J]. CIESC Journal, 2024, 75(7): 2613-2623.
[9]	Han ZHANG, Shuning ZHANG, Ke LIU, Guanlong DENG. Particle size prediction of cobalt oxalate synthesis process based on slow feature analysis and least squares support vector regression [J]. CIESC Journal, 2024, 75(6): 2313-2321.
[10]	Jianwen ZHANG, Tingsheng ZHAO, Pingyu WAN, Qianlin WANG, Zhan DOU, Bo XU. Discussion on integrated security control in process industry [J]. CIESC Journal, 2024, 75(6): 2375-2384.
[11]	Yuhang HE, Dan XIE, Yangcheng LYU. Research progress of cationic polymerization in microreactor [J]. CIESC Journal, 2024, 75(4): 1302-1316.
[12]	Sirui CHEN, Jingliang BI, Lei WANG, Yuanyuan LI, Gui LU. Unsupervised-feature extraction of gas-liquid two-phase flow pattern based on convolutional autoencoder: principle and application [J]. CIESC Journal, 2024, 75(3): 847-857.
[13]	Yibin DONG, Jingchao XIONG, Jingyu WANG, Shoukang WANG, Yafei WANG, Qunxing HUANG. LiDAR measurement based on model predictive control for boiler combustion optimization [J]. CIESC Journal, 2024, 75(3): 924-935.
[14]	Pei WANG, Ruiming DUAN, Guangru ZHANG, Wanqin JIN. Modeling and simulation analysis of solar driven membrane separation biomethane hydrogen production process [J]. CIESC Journal, 2024, 75(3): 967-973.
[15]	Yongjun XIAO, Zhaochong SHI, Ren WAN, Fan SONG, Changjun PENG, Honglai LIU. Prediction of self-diffusion coefficients of ionic liquids using back-propagation neural networks [J]. CIESC Journal, 2024, 75(2): 429-438.