Improved conditional Gaussian regression soft sensor based on just-in-time learning

doi:10.11949/0438-1157.20240127

Abstract

Abstract:

Data-driven online soft sensing is an important research direction in current industrial intelligent sensing. In the practical use of algorithms, process mode switching and data drift might reduce the performance of soft sensors. Traditional adaptive approaches confront limitations, such as a limited variety of models and a tendency to forget previously acquired modes. A sample temporal and spatial weighted conditional gaussian regression (STWCGR) soft sensor algorithm based on just-in-time learning is proposed to overcome these issues. This algorithm uses probability density and conditional probability for soft sensing modeling and prediction. First, a sample spatiotemporal mixed-weight technique is used to pick local modeling data in accordance with the just-in-time learning principle. Then, the local Gaussian probability density models are accumulated to fit the data distribution by incorporating the concept of Gaussian mixture regression. Finally, momentum updates and mode updates are introduced to enhance prediction stability and endow the model with adaptability to new working conditions. The efficacy of the suggested algorithm is confirmed by simulation studies with respect to forecast precision, stability, and flexibility to accommodate new modes.

Key words: AI perception, data-driven soft sensor, prediction, just-in-time learning, Gaussian mixture regression

摘要：

基于数据驱动的在线软测量是当前工业智能化感知的重要研究方向。在算法实际部署中，过程模态切换以及数据漂移都会导致软测量性能下降，传统自适应方法又存在模型单一、模态遗忘等不足。针对上述问题提出一种基于即时学习的样本时空加权条件高斯回归（STWCGR）软测量算法。该方法用概率密度估计和条件概率计算实现软测量建模和预测：首先根据即时学习思想通过样本时空混合加权方法筛选局部建模数据，然后结合高斯混合回归思想累积局部单高斯概率密度模型对数据分布进行拟合，最后引入预测动量更新和模态更新策略提高预测稳定性并赋予模型对新工况的学习适应能力。通过仿真实验验证了所提方法在预测精度、稳定性以及新模态适应能力上的有效性。

关键词: 智能感知, 数据驱动软测量, 预测, 即时学习, 高斯混合回归

CLC Number:

TP 274

Hongtao LI, Zhenlei WANG, Xin WANG. Improved conditional Gaussian regression soft sensor based on just-in-time learning[J]. CIESC Journal, 2024, 75(6): 2299-2312.

黎宏陶, 王振雷, 王昕. 基于即时学习的改进条件高斯回归软测量[J]. 化工学报, 2024, 75(6): 2299-2312.

Figures/Tables 20

Fig.1 Flowchart for soft sensing based on STWCGR

Table 1 Parameters of Gaussian components

高斯模态	$μ$	$Σ$
1	${[2, 2]}^{T}$	$[\begin{matrix} 1 & 0.5 \\ 0.5 & 1 \end{matrix}]$
2	${[- 4, 2]}^{T}$	$[\begin{matrix} 1 & - 0.5 \\ - 0.5 & 1 \end{matrix}]$
3	${[- 2, 6]}^{T}$	$[\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}]$
4	${[8, - 2]}^{T}$	$[\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}]$

Table 1 Parameters of Gaussian components

高斯模态	$μ$	$Σ$
1	${[2, 2]}^{T}$	$[\begin{matrix} 1 & 0.5 \\ 0.5 & 1 \end{matrix}]$
2	${[- 4, 2]}^{T}$	$[\begin{matrix} 1 & - 0.5 \\ - 0.5 & 1 \end{matrix}]$
3	${[- 2, 6]}^{T}$	$[\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}]$
4	${[8, - 2]}^{T}$	$[\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}]$

Fig.2 Visualization of numerical example Ⅰ

Fig.3 Prediction results of numerical example Ⅰ

Fig.4 Prediction results of numerical example Ⅱ based on STWCGR

Fig.5 Prediction results of numerical example Ⅱ based on WGR

Table 2 Soft sensing performance in numerical example Ⅱ

项目	WGR $f_{1}$	STWCGR $f_{1}$	WGR $f_{2}$	STWCGR $f_{2}$
RMSE	0.042	0.190	2.702	0.804
$R^{2}$	0.998	0.966	-1.937	0.732

Table 2 Soft sensing performance in numerical example Ⅱ

项目	WGR $f_{1}$	STWCGR $f_{1}$	WGR $f_{2}$	STWCGR $f_{2}$
RMSE	0.042	0.190	2.702	0.804
$R^{2}$	0.998	0.966	-1.937	0.732

Fig.6 Flowchart of debutanizer column

Table 3 Description of variables for debutanizer column

变量名	含义
U1	塔顶温度
U2	塔顶压力
U3	回流流量
U4	向下一过程流量
U5	第6隔板温度
U6	塔釜温度A
U7	塔釜温度B
Y	塔釜丁烷含量

Fig.7 Prediction results of debutanizer column process based on STWCGR

Table 4 Soft sensing performance in debutanizer column process

项目	LSTM	GRU	JIT-PLS	WGR	STWCGR
RMSE	0.148	0.132	0.073	0.067	0.062
$R^{2}$	0.270	0.226	0.787	0.820	0.872

Table 4 Soft sensing performance in debutanizer column process

项目	LSTM	GRU	JIT-PLS	WGR	STWCGR
RMSE	0.148	0.132	0.073	0.067	0.062
$R^{2}$	0.270	0.226	0.787	0.820	0.872

Fig.8 Prediction results of debutanizer column process based on deep learning

Fig.9 Prediction results of ablation experiment

Table 5 Soft sensing performance of ablation experiment

项目	TWCGR^*	TWCGR	STWCGR^*	STWCGR
RMSE	0.097	0.101	0.063	0.062
$R^{2}$	0.688	0.666	0.871	0.872

Table 5 Soft sensing performance of ablation experiment

项目	TWCGR^*	TWCGR	STWCGR^*	STWCGR
RMSE	0.097	0.101	0.063	0.062
$R^{2}$	0.688	0.666	0.871	0.872

Fig.10 Flowchart of TE process

Table 6 Description of variables for TE process

变量名	含义	变量名	含义
X1	物料A 流量	X13	分离器
X2	物料D 流量	X14	分离器塔底流量
X3	物料E 流量	X15	汽提塔液位
X4	A、B、C总进料	X16	汽提塔压力
X5	再循环流量	X17	汽提塔塔底流量
X6	反应器进料量	X18	汽提塔温度
X7	反应器压力	X19	蒸汽流量
X8	反应器液位	X20	压缩机功率
X9	反应器温度	X21	反应器冷凝水出口温度
X10	排放速度	X22	冷凝器冷却水出口温度
X11	分离器温度	Y	产物中G组分摩尔分数
X12	分离器液位	Y	产物中G组分摩尔分数

Fig.11 Prediction results of TE process based on STWCGR

Fig.12 Prediction results of TE process based on LSTM

Table 7 Process variables of ethylene distillation tower

变量	含义
X1	塔顶压力
X2	塔顶温度
X3	142^#温度
X4	168^#温度
X5	塔釜采出温度
X6	再沸器出口物料温度
Y	塔顶乙烷浓度

Fig.13 Prediction results of ethane concentration

References 31

1	Kadlec P, Gabrys B, Strandt S. Data-driven soft sensors in the process industry[J]. Computers & Chemical Engineering, 2009, 33(4): 795-814.
2	Sun Q Q, Ge Z Q. A survey on deep learning for data-driven soft sensors[J]. IEEE Transactions on Industrial Informatics, 2021, 17(9): 5853-5866.
3	王晓阳, 王昕, 王振雷, 等. 一种乙烯裂解炉的多工况监控方法[J]. 计算机与应用化学, 2012, 29(9): 1119-1122.
	Wang X Y, Wang X, Wang Z L, et al. A novel method for monitoring ethylene cracking furnace with multiple operation modes[J]. Computers and Applied Chemistry, 2012, 29(9): 1119-1122.
4	赵俊峰. 乙烯裂解炉炉管智能结焦诊断方法研究[D]. 广州: 广东工业大学, 2020.
	Zhao J F. Study on intelligent coking diagnosis method of ethylene cracking furnace tubes[D]. Guangzhou: Guangdong University of Technology, 2020.
5	Gama J, Žliobaitė I, Bifet A, et al. A survey on concept drift adaptation[J]. ACM Computing Surveys, 2014, 46(4): 1-37.
6	Lü Y, Yang H Z. A multi-model approach for soft sensor development based on feature extraction using weighted kernel fisher criterion[J]. Chinese Journal of Chemical Engineering, 2014, 22(2): 146-152.
7	Gholami A R, Shahbazian M. Soft sensor design based on fuzzy C-Means and RFN_SVR for a stripper column[J]. Journal of Natural Gas Science and Engineering, 2015, 25: 23-29.
8	Deng X G, Chen Y X, Wang P, et al. Soft sensor modeling for unobserved multimode nonlinear processes based on modified kernel partial least squares with latent factor clustering[J]. IEEE Access, 2020, 8: 35864-35872.
9	Stržinar Ž, Pregelj B, Škrjanc I. Soft sensor for non-invasive detection of process events based on Eigenresponse Fuzzy Clustering[J]. Applied Soft Computing, 2023, 132: 109859.
10	Yuan X F, Ou C, Wang Y L, et al. A layer-wise data augmentation strategy for deep learning networks and its soft sensor application in an industrial hydrocracking process[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8): 3296-3305.
11	Dai Y, Yu Q, Yang T, et al. Enhanced soft sensor with qualified augmented data using centroid measurement criterion[C]//2021 8th International Conference on Information, Cybernetics, and Computational Social Systems (ICCSS). Beijing, China. IEEE, 2021: 435-440.
12	Zhang X R, Song C Y, Zhao J, et al. Gaussian mixture continuously adaptive regression for multimode processes soft sensing under time-varying virtual drift[J]. Journal of Process Control, 2023, 124: 1-13.
13	Yu J, Qin S J. Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models[J]. AIChE Journal, 2008, 54(7): 1811-1829.
14	Yuan X F, Ge Z Q, Song Z H. Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression[J]. Chemometrics and Intelligent Laboratory Systems, 2014, 138: 97-109.
15	Zhu J L, Ge Z Q, Song Z H. Variational Bayesian Gaussian mixture regression for soft sensing key variables in non-Gaussian industrial processes[J]. IEEE Transactions on Control Systems Technology, 2017, 25(3): 1092-1099.
16	Shao W M, Yao L, Ge Z Q, et al. Parallel computing and SGD-based DPMM for soft sensor development with large-scale semisupervised data[J]. IEEE Transactions on Industrial Electronics, 2019, 66(8): 6362-6373.
17	Lu J, Liu A J, Dong F, et al. Learning under concept drift: a review[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(12): 2346-2363.
18	Kadlec P, Grbić R, Gabrys B. Review of adaptation mechanisms for data-driven soft sensors[J]. Computers & Chemical Engineering, 2011, 35(1): 1-24.
19	Minku L L, White A P, Yao X. The impact of diversity on online ensemble learning in the presence of concept drift[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(5): 730-742.
20	Shao W M, Tian X M. Semi-supervised selective ensemble learning based on distance to model for nonlinear soft sensor development[J]. Neurocomputing, 2017, 222: 91-104.
21	Krogh A, Vedelsby J. Neural network ensembles, cross validation, and active learning[C]//Advances in Neural Information Processing Systems. MIT Press, 1994.
22	Yuan X F, Ge Z Q, Song Z H. Locally weighted kernel principal component regression model for soft sensing of nonlinear time-variant processes[J]. Industrial & Engineering Chemistry Research, 2014, 53(35): 13736-13749.
23	Hazama K, Kano M. Covariance-based locally weighted partial least squares for high-performance adaptive modeling[J]. Chemometrics and Intelligent Laboratory Systems, 2015, 146: 55-62.
24	Zhang X, Li Y, Kano M. Quality prediction in complex batch processes with just-in-time learning model based on non-gaussian dissimilarity measure[J]. Industrial & Engineering Chemistry Research, 2015, 54(31): 7694-7705.
25	Xiong W L, Zhang W, Xu B G, et al. JITL based MWGPR soft sensor for multi-mode process with dual-updating strategy[J]. Computers & Chemical Engineering, 2016, 90: 260-267.
26	Yeo W S, Saptoro A, Kumar P, et al. Just-in-time based soft sensors for process industries: a status report and recommendations[J]. Journal of Process Control, 2023, 128: 103025.
27	Jin H P, Chen X G, Yang J W, et al. Adaptive soft sensor modeling framework based on just-in-time learning and kernel partial least squares regression for nonlinear multiphase batch processes[J]. Computers & Chemical Engineering, 2014, 71: 77-93.
28	Jiang J S, Jiang Q C. Variational Bayesian probabilistic modeling framework for data-driven distributed process monitoring[J]. Control Engineering Practice, 2021, 110: 104778.
29	Guo F, Wei B, Huang B. A just-in-time modeling approach for multimode soft sensor based on Gaussian mixture variational autoencoder[J]. Computers & Chemical Engineering, 2021, 146: 107230.
30	Yuan X F, Wang Y L, Yang C H, et al. Probabilistic density-based regression model for soft sensing of nonlinear industrial processes[J]. Journal of Process Control, 2017, 57: 15-25.
31	Bishop C M. Pattern Recognition and Machine Learning[M]. New York: Springer, 2006.

[1]	Han ZHANG, Shuning ZHANG, Ke LIU, Guanlong DENG. Particle size prediction of cobalt oxalate synthesis process based on slow feature analysis and least squares support vector regression [J]. CIESC Journal, 2024, 75(6): 2313-2321.
[2]	Xi MENG, Yan WANG, Zijian SUN, Junfei QIAO. Prediction of NO _x emissions for municipal solid waste incineration processes using attention modular neural network [J]. CIESC Journal, 2024, 75(2): 593-603.
[3]	Yongjun XIAO, Zhaochong SHI, Ren WAN, Fan SONG, Changjun PENG, Honglai LIU. Prediction of self-diffusion coefficients of ionic liquids using back-propagation neural networks [J]. CIESC Journal, 2024, 75(2): 429-438.
[4]	Lin WANG, Rongding JIANG, Chunxiao ZHANG, Xiuzhen LI, Yingying TAN. Evaluation and predictive study of the mixing rules for vapor-liquid equilibrium of R1234yf mixtures [J]. CIESC Journal, 2024, 75(2): 475-483.
[5]	Qichao LIU, Shibo ZHANG, Yunlong ZHOU, Yuqing LI, Cong CHEN, Yiwen RAN. Gas-liquid two-phase flow regimes and transformation mechanism in horizontal tube under fluctuating vibration [J]. CIESC Journal, 2024, 75(2): 493-504.
[6]	Gang YIN, Zhongyou QIAN, Wenqi CAO, Pengcheng QUAN, Hengquan XU, Feiya YAN, Min WANG, Yu XIANG, Dongmei XIANG, Jian LU, Yuhai ZUO, Wen HE, Runting LU. Health state diagnosis of aluminum electrolytic cells based on Adaboost-PSO-SVM [J]. CIESC Journal, 2024, 75(1): 354-365.
[7]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[8]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[9]	Yuying GUO, Jiaqiang JING, Wanni HUANG, Ping ZHANG, Jie SUN, Yu ZHU, Junxuan FENG, Hongjiang LU. Water-lubricated drag reduction and pressure drop model modification for heavy oil pipeline [J]. CIESC Journal, 2023, 74(7): 2898-2907.
[10]	Yuan YU, Weiwei CHEN, Junjie FU, Jiaxiang LIU, Zhiwei JIAO. Study and prediction of flow field in the annular region of geometrically similar turbo air classifier [J]. CIESC Journal, 2023, 74(6): 2363-2373.
[11]	Yanhui LI, Shaoming DING, Zhouyang BAI, Yinan ZHANG, Zhihong YU, Limei XING, Pengfei GAO, Yongzhen WANG. Corrosion micro-nano scale kinetics model development and application in non-conventional supercritical boilers [J]. CIESC Journal, 2023, 74(6): 2436-2446.
[12]	Xinyuan WU, Qilei LIU, Boyuan CAO, Lei ZHANG, Jian DU. Group2vec: group vector representation and its property prediction applications based on unsupervised machine learning [J]. CIESC Journal, 2023, 74(3): 1187-1194.
[13]	Jiahui CHEN, Xinze YANG, Guzhong CHEN, Zhen SONG, Zhiwen QI. A critical discussion on developing molecular property prediction models: density of ionic liquids as example [J]. CIESC Journal, 2023, 74(2): 630-641.
[14]	Kenian SHI, Jingyuan ZHENG, Yu QIAN, Siyu YANG. Two-stage stochastic programming of steam power system based on Markov chain [J]. CIESC Journal, 2023, 74(2): 807-817.
[15]	Xinwei MA, Xingsen MU, Zhu LONG, Shengqiang SHEN. Prediction of heat transfer coefficient of horizontal tube falling film evaporation based on GA-BP neural network [J]. CIESC Journal, 2023, 74(12): 4840-4851.