A semi-supervised soft sensor modeling method based on the Tri-training GPR

doi:10.11949/0438-1157.20240322

Abstract

Abstract:

Ensemble learning often achieves significantly superior generalization capabilities than a single learner by building and combining multiple learners. However, it is still a challenge to build a high-performance ensemble learning soft sensor model when the proportion of labeled data is small. In order to solve this problem, this paper proposes a soft-sensor modeling method based on the semi-supervised ensemble learning: Tri-training Gaussian process regression (Tri-training GPR) model. The modeling strategy gives full play to the advantages of semi-supervised learning, reducing the demand for labeled sample data in the modeling process. Under low data labeling rate, the labeled sample data set for modeling can still be expanded by filtering unlabeled data. Furthermore, a new idea of selecting high-confidence samples is proposed by combining the advantages of semi-supervised learning and ensemble learning. The proposed method was applied to the penicillin fermentation and debutanization tower process, and the soft sensor models for predicting penicillin and butane concentrations were established. Compared with the traditional modeling methods, the proposed method obtained better prediction results, which verified the effectiveness of the model.

Key words: soft senor, ensemble learning, semi-supervised learning, Tri-training, Gaussian process regression, process control, kinetic modeling, chemical processes

摘要：

集成学习因通过构建并结合多个学习器，常获得比单一学习器显著优越的泛化能力。但是在标记数据比例较少时，建立高性能的集成学习软测量模型依然是个挑战。针对这一个问题，提出一种基于半监督集成学习的软测量建模方法——Tri-training GPR 模型。该建模策略充分发挥了半监督学习的优势，减轻建模过程对标记样本数据的需求，在低数据标签率下，仍能通过对无标记数据进行筛选从而扩充可用于建模的有标记样本数据集，并进一步结合半监督学习和集成学习的优势，提出一种新的选择高置信度样本的思路。将所提方法应用于青霉素发酵和脱丁烷塔过程，建立青霉素和丁烷浓度预测软测量模型，与传统的建模方法相比获得了更优的预测结果，验证了模型的有效性。

关键词: 软测量, 集成学习, 半监督学习, Tri-training, 高斯过程回归, 过程控制, 动力学模型, 化学过程

CLC Number:

TQ 018

Junxia MA, Lintao LI, Weili XIONG. A semi-supervised soft sensor modeling method based on the Tri-training GPR[J]. CIESC Journal, 2024, 75(7): 2613-2623.

马君霞, 李林涛, 熊伟丽. 基于Tri-training GPR的半监督软测量建模方法[J]. 化工学报, 2024, 75(7): 2613-2623.

Figures/Tables 15

Fig.1 The framework of Tri-training GPR model

Fig.2 The process flow diagram of penicillin fermentation

Table 1 Variables used to establish a penicillin concentration prediction model

序号	变量描述
$x_{1}$	搅拌功率/ $W$
$x_{2}$	底物进料速度/ $(L / h)$
$x_{3}$	溶解氧浓度/ $(g / L)$
$x_{4}$	产生热量/ $k c a l$
$x_{5}$	培养体积/ $L$
$x_{6}$	二氧化碳浓度/ $(g / L)$
$x_{7}$	pH
$x_{8}$	青霉素浓度/ $(g / L)$

Table 1 Variables used to establish a penicillin concentration prediction model

序号	变量描述
$x_{1}$	搅拌功率/ $W$
$x_{2}$	底物进料速度/ $(L / h)$
$x_{3}$	溶解氧浓度/ $(g / L)$
$x_{4}$	产生热量/ $k c a l$
$x_{5}$	培养体积/ $L$
$x_{6}$	二氧化碳浓度/ $(g / L)$
$x_{7}$	pH
$x_{8}$	青霉素浓度/ $(g / L)$

Table 2 Comparison of four modeling methods for predicting RMSE under different label rates

方法	预测均方根误差
方法	50%	35%	25%	15%	10%
PLS	0.0329	0.0365	0.0370	0.0416	0.0441
GPR	0.0182	0.0193	0.0228	0.0274	0.0317
SSEGPR	0.0190	0.0205	0.0230	0.0266	0.0305
Tri-training GPR	0.0170	0.0181	0.0203	0.0254	0.0290

Fig.3 The prediction errors under different label rates with three modeling methods for penicillin concentration

Fig.4 The tracking effect of three modeling methods on penicillin concentration under different label rates

Table 3 Comparison of three modeling methods for TP under different label rates

方法	TP
方法	50%	25%	10%
PLS	0.9942	0.9935	0.9926
GPR	0.9986	0.9954	0.9930
Tri-training GPR	0.9987	0.9980	0.9952

Fig.5 Scatter plot of prediction results of penicillin concentration using different modeling methods at a label rate of 25%

Table 4 Comparison of predicted RMSE of Tri-training GPR under different signal-to-noise ratios and different label rates

信噪比	预测均方根误差
信噪比	50%	35%	25%	15%	10%
80	0.0170	0.0181	0.0203	0.0254	0.0290
20	0.0181	0.0245	0.0262	0.0346	0.0372
10	0.0224	0.0381	0.0433	0.0500	0.0591

Fig.6 The process flow diagram of debutanizer tower

Table 5 The auxiliary variables for establishing a butane concentration prediction model

序号	变量描述
$x_{1}$	塔顶温度
$x_{2}$	塔顶压力
$x_{3}$	回流的流量
$x_{4}$	流入下一过程的流量
$x_{5}$	第6级塔板温度
$x_{6}$	塔板温度1
$x_{7}$	塔板温度2

Table 5 The auxiliary variables for establishing a butane concentration prediction model

序号	变量描述
$x_{1}$	塔顶温度
$x_{2}$	塔顶压力
$x_{3}$	回流的流量
$x_{4}$	流入下一过程的流量
$x_{5}$	第6级塔板温度
$x_{6}$	塔板温度1
$x_{7}$	塔板温度2

Table 6 Comparison of RMSE prediction for butane concentration using four models under different label rates

方法	预测均方根误差
方法	60%	50%	30%	10%
PLS	0.0916	0.0953	0.0976	0.1057
GPR	0.0721	0.0789	0.0844	0.0983
SSEGPR	0.0708	0.0773	0.0903	0.1186
Tri-training GPR	0.0713	0.0770	0.0827	0.0977

Fig.7 The predicting butane concentration error under different label rates with three modeling methods

Table 7 Comparison of three modeling methods for TP under different label rates

方法	TP
方法	60%	50%	30%	10%
PLS	0.2341	0.2040	0.1936	0.1192
GPR	0.5693	0.5353	0.4905	0.1333
Tri-training GPR	0.7348	0.7087	0.5786	0.2733

Fig.8 The tracking effect of three modeling methods on butane concentration under a label rate of 30%

References 32

1	赵杨. 污水处理过程的智能优化与控制方法研究[D]. 无锡: 江南大学, 2022.
	Zhao Y. Research on intelligent optimization and control method of wastewater treatment process[D]. Wuxi: Jiangnan University, 2022.
2	姚邹静, 赵春晖, 李元龙, 等. 面向工业软测量应用的定制化生成对抗数据填补模型[J]. 控制与决策, 2021, 36(12): 2929-2936.
	Yao Z J, Zhao C H, Li Y L, et al. Customized generative adversarial data imputation model for industrial soft sensing[J]. Control and Decision, 2021, 36(12): 2929-2936.
3	蒋昕祎, 李绍军, 金宇辉. 基于慢特征重构与改进DPLS的软测量建模[J]. 华东理工大学学报(自然科学版), 2018, 44(4): 535-542.
	Jiang X Y, Li S J, Jin Y H. Soft sensor modeling based on enhancing DPLS and slow feature reconstruction[J]. Journal of East China University of Science and Technology (Natural Science Edition), 2018, 44(4): 535-542.
4	郭明. 软测量技术的研究及应用[D]. 杭州: 浙江工业大学, 2019.
	Guo M. Research and applications of soft measurement technique[D]. Hangzhou: Zhejiang University of Technology, 2019.
5	郭润元. 数据与知识混合驱动的深度学习工业软测量方法研究[D]. 西安: 西安理工大学, 2023.
	Guo R Y. Deep learning-based industrial soft-sensing method driven by hybrid data and knowledge[D]. Xi’an: Xi’an University of Technology, 2023.
6	Shao W M, Tian X M. Adaptive soft sensor for quality prediction of chemical processes based on selective ensemble of local partial least squares models[J]. Chemical Engineering Research and Design, 2015, 95: 113-132.
7	邵伟明, 田学民, 宋执环. 基于集成学习的多产品化工过程软测量建模方法[J]. 化工学报, 2018, 69(6): 2551-2559.
	Shao W M, Tian X M, Song Z H. Ensemble learning-based soft sensor method for multi-product chemical processes[J]. CIESC Journal, 2018, 69(6): 2551-2559.
8	阎高伟, 贺敏, 汤健, 等. 基于最大均值差异多源域迁移学习的湿式球磨机负荷参数软测量[J]. 控制与决策, 2018, 33(10): 1795-1800.
	Yan G W, He M, Tang J, et al. Soft sensor of wet ball mill load based on maximum mean discrepancy multi-source domain transfer learning[J]. Control and Decision, 2018, 33(10): 1795-1800.
9	Zhang F, Li N Q, Li L H, et al. A local semi-supervised ensemble learning strategy for the data-driven soft sensor of the power prediction in wind power generation[J]. Fuel, 2023, 333: 126435.
10	Sun Q Q, Ge Z Q. Deep learning for industrial KPI prediction: when ensemble learning meets semi-supervised data[J]. IEEE Transactions on Industrial Informatics, 2021, 17(1): 260-269.
11	罗常伟, 王双双, 尹峻松, 等. 集成学习研究现状及展望[J]. 指挥与控制学报, 2023, 9(1): 502002.
	Luo C W, Wang S S, Yin J S, et al. Research status and prospect of ensemble learning[J]. Journal of Command and Control, 2023, 9(1): 502002.
12	Anh N T N, Thang T N, Solanki V K. Machine learning and ensemble methods[J]. SpringerBriefs in Applied Sciences and Technology, 2022: 9-18.
13	Ganaie M A, Hu M H, Malik A K, et al. Ensemble deep learning: a review[J]. Engineering Applications of Artificial Intelligence, 2022, 115: 105151.
14	徐继伟, 杨云. 集成学习方法: 研究综述[J]. 云南大学学报(自然科学版), 2018, 40(6): 1082-1092.
	Xu J W, Yang Y. A survey of ensemble learning approaches[J]. Journal of Yunnan University (Natural Sciences Edition), 2018, 40(6): 1082-1092.
15	张春霞, 张讲社. 选择性集成学习算法综述[J]. 计算机学报, 2011, 34(8): 1399-1410.
	Zhang C X, Zhang J S. A survey of selective ensemble learning algorithms[J]. Chinese Journal of Computers, 2011, 34(8): 1399-1410.
16	Ge Z Q, Song Z H. Subspace partial least squares model for multivariate spectroscopic calibration[J]. Chemometrics and Intelligent Laboratory Systems, 2013, 125: 51-57.
17	田慧欣, 李坤, 孟博. 一种用于软测量建模的增量学习集成算法[J]. 控制与决策, 2015, 30(8): 1523-1526.
	Tian H X, Li K, Meng B. An incremental learning ensemble algorithm for soft sensor modeling[J]. Control and Decision, 2015, 30(8): 1523-1526.
18	金怀平, 李建刚, 钱斌, 等. 基于多模态扰动的集成即时学习软测量建模[J]. 信息与控制, 2020, 49(3): 257-266.
	Jin H P, Li J G, Qian B, et al. Soft sensor development based on ensemble just-in-time learning with multimodal perturbation[J]. Information and Control, 2020, 49(3): 257-266.
19	王光, 单发顺, 钱禹丞, 等. 基于集成学习传递熵的化工过程微小故障检测方法[J]. 化工学报, 2023, 74(7): 2967-2978.
	Wang G, Shan F S, Qian Y C, et al. Incipient fault detection method for chemical process based on ensemble learning transfer entropy[J]. CIESC Journal, 2023, 74(7): 2967-2978.
20	Zhou Z H, Wu J X, Tang W. Ensembling neural networks: many could be better than all[J]. Artificial Intelligence, 2002, 137(1/2): 239-263.
21	汤健, 乔俊飞. 基于选择性集成核学习算法的固废焚烧过程二𫫇英排放浓度软测量[J]. 化工学报, 2019, 70(2): 696-706.
	Tang J, Qiao J F. Dioxin emission concentration soft measuring approach of municipal solid waste incineration based on selective ensemble kernel learning algorithm[J]. CIESC Journal, 2019, 70(2): 696-706.
22	盛晓晨. 基于多模型集成的软测量建模[D]. 无锡: 江南大学, 2021.
	Sheng X C. Multiple model ensemble based soft sensor development[D]. Wuxi: Jiangnan University, 2021.
23	孙子文, 金浩. 深度自编码网络的集成学习ICPS入侵检测模型[J]. 信息与控制, 2021, 50(5): 591-601.
	Sun Z W, Jin H. Integrated learning ICPS intrusion detection model of deep auto-encoder network[J]. Information and Control, 2021, 50(5): 591-601.
24	Chi S Q, Li X H, Tian Y, et al. Semi-supervised learning to improve generalizability of risk prediction models[J]. Journal of Biomedical Informatics, 2019, 92: 103117.
25	Zhu J L, Ge Z Q, Song Z H. Quantum statistic based semi-supervised learning approach for industrial soft sensor development[J]. Control Engineering Practice, 2018, 74: 144-152.
26	Shao W M, Ge Z Q, Song Z H, et al. Nonlinear industrial soft sensor development based on semi-supervised probabilistic mixture of extreme learning machines[J]. Control Engineering Practice, 2019, 91: 104098.
27	Blum A, Mitchell T. Combining labeled and unlabeled data with co-training[C]//Proceedings of the Eleventh Annual Conference on Computational Learning Theory. Madison, Wisconsin, USA. ACM, 1998: 92-100.
28	Bao L, Yuan X F, Ge Z Q. Co-training partial least squares model for semi-supervised soft sensor development[J]. Chemometrics and Intelligent Laboratory Systems, 2015, 147: 75-85.
29	李东, 刘乙奇, 黄道平. 基于Tri-training MPLS的半监督软测量模型[J]. 华东理工大学学报(自然科学版), 2021, 47(2): 217-224.
	Li D, Liu Y Q, Huang D P. Semi-supervised soft sensor model based on Tri-training MPLS[J]. Journal of East China University of Science and Technology, 2021, 47(2): 217-224.
30	赵帅. 基于集成学习的高斯过程回归软测量建模方法研究[D]. 无锡: 江南大学, 2018.
	Zhao S. Research of Gaussian process regression soft sensor modeling based on ensemble learning[D]. Wuxi: Jiangnan University, 2018.
31	陈雄挺, 李扬, 史琳林. 基于图分割与协同训练的工业过程半监督软测量方法[J]. 中国仪器仪表, 2023(10): 36-42.
	Chen X T, Li Y, Shi L L. Semi-supervised soft sensor modelling method based on graph segmentation and co-training[J]. China Instrumentation, 2023(10): 36-42.
32	Zhou Z H, Li M. Semisupervised regression with cotraining-style algorithms[J]. IEEE Transactions on Knowledge and Data Engineering, 2007, 19(11): 1479-1493.

[1]	Hongrui LI, Chunxi HUANG, Xiaodong HONG, Zuwei LIAO, Jingdai WANG, Yongrong YANG. An adaptive variable-step homotopy-based algorithm for process simulation with cyclic streams [J]. CIESC Journal, 2024, 75(7): 2604-2612.
[2]	Han ZHANG, Shuning ZHANG, Ke LIU, Guanlong DENG. Particle size prediction of cobalt oxalate synthesis process based on slow feature analysis and least squares support vector regression [J]. CIESC Journal, 2024, 75(6): 2313-2321.
[3]	Jianwen ZHANG, Tingsheng ZHAO, Pingyu WAN, Qianlin WANG, Zhan DOU, Bo XU. Discussion on integrated security control in process industry [J]. CIESC Journal, 2024, 75(6): 2375-2384.
[4]	Yuhang HE, Dan XIE, Yangcheng LYU. Research progress of cationic polymerization in microreactor [J]. CIESC Journal, 2024, 75(4): 1302-1316.
[5]	Xueyi MA, Keqin LIU, Jijiang HU, Zhen YAO. CFD studies on the mixing and reaction in a solution polymerization reactor for POE production [J]. CIESC Journal, 2024, 75(1): 322-337.
[6]	Zhenghao JIN, Lijie FENG, Shuhong LI. Energy and exergy analysis of a solution cross-type absorption-resorption heat pump using NH₃/H₂O as working fluid [J]. CIESC Journal, 2023, 74(S1): 53-63.
[7]	Fei KANG, Weiguang LYU, Feng JU, Zhi SUN. Research on discharge path and evaluation of spent lithium-ion batteries [J]. CIESC Journal, 2023, 74(9): 3903-3911.
[8]	Yue CAO, Chong YU, Zhi LI, Minglei YANG. Industrial data driven transition state detection with multi-mode switching of a hydrocracking unit [J]. CIESC Journal, 2023, 74(9): 3841-3854.
[9]	Guang WANG, Fashun SHAN, Yucheng QIAN, Jianfang JIAO. Incipient fault detection method for chemical process based on ensemble learning transfer entropy [J]. CIESC Journal, 2023, 74(7): 2967-2978.
[10]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[11]	Chengze WANG, Kaili GU, Jinhua ZHANG, Jianxuan SHI, Yiwei LIU, Jinxiang LI. Sulfidation couples with aging to enhance the reactivity of zerovalent iron toward Cr(Ⅵ) in water [J]. CIESC Journal, 2023, 74(5): 2197-2206.
[12]	Xiaodan SU, Ganyu ZHU, Huiquan LI, Guangming ZHENG, Ziheng MENG, Fang LI, Yunrui YANG, Benjun XI, Yu CUI. Optimization of wet process phosphoric acid hemihydrate process and crystallization of gypsum [J]. CIESC Journal, 2023, 74(4): 1805-1817.
[13]	Cheng YUN, Qianlin WANG, Feng CHEN, Xin ZHANG, Zhan DOU, Tingjun YAN. Deep-mining risk evolution path of chemical processes based on community structure [J]. CIESC Journal, 2023, 74(4): 1639-1650.
[14]	Zhongqiu ZHANG, Hongguang LI, Yilin SHI. A multi-task learning approach for complex chemical processes based on manual predictive manipulating strategies [J]. CIESC Journal, 2023, 74(3): 1195-1204.
[15]	Jianghuai ZHANG, Zhong ZHAO. Robust minimum covariance constrained control for C₃ hydrogenation process and application [J]. CIESC Journal, 2023, 74(3): 1216-1227.