基于协同训练的半监督异构自适应软测量建模方法的研究

doi:10.11949/0438-1157.20191378

摘要/Abstract

摘要：

软测量技术被广泛应用到工业过程中重要且难以在线测量变量的预测。然而，由于工业过程的复杂性,非线性和高昂的数据获取成本，使得建模所需的输入和输出变量数据比例严重不平衡。因此，本文在已有的co-training模型的基础上，将协同训练算法与前馈神经网络（BP）算法相结合，提出了针对非线性问题的co-training BP模型。然而，由于软测量模型应用过程的时变性和不确定性，以及外部环境等因素的影响，会造成数据突变、延迟和波动性大等情况，导致模型预测性能的衰减。因此，本文提出了一种半监督异构的自适应co-training RPLS-RBP模型。一方面，该模型使用奇偶分组的方法将标记数据进行两部分均分。另一方面，递归PLS（RPLS）与递归BP（RBP）同时用于标记数据的建模和预测。为了验证模型的预测性能，所提出模型在一个污水处理的仿真基准平台(BSM1)和一个实际污水厂(UCI)的数据中得到了验证。结果表明，所提模型具有较好的预测性能。

关键词: 软测量, 半监督, 协同训练, 递归PLS, 递归BP

Abstract:

Soft-sensing technology is widely applied to the prediction of important and difficult to measure variables online in industrial processes. However, due to the complexity of industrial processes, non-linearity and high costs to acquire data, the ratio of input and output variables data required for modeling is seriously unbalanced. Therefore, depending on the existing co-training model, this paper combines the co-training algorithm with the back propagation neural network (BP) algorithm to propose a co-training BP model for nonlinear problems. However, due to the time variability and uncertainty of the application process, as well as the negative influences of external environment, and so on, the data exhibit mutation, delay and high volatility, even the prediction performance of the model deteriorated. Thus, this paper proposed a semi-supervised heterogeneous adaptive co-training RPLS-RBP model. On the one hand, the model used odd-even grouping to equalize two parts of the labeled data. On the other hand, RPLS and RBP are used simultaneously for modeling and the prediction on labeled data. To demonstrate the prediction performance of the model, the proposed model is verified by a simulation benchmark platform (Benchmark Simulation Model-1) and a real sewage treatment plant (UCI database). The results show that the proposed model achieved better prediction performance.

Key words: soft-sensor, semi-supervised, co-training, recursive PLS, recursive BP

中图分类号:

TP 277

李东, 黄道平, 刘乙奇. 基于协同训练的半监督异构自适应软测量建模方法的研究[J]. 化工学报, 2020, 71(5): 2128-2138.

Dong LI, Daoping HUANG, Yiqi LIU. Research on semi-supervised heterogeneous adaptive co-training soft-sensor model[J]. CIESC Journal, 2020, 71(5): 2128-2138.

图/表 8

图1 训练过程(a)和预测过程(b)的图形说明

Fig.1 Graphical description of training process (a) and prediction process (b)

表1 异构自适应co-training RPLS-RBP混合回归模型的详细流程

Table 1 Detailed flow of heterogeneous adaptive co-training RPLS-RBP hybrid regression model

异构自适应co-training RPLS-RBP混合回归模型流程

输入: 标记样本集 $L$ (包含输入变量 $x i$ 和输出变量 $y i$ ),未标记样本集 $U$ （仅包含输入 $x u$ ），测试样本集 $P$ ,最大迭代次数 $T$

训练过程：

用奇偶分配的方法将 $L$ 均分为两部分：L₁和L₂

$L 1 = x 1, y 1, x 3, y 3 ? x L - 1, y L - 1$

$L 2 = x 2, y 2, x 4, y 4 ? x L, y L$

进行 $T$ 次迭代:

$h 1 = P L S L 1, h 2 = B P L 2$

For $? x u ∈ U$

$h 1 x u = y ? u 1, h 2 x u = y ? u 2$

$h 1' = P L S L 1, x u, y ? u 1, h 2' = B P L 2, x u, y ? u 2$

$R 1 = ∑ x i ∈ L 1 y i - h 1 x i 2 L$ ， $R 2 = ∑ x j ∈ L 2 y j - h 2 x j 2 L$

$R 1' = ∑ x i ∈ L 1 y i - h 1' x i 2 L$ ， $R 2' = ∑ x j ∈ L 2 y j - h 2' x j 2 L$

$R = R 1 - R 2$ ， $R' = R 1' - R 2'$

$? x u = R - R'$

取最大的 $? x u$ 对应的 $x u$

$y ˉ u 1 = h 1 x u, y ˉ u 2 = h 2 x u$

$L 1 = L 1 ? y ˉ u 2, L 2 = L 2 ? y ˉ u 1$

$U = U - x u$

End of for

达到最高迭代次数，结束迭代

输出新的标记样本集L₁和L₂

$x ∈ P$

$h 1 = R P L S (L 1)$ , $h 2 = R B P (L 2)$

$y p = 12 (h 1 (x) + h 2 (x))$

表1 异构自适应co-training RPLS-RBP混合回归模型的详细流程

Table 1 Detailed flow of heterogeneous adaptive co-training RPLS-RBP hybrid regression model

异构自适应co-training RPLS-RBP混合回归模型流程

输入: 标记样本集 $L$ (包含输入变量 $x i$ 和输出变量 $y i$ ),未标记样本集 $U$ （仅包含输入 $x u$ ），测试样本集 $P$ ,最大迭代次数 $T$

训练过程：

用奇偶分配的方法将 $L$ 均分为两部分：L₁和L₂

$L 1 = x 1, y 1, x 3, y 3 ? x L - 1, y L - 1$

$L 2 = x 2, y 2, x 4, y 4 ? x L, y L$

进行 $T$ 次迭代:

$h 1 = P L S L 1, h 2 = B P L 2$

For $? x u ∈ U$

$h 1 x u = y ? u 1, h 2 x u = y ? u 2$

$h 1' = P L S L 1, x u, y ? u 1, h 2' = B P L 2, x u, y ? u 2$

$R 1 = ∑ x i ∈ L 1 y i - h 1 x i 2 L$ ， $R 2 = ∑ x j ∈ L 2 y j - h 2 x j 2 L$

$R 1' = ∑ x i ∈ L 1 y i - h 1' x i 2 L$ ， $R 2' = ∑ x j ∈ L 2 y j - h 2' x j 2 L$

$R = R 1 - R 2$ ， $R' = R 1' - R 2'$

$? x u = R - R'$

取最大的 $? x u$ 对应的 $x u$

$y ˉ u 1 = h 1 x u, y ˉ u 2 = h 2 x u$

$L 1 = L 1 ? y ˉ u 2, L 2 = L 2 ? y ˉ u 1$

$U = U - x u$

End of for

达到最高迭代次数，结束迭代

输出新的标记样本集L₁和L₂

$x ∈ P$

$h 1 = R P L S (L 1)$ , $h 2 = R B P (L 2)$

$y p = 12 (h 1 (x) + h 2 (x))$

表2 不同的标记样本率下的RMSE值

Table 2 RMSE values at different labeled data rate

Labeled data rate	co-training PLS					co-training BP					co-training RRPLS-RBP
Labeled data rate	SS	SNH	SNO	COD	BOD₅	SS	SNH	SNO	COD	BOD₅	SS	SNH	SNO	COD	BOD₅
10%	0.095	0.053	0.054	0.064	0.053	0.304	0.07	0.027	0.089	0.029	0.007	0.004	0.003	0.004	0.004
20%	3.833	12.994	8.345	2.456	1.922	16.845	10.073	8.328	3.293	1.615	0.974	0.661	0.638	0.69	0.587
30%	2.066	1.527	0.892	0.801	0.751	7.227	2.457	2.4	0.741	0.421	0.311	0.174	0.174	0.18	0.182
40%	16.864	10.756	12.205	7.173	6.059	54.485	11.066	2.182	1.432	6.136	0.263	0.293	0.252	0.32	0.238
50%	0.043	0.088	0.04	0.037	0.044	0.207	0.055	0.037	0.037	0.052	0.003	0.005	0.002	0.005	0.003

图2 输出变量的预测曲线与实际曲线的对比图（标记样本率为50%）

Fig.2 Prediction profiles of output variables compared with real values(labeled data rate is 50%)

表3 输出变量的RMSE、R值和耗时（标记样本率为50%）

Table 3 RMSE, R values and time consuming of the output variables (labeled data rate is 50%)

预测变量	co-training PLS		co-training BP		co-training RPLS-RBP
预测变量	RMSE	R	RMSE	R	RMSE	R
SS	0.053	0.814	0.029	0.829	0.004	0.941
SNH	1.922	0.922	1.615	0.921	0.587	0.968
SNO	0.751	0.917	0.421	0.935	0.182	0.973
COD	6.059	0.947	6.136	0.753	0.238	0.979
BOD₅	0.044	0.987	0.052	0.966	0.003	0.991
耗时/s	34.368		58.994		182.318

图3 不同标记样本率下RMSE值的变化曲线

Fig.3 Change curves of RMSE value under different labeled data rate

图4 输出变量的预测曲线图（标记样本率为50%）

Fig.4 Prediction curve of output variables (labeled data rate is 50%)

图5 输出变量的RMSE和R的条形图（标记样本率为50%）

Fig.5 Bar graphs of RMSE and R values (labeled data rate is 50%)

参考文献 35

1	黄道平, 刘乙奇, 李艳, 等. 软测量在污水处理过程中的研究与应用[J]. 化工学报, 2011, 62(1): 1-9.
	Huang D P, Liu Y Q, Li Y, et al. Soft sensor research and its application in wastewater treatment[J]. CIESC Journal, 2011, 62(1): 1-9.
2	曹鹏飞, 罗雄麟. 化工过程软测量建模方法研究进展[J].化工学报, 2013, 64(3): 788-800.
	Cao P F, Luo X L. Modeling of soft sensor for chemical process[J]. CIESC Journal, 2013, 64(3): 788-800.
3	汤健, 乔俊飞, 柴天佑, 等. 基于虚拟样本生成技术的多组分机械信号建模[J]. 自动化学报, 2018, 44(9): 35-55.
	Tang J, Qiao J F, Chai T Y, et al. Modeling multiple components mechanical signals by means of virtual sample generation technique[J]. Acta Automatica Sinica, 2018, 44(9): 35-55.
4	Wang D, Liu J, Srinivasan R. Data-driven soft sensor approach for quality prediction in a refining process[J]. IEEE Transactions on Industrial Informatics, 2010, 6(1): 11-17.
5	Chao S, Fan Y, Huang D, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of Process Control, 2014, 24(3): 223–233.
6	谢代梁, 王保良, 黄志尧, 等. 主成分回归在中药过程软测量中的应用研究[J]. 仪器仪表学报, 2004, 25(z3): 671-672.
	Xie D Y, Wang B L, Huang Z Y, et al. Application of principle component regression to soft-sensing of Chinese traditional medicine production process[J]. Chinese Journal of Scientific Instrument, 2004, 25(z3): 671-672.
7	刘瑞兰, 徐艳, 戎舟. 基于稀疏最小二乘支持向量机的软测量建模[J]. 化工学报, 2015, 66(4): 1402-1406.
	Liu R L, Xu Y, Rong Z. Modeling soft sensor based on sparse least square support vector machine[J]. CIESC Journal, 2015, 66(4): 1402-1406.
8	Liu Y, Xiao H, Pan Y, et al. Development of multiple-step soft-sensors using a Gaussian process model with application for fault prognosis[J]. Chemometrics & Intelligent Laboratory Systems, 2016, 157: 85-95.
9	马建, 邓晓刚, 王磊. 基于深度集成支持向量机的工业过程软测量方法[J]. 化工学报, 2018, 69(3): 1121-1128.
	Ma J, Deng X G, Wang L. Industrial process soft sensor method based on deep learning ensemble support vector machine[J]. CIESC Journal, 2018, 69(3): 1121-1128.
10	Liu Y Q, Liu B, Zhao X J, et al. Development of RVM-based multiple-output soft sensors with serial and parallel stacking strategies[J]. IEEE Transactions on Control Systems Technology2018, 27(6): 2727 - 2734.
11	Yao L, Ge Z Q. Deep learning of semisupervised process data with hierarchical extreme learning machine and soft sensor application[J]. IEEE Transactions on Industrial Electronics. 2017, 65(2): 1490-1498.
12	Yuan X F, Ge Z Q, Huang B, et al. Semisupervised JITL framework for nonlinear industrial soft sensing based on locally semisupervised weighted PCR[J]. IEEE Transactions on Industrial Informatics. 2016, 13(2): 532-541.
13	Zhong W M, Jiang C, Peng X, et al. Online quality prediction of industrial terephthalic acid hydropurification process using modified regularized slow-feature analysis[J]. Industrial & Engineering Chemistry Research, 2018, 57(29): 9604-9614.
14	Tu E, Zhang Y, Zhu L, et al. A graph-based semi-supervised k nearest-neighbor method for nonlinear manifold distributed data classification[J]. Information Sciences, 2016, 367/368: 673-688.
15	Li C, Zhu J, Bo Z. Max-margin deep generative models for (semi-)supervised learning[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2018, 40(11): 2672-2775.
16	Shi Y, Yao K, Hu C, et al. Semi-supervised slot tagging in spoken language understanding using recurrent transductive support vector machines[C]//2015 IEEE Workshop on Automatic Speech Recognition & Understanding. Scottsdale, AZ, USA, 2016.
17	Tanha J, Someren M V, Afsarmanesh H. Semi-supervised self-training for decision tree classifiers[J]. International Journal of Machine Learning & Cybernetics, 2017, 8(1): 355-370.
18	Zhan Y Z, Chen Y B. Co-training semi-supervised active learning algorithm with noise filter[J]. Pattern Recognition & Artificial Intelligence, 2009, 22(5): 750-755.
19	Zhou Z H, Li M. Semi-supervised regression with co-training[C]//Proceeding of the 19th International Joint Conference on Artificial Intelligence. Morgan Kaufmann Publishers Inc., 2005.
20	Bao L, Yuan Y F, Ge Z Q. Co-training partial least squares model for semi-supervised soft sensor development[J]. Chemometrics & Intelligent Laboratory Systems, 2015, 147: 75-85.
21	Goldman S, Zhou Y. Enhancing supervised learning with unlabeled data[C]// Proceedings of the Seventeenth International Conference on Machine Learning. 2000.
22	Nigam K, Ghani R. Analyzing the effectiveness and applicability of co-training[C]//International Conference on Information & Knowledge Management. 2000.
23	Zhou Z H, Li M. Tri-training: exploiting unlabeled data using three classifiers[J]. IEEE Transactions on Knowledge & Data Engineering, 2005, 17(11): 1529-1541.
24	徐欧官, 陈祥华, 傅永峰, 等. 基于模型性能评估的递推PLS建模及应用[J]. 化工学报, 2014, 65(12): 4875-4882.
	Xu O G, Chen X H, Fu Y F, et al. Recursive PLS modeling based on model performance assessment and its application[J]. CIESC Journal, 2014, 65(12): 4875-4882.
25	Yang J X, Zhou J T. Prediction of chaotic time series of bridge monitoring system based on multi-step recursive BP neural network[J]. Advanced Materials Research, 2011, 159: 138-143.
26	周云龙, 孙斌, 陆军, 等. 改进BP神经网络在气液两相流流型识别中的应用[J]. 化工学报, 2005, 56(1): 110-115.
	Zhou Y L, Sun B, Lu J, et al. Application of improved BP neural network in identification of air-water two-phase flow patterns[J]. Journal of Chemical Industry and Engineering(China), 2005, 56(1): 110-115.
27	Zhu F G, Lu Y Z. A new recursive learning algorithm of Bp. network[C]// Singapore International Conference on Intelligent Control and Instrumentation. Singapore, 1992.
28	Xu J, He H, Man H. DCPE co-training for classification[J]. Neurocomputing, 2012, 86(4): 75-85.
29	He H T, Luo X N, Ma F T, et al. Network traffic classification based on ensemble learning and co-training[J]. Science in China(Series F), 2009, 52(2): 338-346.
30	Li X M, Lu H L, Yang J H, et al. Semi-supervised LIBS quantitative analysis method based on co-training regression model with selection of effective unlabeled samples[J]. Plasma Science and Technology, 2018, 21(3): 034015.
31	Liu Y Q, Liu B, Zhao X J, et al. A mixture of variational canonical correlation analysis for nonlinear and quality-relevant process monitoring[J]. IEEE Transactions on Industrial Electronics, 2018, 65(8): 6478-6486.
32	Wang G, Wu J, Yin S, et al. Comparison between BP neural network and multiple linear regression method[C]//Proceeding of the First International Conference on Information Computing and Applications. 2010.
33	Liu Y, Pan Y, Huang D. Development of a novel adaptive soft-sensor using variational Bayesian PLS with accounting for online identification of key variables[J]. Industrial & Engineering Chemistry Research, 2015, 54(1): 338-350.
34	吴菁, 刘乙奇, 刘坚, 等. 基于动态多核相关向量机的软测量建模研究[J]. 化工学报, 2019, 70(4): 237-249.
	Wu J, Liu Y Q, Liu J, et al. Study on the soft sensor of multi-kernel relevance vector machine based on time difference[J]. CIESC Journal, 2019, 70(4): 237-249.
35	Liu Y, Huang D, Li Y. Development of interval soft sensors using enhanced just-in-time learning and inductive confidence predictor[J]. Industrial & Engineering Chemistry Research, 2012, 51(8): 3356-3367.

[1]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[2]	邵伟明, 韩文学, 宋伟, 杨勇, 陈灿, 赵东亚. 基于分布式贝叶斯隐马尔可夫回归的动态软测量建模方法[J]. 化工学报, 2023, 74(6): 2495-2502.
[3]	罗顺桦, 王振雷, 王昕. 基于二子空间协同训练算法的半监督软测量建模[J]. 化工学报, 2022, 73(3): 1270-1279.
[4]	刘聪, 谢莉, 杨慧中. 基于改进DPC的青霉素发酵过程多模型软测量建模[J]. 化工学报, 2021, 72(3): 1606-1615.
[5]	陈忠圣, 朱梅玉, 贺彦林, 徐圆, 朱群雄. 基于分位数回归CGAN的虚拟样本生成方法及其过程建模应用[J]. 化工学报, 2021, 72(3): 1529-1538.
[6]	杜宇浩, 阎高伟, 李荣, 王芳. 基于局部线性嵌入的测地线流式核多工况软测量建模方法[J]. 化工学报, 2020, 71(3): 1278-1287.
[7]	杨逸俊,王振雷,王昕. 基于最近邻与神经网络融合模型的软测量建模方法[J]. 化工学报, 2020, 71(12): 5696-5705.
[8]	代学志,熊伟丽. 基于核极限学习机的快速主动学习方法及其软测量应用[J]. 化工学报, 2020, 71(11): 5226-5236.
[9]	侯延彬,高宪文,李翔宇. 采油过程多尺度状态特征生成的有杆泵动态液面预测[J]. 化工学报, 2019, 70(S2): 311-321.
[10]	廉小亲, 王俐伟, 安飒, 魏伟, 刘载文. 基于SOM-RBF神经网络的COD软测量方法[J]. 化工学报, 2019, 70(9): 3465-3472.
[11]	秦美华, 朱红求, 李勇刚, 陈俊名, 张凤雪, 李文婷. 基于STA-K均值聚类的电化学废水处理过程离子浓度软测量[J]. 化工学报, 2019, 70(9): 3458-3464.
[12]	吴菁, 刘乙奇, 刘坚, 黄道平, 邱禹, 于广平. 基于动态多核相关向量机的软测量建模研究[J]. 化工学报, 2019, 70(4): 1472-1484.
[13]	吉文鹏, 杨慧中. 基于改进扩张搜索聚类算法的多流形软测量建模[J]. 化工学报, 2019, 70(2): 723-729.
[14]	朱宝, 乔俊飞. 基于自编码神经网络特征提取的回声状态网络研究及过程建模应用[J]. 化工学报, 2019, 70(12): 4770-4776.
[15]	赵荣荣, 赵忠盖, 刘飞. 基于k-近邻互信息的发酵过程高斯过程回归建模[J]. 化工学报, 2019, 70(12): 4741-4748.