A fast active learning method based on kernel extreme learning machine and its application for soft sensing

doi:10.11949/0438-1157.20200260

Abstract

Abstract:

To improve the efficiency of active learning and reduce the cost of manual marking, a fast active learning method based on kernel extreme learning machine is proposed and applied to soft sensor research. Firstly, the information of unlabeled samples is evaluated by kernel extreme learning machine, and the confidence of unlabeled samples is taken as the evaluation criterion of sample selection. The most valuable unlabeled samples to improve the performance of the model are selected for labeling. Secondly, considering the operation information of each iteration process, the matrix inversion formula is introduced to optimize the sample selection strategy to improve the efficiency of sample evaluation. Finally, the matrix similarity theory is applied to measure the information of the labeled sample data in the iterative process, and it is used as the basis for the termination of the iterative process to improve the performance of the model with the minimum cost of labeling. The proposed method is applied to the study of H₂S and SO₂ concentration soft sensor in the sulfur recovery process. The simulation results show that the proposed method not only has low marking cost, but also improves the speed of iteration and the performance of active learning algorithm. By carrying out this research work, a new method is provided for the application of soft-sensing technology under the condition of less labeled samples.

Key words: active learning, process control, optimization, kernel extreme learning machine, soft sensor, chemical processes

摘要：

为提高主动学习方法的运行效率和降低人工标记成本，提出一种基于核极限学习机的快速主动学习方法，并将其应用于软测量建模中。首先，采用核极限学习机对无标记样本进行信息评估，将无标记样本的置信度作为样本选择评价准则，选择对改善模型性能最有价值的无标记样本进行标记；其次，充分考虑每次迭代过程的运算信息，引入矩阵反演公式优化样本选择策略，提升迭代过程样本评估的运行效率；最后，应用矩阵相似度理论对迭代过程的已标记样本数据进行信息度量，并将其作为迭代终止依据，以最小的标记代价提升模型性能。将所提方法应用于硫回收过程H₂S和SO₂浓度软测量研究中，仿真结果表明：所提方法不仅标记代价小，而且提高了迭代的快速性，比较全面地提升了主动学习算法的性能。通过开展本研究工作，为少标记样本情况下的软测量技术应用提供了一种新方法。

关键词: 主动学习, 过程控制, 优化, 核极限学习机, 软测量, 化学过程

CLC Number:

TP 274

Xuezhi DAI,Weili XIONG. A fast active learning method based on kernel extreme learning machine and its application for soft sensing[J]. CIESC Journal, 2020, 71(11): 5226-5236.

代学志,熊伟丽. 基于核极限学习机的快速主动学习方法及其软测量应用[J]. 化工学报, 2020, 71(11): 5226-5236.

Figures/Tables 12

References 31

1	汤健, 柴天佑, 刘卓, 等. 基于更新样本智能识别算法的自适应集成建模[J]. 自动化学报, 2016, 42(7): 1040-1052.
	Tang J, Chai T Y, Liu Z, et al. Adaptive ensemble modelling approach based on updating sample intelligent identification[J]. Acta Automatica Sinica, 2016, 42(7): 1040-1052.
2	Zheng J H, Song Z H. Semi-supervised learning for probabilistic partial least squares regression model and soft sensor application[J]. Journal of Process Control, 2018, 64: 123-131.
3	Li H, You S, Zhang H, et al. Analyzing the impact of heating emissions on air quality index based on principal component regression[J]. Journal of Cleaner Production, 2018, 171: 1577-1592.
4	Popli K, Maries V, Afacan A, et al. Development of a vision-based online soft sensor for oil sands flotation using support vector regression and its application in the dynamic monitoring of bitumen extraction[J]. Canadian Journal of Chemical Engineering, 2018, 96(7): 1532-1540.
5	Zhang Z, Wang T, Liu X. Melt index prediction by aggregated RBF neural networks trained with chaotic theory[J]. Neurocomputing, 2014, 131(5): 368-376.
6	Ranjan R, Huang B, Fatehi A. Robust gaussian process modeling using EM algorithm[J]. Journal of Process Control, 2016, 42: 125-136.
7	张晓晗, 汪平江, 顾祥柏,等. 基于主元提取的鲁棒极限学习机研究及其化工建模应用[J]. 化工学报, 2019, 70(2): 55-60.
	Zhang X H, Wang P J, Gu X B, et al. Research on principal components extraction based robust extreme learning machine(PCE-RELM) and its application to modeling chemical processes[J]. CIESC Journal, 2019, 70(2): 55-60.
8	王宇红, 狄克松, 张姗, 等. 基于DBN-ELM的聚丙烯熔融指数的软测量[J]. 化工学报, 2016, 67(12): 5163-5168.
	Wang Y H, Di K S, Zhang S, et al. Melt index prediction of polypropylene based on DBN-ELM[J]. CIESC Journal, 2016, 67(12): 5163-5168.
9	赵立杰, 王海龙, 陈斌. 基于流形正则化半监督学习的污水处理操作工况识别方法[J]. 化工学报, 2016, 67(6): 2462-2468.
	Zhao L J, Wang H L, Chen B. Identification of wastewater operational conditions based on manifold regularization semi-supervised learning[J]. CIESC Journal, 2016, 67(6): 2462-2468.
10	Cohn D A, Ghahramani Z, Jordan M I. Active learning with statistical models[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 705-712.
11	Chiu S C, Jin Z, Gu Y. Active learning combining uncertainty and diversity for multi-class image classification[J]. IET Computer Vision, 2015, 9(3): 400-407.
12	Yin L, Wang H, Fan W, et al. Active learning based support vector data description method for robust novelty detection[J] Knowledge-Based Systems, 2018, 153: 40-52.
13	李洋, 方滨兴, 郭莉, 等. 基于主动学习和TCM-KNN方法的有指导入侵检测技术[J]. 计算机学报, 2016, 30(8): 1464-1473.
	Li Y, Fang B X, Guo L, et al. Intrusion detection based on active learning and TCM-KNN algorithm[J]. Chinese Journal of Computers, 2016, 30(8): 1464-1473.
14	Shi X D, Xiong W L. Approximate linear dependence criteria with active learning for smart soft sensor design[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 180: 88-95.
15	Ge Z Q. Active probabilistic sample selection for intelligent soft sensing of industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2016, 151: 181-189.
16	Cohn D A. Neural network exploration using optimal experiment design[J]. Neural Networks, 1996, 9(6):1071-1083.
17	Zhang L, Chen C, Bu J, et al. Active learning based on locally linear reconstruction[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(10): 2026-2038.
18	Appice A, Loglisci C, Malerba D. Active learning via collective inference in network regression problems[J]. Information Sciences, 2018, 460–461: 293-317.
19	Demir B, Bruzzone L. A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5): 2323-2334.
20	Norkus M, Fay D, Murphy M J, et al. On the application of active learning and gaussian processes in postcryopreservation cell membrane integrity experiments[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2012, 9(3): 846-856.
21	Demir B, Bruzzone L. A multiple criteria active learning method for support vector regression[J]. Pattern Recognition, 2014, 47(7): 2558-2567.
22	Tang Q, Li D, Xi Y. A new active learning strategy for soft sensor modeling based on feature reconstruction and uncertainty evaluation[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 172: 43-51.
23	Huang G B, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems Man & Cybernetics Part B, 2012, 42(2): 513-529.
24	杨锡运, 关文渊, 刘玉奇, 等. 基于粒子群优化的核极限学习机模型的风电功率区间预测方法[J]. 中国电机工程学报, 2015, 35: 146-153.
	Yang X Y, Guan W Y, Liu Y Q, et al. Prediction intervals forecasts of wind power based on PSO-KELM[J]. Proceedings of the CSEE, 2015, 35: 146-153.
25	张雷, 张小刚, 陈华. 基于Gath-Geva算法和核极限学习机的多阶段间歇过程软测量[J]. 化工学报, 2018, 69(6): 2576-2585.
	Zhang L, Zhang X G, Chen H. Soft sensors for multi-stage batch processes based on Gath-Geva algorithm and kernel extreme learning machine[J]. CIESC Journal, 2018, 69(6): 2576-2585.
26	Qian K, Zhang Z, Baird A, et al. Active learning for bird sound classification via a kernel-based extreme learning machine[J]. The Journal of the Acoustical Society of America, 2017, 142(4): 1796-1804.
27	Leng Q, Qi H, Miao J, et al. One-class classification with extreme learning machine[J]. Mathematical Problems in Engineering, 2015, 4: 1–11.
28	王培良, 叶晓丰, 杨泽宇. 基于Block-RPLS模型自适应更新的质量预测方法[J]. 控制与决策, 2018, 33(3): 455-462.
	Wang P L, Ye X F, Yang Y Z. Quality prediction method based on adaptive updating of Block-RPLS model[J]. Control and Decision, 2018, 33(3): 455-462.
29	Morey A, Pradhan S, Kumar R A, et al. Pollutant monitoring in tail gas of sulfur recovery unit with statistical and soft computing models[J]. Chemical Engineering Communications, 2018, 206(1): 69-85.
30	Douak F, Melgani F, Alajlanc N, et al. Active learning for spectroscopic data regression[J]. Journal of Chemometrics, 2012, 26(7): 374-383
31	Douak F, Melgani F, Benoudjit N. Kernel ridge regression with active learning for wind speed prediction[J]. Applied Energy, 2013, 103: 328-340.

变量	描述	单位
u₁	MEA_GAS气体流	m³/s
u₂	初级空气流	m³/s
u₃	二级空气流	m³/s
u₄	SWS区域气体流	m³/s
u₅	SWS区域气体流	m³/s
y₁	H₂S浓度	mol/m³
y₂	SO₂浓度	mol/m³

变量	描述	单位
u₁	MEA_GAS气体流	m³/s
u₂	初级空气流	m³/s
u₃	二级空气流	m³/s
u₄	SWS区域气体流	m³/s
u₅	SWS区域气体流	m³/s
y₁	H₂S浓度	mol/m³
y₂	SO₂浓度	mol/m³

迭代次数	性能指标	Random	RSAL	DAL	Proposed
10	平均RMSE	0.0633	0.0415	0.0456	0.0362
	标准差/%	1.29	0.628	0.756	0.169
	运行时间/s	—	2.63	1.98	0.58
20	平均RMSE	0.0572	0.0320	0.0291	0.0282
	标准差/%	0.843	0.133	0.143	0.0920
	运行时间/s	—	10.50	6.77	1.41
30	平均RMSE	0.0472	0.0294	0.0274	0.0263
	标准差/%	0.984	0.123	0.0852	0.0311
	运行时间/s	—	28.68	13.39	3.40
40	平均RMSE	0.0418	0.0288	0.0264	0.0262
	标准差/%	1.20	0.144	0.0040	0.0229
	运行时间/s	—	45.53	21.28	6.02

迭代次数	性能指标	Random	RSAL	DAL	Proposed
10	平均RMSE	0.0633	0.0415	0.0456	0.0362
	标准差/%	1.29	0.628	0.756	0.169
	运行时间/s	—	2.63	1.98	0.58
20	平均RMSE	0.0572	0.0320	0.0291	0.0282
	标准差/%	0.843	0.133	0.143	0.0920
	运行时间/s	—	10.50	6.77	1.41
30	平均RMSE	0.0472	0.0294	0.0274	0.0263
	标准差/%	0.984	0.123	0.0852	0.0311
	运行时间/s	—	28.68	13.39	3.40
40	平均RMSE	0.0418	0.0288	0.0264	0.0262
	标准差/%	1.20	0.144	0.0040	0.0229
	运行时间/s	—	45.53	21.28	6.02

迭代次数	性能指标	Random	RSAL	DAL	Proposed
10	平均RMSE	0.0630	0.0562	0.0540	0.0488
	标准差/%	0.690	0.348	0.517	0.236
	运行时间/s	—	2.56	2.08	0.650
20	平均RMSE	0.0572	0.0468	0.0416	0.0416
	标准差/%	0.668	0.440	0.123	0.169
	运行时间/s	—	10.21	7.11	1.95
30	平均RMSE	0.0504	0.0402	0.0381	0.0374
	标准差/%	0.537	0.0788	0.0512	0.0541
	运行时间/s	—	25.32	13.19	3.15
40	平均RMSE	0.0436	0.0381	0.0365	0.0363
	标准差/%	0.269	0.136	0.0444	0.0399
	运行时间/s	—	46.67	21.38	5.82