基于核极限学习机的快速主动学习方法及其软测量应用

doi:10.11949/0438-1157.20200260

摘要/Abstract

摘要：

为提高主动学习方法的运行效率和降低人工标记成本，提出一种基于核极限学习机的快速主动学习方法，并将其应用于软测量建模中。首先，采用核极限学习机对无标记样本进行信息评估，将无标记样本的置信度作为样本选择评价准则，选择对改善模型性能最有价值的无标记样本进行标记；其次，充分考虑每次迭代过程的运算信息，引入矩阵反演公式优化样本选择策略，提升迭代过程样本评估的运行效率；最后，应用矩阵相似度理论对迭代过程的已标记样本数据进行信息度量，并将其作为迭代终止依据，以最小的标记代价提升模型性能。将所提方法应用于硫回收过程H₂S和SO₂浓度软测量研究中，仿真结果表明：所提方法不仅标记代价小，而且提高了迭代的快速性，比较全面地提升了主动学习算法的性能。通过开展本研究工作，为少标记样本情况下的软测量技术应用提供了一种新方法。

关键词: 主动学习, 过程控制, 优化, 核极限学习机, 软测量, 化学过程

Abstract:

To improve the efficiency of active learning and reduce the cost of manual marking, a fast active learning method based on kernel extreme learning machine is proposed and applied to soft sensor research. Firstly, the information of unlabeled samples is evaluated by kernel extreme learning machine, and the confidence of unlabeled samples is taken as the evaluation criterion of sample selection. The most valuable unlabeled samples to improve the performance of the model are selected for labeling. Secondly, considering the operation information of each iteration process, the matrix inversion formula is introduced to optimize the sample selection strategy to improve the efficiency of sample evaluation. Finally, the matrix similarity theory is applied to measure the information of the labeled sample data in the iterative process, and it is used as the basis for the termination of the iterative process to improve the performance of the model with the minimum cost of labeling. The proposed method is applied to the study of H₂S and SO₂ concentration soft sensor in the sulfur recovery process. The simulation results show that the proposed method not only has low marking cost, but also improves the speed of iteration and the performance of active learning algorithm. By carrying out this research work, a new method is provided for the application of soft-sensing technology under the condition of less labeled samples.

Key words: active learning, process control, optimization, kernel extreme learning machine, soft sensor, chemical processes

中图分类号:

TP 274

代学志,熊伟丽. 基于核极限学习机的快速主动学习方法及其软测量应用[J]. 化工学报, 2020, 71(11): 5226-5236.

Xuezhi DAI,Weili XIONG. A fast active learning method based on kernel extreme learning machine and its application for soft sensing[J]. CIESC Journal, 2020, 71(11): 5226-5236.

图/表 12

图1 基于KELM的快速主动学习算法流程

Fig.1 The flow of quick active learning algorithm based on KELM

图2 已标记样本和无标记样本的置信度关系图（数值仿真实验）

Fig.2 Confidence relationship between labeled samples and unlabeled samples(Numerical simulation experiment)

图3 SRU的简化工艺流程

Fig.3 Simplified process flow of SRU

表1 SRU的过程变量描述

Table 1 Process variable description of SRU

变量	描述	单位
u₁	MEA_GAS气体流	m³/s
u₂	初级空气流	m³/s
u₃	二级空气流	m³/s
u₄	SWS区域气体流	m³/s
u₅	SWS区域气体流	m³/s
y₁	H₂S浓度	mol/m³
y₂	SO₂浓度	mol/m³

图4 已标记样本和无标记样本的置信度关系图（硫回收仿真实验）

Fig.4 Confidence relationship between labeled samples and unlabeled samples(sulfur recovery simulation experiment)

图5 在不同学习步长下模型性能随迭代次数的变化

Fig.5 Change of model performance with the number of iterations under different learning steps

图6 在不同学习步长下模型性能随标记样本个数的变化

Fig.6 Change of model performance with number of labeled samples under different learning steps

图7 无标记样本的置信度随迭代次数的变化

Fig.7 Change of confidence of unlabeled samples with the number of iterations

图8 不同终止阈值对模型性能的影响

Fig.8 Effect of different termination thresholds on model performance

图9 平均RMSE随迭代次数的变化

Fig.9 The change of mean RMSE with the number of iterations

表2 四种主动学习方法在迭代过程中对H2S浓度预测的性能指标

Table 2 Performance index of four active learning methods for H2S concentration prediction in iterative process

迭代次数	性能指标	Random	RSAL	DAL	Proposed
10	平均RMSE	0.0633	0.0415	0.0456	0.0362
	标准差/%	1.29	0.628	0.756	0.169
	运行时间/s	—	2.63	1.98	0.58
20	平均RMSE	0.0572	0.0320	0.0291	0.0282
	标准差/%	0.843	0.133	0.143	0.0920
	运行时间/s	—	10.50	6.77	1.41
30	平均RMSE	0.0472	0.0294	0.0274	0.0263
	标准差/%	0.984	0.123	0.0852	0.0311
	运行时间/s	—	28.68	13.39	3.40
40	平均RMSE	0.0418	0.0288	0.0264	0.0262
	标准差/%	1.20	0.144	0.0040	0.0229
	运行时间/s	—	45.53	21.28	6.02

表3 四种主动学习方法在迭代过程中对SO2浓度预测的性能指标

Table 3 Performance index of four active learning methods for SO2 concentration prediction in iterative process

迭代次数	性能指标	Random	RSAL	DAL	Proposed
10	平均RMSE	0.0630	0.0562	0.0540	0.0488
	标准差/%	0.690	0.348	0.517	0.236
	运行时间/s	—	2.56	2.08	0.650
20	平均RMSE	0.0572	0.0468	0.0416	0.0416
	标准差/%	0.668	0.440	0.123	0.169
	运行时间/s	—	10.21	7.11	1.95
30	平均RMSE	0.0504	0.0402	0.0381	0.0374
	标准差/%	0.537	0.0788	0.0512	0.0541
	运行时间/s	—	25.32	13.19	3.15
40	平均RMSE	0.0436	0.0381	0.0365	0.0363
	标准差/%	0.269	0.136	0.0444	0.0399
	运行时间/s	—	46.67	21.38	5.82

参考文献 31

1	汤健, 柴天佑, 刘卓, 等. 基于更新样本智能识别算法的自适应集成建模[J]. 自动化学报, 2016, 42(7): 1040-1052.
	Tang J, Chai T Y, Liu Z, et al. Adaptive ensemble modelling approach based on updating sample intelligent identification[J]. Acta Automatica Sinica, 2016, 42(7): 1040-1052.
2	Zheng J H, Song Z H. Semi-supervised learning for probabilistic partial least squares regression model and soft sensor application[J]. Journal of Process Control, 2018, 64: 123-131.
3	Li H, You S, Zhang H, et al. Analyzing the impact of heating emissions on air quality index based on principal component regression[J]. Journal of Cleaner Production, 2018, 171: 1577-1592.
4	Popli K, Maries V, Afacan A, et al. Development of a vision-based online soft sensor for oil sands flotation using support vector regression and its application in the dynamic monitoring of bitumen extraction[J]. Canadian Journal of Chemical Engineering, 2018, 96(7): 1532-1540.
5	Zhang Z, Wang T, Liu X. Melt index prediction by aggregated RBF neural networks trained with chaotic theory[J]. Neurocomputing, 2014, 131(5): 368-376.
6	Ranjan R, Huang B, Fatehi A. Robust gaussian process modeling using EM algorithm[J]. Journal of Process Control, 2016, 42: 125-136.
7	张晓晗, 汪平江, 顾祥柏,等. 基于主元提取的鲁棒极限学习机研究及其化工建模应用[J]. 化工学报, 2019, 70(2): 55-60.
	Zhang X H, Wang P J, Gu X B, et al. Research on principal components extraction based robust extreme learning machine(PCE-RELM) and its application to modeling chemical processes[J]. CIESC Journal, 2019, 70(2): 55-60.
8	王宇红, 狄克松, 张姗, 等. 基于DBN-ELM的聚丙烯熔融指数的软测量[J]. 化工学报, 2016, 67(12): 5163-5168.
	Wang Y H, Di K S, Zhang S, et al. Melt index prediction of polypropylene based on DBN-ELM[J]. CIESC Journal, 2016, 67(12): 5163-5168.
9	赵立杰, 王海龙, 陈斌. 基于流形正则化半监督学习的污水处理操作工况识别方法[J]. 化工学报, 2016, 67(6): 2462-2468.
	Zhao L J, Wang H L, Chen B. Identification of wastewater operational conditions based on manifold regularization semi-supervised learning[J]. CIESC Journal, 2016, 67(6): 2462-2468.
10	Cohn D A, Ghahramani Z, Jordan M I. Active learning with statistical models[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 705-712.
11	Chiu S C, Jin Z, Gu Y. Active learning combining uncertainty and diversity for multi-class image classification[J]. IET Computer Vision, 2015, 9(3): 400-407.
12	Yin L, Wang H, Fan W, et al. Active learning based support vector data description method for robust novelty detection[J] Knowledge-Based Systems, 2018, 153: 40-52.
13	李洋, 方滨兴, 郭莉, 等. 基于主动学习和TCM-KNN方法的有指导入侵检测技术[J]. 计算机学报, 2016, 30(8): 1464-1473.
	Li Y, Fang B X, Guo L, et al. Intrusion detection based on active learning and TCM-KNN algorithm[J]. Chinese Journal of Computers, 2016, 30(8): 1464-1473.
14	Shi X D, Xiong W L. Approximate linear dependence criteria with active learning for smart soft sensor design[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 180: 88-95.
15	Ge Z Q. Active probabilistic sample selection for intelligent soft sensing of industrial processes[J]. Chemometrics and Intelligent Laboratory Systems, 2016, 151: 181-189.
16	Cohn D A. Neural network exploration using optimal experiment design[J]. Neural Networks, 1996, 9(6):1071-1083.
17	Zhang L, Chen C, Bu J, et al. Active learning based on locally linear reconstruction[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(10): 2026-2038.
18	Appice A, Loglisci C, Malerba D. Active learning via collective inference in network regression problems[J]. Information Sciences, 2018, 460–461: 293-317.
19	Demir B, Bruzzone L. A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5): 2323-2334.
20	Norkus M, Fay D, Murphy M J, et al. On the application of active learning and gaussian processes in postcryopreservation cell membrane integrity experiments[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2012, 9(3): 846-856.
21	Demir B, Bruzzone L. A multiple criteria active learning method for support vector regression[J]. Pattern Recognition, 2014, 47(7): 2558-2567.
22	Tang Q, Li D, Xi Y. A new active learning strategy for soft sensor modeling based on feature reconstruction and uncertainty evaluation[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 172: 43-51.
23	Huang G B, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems Man & Cybernetics Part B, 2012, 42(2): 513-529.
24	杨锡运, 关文渊, 刘玉奇, 等. 基于粒子群优化的核极限学习机模型的风电功率区间预测方法[J]. 中国电机工程学报, 2015, 35: 146-153.
	Yang X Y, Guan W Y, Liu Y Q, et al. Prediction intervals forecasts of wind power based on PSO-KELM[J]. Proceedings of the CSEE, 2015, 35: 146-153.
25	张雷, 张小刚, 陈华. 基于Gath-Geva算法和核极限学习机的多阶段间歇过程软测量[J]. 化工学报, 2018, 69(6): 2576-2585.
	Zhang L, Zhang X G, Chen H. Soft sensors for multi-stage batch processes based on Gath-Geva algorithm and kernel extreme learning machine[J]. CIESC Journal, 2018, 69(6): 2576-2585.
26	Qian K, Zhang Z, Baird A, et al. Active learning for bird sound classification via a kernel-based extreme learning machine[J]. The Journal of the Acoustical Society of America, 2017, 142(4): 1796-1804.
27	Leng Q, Qi H, Miao J, et al. One-class classification with extreme learning machine[J]. Mathematical Problems in Engineering, 2015, 4: 1–11.
28	王培良, 叶晓丰, 杨泽宇. 基于Block-RPLS模型自适应更新的质量预测方法[J]. 控制与决策, 2018, 33(3): 455-462.
	Wang P L, Ye X F, Yang Y Z. Quality prediction method based on adaptive updating of Block-RPLS model[J]. Control and Decision, 2018, 33(3): 455-462.
29	Morey A, Pradhan S, Kumar R A, et al. Pollutant monitoring in tail gas of sulfur recovery unit with statistical and soft computing models[J]. Chemical Engineering Communications, 2018, 206(1): 69-85.
30	Douak F, Melgani F, Alajlanc N, et al. Active learning for spectroscopic data regression[J]. Journal of Chemometrics, 2012, 26(7): 374-383
31	Douak F, Melgani F, Benoudjit N. Kernel ridge regression with active learning for wind speed prediction[J]. Applied Energy, 2013, 103: 328-340.

[1]	杨欣, 王文, 徐凯, 马凡华. 高压氢气加注过程中温度特征仿真分析[J]. 化工学报, 2023, 74(S1): 280-286.
[2]	康飞, 吕伟光, 巨锋, 孙峙. 废锂离子电池放电路径与评价研究[J]. 化工学报, 2023, 74(9): 3903-3911.
[3]	陈哲文, 魏俊杰, 张玉明. 超临界水煤气化耦合SOFC发电系统集成及其能量转化机制[J]. 化工学报, 2023, 74(9): 3888-3902.
[4]	曹跃, 余冲, 李智, 杨明磊. 工业数据驱动的加氢裂化装置多工况切换过渡状态检测[J]. 化工学报, 2023, 74(9): 3841-3854.
[5]	齐聪, 丁子, 余杰, 汤茂清, 梁林. 基于选择吸收纳米薄膜的太阳能温差发电特性研究[J]. 化工学报, 2023, 74(9): 3921-3930.
[6]	何松, 刘乔迈, 谢广烁, 王斯民, 肖娟. 高浓度水煤浆管道气膜减阻两相流模拟及代理辅助优化[J]. 化工学报, 2023, 74(9): 3766-3774.
[7]	邢雷, 苗春雨, 蒋明虎, 赵立新, 李新亚. 井下微型气液旋流分离器优化设计与性能分析[J]. 化工学报, 2023, 74(8): 3394-3406.
[8]	张曼铮, 肖猛, 闫沛伟, 苗政, 徐进良, 纪献兵. 危废焚烧处理耦合有机朗肯循环系统工质筛选与热力学优化[J]. 化工学报, 2023, 74(8): 3502-3512.
[9]	诸程瑛, 王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报, 2023, 74(8): 3429-3437.
[10]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[11]	陈国泽, 卫东, 郭倩, 向志平. 负载跟踪状态下的铝空气电池堆最优功率点优化方法[J]. 化工学报, 2023, 74(8): 3533-3542.
[12]	刘文竹, 云和明, 王宝雪, 胡明哲, 仲崇龙. 基于场协同和耗散的微通道拓扑优化研究[J]. 化工学报, 2023, 74(8): 3329-3341.
[13]	吴文涛, 褚良永, 张玲洁, 谭伟民, 沈丽明, 暴宁钟. 腰果酚生物基自愈合微胶囊的高效制备工艺研究[J]. 化工学报, 2023, 74(7): 3103-3115.
[14]	汤晓玲, 王嘉瑞, 朱玄烨, 郑仁朝. 基于Pickering乳液的卤醇脱卤酶催化合成手性环氧氯丙烷[J]. 化工学报, 2023, 74(7): 2926-2934.
[15]	文兆伦, 李沛睿, 张忠林, 杜晓, 侯起旺, 刘叶刚, 郝晓刚, 官国清. 基于自热再生的隔壁塔深冷空分工艺设计及优化[J]. 化工学报, 2023, 74(7): 2988-2998.