Gaussian process ensemble soft-sensor modeling based on improved Bagging algorithm

doi:10.11949/j.issn.0438-1157.20151223

Abstract

Abstract:

In order to improve the accuracy and generalization ability of soft-sensor for complex industrial process, a Gaussian process ensemble soft-sensor modeling algorithm based on the improved bagging algorithm is proposed. This algorithm uses Gaussian process regression algorithm to build base learners and the resample method of bagging algorithm to form training subsets of base learners. A criteria for feature ordering base on normalized mutual information is proposed with selecting input features of base learners, which can implement supervised feature perturbance in the ensemble modeling for the sake of improving the diversity between base learners. When estimating the output of the test sample according to the output variances given by Gaussian process base learners, several base learners are selected adaptively to calculate the output of ensemble model. A soft-sensor modeling simulation using the data from the reactors of industrial Bisphenol-A production units shows the effectiveness of the algorithm.

Key words: algorithm, soft-sensor, model, Gaussian process, reactors

摘要：

为提高对工况复杂的工业过程进行软测量建模的模型精度和泛化能力,提出了一种基于改进Bagging算法的高斯过程集成软测量建模方法。该算法采用高斯过程回归算法建立集成学习模型的基学习器,并在Bagging算法对训练样本重采样生成基学习器训练子集的基础上,采用基于正则化互信息的特征排序指标进行基学习器的输入特征抽取,实现有监督的特征扰动,从而改善学习器的差异度。待测样本进行软测量估计时,根据各高斯过程基学习器输出的方差自适应地选择基学习器进行集成输出。采用工业双酚A生产装置反应器的现场数据建模仿真,结果表明该方法是有效的。

关键词: 算法, 软测量, 模型, 高斯过程, 反应器

CLC Number:

TP274

SUN Maowei, YANG Huizhong. Gaussian process ensemble soft-sensor modeling based on improved Bagging algorithm[J]. CIESC Journal, 2016, 67(4): 1386-1391.

孙茂伟, 杨慧中. 基于改进Bagging算法的高斯过程集成软测量建模[J]. 化工学报, 2016, 67(4): 1386-1391.

References 20

[1]	曹鹏飞, 罗雄麟. 化工过程软测量建模方法研究进展[J]. 化工学报, 2013, 64(3): 788-800. DOI: 10.3969/j.issn.0438-1157. 2013. 03.003. CAO P F, LUO X L. Modeling of soft sensor for chemical process[J]. CIESC Journal, 2013, 64(3): 788-800. DOI: 10.3969/j.issn.0438-1157. 2013.03.003.
[2]	王海宁, 夏陆岳, 周猛飞, 等. 过程工业软测量中的多模型融合建模方法[J]. 化工进展, 2014, 33(12): 3157-3163. DOI: 10.3969/j.issn.1000-6613.2014.12.005. WANG H N, XIA L Y, ZHOU M F, et al. Multi-model fusion modeling method for process industries soft sensor[J]. Chemical Industry and Engineering Progress, 2014, 33(12): 3157-3163. DOI: 10.3969/j.issn. 1000-6613.2014.12.005.
[3]	周鑫, 谭帅, 杨琦, 等. 基于Bagging集成的球团矿烧结过程混合模型[J]. 控制工程, 2015, 22(3): 516-520. DOI: 10.14107/j.cnki.kzgc.140039. ZHOU X, TAN S, YANG Q, et al. Modeling for pellets induration process based on Bagging method[J]. Control Engineering of China, 2015, 22(3): 516-520. DOI: 10.14107/j.cnki.kzgc.140039.
[4]	李毓, 徐成贤. 修剪Bagging集成的方法及其应用[J]. 系统工程理论与实践, 2008, 28(7): 105-110. DOI: 10.3321/j.issn: 1000-6788. 2008.07.014. LI Y, XU C X. A method for pruning Bagging ensembles and its applications[J]. Systems Engineering—Theory and Practice, 2008, 28(7): 105-110. DOI: 10.3321/j.issn:1000-6788.2008.07.014.
[5]	陈定三, 杨慧中. 基于局部重构融合流形聚类的多模型软测量建模[J]. 化工学报, 2011, 62(8): 2281-2286. DOI: 10.3969/j.issn. 0438-1157.2011.08.034. CHEN D S, YANG H Z. Multiple model soft sensor based on local reconstruction and fusion manifold clustering[J]. CIESC Journal, 2011, 62(8): 2281-2286. DOI: 10.3969/j.issn.0438-1157.2011.08.034.
[6]	吕业, 邓玉俊, 杨慧中. 基于类别特征提取的组合支持向量机模型[J]. 化工学报, 2011, 62(8): 2164-2169. DOI: 10.3969/j.issn. 0438-1157.2011.08.013. LÜ Y, DENG Y J, YANG H Z. Compositional support vector machine model based on feature extraction of categories[J]. CIESC Journal, 2011, 62(8): 2164-2169. DOI: 10.3969/j.issn.0438-1157.2011.08.013.
[7]	孙博, 王建东, 陈海燕, 等. 集成学习中的多样性度量[J]. 控制与决策, 2014, 29(3): 385-395. DOI: 10.13195/j.kzyjc.2013.1334. SUN B, WANG J D, CHEN H Y, et al. Diversity measures in ensemble learning[J]. Control and Decision, 2014, 29(3): 385-395. DOI: 10.13195/j.kzyjc.2013.1334.
[8]	安睿. 基于Bagging的电力信息安全态势分析系统的研究与实现[D]. 北京: 华北电力大学, 2012. AN R. Research and implementation on Bagging of electrical information security situation analysis system[D]. Beijing: North China Electric Power University, 2012.
[9]	刘余霞, 吕虹, 胡涛, 等. 基于Bagging集成学习的字符识别方法[J]. 计算机工程与应用, 2012, 48(33): 194-196, 211. DOI: 10.3778/j.issn.1002-8331.1207-0203. LIU Y X, LÜ H, HU T, et al. Research on character recognition based on Bagging ensemble learning[J]. Computer Engineering and Applications, 2012, 48(33): 194-196, 211. DOI: 10.3778/j.issn. 1002-8331.1207-0203.
[10]	李雅芹, 杨慧中. 一种基于Bagging算法的高斯过程集成建模方法[J]. 东南大学学报(自然科学版), 2011, 41(S1): 93-96. DOI: 10.3969/j.issn.1001-0505.2011.S1.020. LI Y Q, YANG H Z. Ensemble modeling method based on Bagging algorithm and Gaussian process[J]. Journal of Southeast University (Natural Science Edition), 2011, 41(S1): 93-96. DOI: 10.3969/j.issn. 1001-0505. 2011.S1.020.
[11]	亓慧, 王文剑, 郭虎升. 一种基于特征选择的SVM Bagging集成方法[J]. 小型微型计算机系统, 2014, 35(11): 2533-2537. QI H, WANG W J, GUO H S. An SVM Bagging ensemble learning algorithm based on feature selection[J]. Journal of Chinese Computer Systems, 2014, 35(11): 2533-2537.
[12]	徐峻岭, 周毓明, 陈林, 等. 基于互信息的无监督特征选择[J]. 计算机研究与发展, 2012, 49(2): 372-382. XU J L, ZHOU Y M, CHEN L, et al. An unsupervised feature selection approach based on mutual information[J]. Journal of Computer Research and Development, 2012, 49(2): 372-382.
[13]	PENG H C, LONG F H, DING C. Feature selection based on mutual information: criteria of Max-Dependency, Max-Relevance, and Min-Redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238. DOI: 10.1109/TPAMI. 2005.159.
[14]	ESTEVEZ P A, TESMER M, PEREZ C A, et al. Normalized mutual information feature selection[J]. IEEE Transactions on Neural Networks, 2009, 20(2): 189-201. DOI: 10.1109/TNN.2008.2005601.
[15]	何志昆, 刘光斌, 赵曦晶, 等. 高斯过程回归方法综述[J]. 控制与决策, 2013, 28(8): 1121-1129, 1137. DOI: 10.13195/j.kzyjc. 2013.08.018. HE Z K, LIU G B, ZHAO X J, et al. Overview of Gaussian process regression[J]. Control and Decision, 2013, 28(8): 1121-1129, 1137. DOI: 10.13195/j.kzyjc.2013.08.018.
[16]	NI W D, TAN S K, NG W J, et al. Moving-window GPR for nonlinear dynamic system modeling with dual updating and dual preprocessing[J]. Industrial and Engineering Chemistry Research, 2012, 51(18): 6416-6428. DOI: 10.1021/ie201898a.
[17]	雷瑜, 杨慧中. 基于高斯过程和贝叶斯决策的组合模型软测量[J]. 化工学报, 2013, 64(12): 4434-4438. DOI: 10.3969/j.issn.0438-1157. 2013.12.025. LEI Y, YANG H Z. Combination model soft sensor based on Gaussian process and Bayesian committee machine[J]. CIESC Journal, 2013, 64(12): 4434-4438. DOI: 10.3969/j.issn.0438-1157. 2013.12.025.
[18]	韩敏, 刘晓欣. 基于Copula熵的互信息估计方法[J]. 控制理论与应用, 2013, 30(7): 875-879. DOI: 10.7641/CTA.2013.21262. HAN M, LIU X X. Mutual information estimation based on Copula entropy[J]. Control Theory and Applications, 2013, 30(7): 875-879. DOI: 10.7641/CTA.2013.21262.
[19]	范雪莉, 冯海泓,原猛. 基于互信息的主成分分析特征选择算法[J]. 控制与决策, 2013, 28(6): 915-919. FAN X L, FENG H H, YUAN M. PCA based on mutual information for feature selection[J]. Control and Decision, 2013, 28(6): 915-919.
[20]	洪智勇, 刘灿涛, 邓宝林. 基于二次Renyi熵的正则化互信息特征选择方法[J]. 计算机应用, 2010, 30(5): 1273-1276. HONG Z Y, LIU C T, DENG B L. Normalized mutual information feature selection method based on Renyi's quadratic entropy[J]. Journal of Computer Applications, 2010, 30(5): 1273-1276.64-2169. DOI: 10.3969/j.issn.0438-1157.2011.08.013.
[7]	孙博, 王建东, 陈海燕, 等. 集成学习中的多样性度量[J]. 控制与决策, 2014, 29(3): 385-395. DOI: 10.13195/j.kzyjc.2013.1334.
	SUN B, WANG J D, CHEN H Y, et al. Diversity measures in ensemble learning[J]. Control and Decision, 2014, 29(3): 385-395. DOI: 10.13195/j.kzyjc.2013.1334.
[8]	安睿. 基于bagging的电力信息安全态势分析系统的研究与实现[D]. 北京: 华北电力大学, 2012.
	AN R. Research and implementation on bagging of electrical information security situation analysis system[D]. Beijing: North China Electric Power University, 2012.
[9]	刘余霞, 吕虹, 胡涛, 等. 基于Bagging集成学习的字符识别方法[J]. 计算机工程与应用, 2012, 48(33): 194-196, 211. DOI: 10.3778/j.issn.1002-8331.1207-0203.
	LIU Y X, LV H, HU T, et al. Research on character recognition based on Bagging ensemble learning[J]. Computer Engineering and Applications, 2012, 48(33): 194-196, 211. DOI: 10.3778/j.issn.1002-8331.1207-0203.
[10]	李雅芹, 杨慧中. 一种基于Bagging算法的高斯过程集成建模方法[J]. 东南大学学报(自然科学版), 2011, 41(S1): 93-96. DOI: 10.3969/j.issn.1001-0505.2011.S1.020.
	LI Y Q, YANG H Z. Ensemble modeling method based on Bagging algorithm and Gaussian process[J]. Journal of Southeast University (Natural Science Edition), 2011, 41(S1): 93-96. DOI: 10.3969/j.issn.1001-0505.2011.S1.020.
[11]	亓慧, 王文剑, 郭虎升. 一种基于特征选择的SVM Bagging集成方法[J]. 小型微型计算机系统, 2014, 35(11): 2533-2537.
	QI H, WANG W J, GUO H S. An SVM Bagging ensemble learning algorithm based on feature selection[J]. Journal of Chinese Computer Systems, 2014, 35(11): 2533-2537.
[12]	徐峻岭, 周毓明, 陈林, 等. 基于互信息的无监督特征选择[J]. 计算机研究与发展, 2012, 49(2): 372-382.
	XU J L, ZHOU Y M, CHEN L, et al. An unsupervised feature selection approach based on mutual information[J]. Journal of Computer Research and Development, 2012, 49(2): 372-382.
[13]	PENG H C, LONG F H, DING C. Feature selection based on mutual information: criteria of Max-Dependency, Max-Relevance, and Min-Redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238. DOI: 10.1109/TPAMI.2005.159.
[14]	ESTEVEZ P A, TESMER M, PEREZ C A, et al. Normalized mutual information feature selection[J]. IEEE Transactions on Neural Networks, 2009, 20(2): 189-201. DOI: 10.1109/TNN.2008.2005601.
[15]	何志昆, 刘光斌, 赵曦晶, 等. 高斯过程回归方法综述[J]. 控制与决策, 2013, 28(8): 1121-1129, 1137. DOI: 10.13195/j.kzyjc.2013.08.018.
	HE Z K, LIU G B, ZHAO X J, et al. Overview of Gaussian process regression[J]. Control and Decision, 2013, 28(8): 1121-1129, 1137. DOI: 10.13195/j.kzyjc.2013.08.018.
[16]	NI W D, TAN S K, NG W J, et al. Moving-window GPR for nonlinear dynamic system modeling with dual updating and dual preprocessing[J]. Industrial and Engineering Chemistry Research, 2012, 51(18): 6416-6428. DOI: 10.1021/ie201898a.
[17]	雷瑜, 杨慧中. 基于高斯过程和贝叶斯决策的组合模型软测量[J]. 化工学报, 2013, 64(12): 4434-4438. DOI: 10.3969/j.issn.0438-1157.2013.12.025.
	LEI Y, YANG H Z. Combination model soft sensor based on Gaussian process and Bayesian committee machine[J]. CIESC Journal, 2013, 64(12): 4434-4438. DOI: 10.3969/j.issn.0438-1157.2013.12.025.
[18]	韩敏, 刘晓欣. 基于Copula熵的互信息估计方法[J]. 控制理论与应用, 2013, 30(7): 875-879. DOI: 10.7641/CTA.2013.21262.
	HAN M, LIU X X. Mutual information estimation based on Copula entropy[J]. Control Theory and Applications, 2013, 30(7): 875-879. DOI: 10.7641/CTA.2013.21262.
[19]	范雪莉, 冯海泓,原猛. 基于互信息的主成分分析特征选择算法[J]. 控制与决策, 2013, 28(6): 915-919.
	FAN X L, FENG H H, YUAN M. PCA based on mutual information for feature selection[J]. Control and Decision, 2013, 28(6): 915-919.
[20]	洪智勇, 刘灿涛, 邓宝林. 基于二次Renyi熵的正则化互信息特征选择方法[J]. 计算机应用, 2010, 30(5): 1273-1276.
	HONG Z Y, LIU C T, DENG B L. Normalized mutual information feature selection method based on Renyi's quadratic entropy[J]. Journal of Computer Applications, 2010, 30(5): 1273-1276.