Study on the soft sensor of multi-kernel relevance vector machine based on time difference

doi:10.11949/j.issn.0438-1157.20181240

CIESC Journal ›› 2019, Vol. 70 ›› Issue (4): 1472-1484.DOI: 10.11949/j.issn.0438-1157.20181240

• Process system engineering • Previous Articles Next Articles

Study on the soft sensor of multi-kernel relevance vector machine based on time difference

Jing WU^1,³(),Yiqi LIU^1,²,Jian LIU⁴,Daoping HUANG¹(),Yu QIU¹,Guangping YU⁴

1. School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, Guangdong, China
2. State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, Zhejiang, China
3. School of Information Engineering, Guizhou Minzu University, Guiyang 550025, Guizhou, China
4. Shenyang Institute of Automation, Chinese Academy of Sciences, Guangzhou 511458, Guangdong, China

Received:2018-10-18 Revised:2018-12-25 Online:2019-04-05 Published:2019-04-05
Contact: Daoping HUANG

基于动态多核相关向量机的软测量建模研究

吴菁^1,³(),刘乙奇^1,²,刘坚⁴,黄道平¹(),邱禹¹,于广平⁴

1. 华南理工大学自动化科学与工程学院，广东广州 510640
2. 浙江大学工业控制技术国家重点实验室，浙江杭州 310027
3. 贵州民族大学数据科学与信息工程学院, 贵州贵阳 550025
4. 广州中国科学院沈阳自动化研究所分所，广东广州 511458

通讯作者: 黄道平
作者简介:<named-content content-type="corresp-name">吴菁</named-content>（1988—），女，博士研究生，讲师，<email>ipicq@163.com</email>|黄道平（1961—），男，博士，教授，<email>audhuang@scut.edu.cn</email>
基金资助:
国家自然科学基金项目(61873096,61673181,61533002);广东省科技计划项目(2016A020221007);2017中央高校基本科研业务资助项目一面上项目(2017MS053);广州市科技计划项目(201804010256);工业控制技术国家重点实验室课题(ICT1800372);贵州省科技厅联合基金项目(黔科合LH字[2014]7379)

Abstract

Abstract:

Considering the characteristics of strong multivariable coupling, significant non-linearity and parameter time-varying in the wastewater treatment processes, a multi-kernel relevance vector machine (MRVM) is proposed for soft-sensor modeling. Particle swarm optimization algorithm is further used to optimize multi-kernel weights and kernel parameters. Meanwhile, the time difference (TD) method is introduced to improve the dynamic characteristics of MRVM. The proposed model was demonstrated through a WWTP simulated case study by comparison with relevance vector machine (RVM) with a single kernel, back propagation (BP) neural network and the genetic algorithm-based support vector machine (GA-SVM). Results showed that the proposed model achieved better prediction accuracy. Finally, the robustness of the models is discussed in the context of data drift and anomalies.

Key words: soft sensor, wastewater treatment, multi-kernel, relevance vector machine, time difference

摘要：

针对污水处理过程中存在的多变量耦合、强非线性以及参数时变等问题，提出基于多核学习相关向量机的软测量建模方法，并采用粒子群算法对多核权重以及核参数进行优化。同时，引入时间差分(time difference)方法改进多核相关向量机的动态特性。为了验证所提模型的有效性，通过一仿真案例与单核相关向量机、多层前馈神经网络和基于遗传算法的支持向量机进行对比研究。结果表明，所提模型具有更好的预测效果。最后，对模型的鲁棒性在数据漂移和异常的场景下进行了讨论。

关键词: 软测量, 污水处理, 多核, 相关向量机, 时差建模

CLC Number:

TP 277

Jing WU, Yiqi LIU, Jian LIU, Daoping HUANG, Yu QIU, Guangping YU. Study on the soft sensor of multi-kernel relevance vector machine based on time difference[J]. CIESC Journal, 2019, 70(4): 1472-1484.

吴菁, 刘乙奇, 刘坚, 黄道平, 邱禹, 于广平. 基于动态多核相关向量机的软测量建模研究[J]. 化工学报, 2019, 70(4): 1472-1484.

Figures/Tables 18

Fig.1 Schematic of linear combination of mutil-kernel functions

Fig.2 Framework of time difference mutil-kernel relevance vector machine soft-sensor model

Fig.3 General overview of the BSM1 plant

Table 1 List of secondary variables

序号	变量描述	符号	序号	变量描述	符号
1	入水悬浮固体浓度(mg SS/L)	SSin	11	第一个反应池悬浮固体浓度(mg SS/L)	SSr1
2	入水总化学需氧量 (mg COD/L)	CODin	12	第二个反应池NH₄⁺ + NH₃ (mg N/L)	SNHr2
3	入水NH₄⁺ + NH₃ (mg N/L)	SNHin	13	第二个反应池氨氮(mg N/L)	SNOr2
4	出水悬浮固体浓度(mg SS/L)	SSe	14	第二个反应池溶解氧(g COD/m³)	SOr2
5	出水总化学需氧量 (mg COD/L)	CODe	15	第五个反应池NH₄⁺ + NH₃ (mg N/L)	SNHr5
6	出水NH₄⁺ + NH₃ (mg N/L)	SNHe	16	第五个反应池氨氮(mg N/L)	SNOr5
7	出水氨氮 (mg N/L)	SNOe	17	第五个反应池溶解氧(g COD/m³)	SOr5
8	第一个反应池NH₄⁺ + NH₃ (mg N/L)	SNHr1	18	入水流水(m³/d)	Qin
9	第一个反应池氨氮(mg N/L)	SNOr1	19	内部循环流水(m³/d)	Qintr
10	第一个反应池溶解氧(g COD/m³)	SOr1	20	出水流水(m³/d)	Qe

Table 2 List of chosen kernel functions

序号	核函数名称	表达式
k₁	Gauss	$k (x i, x j) = e x p - 2 τ - 2 x i - x j 2$
k₂	cauchy	$k (x i, x j) = 1 + τ - 2 x i - x j 2 - 1$
k₃	bubble	$k (x i, x j) = τ - 2 x i - x j$
k₄	poly	$k (x i, x j) = (〈 x i, x j 〉 + C) d$
k₅	change points	$k (x i, x j) = k a × L x + k b × L ˉ x$
k₆	spline	$k (x i, x j) = 1 + x i x j + x i x j m i n (x i, x j) - x i + x j 2 m i n (x i, x j) 2 + m i n (x i, x j) 3 3$

Table 2 List of chosen kernel functions

序号	核函数名称	表达式
k₁	Gauss	$k (x i, x j) = e x p - 2 τ - 2 x i - x j 2$
k₂	cauchy	$k (x i, x j) = 1 + τ - 2 x i - x j 2 - 1$
k₃	bubble	$k (x i, x j) = τ - 2 x i - x j$
k₄	poly	$k (x i, x j) = (〈 x i, x j 〉 + C) d$
k₅	change points	$k (x i, x j) = k a × L x + k b × L ˉ x$
k₆	spline	$k (x i, x j) = 1 + x i x j + x i x j m i n (x i, x j) - x i + x j 2 m i n (x i, x j) 2 + m i n (x i, x j) 3 3$

Table 3 Parameters of PSO

数据集	参数优化值
晴天数据	width=5, $υ$ =[1,1,0,0.272899,0,0]
雨天数据	width=25, $υ$ =[0,0.064016,0,1,0,0.61665]
暴雨数据	width=25, $υ$ =[0,0.905672,0,1,0,0.889637]

Table 3 Parameters of PSO

数据集	参数优化值
晴天数据	width=5, $υ$ =[1,1,0,0.272899,0,0]
雨天数据	width=25, $υ$ =[0,0.064016,0,1,0,0.61665]
暴雨数据	width=25, $υ$ =[0,0.905672,0,1,0,0.889637]

Table 4 Parameters definition of the comparison model

模型名称	参数设置
BP	隐含层激励函数=‘logsig’；输出层激励函数=‘purelin’；学习速率lr=0.1；隐含层神经元数=‘6’；训练次数=‘100’。
GA-SVM	高斯核；正则化参数c= 96.6849；准确度阈值g= 0.0286；核参数p=0.01。
RVM	kernel=‘gauss’；width=10。

Fig.4 Predicted results of BOD5 for all models under dry weather

Table 5 Comparison of RMSE and relevance results of each model

模型	模型名称评价指标	晴天		雨天		暴雨
模型	模型名称评价指标	RMSE	r	RMSE	r	RMSE	r
基础模型	BP	0.0301	0.9986	0.9422	0.5452	0.7042	0.5987
	GA-SVM	0.0414	0.9966	0.8579	0.6506	0.4398	0.9030
	RVM	0.0196	0.9990	0.8755	0.6284	0.5661	0.7774
	MRVM	0.0154	0.9991	0.5632	0.9235	0.3547	0.9221
自适应模型	TD-BP	0.0031	0.999973	0.0362	0.9992	0.0539	0.9975
	TD-GA-SVM	0.0036	0.999957	0.0170	0.9998	0.0344	0.9991
	TD-RVM	0.0037	0.999952	0.0161	0.9998	0.0264	0.9994
	TD-MRVM	0.0032	0.999962	0.0158	0.9998	0.0257	0.9994

Fig.5 Predicted results of BOD5 for all models under rain weather

Fig.6 Predicted results of BOD5 for all models under storm weather

Fig.7 Illustration of associate coefficient between input and output variates

Fig.8 Comparison of SSe outliers models’ error difference

Table 6 Comparison of predictive results on SSe outliers models

模型名称评价指标	突变5%		突变10%		突变20%		突变40%
模型名称评价指标	RMSE	r	RMSE	r	RMSE	r	RMSE	r
BP	0.1195	0.9482	0.2156	0.9500	0.4053	0.8566	0.7486	0.6892
GA-SVM	0.1092	0.9860	0.1848	0.9616	0.3320	0.8939	0.6004	0.7639
RVM	0.0769	0.9924	0.1378	0.9763	0.2507	0.9302	0.4684	0.8218
TD-MRVM	0.0080	0.9998	0.0144	0.9992	0.0271	0.9973	0.0499	0.9910

Fig.9 Detail of TD-MRVM’s error difference

Fig.10 Comparison of SSe drifting models’ error difference

Table 7 Comparison of predictive results on SSe drifting models

模型名称评价指标	漂移1°		漂移2°		漂移5°		漂移10°
模型名称评价指标	RMSE	r	RMSE	r	RMSE	r	RMSE	r
BP	0.1634	0.9682	0.3003	0.8946	0.6872	0.5933	1.2211	0.2682
GA-SVM	0.1587	0.9860	0.2741	0. 8992	0.6028	0.6359	1.0593	0.3501
RVM	0.1063	0.9863	0.1927	0.9548	0.4274	0.8045	0.7056	0.6004
TD-MRVM	0.0041	0.9999	0.0058	0.9999	0.0121	0.9995	0.0229	0.9981

Table 8 Comparison of predictive results on white noise effect

模型	晴天				雨天				暴雨
模型	0	5%	7.5%	10%	0	5%	7.5%	10%	0	5%	7.5%	10%
TD-BP	0.0031	0.0371	0.0389	0.0440	0.0362	0.0676	0.0687	0.0837	0.0539	0.1059	0.1059	0.1048
TD-GA-SVM	0.0036	0.0374	0.0361	0.0356	0.0170	0.0655	0.0658	0.0659	0.0344	0.1082	0.1126	0.1128
TD-RVM	0.0037	0.0537	0.0521	0.0511	0.0161	0.0861	0.0865	0.0869	0.0264	0.1090	0.1094	0.1104
TD-MRVM	0.0032	0.0380	0.0367	0.0360	0.0158	0.0625	0.0622	0.0621	0.0257	0.1014	0.1015	0.1017

References 32

1	肖红军, 刘乙奇, 黄道平. 面向污水处理的动态变分贝叶斯混合因子故障诊断[J]. 控制理论与应用, 2016, 33(11): 1519-1526.
	XiaoH J, LiuY Q, HuangD P. Dynamic fault diagnosis via variational Bayesian mixture factor analysis with application to wastewater treatment[J]. Control Theory & Applications, 2016, 33(11): 1519-1526.
2	黄道平, 刘乙奇, 李艳.软测量在污水处理过程中的研究与应用[J].化工学报, 2011, 62 (1): 1-9.
	HuangD P, LiuY Q, LiY. Soft sensor research and its application in wastewater treatment[J]. CIESC Journal, 2011, 62(1): 1-9.
3	曹鹏飞, 罗雄麟. 化工过程软测量建模方法研究进展[J]. 化工学报, 2013, 64(3): 788-800.
	CaoP F, LuoX L. Modeling of soft sensor for chemical process[J]. CIESC Journal, 2013, 64(3): 788-800.
4	HaimiH, MulasM, CoronaF, et al. Data-derived soft-sensors for biological wastewater treatment plants: an overview[J]. Environmental Modelling & Software, 2013, 47: 88-107.
5	VapnikV N, VapnikV. Statistical Learning Theory[M]. New York: Wiley, 1998.
6	HuangY R, ZhangS D. Dissolved oxygen intelligent optimization control system in the aeration tank of wastewater treatment[J]. Information and Control, 2011, 40(3): 393-400.
7	曹巍, 赵英凯, 高世伟. 基于模糊核聚类的多类支持向量机[J]. 化工学报, 2010, 61(2): 420-424.
	CaoW, ZhaoY K, GaoS W. Multi-class support vector machines based on fuzzy kernel cluster[J]. CIESC Journal, 2010, 61(2): 420-424.
8	刘瑞兰, 徐艳, 戎舟. 基于稀疏最小二乘支持向量机的软测量建模[J]. 化工学报, 2015, 66(4): 1402-1406.
	LiuR L, XuY, RongZ. Modeling soft sensor based on sparse least square support vector machine[J]. CIESC Journal, 2015, 66(4): 1402-1406.
9	马建, 邓晓刚, 王磊. 基于深度集成支持向量机的工业过程软测量方法[J]. 化工学报, 2018, 69(3): 1121-1128.
	MaJ, DengX G, WangL. Industrial process soft sensor method based on deep learning ensemble support vector machine[J]. CIESC Journal, 2018, 69(3): 1121-1128.
10	TippingM E. Sparse Bayesian learning and the relevance vector machine[J]. Journal of Machine Learning Research, 2001, (1): 211-244.
11	GeZ Q, SongZ H. Nonlinear soft sensor development based on relevance vector machine[J]. Industrial & Engineering Chemistry Research, 2010, 49(18): 8685-8693.
12	MotaiY. Kernel association for classification and prediction: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(2): 208-223.
13	LiuY, ChenJ, SunZ, et al. A probabilistic self-validating soft-sensor with application to wastewater treatment[J]. Computers & Chemical Engineering, 2014, 71(71): 263-280.
14	LiuY. Adaptive just-in-time and relevant vector machine based soft-sensors with adaptive differential evolution algorithms for parameter optimization[J]. Chemical Engineering Science, 2017, 172: 571-584.
15	许玉格, 曹涛, 罗飞. 基于相关向量机的污水处理出水水质预测模型[J]. 华南理工大学学报(自然科学版), 2014, 42(5): 103-108.
	XuY G , CaoT, LuoF. Wastewater effluent quality prediction model base on relevance vector machine[J]. Journal of South China University of Technology(Natural Science Edition), 2014, 42(5): 103-108.
16	汪洪桥, 孙富春, 蔡艳宁, 等. 多核学习方法[J]. 自动化学报, 2010, 36(8): 1037-1050.
	WangH Q, SunF C, CaiY N, et al. On multiple kernel learning methods[J]. Acta Automatica Sinica, 2010, 36(8): 1037-1050.
17	杨柳, 张磊, 张少勋, 等. 单核和多核相关向量机的比较研究[J]. 计算机工程, 2010, 36(12): 195-197.
	YangL, ZhangL, ZhangS X, et al. Comparison research of single kernel and multi-kernel relevance vector machine[J]. Computer Engineering, 2010, 36(12): 195-197.
18	LeeW J, VerzakovS, DuinR P W. Kernel combination versus classifier combination[C]//International Workshop on Multiple Classifier Systems. Heidelberg, Berlin: Springer, 2007: 22-31.
19	GönenM, AlpaydınE. Localized multiple kernel learning[C]// Proceedings of the 25th International Conference on Machine Learning. Helsinki, Finland: ACM, 2008: 352-359
20	KloftM, BrefeldU, LaskovP, et al. Non-sparse multiple kernel learning[C]// Proceedings of the Workshop on Kernel Learning: Automatic Selection of Optimal Kernels. Whistler, Canada: The MIT Press, 2008: 1-4
21	曹涛. 基于相关向量机的污水软测量建模研究[D]. 广州: 华南理工大学, 2015.
	CaoT. Study of soft sensor modeling for wastewater treatment process base on relevance vector machine[D]. Guangzhou: South China University of Technology, 2015.
22	LinY, XiaK, JiangX, et al. Landslide susceptibility mapping based on particle swarm optimization of multiple kernel relevance vector machines: case of a low Hill Area in Sichuan Province, China[J]. ISPRS International Journal of Geo-Information, 2016, 5(10): 191.
23	赵朋程, 刘彬, 高伟, 等. 用于水泥熟料fCaO预测的多核最小二乘支持向量机模型[J]. 化工学报, 2016, 67(6): 2480-2487.
	ZhaoP C, LiuB, GaoW, et al. Multiple kernel least square support vector machine model for prediction of cement clinker lime content[J]. CIESC Journal, 2016, 67(6): 2480-2487.
24	KaltwangS, TodorovicS, PanticM. Doubly sparse relevance vector machine for continuous facial behavior estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(9): 1748-1761.
25	SouzaF A A, AraujoR, MendesJ. Review of soft sensor methods for regression applications[J]. Chemometrics and Intelligent Laboratory Systems, 2016, 152: 69-79.
26	KanekoH, FunatsuK. Moving window and just-in-time soft sensor model based on time differences considering a small number of measurements[J]. Industrial & Engineering Chemistry Research, 2015, 54(2): 700-704.
27	QinS J. Recursive PLS algorithms for adaptive data modeling[J]. Computers & Chemical Engineering, 1998, 22(4/5): 503-514.
28	刘乙奇. 自确认软测量模型研究及其在污水处理中的应用 [D]. 广州: 华南理工大学, 2013.
	LiuY Q. Research on self-validating soft sensor modeling and its application in wastewater treatment process[D]. Guangzhou: South China University of Technology, 2013.
29	KanekoH, FunatsuK. Maintenance-free soft sensor models with time difference of process variables[J]. Chemometrics and Intelligent Laboratory Systems, 2011, 107(2): 312-317.
30	KanekoH, FunatsuK. Classification of the degradation of soft sensor models and discussion on adaptive models[J]. AIChE Journal, 2013, 59(7): 2339-2347.
31	XiongW, LiY, ZhaoY, et al. Adaptive soft sensor based on time difference Gaussian process regression with local time-delay reconstruction[J]. Chemical Engineering Research and Design, 2017, 117: 670-680.
32	DuvenaudD. Automatic model construction with Gaussian processes[D]. Cambridge: University of Cambridge, 2014.

[1]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[2]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[3]	ZHAO Yang, XIONG Weili. Multi-objective optimization control of wastewater treatment process based on multi-strategy adaptive differential evolution algorithm [J]. CIESC Journal, 2021, 72(4): 2167-2177.
[4]	WANG Ying, LI Qian, CAO Lixia, LI Yanxiang, LI Wangliang. Progress of biomass-based materials for uranium adsorption [J]. CIESC Journal, 2021, 72(3): 1205-1216.
[5]	LIU Cong, XIE Li, YANG Huizhong. Multi-model soft sensor development for penicillin fermentation process based on improved density peak clustering [J]. CIESC Journal, 2021, 72(3): 1606-1615.
[6]	Tangwei CHEN, Zhicheng PAN, Ying CHEN, Min LIU, Tingting CHEN, Yaping ZHONG. Effect of hydrocyclone split ratio on carbon release performance of excess sludge [J]. CIESC Journal, 2021, 72(11): 5761-5769.
[7]	Yuhao DU, Gaowei YAN, Rong LI, Fang WANG. Multiple working conditions soft sensor modeling method of geodesic flow kernel based on locally linear embedding [J]. CIESC Journal, 2020, 71(3): 1278-1287.
[8]	Lu ZHANG, Jiacheng ZHANG, Honggui HAN, Junfei QIAO. FNN-based process control for biochemical phosphorus in WWTP [J]. CIESC Journal, 2020, 71(3): 1217-1225.
[9]	YANG Yijun,WANG Zhenlei,WANG Xin. Soft sensor modeling method based on hybrid model of nearest neighbor and neural network [J]. CIESC Journal, 2020, 71(12): 5696-5705.
[10]	Xuezhi DAI,Weili XIONG. A fast active learning method based on kernel extreme learning machine and its application for soft sensing [J]. CIESC Journal, 2020, 71(11): 5226-5236.
[11]	Wei CHAI, Longhang GUO, Binbin CHI. Interval model for predicting effluent quality variables of wastewater treatment plants [J]. CIESC Journal, 2019, 70(9): 3449-3457.
[12]	Xiaoqin LIAN, Liwei WANG, Sa AN, Wei WEI, Zaiwen LIU. On soft sensor of chemical oxygen demand by SOM-RBF neural network [J]. CIESC Journal, 2019, 70(9): 3465-3472.
[13]	Meihua QIN, Hongqiu ZHU, Yonggang LI, Junming CHEN, Fengxue ZHANG, Wenting LI. Soft-sensor method for ion concentration of electrochemical wastewater treatment based on STA-K-means clustering [J]. CIESC Journal, 2019, 70(9): 3458-3464.
[14]	Fei LI, Cuili YANG, Wenjing LI, Junfei QIAO. Optimal control of wastewater treatment process using NSGAII algorithm based on multi-objective uniform distribution [J]. CIESC Journal, 2019, 70(5): 1868-1878.
[15]	Wenpeng JI, Huizhong YANG. Multi-manifold soft sensor based on modified expanding search clustering algorithm [J]. CIESC Journal, 2019, 70(2): 723-729.

Study on the soft sensor of multi-kernel relevance vector machine based on time difference

基于动态多核相关向量机的软测量建模研究

RichHTML

PDF (PC)

Knowledge

Cited

Abstract

Cite this article

share this article

Figures/Tables 18

References 32

Related Articles 15

Recommended Articles

Metrics

Comments