Soft sensor modeling method based on hybrid model of nearest neighbor and neural network

doi:10.11949/0438-1157.20200401

Abstract

Abstract:

Soft-sensing modeling can effectively solve the problems of large measurement lag, high price, and complex maintenance of online analytical instruments in the production process. At present, neural network based on data-driven is one of the main tools of soft sensor. In the process of modeling data collection, the collection of dominant variables is much more difficult than that of auxiliary variables, resulting in a large amount of unlabeled data. However, traditional soft sensor modeling methods ignore these unlabeled data and only use a small amount of labeled data for modeling, which has negative effect on the prediction accuracy of the model. To solve the problem of label missing, the nearest neighbor algorithm is used to pseudo label the unlabeled data. At the same time, a network structure is designed by combining convolution operation and gated recurrent unit neural network (GRU) to further utilize the unlabeled data, extract the dynamic feature from data at different time, and improve the prediction accuracy of the neural network. Finally, the method is applied to the prediction of propane concentration on the top of propylene distillation column. The results show that the model can solve the problem of label missing in the nonlinear dynamic system and has higher prediction accuracy.

Key words: soft sensor, dynamic modeling, process systems, nearest neighbor algorithm, gated recurrent unit neural network

摘要：

软测量建模能够有效地解决生产过程中在线分析仪表测量滞后大、价格昂贵、维护保养复杂等问题。目前，基于数据驱动的神经网络是软测量建模的主要工具之一。而在建模数据的采集过程中，主导变量的采集相对辅助变量要困难得多，由此产生了大量缺失标签的数据。但传统的软测量建模方法却忽视了这些无标签数据，只利用少量的有标签数据建模，从而影响了模型的预测精度。为了解决标签缺失的问题，采用最近邻算法对无标签数据进行伪标记，同时设计了由卷积操作与门限循环单元神经网络（GRU）结合的网络结构来进一步利用无标签数据，提取不同时刻数据中的动态特征，提高神经网络的预测精度。最后将该方法应用于丙烯精馏塔塔顶丙烷浓度的预测，实验结果表明该模型能有效处理非线性动态系统的标签缺失问题，具有更高的预测精度。

关键词: 软测量, 动态建模, 过程系统, 最近邻算法, 门限循环单元神经网络

CLC Number:

TP 273

YANG Yijun,WANG Zhenlei,WANG Xin. Soft sensor modeling method based on hybrid model of nearest neighbor and neural network[J]. CIESC Journal, 2020, 71(12): 5696-5705.

杨逸俊,王振雷,王昕. 基于最近邻与神经网络融合模型的软测量建模方法[J]. 化工学报, 2020, 71(12): 5696-5705.

Figures/Tables 14

Fig.1 Structure of HNN

Table 1 Pseudocode of KNN

算法：KNN伪标记及组成新样本

输入：有标签数据集 $D l = x i, y i i = 1 l$ ；需要伪标记的无标签数据集

$D u = x j j = 1 u$ ；

KNN算法中的参数K；

1. for j = 1 : u

2. for i = 1 : l

3. 计算{x_j}与{x_i}间的欧几里德距离S_ji

4. end

5. 根据距离S_ji的大小对有标签数据集D_l进行升序排列

6. 选取前K个有标签数据，记录其距离与标签信息

{s₁,y₁},…,{s_K,y_K}

7. for k = 1 : K

8. $w k = e x p (- s k 2 / 2)$

9. end

10. $y j = ∑ k = 1 K w k y k ∑ k = 1 K w k$

11. end

12. 得到伪标记数据集 $D p = x j, y j j = 1 u$

输出：新数据集 $D K N N = D l ? D p$

Table 1 Pseudocode of KNN

算法：KNN伪标记及组成新样本

输入：有标签数据集 $D l = x i, y i i = 1 l$ ；需要伪标记的无标签数据集

$D u = x j j = 1 u$ ；

KNN算法中的参数K；

1. for j = 1 : u

2. for i = 1 : l

3. 计算{x_j}与{x_i}间的欧几里德距离S_ji

4. end

5. 根据距离S_ji的大小对有标签数据集D_l进行升序排列

6. 选取前K个有标签数据，记录其距离与标签信息

{s₁,y₁},…,{s_K,y_K}

7. for k = 1 : K

8. $w k = e x p (- s k 2 / 2)$

9. end

10. $y j = ∑ k = 1 K w k y k ∑ k = 1 K w k$

11. end

12. 得到伪标记数据集 $D p = x j, y j j = 1 u$

输出：新数据集 $D K N N = D l ? D p$

Fig.2 Structure of GRU

Fig.3 Schematic diagram of convolution operation

Fig.4 Modeling flow chart of HNN

Fig.5 Schematic diagram of HNN online prediction

Fig.6 Comparison of prediction effects of four models

Table 2 Performance evaluation of four models

Model	MSE	MAE	MAPE
NARX	3.1788	1.4757	219.5918
SVM	0.6522	0.7681	133.9137
GRU	0.0197	0.1083	15.0631
HNN	0.0026	0.0396	6.4383

Fig.7 Schematic diagram of propylene rectification tower

Fig.8 Prediction of propane concentration by NARX

Fig.9 Prediction of propane concentration by SVM

Fig.10 Prediction of propane concentration by GRU

Table 3 Performance evaluation of four models for propane concentration prediction

Model	MSE	MAE	MAPE
NARX	7.1124 $×$ 10^-5	7.0758 $×$ 10^-3	2.4195
SVM	4.8657 $×$ 10^-5	4.7382 $×$ 10^-3	1.4910
GRU	3.6935 $×$ 10^-5	4.6781 $×$ 10^-3	1.5813
HNN	7.9529 $×$ 10^-6	1.9485 $×$ 10^-3	0.6479

Table 3 Performance evaluation of four models for propane concentration prediction

Model	MSE	MAE	MAPE
NARX	7.1124 $×$ 10^-5	7.0758 $×$ 10^-3	2.4195
SVM	4.8657 $×$ 10^-5	4.7382 $×$ 10^-3	1.4910
GRU	3.6935 $×$ 10^-5	4.6781 $×$ 10^-3	1.5813
HNN	7.9529 $×$ 10^-6	1.9485 $×$ 10^-3	0.6479

Fig.11 Prediction of propane concentration by HNN

References 30

1	Qi Q, Hirai S, Ho V A. Wrinkled soft sensor with variable afferent morphology[J]. IEEE Robotics and Automation Letters, 2019, 4(2): 1908-1915.
2	Han S, Kim T, Kim D, et al. Use of deep learning for characterization of microfluidic soft sensors[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 873-880.
3	Chamil A. Design and applications of soft sensors in polymer processing: a review[J]. IEEE Sensors Journal, 2019, 19(8): 2801-2813.
4	刘剑, 张凌波, 王潇凌. 基于混沌菌群算法改进BP神经网络的焦炭塔生焦高度软测量建模[J]. 化工自动化及仪表, 2019, 46(10): 811-815+833.
	Liu J, Zhang L B, Wang X L. Soft sensor modeling of coke tower coke height based on improved BP neural network based on chaos flora algorithm [J]. Control and Instruments in Chemical Industry, 2019, 46 (10): 811-815 + 833.
5	李文静, 李萌, 乔俊飞. 基于互信息和自组织RBF神经网络的出水BOD软测量方法[J]. 化工学报, 2019, 70(2): 687-695.
	Li W J, Li M, Qiao J F. Soft sensing method of effluent BOD based on mutual information and self-organizing RBF neural network [J]. CIESC Journal, 2019, 70 (2): 687-695.
6	廉小亲, 王俐伟, 安飒, 等. 基于SOM-RBF神经网络的COD软测量方法[J]. 化工学报, 2019, 70(9): 3465-3472.
	Lian X Q, Wang L W, An S, et al. Soft sensing method of COD based on SOM-RBF neural network[J]. CIESC Journal, 2019, 70 (9): 3465-3472.
7	Yan X F. Hybrid artificial neural network based on BP-PLSR and its application in development of soft sensors[J]. Chemometrics and Intelligent Laboratory Systems, 2010, 103(2): 152-159.
8	Wang Y, Chen C, Yan X F. Structure and weight optimization of neural network based on CPA-MLR and its application in naphtha dry point soft sensor[J]. Neural Computing and Applications, 2013, 22(1): 75-82.
9	Wang J S, Shen N N, Ren X D, et al. RBF neural network soft-sensor modeling of rotary kiln pellet quality indices optimized by biogeography-based optimization algorithm[J]. Journal of Chemical Engineering of Japan, 2015, 48(1): 7-15.
10	李军, 桑桦. 基于SCKF的Elman递归神经网络在软测量建模中的应用[J]. 信息与控制, 2017, 46(3): 342-349+357.
	Li J, Sang H. Application of Elman recurrent neural network based on SCKF in soft sensor modeling [J]. Information and Control, 2017, 46 (3): 342-349 + 357.
11	谢一飞. 基于神经网络和支持向量机的蔗渣锅炉烟气氧含量软测量模型[D]. 南宁: 广西大学, 2016.
	Xie Y F. Soft sensing model of oxygen content in flue gas of bagasse boiler based on neural network and support vector machine[D]. Nanning: Guangxi University, 2016.
12	李丁园. 回声状态网络结构设计及应用研究[D]. 吉林: 吉林大学, 2019.
	Li D Y. Design and application of echo state network structure[D]. Jilin: Jilin University, 2019.
13	乔俊飞, 李瑞祥, 柴伟, 等. 基于PSO-ESN神经网络的污水BOD预测[J]. 控制工程, 2016, 23(4): 463-467.
	Qiao J F, Li R X, Chai W, et al. BOD prediction of sewage based on PSO-ESN neural network[J]. Control Engineering of China, 2016, 23 (4): 463-467.
14	Luciano S, David A, Jose O, et al. Assessing the health of LiFePO4 traction batteries through monotonic echo state networks[J]. Sensors, 2018, 18(1): 9-16.
15	Antonelo E A, Camponogara E, Foss B. Echo-state networks for data-driven downhole pressure estimation in gas-lift oil wells[J]. Neural Networks, 2017, 85(1): 106-117.
16	Muhammad Q R, Nadarajah M, Li J, et al. Multivariate ensemble forecast framework for demand prediction of anomalous days[J]. IEEE Transactions on Sustainable Energy, 2020, 11(1): 27-36.
17	Sun W, Wang J. Elman neural network soft-sensor model of conversion velocity in polymerization process optimized by chaos whale optimization algorithm[J]. IEEE Access, 2017, 5: 13062-13076.
18	耿志强, 徐猛, 朱群雄, 等. 基于深度学习的复杂化工过程软测量模型研究与应用[J]. 化工学报, 2019, 70(2): 564-571.
	Geng Z Q, Xu M, Zhu Q X, et al. Research and application of soft sensing model for complex chemical process based on deep learning[J]. CIESC Journal, 2019, 70 (2): 564-571.
19	伊金静. 基于深度学习的工业过程软测量[D]. 杭州: 浙江大学, 2019.
	Yi J J. Soft measurement of industrial process based on deep learning[D]. Hangzhou: Zhejiang University, 2019.
20	Wang Y, Shen Y X, Mao S W, et al. LASSO and LSTM integrated temporal model for short-term solar intensity forecasting[J]. IEEE Internet of Things Journal, 2019, 6(2): 2933-2944.
21	Cheng Y W, Zhu H P, Wu J, et al. Machine health monitoring using adaptive kernel spectral clustering and deep long short-term memory recurrent neural networks[J]. IEEE Transactions on Industrial Informatics, 2019, 15(2): 987-997.
22	Zhang Q, Li F, Long F, et al. Vehicle emission forecasting based on wavelet transform and long short-term memory network[J]. IEEE Access, 2018, 6: 56984-56994.
23	Qin D M, Yu J, Zou G J, et al. A novel combined prediction scheme based on CNN and LSTM for urban PM2. 5 concentration[J]. IEEE Access, 2019, 7: 20050-20059.
24	邵伟明, 葛志强, 李浩, 等. 基于循环神经网络的半监督动态软测量建模方法[J]. 电子测量与仪器学报, 2019, 33(11): 7-13.
	Shao W M, Ge Z Q, Li H, et al. Modeling method of semi supervised dynamic soft measurement based on cyclic neural network [J]. Journal of Electronic Measurement and Instrumentation, 2019, 33 (11): 7-13.
25	熊伟丽, 薛明晨, 李妍君. 基于EM算法的半监督局部加权PLS在线建模方法[J]. 系统仿真学报, 2018, 30(1): 8-17.
	Xiong W L, Xue M C, Li Y J. Semi supervised partial weighted PLS online modeling method based on EM algorithm [J]. Journal of System Simulation, 2018, 30 (1): 8-17
26	李浩. 基于深度学习的半监督软测量建模方法[D]. 杭州: 浙江大学, 2019.
	Li H. Modeling method of semi supervised soft sensing based on deep learning[D]. Hangzhou: Zhejiang University, 2019.
27	Bao L, Yuan X, Ge Z. Co-training partial least squares model for semi-supervised soft sensor development[J]. Chemometrics & Intelligent Laboratory Systems, 2015, 147(5): 75-85.
28	Chen X J, Yuan G W, Nie F P, et al. Semi-supervised feature selection via sparse rescaled linear square regression[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 32(1): 165-176.
29	Ge Z, Huang B, Song Z. Nonlinear semisupervised principal component regression for soft sensor modeling and its mixture form[J]. Journal of Chemometrics, 2015, 28(11): 793-804.
30	Ersin Y, Dursun A. Estimation of right censored nonparametric regression solved by kNN imputation: a comparative study[J]. Turkiye Klinikleri Journal of Biostatistics, 2019, 11(2): 83-92.

[1]	Zhewen CHEN, Junjie WEI, Yuming ZHANG. System integration and energy conversion mechanism of the power technology with integrated supercritical water gasification of coal and SOFC [J]. CIESC Journal, 2023, 74(9): 3888-3902.
[2]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[3]	Yuyuan ZHENG, Zhiwei GE, Xiangyu HAN, Liang WANG, Haisheng CHEN. Progress and prospect of medium and high temperature thermochemical energy storage of calcium-based materials [J]. CIESC Journal, 2023, 74(8): 3171-3192.
[4]	Guixian LI, Abo CAO, Wenliang MENG, Dongliang WANG, Yong YANG, Huairong ZHOU. Process design and evaluation of CO₂ to methanol coupled with SOEC [J]. CIESC Journal, 2023, 74(7): 2999-3009.
[5]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[6]	Cheng YUN, Qianlin WANG, Feng CHEN, Xin ZHANG, Zhan DOU, Tingjun YAN. Deep-mining risk evolution path of chemical processes based on community structure [J]. CIESC Journal, 2023, 74(4): 1639-1650.
[7]	Yalin WANG, Yuqing PAN, Chenliang LIU. Intermittent process monitoring based on GSA-LSTM dynamic structure feature extraction [J]. CIESC Journal, 2022, 73(9): 3994-4002.
[8]	Le ZHOU, Chengkai SHEN, Chao WU, Beiping HOU, Zhihuan SONG. Deep fusion feature extraction network and its application in chemical process soft sensing [J]. CIESC Journal, 2022, 73(7): 3156-3165.
[9]	Kun WANG, Hongbo SHI, Shuai TAN, Bing SONG, Yang TAO. Local time difference constrained neighborhood preserving embedding algorithm for fault detection [J]. CIESC Journal, 2022, 73(7): 3109-3119.
[10]	Xin ZHANG, Li ZHOU, Shihui WANG, Xu JI, Kexin BI. Integrated optimization of refinery hydrogen networks with crude oil properties fluctuations [J]. CIESC Journal, 2022, 73(4): 1631-1646.
[11]	Wenliang MENG, Guixian LI, Huairong ZHOU, Jingwei LI, Jian WANG, Ke WANG, Xueying FAN, Dongliang WANG. A novel coal to methanol process with near zero CO₂ emission by pulverized coal gasification integrated green hydrogen [J]. CIESC Journal, 2022, 73(4): 1714-1723.
[12]	Huan GAO, Guoliang DING, Faxian ZHOU, Dawei ZHUANG. Research on dynamic separation characteristics of R410A refrigerant with lubricant [J]. CIESC Journal, 2022, 73(3): 1054-1062.
[13]	Xingshuo ZHANG, Xionglin LUO, Feng XU. Simulation closer to commercial process and prior process analysis based control loop configuration of FCCU reactor-regenerator system [J]. CIESC Journal, 2022, 73(2): 747-758.
[14]	Dehong WANG, Lin SUN, Xionglin LUO. Full-cycle slow-lift limited optimization analysis of multi-effect distillation heat transfer temperature difference in seawater desalination system [J]. CIESC Journal, 2022, 73(12): 5469-5482.
[15]	Dong JI, Jian WANG, Ke WANG, Jingwei LI, Wenliang MENG, Yong YANG, Guixian LI, Dongliang WANG, Huairong ZHOU. Process research of methanol production by CO₂ coupled green hydrogen with different CO₂ capture technologies [J]. CIESC Journal, 2022, 73(10): 4565-4575.