基于LSTM时间序列重建的生产装置异常检测

doi:10.11949/j.issn.0438-1157.20181050

摘要/Abstract

摘要：

工业生产装置通常设置传感器报警阈值进行报警，但是对处于报警阈值以下的时间序列异常难以及时捕捉。基于统计的传统检测方法在解决时间序列异常检测上存在很大挑战，因此提出基于long short term memory (LSTM)时间序列重建的方法进行生产装置的异常检测。该算法首先引入一层LSTM网络对传感器数据的时间序列进行向量表示，采用另一层LSTM网络对时间序列进行逆序重建，然后利用重建值与实际值之间的误差，通过极大似然估计方法对该段序列进行异常概率估计，最终通过学习异常报警阈值实现时间序列异常检测。采用ECG测试数据、能源数据与危险品储罐传感器数据进行了仿真实验，验证了所提方法在不同长度的数据上的有效性。

关键词: 算法, 神经网络, 参数估计, LSTM, 时间序列, 异常检测, 极大似然估计

Abstract:

Industrial production equipment usually sets sensor alarm thresholds for alarms, but it is difficult to capture time series abnormalities below the alarm thresholds. The traditional statistics based detection method has great challenges in these time series anomaly detection. In this paper, an approach to the anomaly detection of process units is proposed by using the long short term memory (LSTM) time series reconstruction. At first, an LSTM network is introduced to vectorize the time series of sensor data, and another LSTM network is utilized to reconstruct the time series in reverse sequence. Then, the errors between the reconstructed values and the actual values are used to estimate the anomaly probability by the maximum likelihood estimation method. Eventually, anormaly detection is achieved by learning the abnormal alarm thresholds. Simulation resutls on the ECG standard testing data, energy data and sensor data of the dangerous goods tank have shown the effectiveness of the proposed method on data with different lengths.

Key words: algorithm, neural networks, parameter estimation, LSTM, time series, anomaly detection, maximum likelihood estimation

中图分类号:

TP 391

窦珊, 张广宇, 熊智华. 基于LSTM时间序列重建的生产装置异常检测[J]. 化工学报, 2019, 70(2): 481-486.

Shan DOU, Guangyu ZHANG, Zhihua XIONG. Anomaly detection of process unit based on LSTM time series reconstruction[J]. CIESC Journal, 2019, 70(2): 481-486.

图/表 7

图1 LSTM网络存储模块

Fig.1 LSTM memory block

图2 算法基本框架

Fig.2 Overall structure

表1 MLE参数估计步骤[25]

Table 1 Procedure of MLE parameter estimation[25]

1.设异常概率 $p$ 随重建误差e呈高斯分布，则其似然函数如式(4)所示；

2.对似然函数取对数并求和，整理后为

$H (μ, Σ) = ∑ e i ∈ V 1 l n p (e i | μ, Σ)$ (8)

3.对 $H μ, Σ$ 求偏导，使得其满足：

$? H (μ, Σ) ? μ = 0 ? H (μ, Σ) ? Σ = 0$

4.求解上述方程得到参数估计值如下所示，其中N为数据集 $N V 1$ 的时间点总数。

$μ = 1 N ∑ i = 1 N e i Σ = 1 N ∑ i = 1 N (e i - μ) (e i - μ) T$ (9)

表1 MLE参数估计步骤[25]

Table 1 Procedure of MLE parameter estimation[25]

1.设异常概率 $p$ 随重建误差e呈高斯分布，则其似然函数如式(4)所示；

2.对似然函数取对数并求和，整理后为

$H (μ, Σ) = ∑ e i ∈ V 1 l n p (e i | μ, Σ)$ (8)

3.对 $H μ, Σ$ 求偏导，使得其满足：

$? H (μ, Σ) ? μ = 0 ? H (μ, Σ) ? Σ = 0$

4.求解上述方程得到参数估计值如下所示，其中N为数据集 $N V 1$ 的时间点总数。

$μ = 1 N ∑ i = 1 N e i Σ = 1 N ∑ i = 1 N (e i - μ) (e i - μ) T$ (9)

图3 ECG标准测试数据

Fig.3 ECG standard test data

图4 能源用量数据

Fig.4 Energy consumption data

图5 危险品储罐传感器数据

Fig.5 Dangerous goods storage sensor data

表2 三组数据集异常检验结果

Table 2 Experimental results

Data	P		R		$F 0.1$
Data	RNN	LSTM	RNN	LSTM	RNN	LSTM
ECG	0.98	1.00	0.45	0.43	0.96	0.97
EN	0.89	0.94	0.10	0.12	0.83	0.88
SE	0.89	0.92	0.20	0.26	0.86	0.89

表2 三组数据集异常检验结果

Table 2 Experimental results

Data	P		R		$F 0.1$
Data	RNN	LSTM	RNN	LSTM	RNN	LSTM
ECG	0.98	1.00	0.45	0.43	0.96	0.97
EN	0.89	0.94	0.10	0.12	0.83	0.88
SE	0.89	0.92	0.20	0.26	0.86	0.89

参考文献 30

1	周大镯, 刘月芬, 马文秀. 时间序列异常检测[J]. 计算机工程与应用, 2008, 44(35): 145-147.
	ZhouD Z, LiuY F, MaW X. Effective time series outlier detection algorithm based on segmentation[J]. Computer Engineering and Applications, 2008, 44(35): 145-147.
2	HawkinsD M. Identification of Outliers[M]. London: Chapman and Hall, 1980: 25.
3	BillorN, HadiA S, VellemanP F. BACON: blocked adaptive computationally efficient outlier nominators[J]. Computational Statistics & Data Analysis, 2000, 34(3): 279-298.
4	KnorrE M, NgR T. A unified notion of outliers: properties and computation[C]//KDD. 1997: 219-222.
5	李钢, 周东华. 基于SPM的多变量连续过程在线故障预测方法[J]. 化工学报, 2008, 59(7): 1829-1833.
	LiG, ZhouD H. SPM-based online fault prediction approach for multivariate continuous processes[J]. Journal of Chemical Industry and Engineering(China), 2008, 59(7): 1829-1833.
6	张浩, 刘振娟, 李宏光, 等. 基于关联变量时滞分析卷积神经网络的生产过程时间序列预测方法[J]. 化工学报, 2017, 68(9): 3501-3510.
	ZhangH, LiuZ J, LiH G, et al. Process time series prediction based on application of correlated process variables to CNN time delayed analyses[J]. CIESC Journal, 2017, 68(9): 3501-3510.
7	PageE S. Continuous inspection schemes[J]. Biometrika, 1954, 41(1/2): 100-115.
8	BassevilleM, NikiforovI V. Detection of Abrupt Changes: Theory and Application[M]. Englewood Cliffs: Prentice Hall, 1993: 30.
9	MaJ, PerkinsS. Time-series novelty detection using one-class support vector machines[C]//Neural Networks, 2003. Proceedings of the International Joint Conference on. IEEE, 2003: 1741-1745.
10	王少军. 时间序列预测的可重构计算研究[D]. 哈尔滨:哈尔滨工业大学, 2012: 35.
	WangS J. Research on reconfigurable computing for time series forecasting[D]. Harbin: Harbin Institute of Technology, 2012: 35.
11	HochreiterS, SchmidhuberJ. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
12	SutskeverI, VinyalsO, LeQ V. Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems, 2014: 3104-3112.
13	MiwaM, BansalM. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016: 1105-1116.
14	ChoiY, BreckE, CardieC. Joint extraction of entities and relations for opinion recognition[C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2006: 431-439.
15	GravesA, SchmidhuberJ. Offline handwriting recognition with multidimensional recurrent neural networks[C]//Advances in Neural Information Processing Systems, 2009: 545-552.
16	MessinaR, LouradourJ. Segmentation-free handwritten Chinese text recognition with LSTM-RNN[C]//Document Analysis and Recognition (ICDAR), 2015 13th International Conference on. IEEE, 2015: 171-175.
17	HuangZ, XuW, YuK. Bidirectional LSTM-CRF models for sequence tagging[J]. arXivpreprint arXiv: 1508.01991, 2015.
18	MaX, HovyE. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016: 1064-1074.
19	ChoK, van MerrienboerB, BahdanauD, et al. On the properties of neural machine translation: encoder–decoder approaches[C]//Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, 2014: 103-111.
20	ChoK, VanM B, GulcehreC, et al. Learning phrase representations using RNN encoder–decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1724-1734.
21	CybenkoG. Approximation by superpositions of a sigmoidal function[J]. Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314.
22	MaasA L, HannunA Y, NgA Y. Rectifier nonlinearities improve neural network acoustic models[J]. Proc. Icml., 2013, 30(1): 3.
23	GravesA. Long short-term memory[M]//Supervised Sequence Labelling with Recurrent Neural Networks. Heidelberg, Berlin: Springer, 2012: 37-45.
24	ChungJ, GulcehreC, ChoK H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXivpreprint arXiv: 1412.3555, 2014.
25	BockR D, AitkinM. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm[J]. Psychometrika, 1981, 46(4): 443-459.
26	MalhotraP, VigL, ShroffG, et al. Long short term memory networks for anomaly detection in time series[C]//23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015. Bruges, Belgium, 2015: 89.
27	GoutteC, GaussierE. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation[C]//European Conference on Information Retrieval. Heidelberg, Berlin: Springer, 2005: 345-359.
28	LeQ V, NgiamJ, CoatesA, et al. On optimization methods for deep learning[C]//Proceedings of the 28th International Conference on International Conference on Machine Learning. Omnipress, 2011: 265-272.
29	KingmaD P, BaJ. Adam: a method for stochastic optimization[J]. arXivpreprint arXiv: 1412.6980, 2014.
30	ConnorJ T, MartinR D, AtlasL E. Recurrent neural networks and robust time series prediction[J]. IEEE Transactions on Neural Networks, 1994, 5(2): 240-254.

[1]	温凯杰, 郭力, 夏诏杰, 陈建华. 一种耦合CFD与深度学习的气固快速模拟方法[J]. 化工学报, 2023, 74(9): 3775-3785.
[2]	曹跃, 余冲, 李智, 杨明磊. 工业数据驱动的加氢裂化装置多工况切换过渡状态检测[J]. 化工学报, 2023, 74(9): 3841-3854.
[3]	诸程瑛, 王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报, 2023, 74(8): 3429-3437.
[4]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[5]	尹刚, 李伊惠, 何飞, 曹文琦, 王民, 颜非亚, 向禹, 卢剑, 罗斌, 卢润廷. 基于KPCA和SVM的铝电解槽漏槽事故预警方法[J]. 化工学报, 2023, 74(8): 3419-3428.
[6]	徐野, 黄文君, 米俊芃, 申川川, 金建祥. 多源信息融合的离心式压缩机喘振诊断方法[J]. 化工学报, 2023, 74(7): 2979-2987.
[7]	高学金, 姚玉卓, 韩华云, 齐咏生. 基于注意力动态卷积自编码器的发酵过程故障监测[J]. 化工学报, 2023, 74(6): 2503-2521.
[8]	周继鹏, 何文军, 李涛. 异形催化剂上乙烯催化氧化失活动力学反应工程计算[J]. 化工学报, 2023, 74(6): 2416-2426.
[9]	黄磊, 孔令学, 白进, 李怀柱, 郭振兴, 白宗庆, 李平, 李文. 油页岩添加对准东高钠煤灰熔融行为影响的研究[J]. 化工学报, 2023, 74(5): 2123-2135.
[10]	贠程, 王倩琳, 陈锋, 张鑫, 窦站, 颜廷俊. 基于社团结构的化工过程风险演化路径深度挖掘[J]. 化工学报, 2023, 74(4): 1639-1650.
[11]	郑书闽, 郭鹏程, 颜建国, 王帅, 李文博, 周淇. 微小通道内过冷流动沸腾阻力特性实验及预测研究[J]. 化工学报, 2023, 74(4): 1549-1560.
[12]	吴心远, 刘奇磊, 曹博渊, 张磊, 都健. Group2vec：基于无监督机器学习的基团向量表示及其物性预测应用[J]. 化工学报, 2023, 74(3): 1187-1194.
[13]	张江淮, 赵众. 碳三加氢装置鲁棒最小协方差约束控制及应用[J]. 化工学报, 2023, 74(3): 1216-1227.
[14]	顾学荣, 刘硕士, 杨思宇. 基于并行EGO和代理模型辅助的多参数优化方法研究[J]. 化工学报, 2023, 74(3): 1205-1215.
[15]	吴选军, 王超, 曹子健, 蔡卫权. 数据与物理信息混合驱动的固定床吸附穿透深度学习模型[J]. 化工学报, 2023, 74(3): 1145-1160.