Automatic structure and parameters tuning method for deep neural network soft sensor in chemical industries

doi:10.11949/j.issn.0438-1157.20171435

Abstract

Abstract:

Deep learning has been applied to the field of soft sensing in process industries. However, the structure and parameters of deep neural network (DNN) have to be tuned manually, which require solid fundamental knowledge about machine learning and rich experiences on parameters tuning. Complicated tuning procedure restricts generalization application of deep learning in chemical industries. A structure and parameters tuning method for DNN soft sensor with little manual intervention was proposed by systematic analysis on selection process of each essential DNN parameter from massive experiments. The presented method could greatly simplify the tuning procedure and offer a reference for engineers to study and use deep learning. Studies on crude-oil distillation and coal gasification process verified effectiveness and generality of the proposed method.

Key words: deep learning, prediction, parameter tuning, algorithm, neural network

摘要：

深度学习在流程工业的软测量领域已经得到了应用。然而，深度神经网络（DNN）的结构和参数需要人工调整，这需要扎实的机器学习知识基础和丰富的参数调整经验，烦琐的调整过程限制了深度学习在化工领域的推广应用。在大量实验的基础上，对DNN的每个关键参数的选取过程进行了系统化的分析，提出了几乎无须人工干预的基于DNN软测量的结构和参数自动调整方法，极大地简化了参数调整过程，能够给工程技术人员学习及应用深度学习提供参考。对原油蒸馏装置及煤气化装置的案例分析验证了所提出方法的有效性和通用性。

关键词: 深度学习, 预测, 参数调整, 算法, 神经网络

CLC Number:

TP273

WANG Kangcheng, SHANG Chao, KE Wensi, JIANG Yongheng, HUANG Dexian. Automatic structure and parameters tuning method for deep neural network soft sensor in chemical industries[J]. CIESC Journal, 2018, 69(3): 900-906.

王康成, 尚超, 柯文思, 江永亨, 黄德先. 化工过程深度神经网络软测量的结构与参数自动调整方法[J]. 化工学报, 2018, 69(3): 900-906.

References 30

[1]	黄德先, 王京春, 金以慧. 过程控制系统[M]. 北京:清华大学出版社, 2011. HUANG D X, WANG J C, JIN Y H. Process Control Systems[M]. Beijing:Tsinghua University Press, 2011.
[2]	黄德先, 叶心宇, 竺建敏, 等. 化工过程先进控制[M]. 北京:化学工业出版社, 2006. HUANG D X, YE X Y, ZHU J M, et al. Advanced Process Control in Chemical Industrial Processes[M]. Beijing:Chemical Industry Press, 2006.
[3]	GALICIA H J, HE Q P, WANG J. A reduced order soft sensor approach and its application to a continuous digester[J]. Journal of Process Control, 2011, 21(4):489-500.
[4]	QIN S J. Neural networks for intelligent sensors and control-practical issues and some solutions[J]. Neural Systems for Control, 1997, 8(1):213-234.
[5]	LLDIKO E F, JEROME H F. A statistical view of some chemometrics regression tools[J]. Technometrics, 1993, 35(2):109-135.
[6]	KADLEC P, GABRYS B, STRANDT S. Data-driven soft sensors in the process industry[J]. Computers & Chemical Engineering, 2009, 33(4):795-814.
[7]	LIN B, RECKE B, KNUDSEN J K H, et al. A systematic approach for soft sensor development[J]. Computers & Chemical Engineering, 2007, 31(5):419-425.
[8]	PARK S, HAN C. A nonlinear soft sensor based on multivariate smoothing procedure for quality estimation in distillation columns[J]. Computers & Chemical Engineering, 2000, 24(2):871-877.
[9]	YANG Y, CHAI T. Soft sensing based on artificial neural network[C]//American Control Conference, 1997. Proceedings of the IEEE. 2002:674-678.
[10]	QIN S J, MCAVOY T J. Nonlinear PLS modeling using neural networks[J]. Computers & Chemical Engineering, 1992, 16(4):379-391.
[11]	YAN W, SHAO H, WANG X. Soft sensing modeling based on support vector machine and Bayesian model selection[J]. Computers & Chemical Engineering, 2004, 28(8):1489-1498.
[12]	HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.
[13]	BENGIO Y. Learning deep architectures for AI[J]. Foundations & Trends® in Machine Learning, 2009, 2(1):1-127.
[14]	PAN Z, LIU Y, LIU G, et al. Topic Network:Topic Model with Deep Learning for Image Classification[C]//International Conference on Knowledge Science, Engineering and Management, 2015:65-73.
[15]	DENG L, HINTON G, KINGSBURY B. New types of deep neural network learning for speech recognition and related applications:an overview[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE. 2013:8599-8603.
[16]	MNIH A, HINTON G. A scalable hierarchical distributed language model[C]//International Conference on Neural Information Processing Systems. Curran Associates Inc. 2008:1081-1088.
[17]	SHANG C, YANG F, HUANG D, et al. Data-driven soft sensor development based on deep learning technique[J]. Journal of Process Control, 2014, 24(3):223-233.
[18]	MAO K Z, TAN K C, SER W. Probabilistic neural-network structure determination for pattern classification[J]. IEEE Transactions on Neural Networks, 2000, 11(4):1009-1016.
[19]	LEUNG F H F, LAM H K, LING S H, et al. Tuning of the structure and parameters of a neural network using an improved genetic algorithm[J]. IEEE Trans. Neural Netw., 2003, 14(1):79-88.
[20]	TSAI J T, CHOU J H, LIU T K. Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm[J]. IEEE Transactions on Neural Networks, 2006, 17(1):69-80.
[21]	BENGIO Y, GUYON G, DROR V, et al. Deep learning of representations for unsupervised and transfer learning[C]//Workshop on Unsupervised & Transfer Learning. 2012.
[22]	BERGSTRA J, BENGIO Y. Random search for hyper-parameter optimization[J]. Journal of Machine Learning Research, 2012, 13(1):281-305.
[23]	BENGIO Y. Practical Recommendations for Gradient-Based Training of Deep Architectures[M]//Neural Networks:Tricks of the Trade. Berlin:Springer Berlin Heidelberg, 2012:133-144.
[24]	BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[J]. Advances in Neural Information Processing Systems, 2007, 19(1):153-160.
[25]	周志华. 机器学习[M]. 北京:清华大学出版社, 2016. ZHOU Z H. Machine Learning[M]. Beijing:Tsinghua University Press, 2016.
[26]	LV F, WEN C, BAO Z, et al. Fault diagnosis based on deep learning[C]//American Control Conference. 2016:6851-6856.
[27]	CARREIRA-PERPINAN M A, HINTON G E. On contrastive divergence learning[C]//Artificial Intelligence & Statistics. 2005.
[28]	HINTON G. A practical guide to training restricted Boltzmann machines[J]. Momentum, 2010, 9(1):926-946.
[29]	MONAGHAN R F D. Dynamic reduced order modeling of entrained flow gasifiers[D]. Cambridge:Massachusetts Institute of Technology, 2010.
[30]	JI P, GAO X, HUANG D, et al. Prediction of syngas compositions in shell coal gasification process via dynamic soft-sensing method[C]//IEEE International Conference on Control and Automation. IEEE. 2013:244-249.