Regularization based functional link neural network and its applications to modeling complex chemical processes

doi:10.11949/0438-1157.20191474

Abstract

Abstract:

In the modeling of chemical process, due to the high dimensionality and non-linearity of the process data, the calculation amount is greatly increased and the modeling difficulty is increased. In order to solve this problem, a regularization based functional link neural network (RFLNN) is proposed. In the proposed RFLNN method, there are two salient features through using the regularization

method

on one hand, computing complexity and the amount of calculation are greatly reduced; on the other hand, the problem of local extreme values and over-fitting is effectively avoided. As a result, the performance in terms of accuracy and learning speed of functional neural network is much improved. In order to verify the effectiveness of the proposed RFLNN method, firstly, an UCI dataset called Real estate valuation is selected; then the proposed RFLNN method is used to develop a model for the complex production process of high density polyethylene (HDPE). Compared with the conventional functional link neural network(FLNN), simulation results of the selected UCI data and industrial data show that the proposed RFLNN can achieve not only fast convergence speed but also high accuracy in processing complex chemical process data.

Key words: neural network, regularization, process modeling, high density polyethylene

摘要：

在化工过程的建模中，由于过程数据的高维度和高非线性，导致计算量大幅提升和建模难度加大。为了解决这一问题，提出了一种基于正则化方法的函数连接神经网络模型（regularization based functional link neural network, RFLNN）。所提出的RFLNN方法里，通过使用正则化的方法对函数连接神经网络的权值进行优化，一方面大幅降低网络计算复杂度和计算量，另一方面极大程度上克服网络局部极值和过拟合的问题，以提高函数连接神经网络的学习速度和精度。为了验证所提出方法的有效性，首先采用UCI数据中Real estate valuation数据对其性能进行测试；随后将所提的方法应用于高密度聚乙烯（high density polyethylene，HDPE）复杂生产过程进行建模。UCI标准数据与工业数据的仿真结果表明，与传统FLNN对比，RFLNN在处理高维复杂化工过程数据时具有收敛速度快、建模精度高等特点。

关键词: 神经网络, 正则化, 过程建模, 高密度聚乙烯

CLC Number:

TQ 29

Yanlin HE, Ye TIAN, Xiangbai GU, Yuan XU, Qunxiong ZHU. Regularization based functional link neural network and its applications to modeling complex chemical processes[J]. CIESC Journal, 2020, 71(3): 1072-1079.

贺彦林, 田业, 顾祥柏, 徐圆, 朱群雄. 基于正则化的函数连接神经网络研究及其复杂化工过程建模应用[J]. 化工学报, 2020, 71(3): 1072-1079.

Figures/Tables 10

Fig.1 Structure of the FLNN model

Fig.2 Structure of R-FLNN model

Fig.3 Flowchart of R-FLNN algorithm

Fig.4 Residual error distribution of real estate valuation data training examples with two models

Table 1 Performance comparisons of models for estate valuation data

模型	网络训练平均相对误差/%	网络泛化平均相对误差/%
FLNN	14.405	14.820
RFLNN	13.294	13.734

Fig.5 Comparisons of training examples with two models

Fig.6 Residual error distribution of training examples with two models

Fig.7 Comparisons of testing examples with two models

Fig.8 Residual error distribution of testing examples with two models

Table 2 Comparisons of model performance for HDPE samples

模型	网络训练平均相对误差/%	网络泛化平均相对误差/%	网络训练时间/s	网络泛化时间/s
FLNN	0.0214	0.0505	0.3918	0.000418
RFLNN	0.0156	0.0365	0.0024	0.000402

References 30

1	Nastase V, Strube M. Transforming Wikipedia into a large scale multilingual concept network[J]. Artificial Intelligence, 2013, 194: 62-85．
2	Li B Y, Li Y X, Wang H G, et al. Compensation of automatic weighing error of belt weigher based on BP neural network[J]．Measurement, 2018, 129: 625-632．
3	Ming Y W, Zhu E, Wang M, et al. DMP-ELMs: data and model parallel extreme learning machines for large-scale learning tasks[J]． Neurocomputing, 2018, 320: 85-97．
4	Ding S F, Su C Y, Yu J Z. An optimizing BP neural network algorithm based on genetic algorithm[J]. Artificial Intelligence Review, 2011, 36(2): 153-162.
5	Moallem P, Monadjemi S A. An efficient MLP-learning algorithm using parallel tangent gradient and improved adaptive learning rates[J]. Connection Science, 2010, 22(4): 373-392.
6	Zhao H Q, Zeng X P, Zhang J S. Adaptive reduced feedback FLNN filter for active control of nonlinear noise process[J]．Digital Signal Processing, 2010, 90: 834-847.
7	朱群雄, 张晓晗, 顾祥柏, 等. 基于特征提取的函数连接神经网络研究及其化工过程建模应用[J]. 化工学报, 2017, 69(3): 907-912.
	Zhu Q X, Zhang X H, Gu X B, et al. Research and application of feature extraction derived functional link neural network[J]. Journal of Chemical Industry and Engineering, 2017, 69(3): 907-912.
8	Zhao H Q, Zeng X P, Zhang J S, et al. An adaptive decision feedback equalizer based on the combination of the FIR and FLNN[J]. Digital Signal Processing, 2011, 21: 679-689.
9	Pao Y. Adaptive pattern recognition and neural networks[J]. Neural Networks, 1989, 4(1): 124-126.
10	He Y L, Xu Y, Geng Z Q, et al. Hybrid robust model based on an improved functional link neural network integrating with partial least square (IFLNN-PLS) and its application to predicting key process variables[J]. ISA Transactions, 2016, 61: 155-166.
11	Vuković N, Petrović M, Miljković Z. A comprehensive experimental evaluation of orthogonal polynomial expanded random vector functional link neural networks for regression[J]. Applied Soft Computing, 2018, 70: 1083-1096.
12	Kumar L, Rath S K. Hybrid functional link artificial neural network approach for predicting maintainability of object-oriented software[J]. Journal of Systems and Software, 2016, 121: 170-190.
13	Majhi R, Panda G, Sahoo G. Development and performance evaluation of FLANN based model for forecasting of stock markets[J]. Expert systems with Applications, 2009, 36(3): 6800-6808.
14	Dehuri S, Roy R, Cho S B, et al. An improved swarm optimized functional link artificial neural network (ISO-FLANN) for classification[J]. Journal of Systems and Software, 2012, 85(6): 1333-1345.
15	Wu C F, Lin C J, Lee C Y. A functional neural fuzzy network for classification applications[J]. Expert Systems with Applications, 2011, 38(5): 6202-6208.
16	Yang X J, Wang L. A modified Tikhonov regularization method[J]. Journal of Computational and Applied Mathematics, 2015, 288: 180-192.
17	杨浩, 马建红. 正则化参数求解方法研究[J]. 计算机测量与控制, 2017, 25(8): 226-229.
	Yang H, Ma J H. Research on method of regularization parameter solution[J]. Computer Measurement & Control, 2017, 25(8): 226-229.
18	Iosifidis A, Tefas A, Pitas I. DropELM: fast neural network regularization with Dropout and DropConnect[J]. Neurocomputing, 2015, 162: 57-66.
19	Gu B, Sheng V S. A robust regularization path algorithm for ν-support vector classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 28(5): 1241-1248.
20	Das A, Maiti J, Banerjee R N. Process control strategies for a steel making furnace using ANN with Bayesian regularization and ANFIS[J]. Expert systems with Applications, 2010, 37(2): 1075-1085.
21	Scardapane S, Comminiello D, Hussain A, et al. Group sparse regularization for deep neural networks[J]. Neurocomputing, 2017, 241: 81-89.
22	Kumar M, Singh S, Rath S K. Classification of microarray data using functional link neural network[J]. Procedia Computer Science, 2015, 57: 727-737.
23	李海波, 潘丰. 基于FLNN的多粘菌素发酵过程建模[J]. 江南大学学报（自然科学版）, 2004, 3(3): 256-260.
	Li H B, Pan F. Modeling the mycetozoan fermentation based on the functional-linked neural network[J]. Journal of Southern Yangtze University (Natural Science Edition), 2004, 3(3): 256-260.
24	Dehuri S, Cho S B. Evolutionarily optimized features in functional link neural network for classification[J]. Expert Systems with Applications, 2010, 37(6): 4379-4391.
25	吕久旭, 吴乐南. 基于函数连接型神经网络的非线性滤波[J]. 长春工业大学学报, 2006, 27(4): 305-307.
	Lyu J X, Wu L N. Nonlinear filtering based on function connected neural network[J]. Journal of Changchun University of Technology, 2006, 27(4): 305-307.
26	叶世伟, 史忠植. 神经网络原理[M]. 北京: 机械工业出版社, 2004: 10-12.
	Ye S W, Shi Z Z. Neural Networks: A Comprehensive Foundation[M]. Beijing: China Machine Press, 2004: 10-12.
27	Ma C, Zhao L, Mei X, et al. Thermal error compensation of high-speed spindle system based on a modified BP neural network[J]. The International Journal of Advanced Manufacturing Technology, 2017, 89(9/10/11/12): 3071-3085.
28	时慧焯, 王希诚. 基于改进的BP神经网络的注塑成型翘曲优化设计[J]. 化工学报, 2011, 62(9): 2562-2567.
	Shi H Z, Wang X C. Warpage optimization of injection modeling based on improved BP neural network[J]. CIESC Journal, 2011, 62(9): 2562-2567.
29	Ghazali R, Bakar Z A, Hassim Y M M, et al. Functional link neural network with modified cuckoo search training algorithm for physical time series forecasting[C]//International Conference on Intelligent Computing. Springer, Cham, 2014: 285-291.
30	Haykin S. 神经网络与机器学习[M]. 申富饶, 徐烨, 郑俊, 等译. 北京: 机械工业出版社, 2011: 198-210.
	Haykin S. Neural Networks and Learning Machines[M]. Shen F R, Xu Y, Zheng J, et al. trans. Beijing: China Machine Press, 2011: 198-210.

[1]	Kaijie WEN, Li GUO, Zhaojie XIA, Jianhua CHEN. A rapid simulation method of gas-solid flow by coupling CFD and deep learning [J]. CIESC Journal, 2023, 74(9): 3775-3785.
[2]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[3]	Chengying ZHU, Zhenlei WANG. Operation optimization of ethylene cracking furnace based on improved deep reinforcement learning algorithm [J]. CIESC Journal, 2023, 74(8): 3429-3437.
[4]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[5]	Ye XU, Wenjun HUANG, Junpeng MI, Chuanchuan SHEN, Jianxiang JIN. Surge diagnosis method of centrifugal compressor based on multi-source data fusion [J]. CIESC Journal, 2023, 74(7): 2979-2987.
[6]	Xuejin GAO, Yuzhuo YAO, Huayun HAN, Yongsheng QI. Fault monitoring of fermentation process based on attention dynamic convolutional autoencoder [J]. CIESC Journal, 2023, 74(6): 2503-2521.
[7]	Lei HUANG, Lingxue KONG, Jin BAI, Huaizhu LI, Zhenxing GUO, Zongqing BAI, Ping LI, Wen LI. Effect of oil shale addition on ash fusion behavior of Zhundong high-sodium coal [J]. CIESC Journal, 2023, 74(5): 2123-2135.
[8]	Cheng YUN, Qianlin WANG, Feng CHEN, Xin ZHANG, Zhan DOU, Tingjun YAN. Deep-mining risk evolution path of chemical processes based on community structure [J]. CIESC Journal, 2023, 74(4): 1639-1650.
[9]	Xuanjun WU, Chao WANG, Zijian CAO, Weiquan CAI. Deep learning model of fixed bed adsorption breakthrough curve hybrid-driven by data and physical information [J]. CIESC Journal, 2023, 74(3): 1145-1160.
[10]	Xinyuan WU, Qilei LIU, Boyuan CAO, Lei ZHANG, Jian DU. Group2vec: group vector representation and its property prediction applications based on unsupervised machine learning [J]. CIESC Journal, 2023, 74(3): 1187-1194.
[11]	Jianghuai ZHANG, Zhong ZHAO. Robust minimum covariance constrained control for C₃ hydrogenation process and application [J]. CIESC Journal, 2023, 74(3): 1216-1227.
[12]	Xuejin GAO, Kun CHENG, Huayun HAN, Huihui Gao, Yongsheng QI. Fault diagnosis of chillers using central loss conditional generative adversarial network [J]. CIESC Journal, 2022, 73(9): 3950-3962.
[13]	Yalin WANG, Yuqing PAN, Chenliang LIU. Intermittent process monitoring based on GSA-LSTM dynamic structure feature extraction [J]. CIESC Journal, 2022, 73(9): 3994-4002.
[14]	Le ZHOU, Chengkai SHEN, Chao WU, Beiping HOU, Zhihuan SONG. Deep fusion feature extraction network and its application in chemical process soft sensing [J]. CIESC Journal, 2022, 73(7): 3156-3165.
[15]	Zhe SUN, Huaqiang JIN, Kang LI, Jiangping GU, Yuejin HUANG, Xi SHEN. Fault diagnosis method of refrigeration and air-conditioning system based on digitized knowledge representation [J]. CIESC Journal, 2022, 73(7): 3131-3144.