Design of self-organizing fuzzy neural network based on hybrid evaluation index

doi:10.11949/j.issn.0438-1157.20190180

Abstract

Abstract:

Aiming at the problem that the fuzzy neural network structure is difficult to adapt when there is no growth and pruning thresholds, this paper proposes a structure design method based on hybrid evaluation index (HEI). First, the initial number, centers and widths of rule neurons are determined by the fuzzy C-means clustering algorithm. Next, a novel relevance evaluation index (REI), which is composed of the Davies bouldin index (DBI) and the Dunn index (DI), is presented to calculate the correlation among the outputs of rule neurons. The learning ability of neural network will be determined by the change of root mean square error (RMSE) during the training process. Then, the HEI is presented based on REI and RMSE. The topology structure of the fuzzy neural network is adjusted according to the HEI. Finally, the feasibility and effectiveness of the structure design method are proved by using the Mackey-Glass time series prediction, nonlinear system identification and PM_2.5 concentration prediction.

Key words: self-organizing fuzzy neural network, hybrid evaluation index (HEI), optimal design, dynamic modeling, prediction

摘要：

针对在无增长和修剪阈值时模糊神经网络结构难以自适应问题，提出一种基于混合评价指标(hybrid evaluation index, HEI)的结构设计方法。首先，通过模糊C均值聚类算法(fuzzy C-means clustering, FCM)确定初始规则层神经元数目及其中心与宽度。其次，基于戴维森堡丁指数(Davies bouldin index, DBI)和邓恩指数(Dunn index, DI)提出一种新的相关性评价指标(relevance evaluation index, REI)来计算规则层各神经元输出之间的相关性，同时根据训练过程中网络输出均方根误差(root mean square error， RMSE)的变化情况来确定网络的学习能力，然后基于REI和RMSE提出了HEI。通过HEI来调整模糊神经网络的拓扑结构，有效解决了在无增长和修剪阈值时网络结构难以动态自调整的问题且避免了网络结构冗余。最后，通过对Mackey-Glass时间序列预测、非线性系统辨识和大气中PM_2.5浓度预测，证明了该结构设计方法的可行性和有效性。

关键词: 自组织模糊神经网络, 混合评价指标, 优化设计, 动态建模, 预测

CLC Number:

TP 183

Junfei QIAO, Zengzeng HE, Shengli DU. Design of self-organizing fuzzy neural network based on hybrid evaluation index[J]. CIESC Journal, 2019, 70(7): 2606-2615.

乔俊飞, 贺增增, 杜胜利. 基于混合评价指标的自组织模糊神经网络设计研究[J]. 化工学报, 2019, 70(7): 2606-2615.

Figures/Tables 13

Fig.1 Structure of fuzzy neural network

Fig.2 Testing results for Mackey-Glass time series prediction

Fig.3 Testing errors for Mackey-Glass time series prediction

Fig.4 Number of rule neurons for Mackey-Glass time series prediction

Table 1 Performance comparisons of different methods for Mackey-Glass time series prediction

Method	Number of rule nurons	Testing RMSE	Training time/s
HEI-SOFNN	5	0.0093	10.71
SOFNN-AGA ^[17]	6	0.0119	21.20
GPFNN ^[34]	7	0.0107	27.33
NFN-FOESA^[16]	7	0.0132	168.35
FAOS-PFNN ^[9]	7	0.0201	27.33

Fig.5 Testing results

Fig.6 Testing errors for nonlinear system identification

Fig.7 Number of rule neurons for nonlinear system identification

Table 2 Performance comparisons of different methods for nonlinear system identification

Method	Number of rule nurons	Testing RMSE	Training time/s
HEI-SOFNN	5	0.0089	7.34
SOFNN-AGA^[17]	7	0.0090	13.10
GDFNN^[6]	8	0.0121	6.56
GPFNN^[34]	8	0.0067	17.26
NFN-FOESA ^[16]	9	0.0060	21.72

Fig.8 Testing results for PM2.5 concentration prediction

Fig.9 Testing errors for PM2.5 concentration prediction

Fig.10 Number of rule neurons for PM2.5 concentration prediction

Table 3 Performance comparisons of different methods for PM2.5 concentration prediction

Method	Testing RMSE	R²	Number of rule nurons
HEI-SOFNN	14.945	0.8823	6
FNN	36.687	0.7991	8
SOG-SOFNN^[35]	17.900	0.8400	10
RFNN^[36]	35.848	0.8052	9

References 36

1	LuoG Z, ZhangR, ChenZ, et al. A novel nonlinear modeling method for permanent-magnet synchronous motors[J]. IEEE Transactions Industrial Electronics, 2016, 63(10): 6490-6498.
2	WangN, ErM J, HanM. Dynamic tanker steering control using generalized ellipsoidal-basis-function-based fuzzy neural networks[J]. IEEE Transactions on Fuzzy Systems, 2015, 23(5): 1414-1427.
3	GengJ, HuangM L, LiM W, et al. Hybridization of seasonal chaotic cloud simulated annealing algorithm in a SVR-based load forecasting model[J]. Neurocomputing, 2015, 151: 1362-1373.
4	CoyleD, PrasadG, McGinnityT M. Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a brain–computer interface[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2009, 39(6): 1458-1471.
5	ZhouY M, DexterA. Off-line identification of nonlinear, dynamic systems using a neuro-fuzzy modelling technique[J]. Fuzzy Sets and Systems, 2013, 225: 74-92.
6	WuS, ErM J, GaoY. A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks[J]. IEEE Transactions on Fuzzy Systems, 2001, 9(4): 578-594.
7	MalekH, EbadzadehM M, RahmatiM. Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm[J]. Applied Intelligence, 2012, 37(2): 280-289.
8	JuangC F, ChenT C, ChengW Y. Speedup of implementing fuzzy neural networks with high-dimensional inputs through parallel processing on graphic processing units[J]. IEEE Transactions on Fuzzy Systems, 2011, 19(4): 717-728.
9	WangN, ErM J, MengX. A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks[J]. Neurocomputing, 2009, 72(16/17/18): 3818-3829.
10	EbadzadehM M, Salimi-BadrA. CFNN: correlated fuzzy neural network[J]. Neurocomputing, 2015, 148: 430-444.
11	WangN. A generalized ellipsoidal basis function based online self-constructing fuzzy neural network[J]. Neural Processing Letters, 2011, 34(1): 13-37.
12	PrasadM, LinY Y, LinC T, et al. A new data-driven neural fuzzy system with collaborative fuzzy clustering mechanism[J]. Neurocomputing, 2015, 167: 558-568.
13	McDonaldS, KerrD, ColemanS, et al. Modelling retinal ganglion cells using self-organising fuzzy neural networks[C]//Neural Networks (IJCNN), 2015 International Joint Conference. IEEE, 2015: 1-8.
14	de Jesús RubioJ. SOFMLS: online self-organizing fuzzy modified least-squares network[J]. IEEE Transactions on Fuzzy Systems, 2009, 17(6): 1296-1309.
15	LinC J, ChenC H. Nonlinear system control using self-evolving neural fuzzy inference networks with reinforcement evolutionary learning[J]. Applied Soft Computing, 2011, 11(8): 5463-5476.
16	ChenC, WangF Y. A self-organizing neuro-fuzzy network based on first order effect sensitivity analysis[J]. Neurocomputing, 2013, 118(11): 21–32.
17	HanH G, LinZ L, QiaoJ F. Modeling of nonlinear systems using the self-organizing fuzzy neural network with adaptive gradient algorithm[J]. Neurocomputing, 2017, 266: 566-578.
18	乔俊飞, 张力, 李文静. 基于尖峰自组织模糊神经网络的需水量预测[J]. 控制与决策, 2018, 33(12): 2197-2202.
	QiaoJ F, ZhangL, LiW J. Prediction of water demand based on spiking self-organizing fuzzy neural network[J]. Control and Decision, 2018, 33(12): 2197-2202.
19	JuangC F, HsiehC D. A fuzzy system constructed by rule generation and iterative linear SVR for antecedent and consequent parameter optimization[J]. IEEE Transactions on Fuzzy Systems, 2012, 20(2): 372-384.
20	KhayatO, EbadzadehM M, ShahdoostiH R, et al. A novel hybrid algorithm for creating self-organizing fuzzy neural networks[J]. Neurocomputing, 2009, 73(1/2/3): 517-524.
21	IzakianH, PedryczW, JamalI. Fuzzy clustering of time series data using dynamic time warping distance[J]. Engineering Applications of Artificial Intelligence, 2015, 39: 235-244.
22	IzakianH, PedryczW. Agreement-based fuzzy C-means for clustering data with blocks of features[J]. Neurocomputing, 2014, 127: 266-280.
23	DunnJ C. A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters[J]. Journal of Cybernetics, 1973, 3: 32-57.
24	NasrM B, ChtourouM. A self-organizing map-based initialization for hybrid training of feedforward neural networks[J]. Applied Soft Computing, 2011, 11(8): 4458-4464.
25	LiuY, ZhouC F, ChenY W. Weight initialization of feedforward neural networks by means of partial least squares[C]//Machine Learning and Cybernetics, 2006 International Conference. IEEE, 2006: 3119-3122.
26	SongQ. Robust initialization of a Jordan network with recurrent constrained learning[J]. IEEE Transactions on Neural Networks, 2011, 22(12): 2460-2473.
27	TalaśkaT, KolasaM, DługoszR, et al. An efficient initialization mechanism of neurons for winner takes all neural network implemented in the CMOS technology[J]. Applied Mathematics and Computation, 2015, 267: 119–138.
28	QiaoJ F, LiS Y, LiW J. Mutual information based weight initialization method for sigmoidal feedforward neural networks[J]. Neurocomputing, 2016, 207: 676-683.
29	WuJ J, LiuH F, XiongH, et al. K-means-based consensus clustering: a unified view[J]. IEEE Transactions on Knowledge and Data Engineering, 2015, 27(1): 155-169.
30	LiuY C, LiZ M, XiongH, et al. Understanding of internal clustering validation measures[C]//Data Mining (ICDM), 2010 IEEE 10th International Conference. IEEE, 2010: 911-916.
31	AzidA, JuahirH, TorimanM E, et al. Prediction of the level of air pollution using principal component analysis and artificial neural network techniques: a case study in Malaysia[J]. Water, Air & Soil Pollution, 2014, 225(8): 2063.
32	QiaoJ F, HanH G. Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach[J]. Automatica, 2012, 48(8): 1729-1734.
33	IslamM M, SattarM A, AminM F, et al. A new adaptive merging and growing algorithm for designing artificial neural networks[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2009, 39(3): 705-722.
34	HanH G, QiaoJ F. A self-organizing fuzzy neural network based on a growing-and-pruning algorithm[J]. IEEE Transactions on Fuzzy Systems, 2010, 18(6): 1129-1143.
35	QiaoJ F, CaiJ, HanH G, et al. Predicting PM2. 5 concentrations at a regional background station using second order self-organizing fuzzy neural network[J]. Atmosphere, 2017, 8(1): 10.
36	HeZ Z, YeX D, GuK, et al. Learn to predict PM2. 5 concentration with image contrast-sensitive features[C]//2018 37th Chinese Control Conference (CCC). IEEE, 2018: 4102-4106.

[1]	Zhewen CHEN, Junjie WEI, Yuming ZHANG. System integration and energy conversion mechanism of the power technology with integrated supercritical water gasification of coal and SOFC [J]. CIESC Journal, 2023, 74(9): 3888-3902.
[2]	Cong QI, Zi DING, Jie YU, Maoqing TANG, Lin LIANG. Study on solar thermoelectric power generation characteristics based on selective absorption nanofilm [J]. CIESC Journal, 2023, 74(9): 3921-3930.
[3]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[4]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[5]	Yuying GUO, Jiaqiang JING, Wanni HUANG, Ping ZHANG, Jie SUN, Yu ZHU, Junxuan FENG, Hongjiang LU. Water-lubricated drag reduction and pressure drop model modification for heavy oil pipeline [J]. CIESC Journal, 2023, 74(7): 2898-2907.
[6]	Zhaolun WEN, Peirui LI, Zhonglin ZHANG, Xiao DU, Qiwang HOU, Yegang LIU, Xiaogang HAO, Guoqing GUAN. Design and optimization of cryogenic air separation process with dividing wall column based on self-heat regeneration [J]. CIESC Journal, 2023, 74(7): 2988-2998.
[7]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[8]	Yanhui LI, Shaoming DING, Zhouyang BAI, Yinan ZHANG, Zhihong YU, Limei XING, Pengfei GAO, Yongzhen WANG. Corrosion micro-nano scale kinetics model development and application in non-conventional supercritical boilers [J]. CIESC Journal, 2023, 74(6): 2436-2446.
[9]	Jinbo JIANG, Xin PENG, Wenxuan XU, Rixiu MEN, Chang LIU, Xudong PENG. Study on leakage characteristics and parameter influence of pump-out spiral groove oil-gas seal [J]. CIESC Journal, 2023, 74(6): 2538-2554.
[10]	Yuan YU, Weiwei CHEN, Junjie FU, Jiaxiang LIU, Zhiwei JIAO. Study and prediction of flow field in the annular region of geometrically similar turbo air classifier [J]. CIESC Journal, 2023, 74(6): 2363-2373.
[11]	Shanghao LIU, Shengkun JIA, Yiqing LUO, Xigang YUAN. Optimization of ternary-distillation sequence based on gradient boosting decision tree [J]. CIESC Journal, 2023, 74(5): 2075-2087.
[12]	Bimao ZHOU, Shisen XU, Xiaoxiao WANG, Gang LIU, Xiaoyu LI, Yongqiang REN, Houzhang TAN. Effect of burner bias angle on distribution characteristics of gasifier slag layer [J]. CIESC Journal, 2023, 74(5): 1939-1949.
[13]	Wenxuan XU, Jinbo JIANG, Xin PENG, Rixiu MEN, Chang LIU, Xudong PENG. Comparative study on leakage and film-forming characteristics of oil-gas seal with three-typical groove in a wide speed range [J]. CIESC Journal, 2023, 74(4): 1660-1679.
[14]	Jiyuan LI, Jinwang LI, Liuwei ZHOU. Heat transfer performance of cold plates with different turbulence structures [J]. CIESC Journal, 2023, 74(4): 1474-1488.
[15]	Junxian CHEN, Zhongli JI, Yu ZHAO, Qian ZHANG, Yan ZHOU, Meng LIU, Zhen LIU. Study on online detection method of particulate matter in natural gas pipeline based on microwave technology [J]. CIESC Journal, 2023, 74(3): 1042-1053.