Virtual sample generation method based on improved megatrend diffusion and hidden layer interpolation with its application

doi:10.11949/0438-1157.20200673

Abstract

Abstract:

The time and economic cost of obtaining difficult-to-measure quality or environmental pollution indices data for complex industrial processes are very high, which leads to the scarcity of labeled modeling samples. Aimed at this problem, a new virtual sample generation method based on improved megatrend diffusion and hidden layer interpolation is proposed. It is applied to the dioxin (DXN) emissions prediction of municipal solid waste cineration process. First, the true sample input/output sample space is expanded by using improved megatrend diffusion technology (MTD) based on the sub-regional Euclidean distance. Next, the virtual sample input is generated using an equal interval interpolation method, and the virtual sample output is obtained by combining the mapping model and the pruning mechanism. Then, the hidden layer interpolation method based on the improved random weight neural network with regularization mechanism is used to obtain the virtual sample output and input, and the virtual sample is deleted by combining with the expansion space. Finally, the above-mentioned complementary input/output virtual samples are mixed with the original true samples to realize the expansion of the modeling data capacity. The validity and rationality of the proposed method are verified by benchmark data set and industrial process DXN data.

Key words: megatrend diffusion, neural network hidden layer interpolation, virtual sample generation, dioxin emission prediction, waste treatment, algorithm, model

摘要：

针对获取复杂工业过程的难以检测质量或环境污染指标数据的时间和经济成本高导致有标记建模样本稀缺的问题，提出了基于改进大趋势扩散和隐含层插值的虚拟样本生成（VSG）方法，并将其应用于城市固废焚烧过程的二英（DXN）排放预测。首先，采用基于子区域欧氏距离改进大趋势扩散（MTD）方法对真实样本输入/输出空间进行扩展；接着，采用等间隔插值方式生成虚拟样本输入，再结合映射模型和删减机制获得虚拟样本输出；然后，采用基于正则化改进的随机权神经网络隐含层插值依次得到虚拟样本输出和输入，再结合扩展空间对虚拟样本进行删减；最后，将上述具有互补性的虚拟样本与原始真实样本进行混合，实现建模数据容量扩充。通过基准数据集和工业过程DXN数据验证了所提方法的有效性和合理性。

关键词: 大趋势扩散, 神经网络隐含层插值, 虚拟样本生成, 二英排放预测, 废物处理, 算法, 模型

CLC Number:

TQ 183

QIAO Junfei,GUO Zihao,TANG Jian. Virtual sample generation method based on improved megatrend diffusion and hidden layer interpolation with its application[J]. CIESC Journal, 2020, 71(12): 5681-5695.

乔俊飞,郭子豪,汤健. 基于改进大趋势扩散和隐含层插值的虚拟样本生成方法及应用[J]. 化工学报, 2020, 71(12): 5681-5695.

Figures/Tables 22

Fig.1 Relationship between expected sample space and real sample space

Fig.2 Strategy of the proposed VSG method

Fig.3 Comparison between original small sample space and extended space

Table 1 Comparison of extended output space and original output space for Benchmark data

项目	混凝土抗压强度/MPa
y_max	79.99
y_vsg-max	100.472
上限扩展率	25.61%
y_min	6.88
y_vsg-min	0
下限扩展率	100%

Table 2 Input and output of virtual samples generated based on equal interval method for Benchmark data (taking the first and second samples with n=3 as example)

样本	浓度/(kg/m³)							龄期/d	混凝土抗压强度/MPa
样本	水泥	高炉渣	粉煤灰	水	超塑化剂	粗集料	细集料	龄期/d	混凝土抗压强度/MPa
第1组数据	540	0	0	162	2.5	1040	676	28	37.104
第2组数据	332.5	142.5	0	228	0	932	594	180	30.674
第1组生成样本	488.125	35.625	0	178.5	1.875	1013	655.5	66	39.207
第2组生成样本	436.25	71.25	0	195	1.25	986	635	104	41.31
第3组生成样本	384.375	106.875	0	211.5	0.625	959	614.5	142	35.992
备注	样本输入	样本输入	样本输入	样本输入	样本输入	样本输入	样本输入	样本输入	样本输出

Fig.4 Relationship between the deleted number and the generated number of the virtual sample output for Benchmark data

Table 3 Virtual sample input and output results based on multiple sets of hidden layer interpolation method without regularization term for Benchmark data (taking the former five samples as example)

n	X_insert-temp								Y_insert-temp
1	292.664	70.481	-69.88	175.089	2.292	802.293	543.142	-35.603	44.01
2	278.301	93.42	-49.115	181.486	4.993	819.112	511.977	-62.01	44.053
3	250.453	99.612	-31.037	187.579	5.812	802.988	520.632	-79.17	41.626
4	233.829	75.385	-36.606	185.512	3.208	763.418	569.621	-64.189	38.377
5	103.508	-17.664	-29.344	153.978	-2.336	735.579	592.56	-21.546	41.288
合格比率	100%	80%	0%	100%	80%	100%	80%	0%	100%

Table 4 Virtual sample input and output results based on multiple sets of hidden layer interpolation method with adding regularization term for Benchmark data (taking the former five samples as example)

n	X_insert-temp								Y_insert-temp
1	583.657	271.588	197.941	234.288	18.838	1172.991	934.768	201.621	35.034
2	584.102	271.682	197.839	234.446	18.824	1172.598	934.191	201.648	34.975
3	584.548	271.775	197.736	234.604	18.809	1172.206	933.614	201.674	34.915
4	584.994	271.869	197.633	234.762	18.795	1171.813	933.037	201.7	34.856
5	585.439	271.962	197.53	234.921	18.78	1171.42	932.46	201.727	34.796
合格比率	100%	100%	100%	100%	100%	100%	100%	100%	100%

Fig.5 Effect of regularization term on sample qualification rate

Table 5 Virtual sample input and output results based on multiple sets of hidden layer interpolation method with deletion operation for Benchmark data (taking the former five samples as example)

n	X_insert								Y_insert
1	583.657	271.588	197.941	234.288	18.838	1172.991	934.768	201.621	35.034
2	584.102	271.682	197.839	234.446	18.824	1172.598	934.191	201.648	34.975
3	584.548	271.775	197.736	234.604	18.809	1172.206	933.614	201.674	34.915
4	584.994	271.869	197.633	234.762	18.795	1171.813	933.037	201.7	34.856
5	585.439	271.962	197.53	234.921	18.78	1171.42	932.46	201.727	34.796

Table 6 Comparison of experimental results for Benchmark data

实验	虚拟样本数量	RMSE均值	RMSE方差	最佳RMSE及插值参数
实验	虚拟样本数量	RMSE均值	RMSE方差	数值	$n e q u a l$	$n i n s e r t$	$n l a y e r$
A	0	23.599	13.226	—	—	—	—
B	60	18.370	11.330	16.577	6	—	—
C	161	16.156	8.831	14.525	3	—	6
D	29	21.128	112.204	15.498	1	1	3
E	32	19.780	93.273	14.466	2	1	3
F	729	16.134	9.692	13.383	6	2	8

Table 6 Comparison of experimental results for Benchmark data

实验	虚拟样本数量	RMSE均值	RMSE方差	最佳RMSE及插值参数
实验	虚拟样本数量	RMSE均值	RMSE方差	数值	$n e q u a l$	$n i n s e r t$	$n l a y e r$
A	0	23.599	13.226	—	—	—	—
B	60	18.370	11.330	16.577	6	—	—
C	161	16.156	8.831	14.525	3	—	6
D	29	21.128	112.204	15.498	1	1	3
E	32	19.780	93.273	14.466	2	1	3
F	729	16.134	9.692	13.383	6	2	8

Fig.6 Prediction curves of test samples with different methods of Benchmark data

Fig.7 Typical MSWI process flow chart based on grate furnace

Fig.8 Comparison between original small sample space and extended space (taking the former 5 input features as example) for DXN data

Table 7 Comparison between extended output space and original output space for DXN data

项目	DXN浓度/(ng TEQ/m³)
y_max	0.083
y_vsg-max	0.133
上限扩展率	60.24%
y_min	0.002
y_vsg-min	0
下限扩展率	100%

Table 8 Virtual samples input and output based on generated at equal interval method (taking the first and second samples with n=3 as example) for DXN data

样本	反应器入口氧气浓度/%	燃烧炉排右空气流量×10^-3/(m³/h)	二次空预器出口温度/℃	干燥炉排入口空气温度/℃	燃烧炉排左侧温度/℃	DXN浓度/ (ng TEQ/m³)
第1组数据	4.8	1.5	14	176	181	0.05
第2组数据	3.2	3.4	24	180	198	0.035
第1组生成样本	12.054	5.184	33.801	209.677	257.477	0.036
第2组生成样本	12.244	5.261	34.361	211.212	259.972	0.0362
第3组生成样本	12.435	5.338	34.921	212.748	262.468	0.0364
备注	样本输入(DXN变量1)	样本输入(DXN变量2)	样本输入(DXN变量3)	样本输入(DXN变量4)	样本输入(DXN变量5)	样本输出

Fig.9 Relationship between the number of deletion and generation of virtual sample output for DXN data

Table 9 Input and output results based on multiple groups of hidden layer interpolation method without regularization term for DXN data (taking the former five samples as example)

n	X_insert-temp					Y_insert-temp
1	-1.839	-0.766	4.944	84.585	90.561	0.0417
2	-1.862	-0.749	4.963	84.512	90.128	0.0413
3	-1.885	-0.731	4.982	84.439	89.695	0.0409
4	-1.908	-0.714	5	84.365	89.263	0.0405
5	-1.931	-0.697	5.019	84.292	88.83	0.0401
合格比率	100%	0%	100%	100%	100%	100%

Table 10 Virtual sample input and output results based on multiple sets of hidden layer interpolation method with adding regularization term for DXN data (taking the former five samples as example)

n	X_insert-temp					Y_insert-temp
1	12.584	5.416	33.933	214.927	262.392	0.0498
2	12.603	5.431	33.986	214.748	262.18	0.0493
3	12.622	5.446	34.039	214.568	261.969	0.0488
4	12.641	5.461	34.091	214.388	261.757	0.0483
5	12.66	5.476	34.144	214.209	261.545	0.0478
合格比率	100%	100%	100%	100%	100%	100%

Fig.10 Effect of regularization term on sample qualification rate for DXN data

Table 11 Virtual sample input and output results based on multiple sets of hidden layer interpolation method with deletion operation for DXN data (taking the former five samples as example)

n	X_insert					Y_insert
1	12.584	5.416	33.933	214.927	262.392	0.0498
2	12.603	5.431	33.986	214.748	262.18	0.0493
3	12.622	5.446	34.039	214.568	261.969	0.0488
4	12.641	5.461	34.091	214.388	261.757	0.0483
5	12.66	5.476	34.144	214.209	261.545	0.0478

Table 12 Comparison of experimental results for DXN data

实验	虚拟样本数量	RMSE均值	RMSE方差	最佳RMSE及插值参数
实验	虚拟样本数量	RMSE均值	RMSE方差	数值	$n e q u a l$	$n i n s e r t$	$n l a y e r$
A	0	0.0383	0.000203	—	—	—
B	136	0.0440	0.000220	0.0379	8
C	61	0.0304	1.108×10^-5	14.525	2	—	2
D	436	0.0288	2.532×10^-5	0.0255	2	9	3
E	81	0.0289	2.073×10^-5	0.0255	1	2	6
F	501	0.0286	1.697×10^-5	0.0254	5	1	7

Table 12 Comparison of experimental results for DXN data

实验	虚拟样本数量	RMSE均值	RMSE方差	最佳RMSE及插值参数
实验	虚拟样本数量	RMSE均值	RMSE方差	数值	$n e q u a l$	$n i n s e r t$	$n l a y e r$
A	0	0.0383	0.000203	—	—	—
B	136	0.0440	0.000220	0.0379	8
C	61	0.0304	1.108×10^-5	14.525	2	—	2
D	436	0.0288	2.532×10^-5	0.0255	2	9	3
E	81	0.0289	2.073×10^-5	0.0255	1	2	6
F	501	0.0286	1.697×10^-5	0.0254	5	1	7

References 36

1	Qiao J F, Guo Z H, Tang J. Dioxin emission concentration measurement approaches for municipal solid wastes incineration process: a survey[J]. Acta Automatica Sinica, 2020, 46(6): 1063-1089.
2	Di A B, Fortuna L, Graziani S, et al. Development of a soft sensor for a thermal cracking unit using a small experimental data set[C]//2007 IEEE International Symposium on Intelligent Signal Processing. IEEE, 2007: 1-6.
3	Vapnik V N. An overview of statistical learning theory[J]. IEEE Transactions on Neural Networks, 1999, 10(5): 988-999.
4	Liu Y, Sun W, Liu J. Greenhouse gas emissions from different municipal solid waste management scenarios in China: based on carbon and energy flow analysis[J]. Waste Management, 2017, 68: 653-661.
5	Bai J, Sun X, Zhang C, et al. Mechanism and kinetics study on the ozonolysis reaction of 2, 3, 7, 8-TCDD in the atmosphere[J]. Journal of Environmental Sciences, 2014, 26(1): 181-188.
6	Zhang H J, Ni Y W, Chen J P, et al. Influence of variation in the operating conditions on PCDD/F distribution in a full-scale MSW incinerator[J]. Chemosphere, 2008, 70(4): 721-730.
7	Martens H A, Dardenne P. Validation and verification of regression in small data sets[J]. Chemometrics and Intelligent Laboratory Systems, 1998, 44(1/2): 99-121.
8	Deng J L. Control problems of grey systems[J]. Systems & Control Letters, 1982, 1(5): 288-294.
9	Chang C J, Li D C, Huang Y H, et al. A novel gray forecasting model based on the box plot for small manufacturing data sets[J]. Applied Mathematics and Computation, 2015, 265: 400-408.
10	Chervonenkis A I A, Vapnik V N. Theory of uniform convergence of frequencies of events to their probabilities and problems of search for an optimal solution from empirical data[J]. Automation and Remote Control, 1971, 32: 207-217.
11	Poggio T, Vetter T. Recognition and structure from one 2D model view: observations on prototypes, object classes and symmetries[R]. Technical Report A. I. Memo 1347. Massachusetts Institute of Technology Cambridge, MA, USA, 1992.
12	Niyogi P, Girosi F, Poggio T. Incorporating prior information in machine learning by creating virtual examples[J]. Proceedings of the IEEE, 1998, 86(11): 2196-2209.
13	Ho K I J, Leung C S, Sum J. Convergence and objective functions of some fault/noise-injection-based online learning algorithms for RBF networks[J]. IEEE Transactions on Neural Networks, 2010, 21(6): 938-947.
14	Song H, Choi K K, Lee I, et al. Adaptive virtual support vector machine for reliability analysis of high-dimensional problems[J]. Structural and Multidisciplinary Optimization, 2013, 47(4): 479-491.
15	Li L J, Peng Y, Qiu G Y, et al. A survey of virtual sample generation technology for face recognition[J]. Artificial Intelligence Review, 2018, 50 (1): 1-20.
16	Chang C J, Li D C, Chen C C, et al. A forecasting model for small non-equigap data sets considering data weights and occurrence possibilities[J]. Computers & Industrial Engineering, 2014, 67(1): 139-145.
17	Li D C, Lin Y S. Using virtual sample generation to build up management knowledge in the early manufacturing stages[J]. European Journal of Operational Research, 2005, 175(1): 413-434.
18	Li D C, Wen I H. A genetic algorithm-based virtual sample generation technique to improve small data set learning[J]. Neurocomputing, 2014, 143(16): 222-230.
19	Chen Z S, Zhu B, He Y L, et al. PSO based virtual sample generation method for small sample sets: applications to regression datasets[J]. Engineering Applications of Artificial Intelligence, 2017, 59: 236-243.
20	Gong H F, Chen Z S, Zhu Q X, et al. A Monte Carlo and PSO based virtual sample generation method for enhancing the energy prediction and energy optimization on small data problem: an empirical study of petrochemical industries[J]. Applied Energy, 2017, 197: 405-415.
21	Coqueret G. Approximate NORTA simulations for virtual sample generation[J]. Expert Systems with Applications, 2017, 73: 69-81.
22	朱宝. 虚拟样本生成技术及建模应用研究[D]. 北京: 北京化工大学, 2017.
	Zhu B. Research on virtual sample generation technologies and their modeling application[D].Beijing: Beijing University of Chemical Technology, 2017.
23	Wang F Y. A big-data perspective on AI: Newton, Merton, and analytics intelligence[J]. IEEE Intelligent Systems, 2012, 27(5): 24-34.
24	Zhu Q, Chen Z, Zhang X. et al. Dealing with small sample size problems in process industry using virtual sample generation: a Kriging-based approach[J]. Soft Computing, 2020, 24: 6889-6902.
25	Tang J, Jia M Y, Liu Z, et al. Modeling high dimensional frequency spectral data based on virtual sample generation technique[C]//IEEE International Conference on Information and Automation. IEEE, 2015: 1090-1095.
26	汤健, 乔俊飞, 柴天佑, 等.基于虚拟样本生成技术的多组分机械信号建模[J]. 自动化学报, 2018, 44(9): 1569-1589.
	Tang J, Qiao J F, Chai T Y, et al. Modeling multiple components mechanical signals by means of virtual sample generation technique[J]. Acta Automatica Sinica, 2018, 44(9): 1569-1589.
27	Li D C, Wu C S, Tsai T I, et al. Using mega-trend-diffusion and artificial samples in small data set learning for early flexible manufacturing system scheduling knowledge[J]. Computers & Operations Research, 2007, 34(4): 966-982.
28	Huang G, Huang G B, Song S, et al. Trends in extreme learning machines: a review[J]. Neural Networks, 2015, 61: 32-48.
29	朱宝, 乔俊飞.基于AANN特征缩放的虚拟样本生成方法及其过程建模应用[J]. 计算机与应用化学, 2019, 36(4): 304-307.
	Zhu B, Qiao J F. Novel virtual sample generation based on feature scaling of auto-associative neural network and its applications to process modeling[J]. Computers and Applied Chemistry, 2019, 36(4): 304-307.
30	Yang J, Yu X, Xie Z Q, et al. A novel virtual sample generation method based on Gaussian distribution[J]. Knowledge-Based Systems, 2011, 24(6): 740-748.
31	Guo Y, Chen T, Yang J, et al. Study on on-line detection of dioxins based on correlation model[J]. Chinese Journal of Environmental Engineering, 2014, 8(8): 3524-3529.
32	Li A D, Hong W, Wang J. Online detection of dioxin and dioxin-related substances using laser desoption / laser ionization-mass spectrometry[J]. Journal of Yanshan University, 2015, 39(6): 511-515
33	Bunsan S, Chen W Y, Chen H W, et al. Modeling the dioxin emission of a municipal solid waste incinerator using neural networks[J]. Chemosphere, 2013, 92(3): 258-264.
34	Xiao X D, Lu J W, Hai J, et al. Prediction of dioxin emissions in flue gas from waste incineration based on support vector regression[J]. Renewable Energy Resources, 2017, 35(8): 1107-1114.
35	Tang J, Qiao J F. Dioxin emission concentration soft measuring approach of municipal solid waste incineration based on selective ensemble kernel learning algorithm[J]. CIESC Journal, 2019, 70(2): 696-706.
36	Chang N B, Chen W C. Prediction of PCDDs/PCDFs emissions from municipal incinerators by genetic programming and neural network modeling[J]. Waste Management and Research, 2000, 18(4): 341-351.

[1]	Jiahao SONG, Wen WANG. Study on coupling operation characteristics of Stirling engine and high temperature heat pipe [J]. CIESC Journal, 2023, 74(S1): 287-294.
[2]	Mengya LIAN, Yingying TAN, Lin WANG, Feng CHEN, Yifei CAO. Heating performance of air preheated integrated ground water heat pump air-conditioning system [J]. CIESC Journal, 2023, 74(S1): 311-319.
[3]	Zhenghao JIN, Lijie FENG, Shuhong LI. Energy and exergy analysis of a solution cross-type absorption-resorption heat pump using NH₃/H₂O as working fluid [J]. CIESC Journal, 2023, 74(S1): 53-63.
[4]	Hao WANG, Zhenlei WANG. Model simplification strategy of cracking furnace coking based on adaptive spectroscopy method [J]. CIESC Journal, 2023, 74(9): 3855-3864.
[5]	Yue CAO, Chong YU, Zhi LI, Minglei YANG. Industrial data driven transition state detection with multi-mode switching of a hydrocracking unit [J]. CIESC Journal, 2023, 74(9): 3841-3854.
[6]	Ke LI, Jian WEN, Biping XIN. Study on influence mechanism of vacuum multi-layer insulation coupled with vapor-cooled shield on self-pressurization process of liquid hydrogen storage tank [J]. CIESC Journal, 2023, 74(9): 3786-3796.
[7]	Xudong YU, Qi LI, Niancu CHEN, Li DU, Siying REN, Ying ZENG. Phase equilibria and calculation of aqueous ternary system KCl + CaCl₂ + H₂O at 298.2, 323.2, and 348.2 K [J]. CIESC Journal, 2023, 74(8): 3256-3265.
[8]	Manzheng ZHANG, Meng XIAO, Peiwei YAN, Zheng MIAO, Jinliang XU, Xianbing JI. Working fluid screening and thermodynamic optimization of hazardous waste incineration coupled organic Rankine cycle system [J]. CIESC Journal, 2023, 74(8): 3502-3512.
[9]	Chengying ZHU, Zhenlei WANG. Operation optimization of ethylene cracking furnace based on improved deep reinforcement learning algorithm [J]. CIESC Journal, 2023, 74(8): 3429-3437.
[10]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[11]	Gang YIN, Yihui LI, Fei HE, Wenqi CAO, Min WANG, Feiya YAN, Yu XIANG, Jian LU, Bin LUO, Runting LU. Early warning method of aluminum reduction cell leakage accident based on KPCA and SVM [J]. CIESC Journal, 2023, 74(8): 3419-3428.
[12]	Guoze CHEN, Dong WEI, Qian GUO, Zhiping XIANG. Optimal power point optimization method for aluminum-air batteries under load tracking condition [J]. CIESC Journal, 2023, 74(8): 3533-3542.
[13]	Jintong LI, Shun QIU, Wenshou SUN. Oxalic acid and UV enhanced arsenic leaching from coal in flue gas desulfurization by coal slurry [J]. CIESC Journal, 2023, 74(8): 3522-3532.
[14]	Chen HAN, Youmin SITU, Bin ZHU, Jianliang XU, Xiaolei GUO, Haifeng LIU. Study of reaction and flow characteristics in multi-nozzle pulverized coal gasifier with co-processing of wastewater [J]. CIESC Journal, 2023, 74(8): 3266-3278.
[15]	Chunyu LIU, Huanyu ZHOU, Yue MA, Changtao YUE. Drying characteristics and mathematical model of CaO-conditioned oil sludge [J]. CIESC Journal, 2023, 74(7): 3018-3027.