基于分位数回归CGAN的虚拟样本生成方法及其过程建模应用

doi:10.11949/0438-1157.20201748

摘要/Abstract

摘要：

针对复杂工业过程因难以检测变量或因时间上和经济上成本因素导致的建模样本稀缺问题，提出了一种将分位数回归（quantile regression）嵌入到条件生成式对抗网络（conditional generative adversarial network， CGAN）的虚拟样本生成方法QRCGAN。首先，在标准CGAN“生成器-判别器”两元对弈结构中嵌入回归器，使模型不仅具备标签样本生成能力，同时也具备处理回归预测问题的能力。其次，以分位数回归神经网络实现回归器，连同判别器和生成器进行同步对抗训练。当模型到达Nash平衡时，在分位数回归神经网络回归器的作用下，生成器能够产生落在一定置信区间的新样本。然后，利用Kullback-Leibler（KL）散度评估生成样本的质量。最后，通过标准函数数据和实际化工过程数据验证所提方法的有效性。

关键词: 虚拟样本生成, CGAN, 分位数回归, 数据稀缺, 软测量, 深度学习

Abstract:

Complex industry processes suffer from the problems on data scarcity of training samples which are collected for modeling, as a result of inaccessibility of difficult-to-measure variables or high cost in time or economy. To tackle the issues, we proposed a novel virtual sample generation embedding quantile regression into conditional generative adversarial networks (QRCGAN). First of all, a regression is embedded in the standard CGAN “generator-discriminator” two-element game structure, so that the model not only has the ability to generate label samples, but also has the ability to handle regression prediction problems. Secondly, the regressor is implemented by the quantile regression neural network (QRNN), together with the discriminator and generator for simultaneous adversarial training. Once the model reaches the Nash equilibrium, with the help of the QRNN regressor, the generator can generate new samples that fall within a certain confidence interval. Moreover, the Kullback-Leibler (KL) divergence was used to evaluate the quality of the generated samples. Finally, the effectiveness of the proposed method is verified by standard function data and actual chemical process data.

Key words: virtual sample generation, CGAN, quantile regression, data scarcity, soft sensing, deep learning

中图分类号:

陈忠圣, 朱梅玉, 贺彦林, 徐圆, 朱群雄. 基于分位数回归CGAN的虚拟样本生成方法及其过程建模应用[J]. 化工学报, 2021, 72(3): 1529-1538.

CHEN Zhongsheng, ZHU Meiyu, HE Yanlin, XU Yuan, ZHU Qunxiong. Quantile regression CGAN based virtual samples generation and its applications to process modeling[J]. CIESC Journal, 2021, 72(3): 1529-1538.

图/表 9

图1 提出的QRCGAN示意图

Fig.1 Schematic diagram of the proposed QRCGAN

表1 基准函数及其定义

Table 1 Benchmarking functions used and its definitions

记号	基准函数	噪声分布	定义区间
LinHomo	y=x + ε	ε ~ N (0,0.05²)	x $∈$ [0, 1]
LinHetero	y=x + ε	ε ~ N (0,0.01x²)	x $∈$ [0, 1]
SinOsc	y=x + 0.2sin(20x) +ε	ε ~ N (0,0.05²)	x $∈$ [0, 1]

表1 基准函数及其定义

Table 1 Benchmarking functions used and its definitions

记号	基准函数	噪声分布	定义区间
LinHomo	y=x + ε	ε ~ N (0,0.05²)	x $∈$ [0, 1]
LinHetero	y=x + ε	ε ~ N (0,0.01x²)	x $∈$ [0, 1]
SinOsc	y=x + 0.2sin(20x) +ε	ε ~ N (0,0.05²)	x $∈$ [0, 1]

图2 基准函数LinHomo在给定不同的条件信号y下的条件密度估计红色实线—逼近分布；绿色实线—真实分布

Fig.2 Conditional density estimation for the benchmark function LinHomo when given different y

图3 基准函数LinHetero在给定不同的条件信号y下的条件密度估计红色实线—逼近分布；绿色实线—真实分布

Fig.3 Conditional density estimation for the benchmark function LinHetero when given different y

图4 基准函数SinOsc在给定不同的条件信号y下的条件密度估计红色实线—逼近分布；绿色实线—真实分布

Fig.4 Conditional density estimation for the benchmark function SinOsc when given different y

图5 KL散度随着生成器更新次数的变化

Fig.5 KL divergence against the updates of generator

图6 虚拟样本云点左侧图—提出的QRCGAN, 右侧图—CGAN

Fig.6 Cloud points of virtual samples

表2 虚拟样本云点质量评估

Table 2 Quality evaluation for cloud points of virtual samples

模型	LinHomo	LinHetero	SinOsc
QRCGAN	0.418	1.315	3.028
CGAN	0.611	1.932	8.328

图7 模型预测结果比较

Fig.7 Comparison of predictions of models

参考文献 22

1	Zhu J, Ge Z, Song Z, et al. Review and big data perspectives on robust data mining approaches for industrial process modeling with outliers and missing data[J]. Annual Reviews in Control, 2018, 46: 107-133.
2	朱群雄, 耿志强, 徐圆, 等. 数据和知识融合驱动的智能过程系统工程研究进展[J]. 北京化工大学学报(自然科学版), 2018, 45(5): 143-152.
	Zhu Q X, Geng Z Q, Xu Y, et al. Progress in intelligent process system engineering driven by data and knowledge fusion[J]. Journal of Beijing University of Chemical Technology (Natural Science Edition), 2018, 45(5): 143-152.
3	Zhu Q X, Zhang X H, He Y L. Novel virtual sample generation based on locally linear embedding for optimizing the small sample problem: case of soft sensor applications[J]. Industrial & Engineering Chemistry Research, 2020, 59(40): 17977-17986.
4	Zhu Q X, Chen Z S, Zhang X H, et al. Dealing with small sample size problems in process industry using virtual sample generation: a Kriging-based approach[J]. Soft Computing, 2020, 24(9): 6889-6902.
5	乔俊飞, 郭子豪, 汤健. 基于改进大趋势扩散和隐含层插值的虚拟样本生成方法及应用[J]. 化工学报, 2020, 71(12): 5681-5695.
	Qiao J F, Guo Z H, Tang J. Virtual sample generation method based on improved megatrend diffusion and hidden layer interpolation with its application[J]. CIESC Journal, 2020, 71(12): 5681-5695.
6	Zhu Q X, Gong H F, Xu Y, et al. A bootstrap based virtual sample generation method for improving the accuracy of modeling complex chemical processes using small datasets[C]// 6th Data Driven Control and Learning Systems Conference. Chongqing, China, 2017.
7	Coqueret G. Approximate NORTA simulations for virtual sample generation[J]. Expert Systems with Applications, 2017, 73: 69-81.
8	Li D C, Wu C S, Tsai T I, et al. Using mega-trend-diffusion and artificial samples in small data set learning for early flexible manufacturing system scheduling knowledge[J]. Computers & Operations Research, 2007, 34(4): 966-982.
9	朱宝, 陈忠圣, 余乐安. 一种新颖的小样本整体趋势扩散技术[J]. 化工学报, 2016, 67(3): 820-826.
	Zhu B, Chen Z S, Yu L A. A novel mega-trend-diffusion for small sample[J]. CIESC Journal, 2016, 67(3): 820-826.
10	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. arXiv, 2014, arXiv: 1406.2661 [stat.ML]. .
11	Kingma D P, Welling M. Auto-encoding variational bayes[J]. arXiv, 2013, arXiv: 1312.6114 [stat.ML]. .
12	Salimans T, Goodfellow I, Zaremba W, et al. Improved techniques for training GANs[J]. arXiv, 2016, arXiv: 1606.03498 [cs.LG]. .
13	Mirza M, Osindero S. Conditional generative adversarial nets[J]. arXiv, 2014, arXiv: 1411.1784 [cs.LG]. .
14	Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv, 2016, arXiv: 1511.06434 [cs.LG]. .
15	Arjovsky M, Chintala S, Bottou L. Wasserstein GAN[J]. arXiv, 2017, arXiv: 1701.07875 [stat.ML]. .
16	Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[J]. arXiv, 2017, arXiv: 1703.10593 [cs.CV]. .
17	Donahue J, Krähenbühl P, Darrell T. Adversarial feature learning[J]. arXiv, 2016, arXiv: 1605.09782 [cs.LG]. .
18	Rezagholiradeh M, Haidar M A. Reg-Gan: semi-supervised learning based on generative adversarial networks for regression [C]// International Conference on Acoustics, Speech and Signal Processing. Calgary, Canada, 2018.
19	庞昊, 高金峰, 杜耀恒. 基于时间卷积网络分位数回归的短期负荷概率密度预测方法[J]. 电网技术, 2020, 44(4): 1343-1350.
	Pang H, Gao J F, Du Y H. A short-term load probability density prediction based on quantile regression of time convolution network[J]. Power System Technology, 2020, 44(4): 1343-1350.
20	Taylor J W. A quantile regression approach to estimating the distribution of multiperiod returns[J]. The Journal of Derivatives, 1999, 7(1): 64-78.
21	Boget Y. Adversarial regression. Generative adversarial networks for non-linear regression: theory and assessment[D]. Switzerland: University of Neuchâtel, 2019.
22	朱群雄, 张晓晗, 顾祥柏, 等. 基于特征提取的函数连接神经网络研究及其化工过程建模应用[J]. 化工学报, 2018, 69(3): 907-912.
	Zhu Q X, Zhang X H, Gu X B, et al. Research and application of feature extraction derived functional link neural network[J]. CIESC Journal, 2018, 69(3): 907-912.

[1]	温凯杰, 郭力, 夏诏杰, 陈建华. 一种耦合CFD与深度学习的气固快速模拟方法[J]. 化工学报, 2023, 74(9): 3775-3785.
[2]	闫琳琦, 王振雷. 基于STA-BiLSTM-LightGBM组合模型的多步预测软测量建模[J]. 化工学报, 2023, 74(8): 3407-3418.
[3]	邵伟明, 韩文学, 宋伟, 杨勇, 陈灿, 赵东亚. 基于分布式贝叶斯隐马尔可夫回归的动态软测量建模方法[J]. 化工学报, 2023, 74(6): 2495-2502.
[4]	罗顺桦, 王振雷, 王昕. 基于二子空间协同训练算法的半监督软测量建模[J]. 化工学报, 2022, 73(3): 1270-1279.
[5]	齐书平, 王文龙, 张磊, 都健. 基于深度学习的金属离子-有机配体配位稳定常数的预测[J]. 化工学报, 2022, 73(12): 5461-5468.
[6]	谢昊源, 黄群星, 林晓青, 李晓东, 严建华. 基于图像深度学习的垃圾热值预测研究[J]. 化工学报, 2021, 72(5): 2773-2782.
[7]	于程远, 吴金奎, 周利, 吉旭, 戴一阳, 党亚固. 基于深度学习预测有机光伏电池能量转换效率[J]. 化工学报, 2021, 72(3): 1487-1495.
[8]	刘聪, 谢莉, 杨慧中. 基于改进DPC的青霉素发酵过程多模型软测量建模[J]. 化工学报, 2021, 72(3): 1606-1615.
[9]	王晓慧, 王延江, 邓晓刚, 张政. 基于加权深度支持向量数据描述的工业过程故障检测[J]. 化工学报, 2021, 72(11): 5707-5716.
[10]	尹林子, 关羽吟, 蒋朝辉, 许雪梅. 基于k-means++的高炉铁水硅含量数据优选方法[J]. 化工学报, 2020, 71(8): 3661-3670.
[11]	李东, 黄道平, 刘乙奇. 基于协同训练的半监督异构自适应软测量建模方法的研究[J]. 化工学报, 2020, 71(5): 2128-2138.
[12]	杜宇浩, 阎高伟, 李荣, 王芳. 基于局部线性嵌入的测地线流式核多工况软测量建模方法[J]. 化工学报, 2020, 71(3): 1278-1287.
[13]	乔俊飞,郭子豪,汤健. 基于改进大趋势扩散和隐含层插值的虚拟样本生成方法及应用[J]. 化工学报, 2020, 71(12): 5681-5695.
[14]	杨逸俊,王振雷,王昕. 基于最近邻与神经网络融合模型的软测量建模方法[J]. 化工学报, 2020, 71(12): 5696-5705.
[15]	代学志,熊伟丽. 基于核极限学习机的快速主动学习方法及其软测量应用[J]. 化工学报, 2020, 71(11): 5226-5236.