Gaussian process regression modeling of fermentation process based on k-nearest neighbor mutual information

doi:10.11949/0438-1157.20190606

Abstract

Abstract:

The concentration of the substrate during fermentation is often not measured online. In this paper, Gaussian process regression (GPR) is used to establish an estimation model of substrate concentration, and its soft measurement is realized. Different from traditional regression models, the GPR model can not only predict the quality value, but also provide the estimation variance. In order to improve the prediction performance of the model in the nonlinear fermentation process with correlated variables, the input variables of the model are selected by the k-nearest neighbor mutual information (k-MI) method before the model development. The application results of penicillin fermentation process show the ideal prediction performance based on the kMI-GPR model.

Key words: fermentation process, Gaussian process regression, k-nearest neighbor mutual information, soft sensor

摘要：

发酵过程中基质浓度往往无法在线测量，采用高斯过程回归（GPR）建立基质浓度的估计模型，实现了其软测量。不同于传统软测量方法对基质浓度的估计，该方法不仅可以得到估计值，还能够得到其估计方差。考虑到发酵过程中各变量之间的非线性、相关性，为了提高模型的预测性能，在模型建立之前首先用k-近邻互信息（k-MI）辅助变量选择方法对模型的输入变量进行选择。从青霉素发酵过程的应用结果来看，采用kMI-GPR方法取得了较好的估计效果。

关键词: 发酵过程, 高斯过程回归, k-近邻互信息, 软测量

CLC Number:

TP 273

Rongrong ZHAO, Zhonggai ZHAO, Fei LIU. Gaussian process regression modeling of fermentation process based on k-nearest neighbor mutual information[J]. CIESC Journal, 2019, 70(12): 4741-4748.

赵荣荣, 赵忠盖, 刘飞. 基于k-近邻互信息的发酵过程高斯过程回归建模[J]. 化工学报, 2019, 70(12): 4741-4748.

Figures/Tables 9

Fig.1 Penicillin fermentation apparatus

Fig.2 Mutual informations between candidate secondary variables and primary variable

Fig.3 Flow chart of substrate concentration soft-sensing modeling

Fig.4 Prediction results by kMI-GPR

Fig.5 Prediction results by PLS-GPR

Fig.6 Prediction results by kMI-SVM

Fig.7 Prediction errors of three diffferent models

Fig.8 Prediction values of three diffferent models(with noise)

Table 1 Prediction errors evaluation of three different models

Model	No noise			With noise
Model	$σ A R E$	$σ R M S E$	$σ M A X E$	$σ A R E$	$σ R M S E$	$σ M A X E$
kMI-GPR	0.5264	0.1410	0.8149	4.5148	0.2746	1.5371
kMI-SVM	11.2389	0.3768	1.2968	5.8723	0.5409	2.7882
PLS-GPR	3.9837	0.5892	3.0999	1.0641	2.5734	14.0338

Table 1 Prediction errors evaluation of three different models

Model	No noise			With noise
Model	$σ A R E$	$σ R M S E$	$σ M A X E$	$σ A R E$	$σ R M S E$	$σ M A X E$
kMI-GPR	0.5264	0.1410	0.8149	4.5148	0.2746	1.5371
kMI-SVM	11.2389	0.3768	1.2968	5.8723	0.5409	2.7882
PLS-GPR	3.9837	0.5892	3.0999	1.0641	2.5734	14.0338

References 30

1	卢涛, 关丹, 胡修玉, 等 . 我国生物发酵产业发展现状及发展趋势[J]. 工业微生物, 2015， (3): 62-66.
	Lu T , Guan D , Hu X Y , et al . Development status and trends of bio-fermentation industry in China[J]. Industrial Microbiology, 2015， (3): 62-66.
2	张嗣良 . 发酵工程技术发展现状与趋势[J]. 生物产业技术, 2011， (1): 25-32.
	Zhang S L . Development status and trend of fermentation engineering technology[J]. Biotechnology & Business, 2011， (1): 25-32.
3	Vanichsriratana W . Optimal control of fed-batch fermentation processes[D]. London：University of Westminster, 1996.
4	Weber R , Brosilow C . The use of secondary measurements to improve control[J]. AIChE Journal, 1972, 18(3): 614-623.
5	Joseph B , Brosilow C B . Inferential control of processes(Ⅰ): Steady state analysis and design[J]. AIChE Journal, 2010, 24(3): 485-492.
6	Joseph B , Brosilow C . Inferential control of processes(3): Construction of optimal and suboptimal dynamic estimators[J]. AIChE Journal, 1978, 24(3): 500-509.
7	Tham M T , Montague G A , Morris A J , et al . Soft sensors for process estimation and inferential control[J]. Journal of Process Control, 1991, 1(1): 3-14.
8	刘毅, 王海清, 李平 . 关键核网络及其在发酵过程在线建模中的应用[J]. 化工学报, 2008, 59(5): 1194-1199.
	Liu Y , Wang H Q , Li P . Key kernel network and its application to online modeling for fermentation processes[J]. Journal of Chemical Industry and Engineering(China), 2008, 59(5): 1194-1199.
9	Kadlec P , Grbic R , Gabrys B . Review of adaptation mechanisms for data-driven soft sensors[J]. Computers & Chemical Engineering, 2011, 35(1): 1-24.
10	Kaneko H , Funatsu K . Development of soft sensor models based on time difference of process variables with accounting for nonlinear relationship[J]. Industrial & Engineering Chemistry Research, 2011, 50(18): 10643-10651.
11	Zhao Y . Studies on modeling and control of continuous biotechnical processes[D]. Norway: Norwegian University of Science and Technology, 1996.
12	Trelea I C , Titica M , Landaud S , et al . Predictive modelling of brewing fermentation: from knowledge-based to black-box models[J]. Mathematics and Computers in Simulation, 2001, 56(4/5): 405-424.
13	房慧, 孙玉坤, 嵇小辅 . 青霉素发酵过程的粒子群模糊神经网络软测量[J]. 自动化仪表, 2011, 32(5): 46-48.
	Fang H , Sun Y K , Ji X F . Soft sensing based on particle swarm fuzzy neural network for penicillin fermentation process[J]. Process Automation Instrumentation, 2011, 32(5): 46-48.
14	杨强大, 侯新宇 . 诺西肽发酵过程中的分阶段软测量建模[J]. 化工学报, 2011, 62(6): 1612-1619.
	Yang Q D , Hou X Y . Staged soft-sensor modeling for nosiheptide fermentation process[J]. CIESC Journal, 2011, 62(6): 1612-1619.
15	桑海峰, 王福利, 何大阔, 等 . 基于多支持向量机的诺西肽发酵中菌体浓度软测量[J]. 系统仿真学报, 2006, 18(7): 1983-1986.
	Sang H F , Wang F L , He D K , et al . Soft sensors of biomass concentration in nosiheptied fermentation process based on multiple support vector machines[J]. Journal of System Simulation, 2006, 18(7): 1983-1986.
16	刘毅, 王海清, 李平 . 用于发酵过程在线建模的自适应局部最小二乘支持向量机回归方法[J]. 化工学报, 2008, 59(8): 2052-2057.
	Liu Y , Wang H Q , Li P . Adaptive local learning based least squares support vector regression with application to online modeling for fermentation processes[J]. Journal of Chemical Industry and Engineering(China), 2008, 59(8): 2052-2057.
17	Jin H , Chen X , Wang L , et al . Adaptive soft sensor development based on online ensemble Gaussian process regression for nonlinear time-varying batch processes[J]. Industrial & Engineering Chemistry Research, 2015, 54(30): 7320-7345.
18	Yu J , Chen K , Rashid M M . A Bayesian model averaging based multi-kernel Gaussian process regression framework for nonlinear state estimation and quality prediction of multiphase batch processes with transient dynamics and uncertainty[J]. Chemical Engineering Science, 2013, 93: 96-109.
19	Liu Y , Chen T , Chen J . Auto-switch Gaussian process regression-based probabilistic soft sensors for industrial multigrade processes with transitions[J]. Industrial & Engineering Chemistry Research, 2015, 54(18): 5037-5047.
20	Mei C L , Yang M , Shu D X , et al . Soft sensor based on Gaussian process regression and its application in erythromycin fermentation process[J]. Chemical Industry & Chemical Engineering Quarterly, 2016, 22(2): 127-135.
21	Zheng R J , Pan F . Soft sensor modeling of product concentration in glutamate fermentation using Gaussian process regression[J]. American Journal of Biochemistry and Biotechnology, 2016, 12(3): 179-187.
22	Jin H P , Chen X G , Yang J W , et al . Hybrid intelligent control of substrate feeding for industrial fed-batch chlortetracyclline fermentation process[J]. ISA Transactions, 2014, 53: 1822-1837.
23	康立宏, 姜帆 . 变温培养对青霉素发酵的影响[J]. 黑龙江医药, 2000, 13(5): 269-270.
	Kang L H , Jiang F . The influence to penicillin fermentation by changing culture temperatures[J]. Heilongjiang Medical Journal, 2000, 13(5): 269-270.
24	Rasmussen C E , Williams C K I . Gaussian Processes for Machine Learning[M]. Cambridge: MIT Press, 2005: 52.
25	Rasmussen C E . Gaussian processes in machine learning[C]// Summer School on Machine Learning. 2003: 63-71.
26	Kong D D , Chen Y J , Li N . Gaussian process regression for tool prediction[J]. MechSystand Signal Processing, 2018, 104: 556-574.
27	范雪莉, 冯海泓, 原猛 . 基于互信息的主成分分析特征选择算法[J]. 控制与决策, 2013, 28(6): 915-919.
	Fan X L , Feng H H , Yuan M . PCA based on mutual information foe feature selection[J]. Control and Decision, 2013, 28(6): 915-919.
28	Shannon C E . Communication theory of secrecy systems[J]. The Bell System Technical Journal, 2014, 28(4): 656-715.
29	Kraskov A , Stgbauer H , Grassberger P . Estimating mutual information[J]. Physical Review E Statistical Nonlinear & Soft Matter Physics, 2004, 696（Pt 2): 066138.
30	Undey C , Tatara E , Cinar A . Intelligent real-time performance monitoring and quality prediction for batch/fed-batch cultivations[J]. Journal of Biotechnology, 2004, 108(1): 61-77.

[1]	Linqi YAN, Zhenlei WANG. Multi-step predictive soft sensor modeling based on STA-BiLSTM-LightGBM combined model [J]. CIESC Journal, 2023, 74(8): 3407-3418.
[2]	Weiming SHAO, Wenxue HAN, Wei SONG, Yong YANG, Can CHEN, Dongya ZHAO. Dynamic soft sensor modeling method based on distributed Bayesian hidden Markov regression [J]. CIESC Journal, 2023, 74(6): 2495-2502.
[3]	WANG Haodong, WANG Xin, WANG Zhenlei, CAO Chenxin. Grading performance assessment method of chemical process based on Ms-NIPLS-GPR [J]. CIESC Journal, 2021, 72(3): 1549-1556.
[4]	LIU Cong, XIE Li, YANG Huizhong. Multi-model soft sensor development for penicillin fermentation process based on improved density peak clustering [J]. CIESC Journal, 2021, 72(3): 1606-1615.
[5]	WANG Kai, DU Wenli, LONG Jian. Online adaptive wavelength selection method and its application in gasoline blending process [J]. CIESC Journal, 2021, 72(2): 1059-1066.
[6]	Yuhao DU, Gaowei YAN, Rong LI, Fang WANG. Multiple working conditions soft sensor modeling method of geodesic flow kernel based on locally linear embedding [J]. CIESC Journal, 2020, 71(3): 1278-1287.
[7]	YANG Yijun,WANG Zhenlei,WANG Xin. Soft sensor modeling method based on hybrid model of nearest neighbor and neural network [J]. CIESC Journal, 2020, 71(12): 5696-5705.
[8]	Xuezhi DAI,Weili XIONG. A fast active learning method based on kernel extreme learning machine and its application for soft sensing [J]. CIESC Journal, 2020, 71(11): 5226-5236.
[9]	Xiaoqin LIAN, Liwei WANG, Sa AN, Wei WEI, Zaiwen LIU. On soft sensor of chemical oxygen demand by SOM-RBF neural network [J]. CIESC Journal, 2019, 70(9): 3465-3472.
[10]	Jing WU, Yiqi LIU, Jian LIU, Daoping HUANG, Yu QIU, Guangping YU. Study on the soft sensor of multi-kernel relevance vector machine based on time difference [J]. CIESC Journal, 2019, 70(4): 1472-1484.
[11]	Wenpeng JI, Huizhong YANG. Multi-manifold soft sensor based on modified expanding search clustering algorithm [J]. CIESC Journal, 2019, 70(2): 723-729.
[12]	Bao ZHU, Junfei QIAO. Features extracted from auto-encoder based echo state network and its applications to process modeling [J]. CIESC Journal, 2019, 70(12): 4770-4776.
[13]	QIU Yu, LIU Yiqi, WU Jing, HUANG Daoping. A self-adaptive multi-output soft sensor modeling based on deep neural network [J]. CIESC Journal, 2018, 69(7): 3101-3113.
[14]	ZHANG Lei, ZHANG Xiaogang, CHEN Hua. Soft sensors for multi-stage batch processes based on Gath-Geva algorithm and kernel extreme learning machine [J]. CIESC Journal, 2018, 69(6): 2576-2585.
[15]	SHAO Weiming, TIAN Xuemin, SONG Zhihuan. Ensemble learning-based soft sensor method for multi-product chemical processes [J]. CIESC Journal, 2018, 69(6): 2551-2559.