Fault monitoring of fermentation process based on attention dynamic convolutional autoencoder

doi:10.11949/0438-1157.20230332

Abstract

Abstract:

The status monitoring of the fermentation process plays a vital role in timely detection of various abnormal faults. However, due to the nonlinear characteristics of the fermentation process data, it is difficult to extract feature information, which increases the difficulty of fault monitoring. In order to solve the above problems, an attention dynamic convolutional autoencoder (ADCAE) based fault monitoring method for fermentation process is proposed. Firstly, a dynamic convolution structure is designed, which can extract low-level features using large-size convolution kernels in the shallow layer, and extract high-level features using small-size convolution kernels in the deep layer, thereby broadening the scope of model feature learning scale. Secondly, a channel convolutional attention (CCA) module is designed, which can extract the nonlinear features of input from different scales, and can be better extract local features in the process of converting channel vectors into weights, which improves the ability to pay attention to effective information. Finally, the dynamic convolution structure and CCA module are integrated into the convolutional autoencoder, so that the model can effectively capture the nonlinear relationship in the variables, so as to better cope with the problem of fault monitoring in the fermentation process. The feasibility of the method was verified by using the simulation platform of penicillin fermentation process and the actual production data of Escherichia coli, and the results showed that the method had good fault monitoring performance.

Key words: fermentation, algorithm, nonlinear, fault monitoring, neural networks, attention mechanism, convolutional autoencoder

摘要：

发酵过程的状态监测对于及时发现各类异常故障起到了至关重要的作用。然而，由于发酵过程数据呈现非线性特性，导致在提取特征信息时存在困难，增加了故障监测的难度。为了解决上述问题，提出了一种基于注意力动态卷积自编码器（attention dynamic convolutional autoencoder， ADCAE）的发酵过程故障监测方法。首先，设计了一种动态卷积结构（dynamic convolution structure），该结构可以在浅层使用大尺寸卷积核提取低级特征，在深层使用小尺寸卷积核提取高级特征，从而拓宽了模型特征学习的尺度；其次，设计了一种通道卷积注意力（channel convolutional attention， CCA）模块，该模块能够从不同尺度提取输入的非线性特征，并且在通道向量转化为权重的过程中可以更好地提取局部特征，提高了对有效信息的关注能力；最后，将动态卷积结构与CCA模块融入卷积自编码器中，使模型能够有效地捕获变量中的非线性关系，从而更好地应对发酵过程中的故障监测问题。利用青霉素发酵过程仿真平台和大肠埃希菌实际生产数据对该方法的可行性进行了验证，结果表明该方法具有较好的故障监测性能。

关键词: 发酵, 算法, 非线性, 故障监测, 神经网络, 注意力机制, 卷积自编码器

CLC Number:

TP 277

Xuejin GAO, Yuzhuo YAO, Huayun HAN, Yongsheng QI. Fault monitoring of fermentation process based on attention dynamic convolutional autoencoder[J]. CIESC Journal, 2023, 74(6): 2503-2521.

高学金, 姚玉卓, 韩华云, 齐咏生. 基于注意力动态卷积自编码器的发酵过程故障监测[J]. 化工学报, 2023, 74(6): 2503-2521.

Figures/Tables 30

References 36

1	Zhang S M, Zhao C H, Gao F R. Incipient fault detection for multiphase batch processes with limited batches[J]. IEEE Transactions on Control Systems Technology, 2019, 27(1): 103-117.
2	Ren J Y, Ni D. A batch-wise LSTM-encoder decoder network for batch process monitoring[J]. Chemical Engineering Research and Design, 2020, 164: 102-112.
3	Zhang S M, Zhao C H. Slow-feature-analysis-based batch process monitoring with comprehensive interpretation of operation condition deviation and dynamic anomaly[J]. IEEE Transactions on Industrial Electronics, 2019, 66(5): 3773-3783.
4	Yu W K, Zhao C H, Huang B. Stationary subspace analysis-based hierarchical model for batch processes monitoring[J]. IEEE Transactions on Control Systems Technology, 2021, 29(1): 444-453.
5	Abbasi M A, Khan A Q, Mustafa G, et al. Data-driven fault diagnostics for industrial processes: an application to penicillin fermentation process[J]. IEEE Access, 2021, 9: 65977-65987.
6	Li L M, Zhao J, Wang C R, et al. Comprehensive evaluation of robotic global performance based on modified principal component analysis[J]. International Journal of Advanced Robotic Systems, 2020, 17(4): 172988141989688.
7	邓佳伟, 邓晓刚, 曹玉苹, 等. 基于加权统计局部核主元分析的非线性化工过程微小故障诊断方法[J]. 化工学报, 2019, 70(7): 2594-2605.
	Deng J W, Deng X G, Cao Y P, et al. Incipient fault diagnosis method of nonlinear chemical process based on weighted statistical local KPCA[J]. CIESC Journal, 2019, 70(7): 2594-2605.
8	Uurtio V, Monteiro J M, Kandola J, et al. A tutorial on canonical correlation methods[J]. ACM Computing Surveys, 2018, 50(6): 1-33.
9	高学金, 何紫鹤, 高慧慧, 等. 基于联合典型变量矩阵的多阶段发酵过程质量相关故障监测[J]. 化工学报, 2022, 73(3): 1300-1314.
	Gao X J, He Z H, Gao H H, et al. Quality-related fault monitoring of multi-phase fermentation process based on joint canonical variable matrix[J]. CIESC Journal, 2022, 73(3): 1300-1314.
10	Xiong W P, Li T C, Zeng Q X, et al. Research on partial least squares method based on deep confidence network in traditional Chinese medicine[J]. Discrete Dynamics in Nature and Society, 2020, 2020: 4142824.
11	胡益, 王丽, 马贺贺, 等. 基于核PLS方法的非线性过程在线监控[J]. 化工学报, 2011, 62(9): 2555-2561.
	Hu Y, Wang L, Ma H H, et al. Online nonlinear process monitoring using kernel partial least squares[J]. CIESC Journal, 2011, 62(9): 2555-2561.
12	常鹏, 王普, 高学金. 基于统计量模式分析的T-KPLS间歇过程故障监控[J]. 化工学报, 2015, 66(1): 265-271.
	Chang P, Wang P, Gao X J. Fault monitoring batch process based on statistics pattern analysis of T-KPLS[J]. CIESC Journal, 2015, 66(1): 265-271.
13	栗志意, 张卫强, 何亮, 等. 基于核函数的IVEC-SVM说话人识别系统研究[J]. 自动化学报, 2014, 40(4): 780-784.
	Li Z Y, Zhang W Q, He L, et al. Speaker recognition with kernel based IVEC-SVM[J]. Acta Automatica Sinica, 2014, 40(4): 780-784.
14	Liu Y, Yang C, Gao Z L, et al. Ensemble deep kernel learning with application to quality prediction in industrial polymerization processes[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 174: 15-21.
15	Lee J M, Yoo C, Choi S W, et al. Nonlinear process monitoring using kernel principal component analysis[J]. Chemical Engineering Science, 2004, 59(1): 223-234.
16	Deng X G, Tian X M, Chen S, et al. Deep principal component analysis based on layerwise feature extraction and its application to nonlinear process monitoring[J]. IEEE Transactions on Control Systems Technology, 2019, 27(6): 2526-2540.
17	Wang Y J, Yu H L, Li X H. Efficient iterative dynamic kernel principal component analysis monitoring method for the batch process with super-large-scale data sets[J]. ACS Omega, 2021, 6(15): 9989-9997.
18	Tan R M, Ottewill J R, Thornhill N F. Monitoring statistics and tuning of kernel principal component analysis with radial basis function kernels[J]. IEEE Access, 2020, 8: 198328-198342.
19	Lee H, Kim C, Jeong D H, et al. Data-driven fault detection for chemical processes using autoencoder with data augmentation[J]. Korean Journal of Chemical Engineering, 2021, 38(12): 2406-2422.
20	Yan W W, Guo P J, Gong L, et al. Nonlinear and robust statistical process monitoring based on variant autoencoders[J]. Chemometrics and Intelligent Laboratory Systems, 2016, 158: 31-40.
21	Masci J, Meier U, Cireşan D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction[M]//Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 52-59.
22	Yang J W, Lee Y D, Koo I S. Convolutional autoencoder-based sensor fault classification[C]//2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN). Prague, Czech Republic: IEEE, 2018: 865-867.
23	Ma X, Liu Z Z, Zheng M X, et al. Application and exploration of self-attention mechanism in dynamic process monitoring[J]. IFAC-PapersOnLine, 2022, 55(6): 139-144.
24	Hassanin M, Anwar S, Radwan I, et al. Visual attention methods in deep learning: an in-depth survey[EB/OL]. 2022. .
25	Zhang H, Zu K K, Lu J, et al. EPSANet: an efficient pyramid squeeze attention block on convolutional neural network[EB/OL]. 2021. .
26	Luo X Y, Li X M, Wang Z Y, et al. Discriminant autoencoder for feature extraction in fault diagnosis[J]. Chemometrics and Intelligent Laboratory Systems, 2019, 192: 103814.
27	Yang S K, Kong X G, Wang Q B, et al. Deep multiple auto-encoder with attention mechanism network: a dynamic domain adaptation method for rotary machine fault diagnosis under different working conditions[J]. Knowledge-Based Systems, 2022, 249: 108639.
28	Yu F, Liu J C, Liu D M, et al. Supervised convolutional autoencoder-based fault-relevant feature learning for fault diagnosis in industrial processes[J]. Journal of the Taiwan Institute of Chemical Engineers, 2022, 132: 104200.
29	Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
30	Liu X, Yu J B, Ye L. Residual attention convolutional autoencoder for feature learning and fault detection in nonlinear industrial processes[J]. Neural Computing and Applications, 2021, 33(19): 12737-12753.
31	Aguado D, Ferrer A, Ferrer J, et al. Multivariate SPC of a sequencing batch reactor for wastewater treatment[J]. Chemometrics and Intelligent Laboratory Systems, 2007, 85(1): 82-93.
32	Chen Q, Wynne R J, Goulding P, et al. The application of principal component analysis and kernel density estimation to enhance process monitoring[J]. Control Engineering Practice, 2000, 8(5): 531-543.
33	于蕾, 邓晓刚, 曹玉苹, 等. 基于变量分组DTW-MCVA的不等长间歇过程故障检测方法[J]. 化工学报, 2019, 70(9): 3441-3448.
	Yu L, Deng X G, Cao Y P, et al. Fault detection method of unequal-length batch process based on VGDTW-MCVA[J]. CIESC Journal, 2019, 70(9): 3441-3448.
34	Birol G, Ündey C, Çinar A. A modular simulation package for fed-batch fermentation: penicillin production[J]. Computers & Chemical Engineering, 2002, 26(11): 1553-1565.
35	刘毅, 王海清. Pensim仿真平台在青霉素发酵过程的应用研究[J]. 系统仿真学报, 2006, 18(12): 3524-3527.
	Liu Y, Wang H Q. Pensim simulator and its application in penicillin fermentation process[J]. Journal of System Simulation, 2006, 18(12): 3524-3527.
36	高学金, 刘腾飞, 徐子东, 等. 基于循环自动编码器的间歇过程故障监测[J]. 化工学报, 2020, 71(7): 3172-3179.
	Gao X J, Liu T F, Xu Z D, et al. Intermittent process fault monitoring based on recurrent autoencoder[J]. CIESC Journal, 2020, 71(7): 3172-3179.

变量编号	变量	单位
x₁	通风速率	L/h
x₂	搅拌功率	W
x₃	底物流加速率	L/h
x₄	补料温度	K
x₅	溶解氧浓度	%(质量)
x₆	排气二氧化碳浓度	mmol/L
x₇	pH	—
x₈	反应器温度	K
x₉	反应热	kcal/h
x₁₀	冷水流加速率	L/h

变量编号	变量	单位
x₁	通风速率	L/h
x₂	搅拌功率	W
x₃	底物流加速率	L/h
x₄	补料温度	K
x₅	溶解氧浓度	%(质量)
x₆	排气二氧化碳浓度	mmol/L
x₇	pH	—
x₈	反应器温度	K
x₉	反应热	kcal/h
x₁₀	冷水流加速率	L/h

故障批次	故障变量	故障类型	幅值/%	持续时间/h
1	底物流加速率	斜坡	5	200~400
2	底物流加速率	阶跃	5	200~400
3	搅拌功率	斜坡	1	200~400
4	搅拌功率	阶跃	1	200~400
5	通风速率	阶跃	5	200~400

故障批次	故障变量	故障类型	幅值/%	持续时间/h
1	底物流加速率	斜坡	5	200~400
2	底物流加速率	阶跃	5	200~400
3	搅拌功率	斜坡	1	200~400
4	搅拌功率	阶跃	1	200~400
5	通风速率	阶跃	5	200~400

网络层	输入维度	输出维度	卷积核	激活函数
卷积层1	1×200×10	8×200×8	（1, 3）	ReLU
卷积层2	8×200×8	15×200×6	（1, 3）	ReLU
卷积层3	15×200×6	22×200×4	（1, 3）	ReLU
反卷积层1	22×200×4	15×200×6	（1, 3）	ReLU
反卷积层2	15×200×6	8×200×8	（1, 3）	ReLU
反卷积层3	8×200×8	1×200×10	（1, 3）	ReLU