CIESC Journal ›› 2025, Vol. 76 ›› Issue (1): 266-282.DOI: 10.11949/0438-1157.20240708
• Process system engineering • Previous Articles Next Articles
Received:
2024-06-24
Revised:
2024-08-19
Online:
2025-02-08
Published:
2025-01-25
Contact:
Jiangang LU
通讯作者:
卢建刚
作者简介:
杨晔(1996—),男,博士研究生,工程师,11832016@zju.edu.cn
基金资助:
CLC Number:
Ye YANG, Jiangang LU. Mooney viscosity prediction modeling based on fusion Transformer[J]. CIESC Journal, 2025, 76(1): 266-282.
杨晔, 卢建刚. 基于融合Transformer的门尼系数预测建模研究[J]. 化工学报, 2025, 76(1): 266-282.
算法1:所提出模型算法伪代码 |
---|
Input: 1. data:实验数据集 2. tsp(·):包含可训练参数 Θtsp的初始化时序特征处理模块 3. temporal(·):包含可训练参数 Θtem的初始化时序编码-解码框架 4. sem(·):包含可训练参数 Θsem的初始化静态特征富集模块 5. f (·):包含可训练参数 Θff的初始化全连接层 Output:门尼系数预测值 for each batch ( X, Y ) from data do 1. 将输入数据 X 分割成时间序列变量 Xts以及静态协变量 Xstatic 2. 将时间序列变量 Xts输入到时序特征处理模块计算时序特征:outtsp=tsp( Xts; Θtsp) 3. 将时序特征outtsp输入到时序编码-解码框架计算时序特征向量:outtem=temporal(outtsp; Θtem) 4. 将静态协变量 Xstatic输入到静态特征富集模块计算静态特征向量:outstatic=sem( Xstatic; Θsem) 5. 将时序特征向量outtem与静态特征向量outstatic进行拼接计算预测输入:predin=[outtem,outstatic] 6. 将预测输入输入到全连接层计算最终预测结果: Ypred=f(predin; Θff) end for |
Table 1 Algorithm 1
算法1:所提出模型算法伪代码 |
---|
Input: 1. data:实验数据集 2. tsp(·):包含可训练参数 Θtsp的初始化时序特征处理模块 3. temporal(·):包含可训练参数 Θtem的初始化时序编码-解码框架 4. sem(·):包含可训练参数 Θsem的初始化静态特征富集模块 5. f (·):包含可训练参数 Θff的初始化全连接层 Output:门尼系数预测值 for each batch ( X, Y ) from data do 1. 将输入数据 X 分割成时间序列变量 Xts以及静态协变量 Xstatic 2. 将时间序列变量 Xts输入到时序特征处理模块计算时序特征:outtsp=tsp( Xts; Θtsp) 3. 将时序特征outtsp输入到时序编码-解码框架计算时序特征向量:outtem=temporal(outtsp; Θtem) 4. 将静态协变量 Xstatic输入到静态特征富集模块计算静态特征向量:outstatic=sem( Xstatic; Θsem) 5. 将时序特征向量outtem与静态特征向量outstatic进行拼接计算预测输入:predin=[outtem,outstatic] 6. 将预测输入输入到全连接层计算最终预测结果: Ypred=f(predin; Θff) end for |
字段名 | 变量类型 | 物理意义 | 备注说明 |
---|---|---|---|
Plan_Date | 静态协变量 | 生产计划日期 | YYYY/MM/DD |
Equip | 静态协变量 | 生产器械编号 | 数字格式 |
Type1 | 静态协变量 | 胶料主类别编号 | 字符串格式 |
Type2 | 静态协变量 | 胶料子类别编号 | 字符串格式 |
Done_Rtime | 静态协变量 | 排胶时间 | 单位为s |
PJ_Time | 静态协变量 | 卸料门开启时长 | 单位为s |
CB_DisTime | 静态协变量 | 加炭黑时长 | 单位为s |
Oil_DisTime | 静态协变量 | 加油时长 | 单位为s |
Mixing_Temp | 时序变量 | 实时温度 | 采样周期为1 s |
Mixing_Power | 时序变量 | 实时功率 | 采样周期为1 s |
Mixing_Press | 时序变量 | 实时压力 | 采样周期为1 s |
Mixing_Speed | 时序变量 | 实时转速 | 采样周期为1 s |
Mixing_Energy | 时序变量 | 实时能量 | 采样周期为1 s |
Item_Check | 静态协变量 | 门尼系数 | 预测目标 |
Table 2 Description of dataset feature fields
字段名 | 变量类型 | 物理意义 | 备注说明 |
---|---|---|---|
Plan_Date | 静态协变量 | 生产计划日期 | YYYY/MM/DD |
Equip | 静态协变量 | 生产器械编号 | 数字格式 |
Type1 | 静态协变量 | 胶料主类别编号 | 字符串格式 |
Type2 | 静态协变量 | 胶料子类别编号 | 字符串格式 |
Done_Rtime | 静态协变量 | 排胶时间 | 单位为s |
PJ_Time | 静态协变量 | 卸料门开启时长 | 单位为s |
CB_DisTime | 静态协变量 | 加炭黑时长 | 单位为s |
Oil_DisTime | 静态协变量 | 加油时长 | 单位为s |
Mixing_Temp | 时序变量 | 实时温度 | 采样周期为1 s |
Mixing_Power | 时序变量 | 实时功率 | 采样周期为1 s |
Mixing_Press | 时序变量 | 实时压力 | 采样周期为1 s |
Mixing_Speed | 时序变量 | 实时转速 | 采样周期为1 s |
Mixing_Energy | 时序变量 | 实时能量 | 采样周期为1 s |
Item_Check | 静态协变量 | 门尼系数 | 预测目标 |
参数 | 搜索范围 |
---|---|
d_model | 64, 128, 256, 512 |
encoder layer num | 1, 2, 3, 4 |
decoder layer num | 1, 2, 3, 4 |
dropout | 0.01, 0.02, 0.05, 0.1, 0.2 |
batch size | 16, 32, 64, 128 |
learning rate | 1×10-3, 1×10-4, 1×10-5 |
kernel size | 1, 2, 3, 4 |
dilation | 1, 2, 3, 4 |
clip | 40, 50, 60, 70 |
epoch | 80, 100, 120, 140 |
Table 3 Search range for experimental parameters
参数 | 搜索范围 |
---|---|
d_model | 64, 128, 256, 512 |
encoder layer num | 1, 2, 3, 4 |
decoder layer num | 1, 2, 3, 4 |
dropout | 0.01, 0.02, 0.05, 0.1, 0.2 |
batch size | 16, 32, 64, 128 |
learning rate | 1×10-3, 1×10-4, 1×10-5 |
kernel size | 1, 2, 3, 4 |
dilation | 1, 2, 3, 4 |
clip | 40, 50, 60, 70 |
epoch | 80, 100, 120, 140 |
参数 | 数值 |
---|---|
d_model | 256 |
encoder layer num | 3 |
decoder layer num | 1 |
dropout | 0.05 |
batch size | 64 |
learning rate | 0.0001 |
kernel size | 3 |
dilation | 3 |
clip | 60 |
epoch | 120 |
Table 4 Experimental parameter settings
参数 | 数值 |
---|---|
d_model | 256 |
encoder layer num | 3 |
decoder layer num | 1 |
dropout | 0.05 |
batch size | 64 |
learning rate | 0.0001 |
kernel size | 3 |
dilation | 3 |
clip | 60 |
epoch | 120 |
性能指标 | LSTM | LightGBM | ResNet1D | Transformer | Conv-Transformer | Fusformer |
---|---|---|---|---|---|---|
RMSE | 5.1946 | 4.7689 | 4.5132 | 4.3664 | 3.3798 | |
MAE | 3.7790 | 3.6576 | 3.5434 | 3.3530 | 2.6169 | |
CORR | 0.8673 | 0.8812 | 0.8891 | 0.9115 | 0.9323 |
Table 5 Comparison of Mooney viscosity prediction accuracy
性能指标 | LSTM | LightGBM | ResNet1D | Transformer | Conv-Transformer | Fusformer |
---|---|---|---|---|---|---|
RMSE | 5.1946 | 4.7689 | 4.5132 | 4.3664 | 3.3798 | |
MAE | 3.7790 | 3.6576 | 3.5434 | 3.3530 | 2.6169 | |
CORR | 0.8673 | 0.8812 | 0.8891 | 0.9115 | 0.9323 |
Ablation module | Model | RMSE | IMV | Δ |
---|---|---|---|---|
LEL | Transformer | 4.3664 | — | — |
Fusformer W/O LEL | 3.8132 | 14.51% | 0.4334 | |
Fusformer | 3.3798 | 28.99% | — | |
RPL&RMA | Transformer | 4.3664 | — | — |
Fusformer W/O RPL&RMA | 3.6656 | 19.12% | 0.2858 | |
Fusformer | 3.3798 | 28.99% | — | |
SEM | Transformer | 4.3664 | — | — |
Fusformer W/O SEM | 3.6243 | 20.47% | 0.2445 | |
Fusformer | 3.3798 | 28.99% | — | |
GLU | Transformer | 4.3664 | — | — |
Fusformer W/O GLU | 3.4562 | 26.37% | 0.0764 | |
Fusformer | 3.3798 | 28.99% | — |
Table 6 Comparison of ablation experiment results for each module
Ablation module | Model | RMSE | IMV | Δ |
---|---|---|---|---|
LEL | Transformer | 4.3664 | — | — |
Fusformer W/O LEL | 3.8132 | 14.51% | 0.4334 | |
Fusformer | 3.3798 | 28.99% | — | |
RPL&RMA | Transformer | 4.3664 | — | — |
Fusformer W/O RPL&RMA | 3.6656 | 19.12% | 0.2858 | |
Fusformer | 3.3798 | 28.99% | — | |
SEM | Transformer | 4.3664 | — | — |
Fusformer W/O SEM | 3.6243 | 20.47% | 0.2445 | |
Fusformer | 3.3798 | 28.99% | — | |
GLU | Transformer | 4.3664 | — | — |
Fusformer W/O GLU | 3.4562 | 26.37% | 0.0764 | |
Fusformer | 3.3798 | 28.99% | — |
1 | Kim J K, Mudambi R. An ecosystem-based analysis of design innovation infringements: South Korea and China in the global tire industry[J]. Journal of International Business Policy, 2020, 3(1): 38-57. |
2 | 陈志宏, 胡浩, 赵敏. 创新, 综合创新, 再创新: 共同实现轮胎全行业高质量发展[J]. 橡胶工业, 2023, 70(9): 643-654. |
Chen Z H, Hu H, Zhao M. Innovation, comprehensive innovation, and re-innovation—jointly achieving high-quality development of entire tire industry[J]. China Rubber Industry, 2023, 70(9): 643-654. | |
3 | 朱孔阳, 方之峻, 魏向阳. 混炼胶门尼黏度的预测与控制[J]. 轮胎工业, 1997, 17(10): 608-611. |
Zhu K Y, Fang Z J, Wei X Y. Prediction and control of mooney viscosity of rubber mix[J]. Tire Industry, 1997, 17(10): 608-611. | |
4 | 朱海亮. 提高顺丁橡胶胶液门尼黏度测定准确性的研究[J]. 橡胶工业, 2017, 64(2): 121-122. |
Zhu H L. Study on improving the accuracy of Mooney viscosity determination of cis-polybutadiene rubber solution[J]. China Rubber Industry, 2017, 64(2): 121-122. | |
5 | Liu Y, Gao Z. Real-time property prediction for an industrial rubber-mixing process with probabilistic ensemble Gaussian process regression models[J]. Journal of Applied Polymer Science, 2015, 132(6):41432. |
6 | Gurdeep Singh H K, Yusup S, Abdullah B, et al. Refining of crude rubber seed oil as a feedstock for biofuel production[J]. Journal of Environmental Management, 2017, 203: 1011-1016. |
7 | 朱锋峰, 张海, 贺德化, 等. 混炼胶门尼黏度的预测[J]. 特种橡胶制品, 1999, 20(3): 47-49. |
Zhu F F, Zhang H, He D H, et al. Prediction of Mooney viscosity of rubber compounds[J]. Special Purpose Rubber Products, 1999, 20(3): 47-49. | |
8 | 李俊, 马铁军, 陈国华, 等. 基于多因素的混炼胶质量预测模型[J]. 橡胶工业, 2006, 53(10): 614-617. |
Li J, Ma T J, Chen G H, et al. Model for predicting mix quality based on multi-factors[J]. China Rubber Industry, 2006, 53(10): 614-617. | |
9 | Liu M, Huang D P, Sun Z H, et al. Combining KPCA with LSSVM for the Mooney-viscosity forecasting[C]//2008 Second International Conference on Genetic and Evolutionary Computing. IEEE, 2008: 522-526. |
10 | 徐唯易. 炼胶工艺参数与门尼黏度关系的研究[D]. 哈尔滨: 哈尔滨工业大学, 2018. |
Xu W Y. Study on the relationship between rubber mixing process parameters and Mooney viscosity[D]. Harbin: Harbin Institute of Technology, 2018. | |
11 | 李高伟, 李佳, 朱金梅, 等. 基于不同算法优化的back propagation神经网络在三元乙丙橡胶混炼胶门尼黏度预测中的应用[J]. 合成橡胶工业, 2023, 46(6): 488-494. |
Li G W, Li J, Zhu J M, et al. Application of back propagation neural network optimized by different algorithms in prediction of Mooney viscosity of ethylene-propylene-diene monomer compound[J]. China Synthetic Rubber Industry, 2023, 46(6): 488-494. | |
12 | 金怀平, 张燕, 董守龙, 等. 基于半监督集成即时学习的橡胶混炼过程门尼黏度软测量研究[J]. 高校化学工程学报, 2022, 36(4): 586-596. |
Jin H P, Zhang Y, Dong S L, et al. Study on semi-supervised ensemble just-in-time learning based soft sensing of Mooney viscosity in rubber mixing process[J]. Journal of Chemical Engineering of Chinese Universities, 2022, 36(4): 586-596. | |
13 | Liu H C, Cui Z X, Yue J G, et al. A sparse data gas sensor array feature mining method for rubber Mooney viscosity measurement[J]. Sensors and Actuators A: Physical, 2024, 373: 115335. |
14 | Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008. |
15 | Gillioz A, Casas J, Mugellini E, et al. Overview of the transformer-based models for NLP tasks[C]//2020 15th Conference on Computer Science and Information Systems (FedCSIS). IEEE, 2020: 179-183. |
16 | Jain LC, Medsker L R. Recurrent Neural Networks: Design and Applications[M]. Boca Raton: CRC press, 1999. |
17 | Gu J X, Wang Z H, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377. |
18 | 石争浩, 李成建, 周亮, 等. Transformer驱动的图像分类研究进展[J]. 中国图象图形学报, 2023(9): 2661-2692. |
Shi Z H, Li C J, Zhou L, et al. Survey on transformer for image classification[J]. Journal of Image and Graphics, 2023(9): 2661-2692. | |
19 | Dong L H, Xu S, Xu B. Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018: 5884-5888. |
20 | Zhou H Y, Zhang S H, Peng J Q, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11106-11115. |
21 | 陆宁云, 王福利, 高福荣, 等. 间歇过程的统计建模与在线监测[J]. 自动化学报, 2006, 32(3): 400-410. |
Lu N Y, Wang F L, Gao F R, et al. Statistical modeling and online monitoring for batch processes[J]. Acta Automatica Sinica, 2006, 32(3): 400-410. | |
22 | 于清溪. 橡胶混炼设备使用现状与工艺发展[J]. 橡塑技术与装备, 2007, 33(5): 6-16. |
Yu Q X. Present service status and process development of rubber mixing equipment[J]. China Rubber/Plastics Technology and Equipment, 2007, 33(5): 6-16. | |
23 | 黄树林. 国产开炼机的发展历程与趋势[J]. 橡胶工业, 2007, 54(7): 440-443. |
Huang S L. Development and trend of domestic roller mixer[J]. China Rubber Industry, 2007, 54(7): 440-443. | |
24 | 马铁军, 蔡群英, 黄有发, 等. 橡胶在密炼机内混炼过程的分析[J]. 华南理工大学学报(自然科学版), 1997, 25(3): 119-122. |
Ma T J, Cai Q Y, Huang Y F, et al. Analysis of rubber mixing processin internal nixers[J]. Journal of South China University of Technology (Natural Science Edition), 1997, 25(3): 119-122. | |
25 | 杨顺根. 密炼机的现状发展与趋势[J]. 橡塑技术与装备, 2008, 34(2): 18-23. |
Yang S G. The present status and development trends of mixer[J]. China Rubber/Plastics Technology and Equipment, 2008, 34(2): 18-23. | |
26 | 张海, 马铁军, 钟荣. 密炼机混炼过程功率曲线的作用[J]. 橡胶工业, 1999, 46(4): 232-234. |
Zhang H, Ma T J, Zhong R. The effect of power curve in the mixing process of internal mixer[J]. China Rubber Industry, 1999, 46(4): 232-234. | |
27 | Montero-Manso P, Hyndman R J. Principles and algorithms for forecasting groups of time series: locality and globality[J]. International Journal of Forecasting, 2021, 37(4): 1632-1653. |
28 | Li Y H, Zhang X F, Chen D M. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 1091-1100. |
29 | Zhang J, Sun J F, Luo X D, et al. Characterizing pseudoperiodic time series through the complex network approach[J]. Physica D: Nonlinear Phenomena, 2008, 237(22): 2856-2865. |
30 | Li S, Jin X, Xuan Y, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, BC, Canada, 2019: 5243-5253. |
31 | Jung S, Moon J, Park S, et al. Self-attention-based deep learning network for regional influenza forecasting[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(2): 922-933. |
32 | Noh S H. Analysis of gradient vanishing of RNNs and performance comparison[J]. Information, 2021, 12(11): 442. |
33 | Rasamoelina A D, Adjailia F, Sinčák P. A review of activation function for artificial neural network[C]//2020 IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE, 2020: 281-286. |
34 | Khadka N. General machine learning practices using Python[D].Oulu: Oulu University of Applied Science 2019. |
35 | Dahouda M K, Joe I. A deep-learned embedding technique for categorical features encoding[J]. IEEE Access, 2021, 9: 114381-114391. |
36 | Henderi H. Comparison of min-max normalization and Z-score normalization in the k-nearest neighbor (kNN) algorithm to test the accuracy of types of breast cancer[J]. International Journal of Informatics and Information Systems, 2021, 4(1): 13-20. |
37 | Bergstra J, Bengio Y. Random search for hyper-parameter optimization[J]. Journal of Machine Learning Research, 2012, 13: 281-305. |
38 | Ke G, Meng Q, Finley T, et al. LightGBM: a highly efficient gradient boosting decision tree[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA, 2017: 3149-3157. |
39 | Smagulova K, James A P. A survey on LSTM memristive neural network architectures and applications[J]. The European Physical Journal Special Topics, 2019, 228(10): 2313-2324. |
40 | Chen D W, Hu F, Nian G K, et al. Deep residual learning for nonlinear regression[J]. Entropy, 2020, 22(2): 193. |
41 | Meyes R, Lu M, de Puiseau C W, et al. Ablation studies in artificial neural networks[EB/OL]. . |
[1] | Ping LIU, Yusheng QIU, Shijing LI, Ruiqi SUN, Chen SHEN. Heat transfer and flow characteristics of nanofluids in microchannels [J]. CIESC Journal, 2025, 76(1): 184-197. |
[2] | Haijun FENG, Bingxuan ZHANG, Jian ZHOU. Predicting and interpreting the toxicity of ionic liquids using graph neural network [J]. CIESC Journal, 2025, 76(1): 93-106. |
[3] | Yushi LI, Yuan CHEN, Yuntang LI, Xudong PENG, Bingqing WANG, Xiaolu LI. Intelligent optimization and deformation analysis of novel flexible dam foil face gas seal [J]. CIESC Journal, 2025, 76(1): 324-334. |
[4] | Haidong LI, Qiqi ZHANG, Lu YANG, Naeem AKRAM, Chenglin CHANG, Wenlong MO, Weifeng SHEN. Detailed design of shell-and-tube heat exchanger using intelligent evolutionary algorithms [J]. CIESC Journal, 2025, 76(1): 241-255. |
[5] | Shaoji WANG, Kuangrong HAO, Lei CHEN. Research on temperature forecasting of polyester fiber esterification process based on federated learning [J]. CIESC Journal, 2025, 76(1): 283-295. |
[6] | Wuling ZHAO, Yi MAN. Research on framework of nanocellulose molecular structure prediction model based on variational encoder [J]. CIESC Journal, 2024, 75(9): 3221-3230. |
[7] | Xin GUO, Wenjing LI, Junfei QIAO. Prediction of effluent parameters in wastewater treatment process using self-organizing modular neural network [J]. CIESC Journal, 2024, 75(9): 3242-3254. |
[8] | Ziliang ZHU, Shuang WANG, Yu'ang JIANG, Mei LIN, Qiuwang WANG. Solid-liquid phase change algorithm with Euler-Lagrange iteration [J]. CIESC Journal, 2024, 75(8): 2763-2776. |
[9] | Qian LI, Rongmin ZHANG, Zijie LIN, Qi ZHAN, Weihua CAI. Prediction and simulation of flow and heat transfer for printed circuit plate heat exchanger based on machine learning [J]. CIESC Journal, 2024, 75(8): 2852-2864. |
[10] | Yanxi LI, Yechun WANG, Xiangdong XIE, Jinzhi WANG, Jiang WANG, Yu ZHOU, Yingxiu PAN, Wentao DING, Liejin GUO. Study on separation characteristics and structure optimization of a volute type multi-channel gas-liquid cyclone separator [J]. CIESC Journal, 2024, 75(8): 2875-2885. |
[11] | Hongrui LI, Chunxi HUANG, Xiaodong HONG, Zuwei LIAO, Jingdai WANG, Yongrong YANG. An adaptive variable-step homotopy-based algorithm for process simulation with cyclic streams [J]. CIESC Journal, 2024, 75(7): 2604-2612. |
[12] | Ziyang LI, Nan ZHENG, Jiabin FANG, Jinjia WEI. Performance analysis and multi-objective optimization of recompression S-CO2 Brayton cycle [J]. CIESC Journal, 2024, 75(6): 2143-2156. |
[13] | Han ZHANG, Shuning ZHANG, Ke LIU, Guanlong DENG. Particle size prediction of cobalt oxalate synthesis process based on slow feature analysis and least squares support vector regression [J]. CIESC Journal, 2024, 75(6): 2313-2321. |
[14] | Zhihong HUANG, Li ZHOU, Shiyang CHAI, Xu JI. Integrating optimization of hydrogenation units in multi-period hydrogen network [J]. CIESC Journal, 2024, 75(5): 1951-1965. |
[15] | Lei XIE, Yongsheng XU, Mei LIN. Comparative study on single-phase flow and heat transfer of different cross-section rib-soft tail structures [J]. CIESC Journal, 2024, 75(5): 1787-1801. |
Viewed | ||||||||||||||||||||||||||||||||||||||||||||||||||
Full text 17
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
Abstract 51
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||