基于融合Transformer的门尼系数预测建模研究

doi:10.11949/0438-1157.20240708

摘要/Abstract

摘要：

门尼系数的精准预测是优化橡胶混炼工艺的关键一环，有助于橡胶轮胎生产过程及时把控胶料质量。为此，提出了一种基于融合Transformer的门尼系数预测模型（fusion Transformer，Fusformer），对橡胶混炼产生的时间序列变量与静态协变量数据进行针对性建模，提取并融合各类数据特征，精确预测门尼系数。针对时间序列变量，模型引入有向图的概念，提出相对位置感知层与相对多头注意力机制，充分地捕捉时序依赖特征；针对静态协变量，模型引入门控线性单元并提出了静态特征富集模块，提取有效静态特征，最终将时序依赖特征与有效静态特征融合并输出预测值。实验利用大型橡胶轮胎企业的20万条实际采样样本测试了模型的预测性能，通过对照实验、消融分析以及参数灵敏度分析论证了模型设计的合理性。

关键词: 门尼系数, 橡胶混炼, 多变量时间序列预测, 神经网络, 静态协变量, 优化设计, 算法

Abstract:

Accurate prediction of Mooney viscosity is a key link in optimizing rubber mixing process, which helps to timely control rubber quality in rubber tire production process. To this end, a Mooney viscosity prediction model based on fusion Transformer (Fusformer) is proposed to conduct targeted modeling of time series variables and static covariates data generated by rubber mixing, extract and fuse various data features, and accurately predict Mooney viscosity. For time series variables, the model introduces the concept of directed graph, proposes relative position perception layer and relative multi-head attention mechanism to fully capture time series dependency features; for static covariates, the model introduces gated linear units and proposes static feature enrichment module to extract effective static features, and finally fuses time series dependency features with effective static features and outputs predicted values. The experiment uses 200000 actual samples from a rubber factory to test the prediction performance of the model, and demonstrates the rationality of the model design through comparative experiments, ablation analysis and parameter sensitivity analysis.

Key words: Mooney viscosity, rubber mixing, multivariate time series prediction, neural network, static covariates, optimal design, algorithm

中图分类号:

TQ 028.8

杨晔, 卢建刚. 基于融合Transformer的门尼系数预测建模研究[J]. 化工学报, 2025, 76(1): 266-282.

Ye YANG, Jiangang LU. Mooney viscosity prediction modeling based on fusion Transformer[J]. CIESC Journal, 2025, 76(1): 266-282.

图/表 39

图1 密炼机结构示意图

Fig.1 Schematic diagram of mixer

图2 橡胶混炼部分工艺参数曲线

Fig.2 Curve of several process parameters in rubber mixing

图3 Fusformer模型框架图

Fig.3 Framework diagram of Fusformer

图4 时序特征处理模块结构示意图

Fig.4 Framework diagram of time series data processing block

图5 忽略上下文信息情况

Fig.5 Ignoring contextual information

图6 考虑上下文信息情况

Fig.6 Considering contextual information

图7 时序编码-解码框架结构示意图

Fig.7 Diagram of temporal encoder-decoder framework

图8 两种乘积结果曲线对比

Fig.8 Comparison of two product result curves

图9 有向图思想对元素相对位置信息进行表征示意图

Fig.9 Schematic diagram of the directed graph representing the relative position information of elements

图10 相对位置感知层计算流程示意图

Fig.10 Schematic diagram of relative position-aware layer calculation process

图11 静态特征富集模块结构示意图

Fig.11 Diagram of static enrichment module

表1 算法1

Table 1 Algorithm 1

算法1：所提出模型算法伪代码

Input：

1. data：实验数据集

2. tsp(·)：包含可训练参数 Θ_tsp的初始化时序特征处理模块

3. temporal(·)：包含可训练参数 Θ_tem的初始化时序编码-解码框架

4. sem(·)：包含可训练参数 Θ_sem的初始化静态特征富集模块

5. f (·)：包含可训练参数 Θ_ff的初始化全连接层

Output：门尼系数预测值

for each batch ( X, Y ) from data do

1. 将输入数据 X 分割成时间序列变量 X_ts以及静态协变量 X_static

2. 将时间序列变量 X_ts输入到时序特征处理模块计算时序特征：out_tsp=tsp( X_ts; Θ_tsp)

3. 将时序特征out_tsp输入到时序编码-解码框架计算时序特征向量：out_tem=temporal(out_tsp; Θ_tem)

4. 将静态协变量 X_static输入到静态特征富集模块计算静态特征向量：out_static=sem( X_static; Θ_sem)

5. 将时序特征向量out_tem与静态特征向量out_static进行拼接计算预测输入：pred_in=[out_tem,out_static]

6. 将预测输入输入到全连接层计算最终预测结果： Y_pred=f(pred_in; Θ_ff)

end for

表2 数据集特征字段说明

Table 2 Description of dataset feature fields

字段名	变量类型	物理意义	备注说明
Plan_Date	静态协变量	生产计划日期	YYYY/MM/DD
Equip	静态协变量	生产器械编号	数字格式
Type1	静态协变量	胶料主类别编号	字符串格式
Type2	静态协变量	胶料子类别编号	字符串格式
Done_Rtime	静态协变量	排胶时间	单位为s
PJ_Time	静态协变量	卸料门开启时长	单位为s
CB_DisTime	静态协变量	加炭黑时长	单位为s
Oil_DisTime	静态协变量	加油时长	单位为s
Mixing_Temp	时序变量	实时温度	采样周期为1 s
Mixing_Power	时序变量	实时功率	采样周期为1 s
Mixing_Press	时序变量	实时压力	采样周期为1 s
Mixing_Speed	时序变量	实时转速	采样周期为1 s
Mixing_Energy	时序变量	实时能量	采样周期为1 s
Item_Check	静态协变量	门尼系数	预测目标

图12 时序特征补齐操作示意图

Fig.12 Schematic diagram of sequence feature padding

表3 实验参数的搜索范围

Table 3 Search range for experimental parameters

参数	搜索范围
d_model	64, 128, 256, 512
encoder layer num	1, 2, 3, 4
decoder layer num	1, 2, 3, 4
dropout	0.01, 0.02, 0.05, 0.1, 0.2
batch size	16, 32, 64, 128
learning rate	1×10^-3, 1×10^-4, 1×10^-5
kernel size	1, 2, 3, 4
dilation	1, 2, 3, 4
clip	40, 50, 60, 70
epoch	80, 100, 120, 140

表4 实验模型参数设置

Table 4 Experimental parameter settings

参数	数值
d_model	256
encoder layer num	3
decoder layer num	1
dropout	0.05
batch size	64
learning rate	0.0001
kernel size	3
dilation	3
clip	60
epoch	120

表5 门尼系数预测精度对比

Table 5 Comparison of Mooney viscosity prediction accuracy

性能指标	LSTM	LightGBM	ResNet1D	Transformer	Conv-Transformer	Fusformer
RMSE	5.1946	4.7689	4.5132	4.3664	4.2379	3.3798
MAE	3.7790	3.6576	3.5434	3.3530	3.2531	2.6169
CORR	0.8673	0.8812	0.8891	0.9115	0.9228	0.9323

图13 采样时长100 s内的样本预测结果

Fig.13 Prediction results with a sampling duration of 100 s

图14 采样时长100~120 s内的样本预测结果

Fig.14 Prediction results with a sampling duration of 100 to 120 s

图15 采样时长120~140 s内的样本预测结果

Fig.15 Prediction results with a sampling duration of 120 s to 140 s

图16 采样时长超过140 s的样本预测结果

Fig.16 Prediction results with a sampling duration exceeding 140 s

表6 各模块消融实验结果对比

Table 6 Comparison of ablation experiment results for each module

Ablation module	Model	RMSE	IMV	Δ
LEL	Transformer	4.3664	—	—
	Fusformer W/O LEL	3.8132	14.51%	0.4334
	Fusformer	3.3798	28.99%	—
RPL&RMA	Transformer	4.3664	—	—
	Fusformer W/O RPL&RMA	3.6656	19.12%	0.2858
	Fusformer	3.3798	28.99%	—
SEM	Transformer	4.3664	—	—
	Fusformer W/O SEM	3.6243	20.47%	0.2445
	Fusformer	3.3798	28.99%	—
GLU	Transformer	4.3664	—	—
	Fusformer W/O GLU	3.4562	26.37%	0.0764
	Fusformer	3.3798	28.99%	—

图17 LEL消融实验训练曲线对比

Fig.17 Comparison of LEL ablation experiment training curves

图18 RPL&RMA消融实验训练曲线对比

Fig.18 Comparison of RPL&RMA ablation experiment training curves

图19 SEM消融实验训练曲线对比

Fig.19 Comparison of SEM ablation experiment training curves

图20 GLU消融实验训练曲线对比

Fig.20 Comparison of GLU ablation experiment training curves

图21 d_model精度影响

Fig.21 The impact of d_model

图22 Encoder layer num精度影响

Fig.22 The impact of encoder layer num

图23 Decoder layer num精度影响

Fig.23 The impact of decoder layer num

图24 Kernel精度影响

Fig.24 The impact of kernel

图25 Dilation精度影响

Fig.25 The impact of dilation

图26 Clip精度影响

Fig.26 The impact of clip

图27 d_model训练曲线

Fig.27 Training curve of d_model

图28 Encoder layer num训练曲线

Fig.28 Training curve of encoder layer num

图29 Decoder layer num训练曲线

Fig.29 Training curve of decoder layer num

图30 Kernel训练曲线

Fig.30 Training curve of kernel

图31 Dilation训练曲线

Fig.31 Training curve of dilation

图32 Clip训练曲线

Fig.32 Training curve of clip

图33 时序输入的各时间段平均权重分布

Fig.33 Average weight distribution of time series input in each time period

参考文献 41

1	Kim J K, Mudambi R. An ecosystem-based analysis of design innovation infringements: South Korea and China in the global tire industry[J]. Journal of International Business Policy, 2020, 3(1): 38-57.
2	陈志宏, 胡浩, 赵敏. 创新, 综合创新, 再创新: 共同实现轮胎全行业高质量发展[J]. 橡胶工业, 2023, 70(9): 643-654.
	Chen Z H, Hu H, Zhao M. Innovation, comprehensive innovation, and re-innovation—jointly achieving high-quality development of entire tire industry[J]. China Rubber Industry, 2023, 70(9): 643-654.
3	朱孔阳, 方之峻, 魏向阳. 混炼胶门尼黏度的预测与控制[J]. 轮胎工业, 1997, 17(10): 608-611.
	Zhu K Y, Fang Z J, Wei X Y. Prediction and control of mooney viscosity of rubber mix[J]. Tire Industry, 1997, 17(10): 608-611.
4	朱海亮. 提高顺丁橡胶胶液门尼黏度测定准确性的研究[J]. 橡胶工业, 2017, 64(2): 121-122.
	Zhu H L. Study on improving the accuracy of Mooney viscosity determination of cis-polybutadiene rubber solution[J]. China Rubber Industry, 2017, 64(2): 121-122.
5	Liu Y, Gao Z. Real-time property prediction for an industrial rubber-mixing process with probabilistic ensemble Gaussian process regression models[J]. Journal of Applied Polymer Science, 2015, 132(6):41432.
6	Gurdeep Singh H K, Yusup S, Abdullah B, et al. Refining of crude rubber seed oil as a feedstock for biofuel production[J]. Journal of Environmental Management, 2017, 203: 1011-1016.
7	朱锋峰, 张海, 贺德化, 等. 混炼胶门尼黏度的预测[J]. 特种橡胶制品, 1999, 20(3): 47-49.
	Zhu F F, Zhang H, He D H, et al. Prediction of Mooney viscosity of rubber compounds[J]. Special Purpose Rubber Products, 1999, 20(3): 47-49.
8	李俊, 马铁军, 陈国华, 等. 基于多因素的混炼胶质量预测模型[J]. 橡胶工业, 2006, 53(10): 614-617.
	Li J, Ma T J, Chen G H, et al. Model for predicting mix quality based on multi-factors[J]. China Rubber Industry, 2006, 53(10): 614-617.
9	Liu M, Huang D P, Sun Z H, et al. Combining KPCA with LSSVM for the Mooney-viscosity forecasting[C]//2008 Second International Conference on Genetic and Evolutionary Computing. IEEE, 2008: 522-526.
10	徐唯易. 炼胶工艺参数与门尼黏度关系的研究[D]. 哈尔滨: 哈尔滨工业大学, 2018.
	Xu W Y. Study on the relationship between rubber mixing process parameters and Mooney viscosity[D]. Harbin: Harbin Institute of Technology, 2018.
11	李高伟, 李佳, 朱金梅, 等. 基于不同算法优化的back propagation神经网络在三元乙丙橡胶混炼胶门尼黏度预测中的应用[J]. 合成橡胶工业, 2023, 46(6): 488-494.
	Li G W, Li J, Zhu J M, et al. Application of back propagation neural network optimized by different algorithms in prediction of Mooney viscosity of ethylene-propylene-diene monomer compound[J]. China Synthetic Rubber Industry, 2023, 46(6): 488-494.
12	金怀平, 张燕, 董守龙, 等. 基于半监督集成即时学习的橡胶混炼过程门尼黏度软测量研究[J]. 高校化学工程学报, 2022, 36(4): 586-596.
	Jin H P, Zhang Y, Dong S L, et al. Study on semi-supervised ensemble just-in-time learning based soft sensing of Mooney viscosity in rubber mixing process[J]. Journal of Chemical Engineering of Chinese Universities, 2022, 36(4): 586-596.
13	Liu H C, Cui Z X, Yue J G, et al. A sparse data gas sensor array feature mining method for rubber Mooney viscosity measurement[J]. Sensors and Actuators A: Physical, 2024, 373: 115335.
14	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
15	Gillioz A, Casas J, Mugellini E, et al. Overview of the transformer-based models for NLP tasks[C]//2020 15th Conference on Computer Science and Information Systems (FedCSIS). IEEE, 2020: 179-183.
16	Jain LC, Medsker L R. Recurrent Neural Networks: Design and Applications[M]. Boca Raton: CRC press, 1999.
17	Gu J X, Wang Z H, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377.
18	石争浩, 李成建, 周亮, 等. Transformer驱动的图像分类研究进展[J]. 中国图象图形学报, 2023(9): 2661-2692.
	Shi Z H, Li C J, Zhou L, et al. Survey on transformer for image classification[J]. Journal of Image and Graphics, 2023(9): 2661-2692.
19	Dong L H, Xu S, Xu B. Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018: 5884-5888.
20	Zhou H Y, Zhang S H, Peng J Q, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11106-11115.
21	陆宁云, 王福利, 高福荣, 等. 间歇过程的统计建模与在线监测[J]. 自动化学报, 2006, 32(3): 400-410.
	Lu N Y, Wang F L, Gao F R, et al. Statistical modeling and online monitoring for batch processes[J]. Acta Automatica Sinica, 2006, 32(3): 400-410.
22	于清溪. 橡胶混炼设备使用现状与工艺发展[J]. 橡塑技术与装备, 2007, 33(5): 6-16.
	Yu Q X. Present service status and process development of rubber mixing equipment[J]. China Rubber/Plastics Technology and Equipment, 2007, 33(5): 6-16.
23	黄树林. 国产开炼机的发展历程与趋势[J]. 橡胶工业, 2007, 54(7): 440-443.
	Huang S L. Development and trend of domestic roller mixer[J]. China Rubber Industry, 2007, 54(7): 440-443.
24	马铁军, 蔡群英, 黄有发, 等. 橡胶在密炼机内混炼过程的分析[J]. 华南理工大学学报(自然科学版), 1997, 25(3): 119-122.
	Ma T J, Cai Q Y, Huang Y F, et al. Analysis of rubber mixing processin internal nixers[J]. Journal of South China University of Technology (Natural Science Edition), 1997, 25(3): 119-122.
25	杨顺根. 密炼机的现状发展与趋势[J]. 橡塑技术与装备, 2008, 34(2): 18-23.
	Yang S G. The present status and development trends of mixer[J]. China Rubber/Plastics Technology and Equipment, 2008, 34(2): 18-23.
26	张海, 马铁军, 钟荣. 密炼机混炼过程功率曲线的作用[J]. 橡胶工业, 1999, 46(4): 232-234.
	Zhang H, Ma T J, Zhong R. The effect of power curve in the mixing process of internal mixer[J]. China Rubber Industry, 1999, 46(4): 232-234.
27	Montero-Manso P, Hyndman R J. Principles and algorithms for forecasting groups of time series: locality and globality[J]. International Journal of Forecasting, 2021, 37(4): 1632-1653.
28	Li Y H, Zhang X F, Chen D M. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 1091-1100.
29	Zhang J, Sun J F, Luo X D, et al. Characterizing pseudoperiodic time series through the complex network approach[J]. Physica D: Nonlinear Phenomena, 2008, 237(22): 2856-2865.
30	Li S, Jin X, Xuan Y, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, BC, Canada, 2019: 5243-5253.
31	Jung S, Moon J, Park S, et al. Self-attention-based deep learning network for regional influenza forecasting[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(2): 922-933.
32	Noh S H. Analysis of gradient vanishing of RNNs and performance comparison[J]. Information, 2021, 12(11): 442.
33	Rasamoelina A D, Adjailia F, Sinčák P. A review of activation function for artificial neural network[C]//2020 IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE, 2020: 281-286.
34	Khadka N. General machine learning practices using Python[D].Oulu: Oulu University of Applied Science 2019.
35	Dahouda M K, Joe I. A deep-learned embedding technique for categorical features encoding[J]. IEEE Access, 2021, 9: 114381-114391.
36	Henderi H. Comparison of min-max normalization and Z-score normalization in the k-nearest neighbor (kNN) algorithm to test the accuracy of types of breast cancer[J]. International Journal of Informatics and Information Systems, 2021, 4(1): 13-20.
37	Bergstra J, Bengio Y. Random search for hyper-parameter optimization[J]. Journal of Machine Learning Research, 2012, 13: 281-305.
38	Ke G, Meng Q, Finley T, et al. LightGBM: a highly efficient gradient boosting decision tree[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA, 2017: 3149-3157.
39	Smagulova K, James A P. A survey on LSTM memristive neural network architectures and applications[J]. The European Physical Journal Special Topics, 2019, 228(10): 2313-2324.
40	Chen D W, Hu F, Nian G K, et al. Deep residual learning for nonlinear regression[J]. Entropy, 2020, 22(2): 193.
41	Meyes R, Lu M, de Puiseau C W, et al. Ablation studies in artificial neural networks[EB/OL]. .

编辑推荐 0

Metrics

阅读次数

全文

HTML			PDF

最新录用	在线预览	正式出版	最新录用	在线预览	正式出版
0	0	5	0	0	18

来源	本网站	其他网站

次数	18	5
比例	78%	22%

摘要

最新录用	在线预览	正式出版

0	0	77

来源	本网站	其他网站

次数	31	46
比例	40%	60%

[1]	冯海军, 章冰璇, 周健. 图神经网络模型预测和解释离子液体毒性的研究[J]. 化工学报, 2025, 76(1): 93-106.
[2]	刘萍, 邱雨生, 李世婧, 孙瑞奇, 申晨. 微通道内纳米流体传热流动特性[J]. 化工学报, 2025, 76(1): 184-197.
[3]	张俊杰, 陈源, 李运堂, 李孝禄, 王冰清, 彭旭东. 超椭圆织构浮动坝箔片端面气膜密封动态性能分析与优化[J]. 化工学报, 2025, 76(1): 296-310.
[4]	李雨诗, 陈源, 李运堂, 彭旭东, 王冰清, 李孝禄. 新型柔性坝箔片端面气膜密封变形协调分析及性能智能优化[J]. 化工学报, 2025, 76(1): 324-334.
[5]	李海东, 张奇琪, 杨路, AKRAM Naeem, 常承林, 莫文龙, 申威峰. 采用智能进化算法的管壳式换热器详细设计[J]. 化工学报, 2025, 76(1): 241-255.
[6]	王绍吉, 郝矿荣, 陈磊. 基于联邦学习的聚酯纤维酯化过程温度预测研究[J]. 化工学报, 2025, 76(1): 283-295.
[7]	郭鑫, 李文静, 乔俊飞. 基于自组织模块化神经网络的污水处理过程出水参数预测[J]. 化工学报, 2024, 75(9): 3242-3254.
[8]	赵武灵, 满奕. 基于变分编码器的纳米纤维素分子结构预测模型框架研究[J]. 化工学报, 2024, 75(9): 3221-3230.
[9]	朱子良, 王爽, 姜宇昂, 林梅, 王秋旺. 欧拉-拉格朗日迭代固-液相变算法[J]. 化工学报, 2024, 75(8): 2763-2776.
[10]	李倩, 张蓉民, 林子杰, 战琪, 蔡伟华. 基于机器学习的印刷电路板式换热器流动换热预测与仿真[J]. 化工学报, 2024, 75(8): 2852-2864.
[11]	李彦熹, 王晔春, 谢向东, 王进芝, 王江, 周煜, 潘盈秀, 丁文涛, 郭烈锦. 蜗壳式多通道气液旋流分离器结构优化及分离特性研究[J]. 化工学报, 2024, 75(8): 2875-2885.
[12]	李洪瑞, 黄纯西, 洪小东, 廖祖维, 王靖岱, 阳永荣. 基于自适应变步长同伦法的循环流程收敛算法[J]. 化工学报, 2024, 75(7): 2604-2612.
[13]	李子扬, 郑楠, 方嘉宾, 魏进家. 再压缩S-CO₂布雷顿循环性能分析及多目标优化[J]. 化工学报, 2024, 75(6): 2143-2156.
[14]	张晗, 张淑宁, 刘珂, 邓冠龙. 基于慢特征分析与最小二乘支持向量回归集成的草酸钴合成过程粒度预报[J]. 化工学报, 2024, 75(6): 2313-2321.
[15]	黄志鸿, 周利, 柴士阳, 吉旭. 耦合加氢装置优化的多周期氢网络集成[J]. 化工学报, 2024, 75(5): 1951-1965.