2025 International Conference on Equipment Intelligent Operation and Maintenance (ICEIOM2025)

175 / 2025-06-11 20:00:39

Multi-modal Large Language Model for Fault Diagnosis in Mechanical Equipment

Multi-modal large language model,Fault diagnosis,Intelligent operation and maintenance,Fine-Tuning

In today's industrial landscape, mechanical faults carry hefty repair costs and pose serious risks to safety, potentially endangering personnel. Developing cutting-edge fault diagnosis technologies can significantly enhance the operational reliability of machinery while slashing maintenance expenses. While deep learning has witnessed advancements in diagnosing mechanical faults, challenges persist regarding the interpretability of diagnoses, adequacy of training datasets, and diversity of input modalities. This research introduces a multi-modal large language model(MLLM) specifically tailored for mechanical fault diagnosis. It aims to identify types of faults, investigate underlying causes, and offer remedial advice. Additionally, it presents a novel approach to transforming datasets to tackle the shortage of training data. A signal-to-text module has been designed to gather multi-modal data from faulty machinery, translating features from digital signals into textual descriptions for the large language model to process, thus addressing the limitation of single-modality inputs. The experimental outcomes highlight that our MLLM excels in evaluating vibration according to the three-tier standard and outperforms three traditional fault diagnosis models. This underscores its promising role in supporting precise fault diagnosis and advancing the intelligence of machinery operation and maintenance.