Search

CN-122008162-A - Industrial equipment intelligent maintenance method, equipment and medium based on multi-mode sensing

CN122008162ACN 122008162 ACN122008162 ACN 122008162ACN-122008162-A

Abstract

The invention relates to the technical field of industrial automation, in particular to an intelligent maintenance method, equipment and medium for industrial equipment based on multi-mode sensing. The method comprises the steps of obtaining multi-source sensing data of target industrial equipment through a multi-source sensor assembly, establishing an equipment virtual model based on the multi-source sensing data, performing dynamic time alignment and three-dimensional space mapping on the data, inputting a fault diagnosis algorithm to obtain fault diagnosis types and confidence degrees, and automatically, cooperatively and manually confirming any maintenance mode according to the confidence degrees. And when the mechanical arm is cooperatively maintained, switching the mechanical arm tool according to the fault type, calibrating the tactile glove sensor of an operator, highlighting the fault point of the virtual model to guide the mechanical arm to be positioned, driving the mechanical arm to perform preliminary operation, collecting the sensing data of the glove, fusing the clamping force and the vibration frequency spectrum to judge the state of the bolt, and adjusting the operation parameters of the mechanical arm according to the feedback data of the glove. The invention can ensure the accuracy, high efficiency and safety of industrial maintenance.

Inventors

  • JING WEN
  • LI JIANGYONG

Assignees

  • 深圳市明日实业有限责任公司

Dates

Publication Date
20260512
Application Date
20260206

Claims (10)

  1. 1. An intelligent maintenance method for industrial equipment based on multi-modal sensing is characterized by being applied to multi-modal sensing equipment, wherein the multi-modal sensing equipment comprises a multi-source sensor assembly, a mechanical arm and a touch glove, and the multi-source sensor assembly comprises: A vision module comprising a 3D-ToF camera; The touch module comprises a pressure-sensitive sensing network arranged at the tail end of the mechanical arm; An acoustic module comprising a microphone array; The intelligent maintenance method for the industrial equipment based on the multi-mode sensing comprises the following steps: The multi-source sensing data of the target industrial equipment are obtained through the multi-source sensor assembly, wherein the multi-source sensing data comprise at least one of three-dimensional point cloud and texture image data of the surface of the equipment, clamping force and vibration spectrum data and frequency abnormal characteristic data; establishing a virtual model of the target industrial equipment based on the multi-source sensing data, and ensuring that a virtual coordinate system of the virtual model is aligned with a physical coordinate system of the target industrial equipment; Performing dynamic time warping and three-dimensional space mapping on the multi-source sensing data, and then inputting a fault diagnosis algorithm to obtain a fault diagnosis type and diagnosis confidence, wherein the fault diagnosis algorithm is a mixed depth network of a convolutional neural network and a gating circulation unit; selecting a matching repair pattern based on the diagnostic confidence, the repair pattern including any one of automatic repair, collaborative repair, and manual validation; when the maintenance mode is collaborative maintenance, switching an operation tool used by the mechanical arm according to the fault diagnosis type; When an operator wears the haptic gloves, starting a calibration program to finish the return-to-zero and the accuracy verification of the sensor parameters on each haptic glove; highlighting a fault point in the virtual model, the fault point corresponding to a work position corresponding to the fault diagnosis type, whereby an operator can operate the robotic arm to the work position based on the fault point; driving the mechanical arm to execute a preliminary operation action corresponding to the fault diagnosis type, and collecting multi-dimensional sensing data of the touch glove; And acquiring multi-dimensional feedback data of the touch glove, and adjusting the operating parameters of the mechanical arm according to the multi-dimensional feedback data, wherein the multi-dimensional data is input by an operator based on the bolt state and the multi-dimensional touch data.
  2. 2. The intelligent maintenance method of industrial equipment based on multi-modal sensing according to claim 1, wherein the step of adjusting the operating parameters of the mechanical arm according to the multi-dimensional feedback data comprises: when the resistances of the 5 proximal knuckle bending sensors are all more than 80kΩ, the fingertip pressure sheet detection pressure is more than 3N, and the dorsum strain sheet data have no fluctuation, the mechanical arm immediately stops the current action, and keeps the working posture; When the dorsum of hand strain gauge detects periodic strain change of 1-2Hz, the resistance of the knuckle bending sensor is less than 30kΩ, and the fingertip pressure is less than 0.5N, the mechanical arm returns to the initial standby position; When the resistance of the 5 proximal knuckle bending sensors is less than 28kΩ, the palm and back of hand strain gauge data have no fluctuation, and the lateral pressures of the fingertips and the thumbs are less than 0.3N, the mechanical arm resumes the operation state before suspension.
  3. 3. The multi-modal awareness based industrial equipment intelligent maintenance method of claim 2, further comprising: when the fingertip pressure is less than 1N, the palm strain is less than 100 mu epsilon, the thumb side pressure is less than 0.5N, and the torque is less than 5Nm, the bolt state is the state needing fastening, the mechanical arm is controlled to increase the torque; When the fingertip pressure is 2-5N, the palm strain is 200-400 mu epsilon, the thumb side pressure is 1-3N, the torque is 5-20Nm, and the bolt state is a normal state, maintaining the current operation parameters; When the fingertip pressure suddenly increases by more than 5N and the palm strain suddenly jumps by more than 500 mu epsilon, the touch glove triggers vibration feedback and LED alarm, and simultaneously automatically sends a pause signal to the mechanical arm.
  4. 4. The intelligent maintenance method of industrial equipment based on multi-modal sensing according to claim 1, wherein after the step of establishing the virtual model of the target industrial equipment based on the multi-source sensing data, comprising: Mapping the perception data in real time in the virtual model, and simulating fault evolution; and uploading the actual maintenance data to the cloud, and automatically updating a fault feature library and a diagnosis model to realize closed-loop optimization.
  5. 5. The multi-modal awareness based industrial equipment intelligent maintenance method of claim 4, further comprising: Predicting the residual life of the components of the target industrial equipment based on the fusion operation parameters of the long-term memory network, and generating maintenance early warning 7 days in advance.
  6. 6. The intelligent maintenance method of industrial equipment based on multi-mode sensing according to claim 1, wherein the convolutional neural network comprises three convolutional layers and two largest pooling layers, the convolutional kernel size is 3×3, 5×5, 3×3 in sequence, the step size is 1, and the pooling kernel size is 2×2; the gating circulation unit comprises two hidden layers, wherein the number of units in each layer is 256, and the dropout proportion is 0.2.
  7. 7. The method of intelligent maintenance of industrial equipment based on multi-modal awareness of claim 1, wherein the step of selecting a matching maintenance pattern based on the diagnostic confidence comprises: When the diagnosis confidence is greater than 95%, entering an automatic maintenance mode; when the diagnosis confidence is 60% -95%, entering a collaborative maintenance mode; and when the diagnosis confidence is less than 60%, entering a manual confirmation mode.
  8. 8. A multi-modal sensing apparatus comprising a multi-source sensor assembly, a robotic arm, and a haptic glove, and a control system coupled to the multi-source sensor assembly, the robotic arm, and the haptic glove, the multi-modal sensing apparatus for implementing the method of any of claims 1-7.
  9. 9. A storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1 to 7.
  10. 10. A smart maintenance device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 7.

Description

Industrial equipment intelligent maintenance method, equipment and medium based on multi-mode sensing Technical Field The invention relates to the technical field of industrial automation, in particular to an intelligent maintenance method, equipment and medium for industrial equipment based on multi-mode sensing. Background The stable operation of industrial equipment directly determines the production efficiency and quality, however, the current maintenance mode mainly comprising manual inspection or single sensor detection has systematic bottlenecks, namely firstly, single perception dimension, incapability of effectively capturing internal hidden faults and key mechanical and acoustic characteristics due to excessive dependence on vision, high missed diagnosis error rate under complex working conditions, secondly, difficult effective fusion of multi-source data (such as vision, acoustics and mechanics) due to non-uniform sampling frequency and space-time coordinates, information distortion, incapability of forming reliable joint diagnosis basis, and stiff decision mechanism, incapability of layering according to the confidence level of fault diagnosis, waste of manpower on simple faults, and easiness in causing safety risks due to blind automation on complex faults. These problems limit the industrial maintenance to the accurate, efficient and safe intelligent upgrade. Disclosure of Invention The embodiment of the invention provides an intelligent maintenance method, intelligent maintenance equipment and intelligent maintenance medium for industrial equipment based on multi-mode sensing, which are used for solving the problems of low industrial maintenance accuracy, low efficiency and low safety in the prior art. The invention discloses an intelligent maintenance method of industrial equipment based on multi-mode perception, which is applied to multi-mode perception equipment, wherein the multi-mode perception equipment comprises a multi-source sensor assembly, a mechanical arm and a touch glove, the multi-source sensor assembly comprises a vision module, a touch module, an acoustic module and a control module, the vision module comprises a 3D-ToF camera, the touch module comprises a pressure-sensitive sensing network arranged at the tail end of the mechanical arm, the acoustic module comprises a microphone array, the intelligent maintenance method of the industrial equipment based on multi-mode perception comprises the steps of acquiring multi-source sensing data of target industrial equipment through the multi-source sensor assembly, the multi-source sensing data comprises at least one of equipment surface three-dimensional point cloud and texture image data, clamping force and vibration spectrum data and frequency abnormality characteristic data, establishing a virtual model of the target industrial equipment based on the multi-source sensing data, ensuring that a virtual coordinate system of the virtual model is aligned with a physical coordinate system of the target industrial equipment, inputting a fault diagnosis algorithm after dynamic time alignment and three-dimensional space mapping is performed on the multi-source sensing data, acquiring a fault diagnosis type and a confidence and a fault diagnosis algorithm after the multi-source sensing data is matched with the three-dimensional space, and the control algorithm is matched with the control system by a user, and the control unit is matched with the control system when the control system is matched with the control system and the control system is matched with the control system to the maintenance mode, the method comprises the steps of finishing sensor parameter zeroing and precision verification on each haptic glove, highlighting fault points in the virtual model, enabling an operator to operate the mechanical arm to the operation position based on the fault points, driving the mechanical arm to execute preliminary operation corresponding to the fault diagnosis type, collecting multidimensional sensing data of the haptic glove, judging and displaying a bolt state in real time based on a data fusion model and the clamping force and vibration spectrum data collected in real time, collecting multidimensional feedback data of the haptic glove, and adjusting operation parameters of the mechanical arm according to the multidimensional feedback data, wherein the multidimensional data is input by the operator based on the bolt state and the multidimensional sensing data. Optionally, the step of adjusting the operating parameter of the mechanical arm according to the multi-dimensional feedback data includes: when the resistances of the 5 proximal knuckle bending sensors are all more than 80kΩ, the fingertip pressure sheet detection pressure is more than 3N, and the dorsum strain sheet data have no fluctuation, the mechanical arm immediately stops the current action, and keeps the working posture; When the dorsum of hand strain gauge detects periodic