CN-122008224-A - Exoskeleton man-machine cooperative control system and method based on multi-mode physiological information fusion

CN122008224ACN 122008224 ACN122008224 ACN 122008224ACN-122008224-A

Abstract

The invention relates to the technical field of robot control, in particular to an exoskeleton human-computer cooperative control system and method based on multi-mode physiological information fusion, wherein a multi-mode physiological signal synchronous acquisition module is used for acquiring a user surface electromyographic signal and a heart rate signal, a fatigue state intelligent detection module is used for carrying out fusion processing on the surface electromyographic signal and the heart rate signal, a movement intention continuous decoding module is used for processing the surface electromyographic signal, a double-decision self-adaptive cooperative control module is used for generating a power-assisting instruction of an exoskeleton robot, and an exoskeleton real-time execution module drives an upper limb exoskeleton to realize follow-up or power-assisting action. The system and the method can realize high-precision fatigue state sensing and accurate movement intention decoding, and can realize integrated human-computer intelligent coordination, so that the individual adaptability and industrial practicability of the system are improved, the musculoskeletal injury risk of workers is effectively reduced, the operation safety and efficiency are improved, and the power-assisted manufacturing industry is intelligent and humanized.

Inventors

CAI YONGQING
LI HAIYANG
ZHANG CHI
XU WEILIN
TANG HAO
TAN QI
CHEN SHAOFENG

Assignees

合肥工业大学

Dates

Publication Date: 20260512
Application Date: 20260320

Claims (10)

1. An exoskeleton man-machine cooperative control system based on multi-mode physiological information fusion is characterized by comprising: The multi-mode physiological signal synchronous acquisition module is used for synchronously acquiring surface electromyographic signals and heart rate signals of a user; the fatigue state intelligent detection module is used for carrying out fusion processing on the surface electromyographic signals and the heart rate signals based on an R2CMSE-SVM model and outputting fatigue grades; the motion intention continuous decoding module is used for processing the surface electromyographic signals based on a MFTCAN-KNR model and estimating angles of a shoulder joint and an elbow joint in real time; The double-decision self-adaptive cooperative control module generates a power-assisted instruction of the exoskeleton robot according to the fatigue grade and the joint angle estimated value; And the exoskeleton real-time execution module is used for receiving the assistance command and driving the upper limb exoskeleton to realize follow-up or assistance actions.
2. The exoskeleton human-computer cooperative control system based on multi-modal physiological information fusion according to claim 1, wherein the multi-modal physiological signal synchronous acquisition module comprises: the myoelectricity acquisition unit is used for acquiring original surface myoelectricity signals of the upper limb target muscle groups; the heart rate acquisition unit is used for acquiring a heart rate sequence of a user and heart rate recovery time after exercise; the hardware synchronous triggering unit is used for realizing the time stamp alignment of the myoelectricity acquisition unit and the heart rate acquisition unit based on the TTL pulse signals and ensuring the time sequence consistency of the bimodal data; And the data preprocessing unit is used for filtering and denoising the original surface electromyographic signals, carrying out sliding window average processing on the heart rate signals, and extracting the instant heart rate value and the heart rate recovery time.
3. The exoskeleton human-computer cooperative control system based on multi-mode physiological information fusion according to claim 1, wherein the fatigue state intelligent detection module comprises: The time-frequency conversion sub-module is used for processing the surface electromyographic signals by adopting continuous wavelet conversion, and Morlet wavelet is selected as a basis function to generate a two-dimensional time-frequency diagram containing time-frequency domain information; The R2CMSE feature extraction submodule comprises a multi-layer convolution structure, a pooling layer and a full-connection layer and is used for extracting high-dimensional deep features from the time-frequency diagram and outputting 64-dimensional feature vectors; The characteristic fusion sub-module is used for splicing the heart rate characteristic value to the tail end of the 64-dimensional characteristic vector to form a fusion characteristic vector; the SVM classification sub-module adopts a radial basis function, classifies the fatigue state of the user based on the fusion feature vector and outputs 0-9-level fatigue grade; and the model training unit is used for carrying out staged training on the R2CMSE-SVM model by adopting a pre-labeling dataset, training the R2CMSE network firstly, and then training the SVM classifier after freezing the R2CMSE parameters.
4. The exoskeleton human-computer cooperative control system based on multi-modal physiological information fusion according to claim 1, wherein the movement intention continuous decoding module comprises: the multi-feature extraction submodule adopts a sliding window mechanism to extract RMS, MAV, WAMP, WL, ZC time domain features of the surface electromyographic signals to form a multi-dimensional feature sequence; The PCA dimension reduction sub-module is used for carrying out principal component analysis on the characteristic sequence, reducing characteristic dimension and reserving main information; MFTCAN a feature learning submodule, which comprises a plurality of cascaded time sequence convolution blocks, wherein each block comprises a causal expansion convolution layer, a time sequence attention mechanism layer and an enhanced residual error connecting layer, and is used for extracting deep space-time features from the feature sequence; The KNR regression prediction submodule is used for searching K nearest neighbor samples most similar to the current input characteristics in a training set based on a K nearest neighbor regression algorithm and predicting the angles of the shoulder joint and the elbow joint at the current moment in a distance weighted average mode; a model optimization unit comprising: the light processing unit is used for carrying out channel cutting or knowledge distillation on the MFTCAN network, so as to reduce calculation delay; the robustness enhancing unit is used for adding Gaussian noise, amplitude scaling or time drift to the surface electromyographic signals in the training stage and improving the anti-interference capability of the model; And the reasoning acceleration unit is used for converting the trained model into TensorRT or OpenVINO format and improving the reasoning speed on the edge computing equipment.
5. The exoskeleton human-computer cooperative control system based on the multi-mode physiological information fusion according to claim 1, wherein the dual-decision self-adaptive cooperative control module obtains the quantized fatigue grade output by the fatigue state intelligent detection module based on the R2CMSE-SVM model in real time, and generates a power-assisted instruction of the exoskeleton robot through the main control unit after the motion intention continuous decoding module predicts the angular displacement vector of the shoulder joint and the elbow joint based on the MFTCAN-KNR model.
6. The exoskeleton human-computer cooperative control system based on multi-modal physiological information fusion according to claim 1, wherein the exoskeleton real-time execution module comprises: The mechanical structure unit comprises a three-degree-of-freedom upper limb exoskeleton body, wherein the shoulder joint has two active degrees of freedom of buckling/stretching and internal rotation/external rotation, the elbow joint has one active degree of freedom of buckling/stretching, the upper arm and the forearm are provided with a length adjustable mechanism and a circumference adjustable support piece, and the wrist joint is provided with a detachable hook head structure; the driving unit adopts a DM8006 high-torque servo motor, is internally provided with a double encoder for storing an absolute position and supports torque, position and speed mixed control in an MIT mode; The main control unit adopts an STM32H7 chip as a core controller, has a main frequency of 550MHz, and is provided with an FPU floating point operation unit and integrates a CAN bus interface and an IMU module; the communication unit is used for realizing high-speed real-time communication between the main control unit and each driving motor based on a CAN bus protocol; the power management unit provides stable power supply for each module and has overcurrent, overvoltage and overheat protection functions.
7. An exoskeleton human-computer cooperative control method based on multi-mode physiological information fusion, which applies the system of any one of claims 1-6, and is characterized by comprising the following steps: S1, synchronously acquiring multi-mode signals, namely synchronously acquiring surface electromyographic signals and heart rate signals of a user through a hardware trigger mechanism; S2, performing intelligent detection on the fatigue state, namely performing continuous wavelet transformation on the surface electromyographic signals to generate a time-frequency diagram, inputting an R2CMSE model to extract deep features, inputting an SVM classifier after fusion with heart rate features, and outputting quantized fatigue grades; S3, continuously decoding the exercise intention, namely extracting RMS, MAV, WAMP, WL, ZC time domain features of the surface electromyographic signals, inputting the time-space feature learning by a PCA dimension reduction network to MFTCAN, and outputting an angle estimated value of the shoulder and elbow joint in real time by KNR regression; s4, self-adaptive cooperative control decision, wherein the double-decision self-adaptive cooperative control module acquires the quantized fatigue grade F which is output by the fatigue state intelligent detection module in real time ) Shoulder joint and elbow joint predicted angular displacement vectors output by motion intention continuous decoding mode Then, generating a power-assisted instruction of the exoskeleton robot through a main control unit (STM 32H7 chip); S5, the exoskeleton executes in real time, namely the assistance command is sent to a servo driving unit through a CAN bus to drive the upper limb exoskeleton to execute corresponding follow-up or assistance actions; S6, monitoring the system state and feeding back in a closed loop, namely monitoring the execution state in real time, and carrying out dynamic fine adjustment on the control parameters according to the actual motion feedback.
8. The method for collaborative control of an exoskeleton human-computer based on multimodal physiological information fusion according to claim 7, wherein step S2 further comprises: S201, processing surface electromyographic signals by adopting continuous wavelet transformation, and selecting Morlet wavelet as a basis function to generate a two-dimensional time-frequency diagram; S202, inputting the time-frequency diagram into a pre-trained R2CMSE network, extracting deep features through multi-layer convolution and pooling operation, and outputting a 64-dimensional feature vector; s203, extracting an instant heart rate value and a heart rate recovery time after exercise from a heart rate signal, and constructing a heart rate characteristic vector; s204, splicing the heart rate feature vector to the tail end of the 64-dimensional feature vector to form a fusion feature vector; S205, inputting the fusion feature vector into a trained SVM classifier, classifying based on RBF kernel functions, and outputting 0-9 levels of fatigue grade; And S206, training the R2CMSE network and the SVM classifier by adopting a staged training strategy, wherein the first stage uses the labeling data set to train the R2CMSE network, and the second stage freezes the R2CMSE network parameters, and only uses the fusion feature vector to train the SVM classifier.
9. The method for collaborative control of an exoskeleton human-computer based on multimodal physiological information fusion according to claim 7, wherein step S3 further comprises: s301, segmenting a surface electromyographic signal by using a sliding window with the length of 200ms and taking 10ms as a step length; S302, calculating RMS, MAV, WAMP, WL, ZC time domain features of signals in each window to construct a multi-dimensional feature sequence; s303, performing PCA dimension reduction on the characteristic sequence, and reserving main component information; s304, inputting the feature sequence after dimension reduction into MFTCAN networks, and extracting deep space-time features through multistage causal expansion convolution and a time sequence attention mechanism; S305, inputting the high-dimensional feature vector output by the MFTCAN network into a KNR regression, searching K nearest neighbor samples in a training set, and predicting the angles of the shoulder joint and the elbow joint at the current moment in a distance weighted average mode; And S306, optimizing the model by adopting a pre-training-combined training strategy, namely pre-training MFTCAN the network by using an MSE loss function, freezing MFTCAN parameters, and training only the KNR regressor.
10. The method for collaborative control of an exoskeleton human-computer based on multimodal physiological information fusion according to claim 7, wherein step S4 further comprises: S401, developing a manifold matrix of a man-machine coupling nonlinear dynamics state, namely constructing a second-order nonlinear affine dynamics model containing man-machine physical interaction characteristics, and defining a system state vector as The state space equation is developed as: ; Wherein, the Is the actual angular displacement vector fed back in real time by the exoskeleton joint encoder; the real-time angular velocity vector is the joint; actively outputting an electromagnetic moment instruction vector for a motor actuator to be solved; The mass inertia matrix is positively fixed for the system symmetry; Is a centrifugal matrix containing Coriolis force and centripetal moment; a joint gravity compensation vector determined for the exoskeleton link centroid distribution; the method is an unstructured disturbance item regulated and controlled by a fatigue grade F of the intelligent fatigue state detection module; S402, risk self-adaptive safety boundary based on fatigue logistic regression Calculation introducing dynamic safety margin The method is used for online adjusting the fault-tolerant space of the exoskeleton for the deviation of the intention of the human body, and a nonlinear mapping formula is defined as follows: ; Wherein: For a dynamic security boundary half-width value, A preset minimum and maximum safety boundary constant; An expansion rate factor that is the increase in the safety margin with fatigue; median constant in response to fatigue risk; S403, constructing a time-varying control barrier function (TV-CBF) constraint surface: Prediction intent using motion intent continuous decoding module output And calculated out Constructing scalar functions As a control barrier function of the system, an intrinsically safe set for demarcating human-computer collaboration: ; Wherein, the For the joint weight diagonal matrix, to ensure that the system state does not cross the safety boundary all the time, the following differential constraint is satisfied: ; Expanding the above to explicitly include control moment Linear inequality constraint of (c): ; wherein h is a barrier function scalar value; attracting gain for the safety boundary; respectively a constraint matrix and a constant vector; s404, variable impedance stiffness self-adaptive adjustment based on energy dissipation theory: On the premise of meeting the constraint of the linear inequality in the step S403, the system adjusts the virtual impedance stiffness matrix in real time by using the fatigue grade F And damping matrix : ; ; Wherein, the Is the initial rated stiffness; small constant for regularization ); Is critical damping ratio; And S405, optimizing optimal control moment decision based on quadratic programming, namely solving the following optimization problem in real time in each 1ms control period of the main control unit in order to balance the intention tracking precision and the fatigue assistance strength: ; the following constraint conditions are satisfied: ; Wherein, the Impedance interaction moment calculated based on variable parameters; compensating feedforward torque for gravity; The power-assisted percentage weight is regulated and controlled by the fatigue grade; a cost function weight matrix; Is a relaxation factor; is a penalty weight constant; The physical saturation limit of the driving motor; s406, deterministic encapsulation of the bottom layer control instruction and 1kHz real-time issuing: After the main control unit completes optimizing calculation by utilizing the built-in FPU unit, the main control unit outputs the optimal moment Final desired joint angle after admittance correction Bit-level encapsulation and structured instruction data frame The fixed-length message structure based on FDCAN protocol is adopted: ; Wherein Heade is fixed as 0x5A, ID corresponds to the physical address of the driver of the shoulder joint (ID: 0x 01) or elbow joint (ID: 0x 02), mode is set as 0x01 when F <4, and Mode is set as 0x01 when F < 4) The time is set to 0x02, and the corresponding control loop is instructed to be started by the driver; To floating point moment Via the formula Mapping to 16-bit signed fixed point number as motor current loop reference; correction angle for generating admittance model Mapping to motor position loop given signal, CRC employing CRC-16-CCITT polynomial pair ID to The segments are verified.

Description

Exoskeleton man-machine cooperative control system and method based on multi-mode physiological information fusion Technical Field The invention relates to the technical field of robot control, in particular to an exoskeleton human-computer cooperative control system and method based on multi-mode physiological information fusion. Background With the rapid development of the manufacturing industry towards intelligentization, high-end transformation and logistics industry, the scenes of first-line workers in high-strength and repeated carrying operations are still common. The long-time and high-load physical labor is extremely easy to cause muscle fatigue of workers, which not only seriously damages the occupational health of the workers and obviously increases the risk of suffering from musculoskeletal diseases, but also causes physical harmony and reaction speed reduction due to fatigue, thereby causing handling accidents and forming double threats to personal safety and production efficiency. In order to reduce the burden on workers, some auxiliary devices have appeared in the prior art. For example, conventional passive braces (e.g., lumbar braces) provide limited physical support and do not actively relieve muscle load. Most of the existing industrial exoskeleton or power-assisted mechanical arms in the market adopt a preset program or rely on a single mechanical sensor (such as a torque sensor and a position sensor) for feedback control. These systems suffer from the following significant drawbacks: (1) The prior system can not accurately sense the muscle fatigue state of the operator in real time. They can't judge whether the operator "needs helping hand" and "how big degree helping hand that needs", lead to human-computer interaction hard, helping hand opportunity and dynamics and actual demand mismatch. (2) The control mode is single, the adaptability is poor, traditional control strategies only pay attention to accurate position following (position control) or only pay attention to constant force output (force control), and flexible and natural coordination is difficult to realize in a complex dynamic working environment. When the operator is in a tired state, the system can increase the operator's burden if it is still aimed at rigid follow-up. (3) The generalization capability of the model is insufficient, and partial researches try to control by utilizing the surface electromyographic signals, but the model is mostly based on a single signal and a general model. Because of the obvious differences of physiological characteristics and muscle activation modes of different individuals, the generalization capability of the models is poor, and the models are difficult to popularize in practical application. Therefore, how to construct an intelligent man-machine cooperative system capable of reading and understanding the physical state of an operator and adaptively providing accurate and flexible assistance according to the fatigue degree and the movement intention is a technical problem to be solved currently. Disclosure of Invention The invention mainly aims to overcome the defects in the prior art and provides an exoskeleton human-computer cooperative control system and method based on multi-mode physiological information fusion. The technical scheme adopted by the invention for realizing the technical purpose is that the exoskeleton man-machine cooperative control system based on multi-mode physiological information fusion comprises: The multi-mode physiological signal synchronous acquisition module is used for synchronously acquiring surface electromyographic signals and heart rate signals of a user; the fatigue state intelligent detection module is used for carrying out fusion processing on the surface electromyographic signals and the heart rate signals based on an R2CMSE-SVM model and outputting fatigue grades; the motion intention continuous decoding module is used for processing the surface electromyographic signals based on a MFTCAN-KNR model and estimating angles of a shoulder joint and an elbow joint in real time; The double-decision self-adaptive cooperative control module generates a power-assisted instruction of the exoskeleton robot according to the fatigue grade and the joint angle estimated value; And the exoskeleton real-time execution module is used for receiving the assistance command and driving the upper limb exoskeleton to realize follow-up or assistance actions. Preferably, the multi-mode physiological signal synchronous acquisition module comprises: the myoelectricity acquisition unit is used for acquiring original surface myoelectricity signals of the upper limb target muscle groups by adopting a Noraxon Ultium EMG system; The heart rate acquisition unit adopts a Polar H10 heart rate monitor and a Polar Unite, and the heart rate watch is used for acquiring a heart rate sequence of a user and heart rate recovery time after exercise; the hardware synchronous triggering unit is