CN-121980135-A - Motion intention recognition method based on multi-mode enhanced contrast learning

CN121980135ACN 121980135 ACN121980135 ACN 121980135ACN-121980135-A

Abstract

The invention discloses a movement intention recognition method based on multi-mode enhanced contrast learning, which realizes accurate movement intention recognition based on any single mode by systematically fusing multi-mode physiological signals and innovative contrast learning architecture. The intention perception contrast learning is carried out, and the discrimination capability of the model in the intention dynamic transfer is obviously improved by constructing positive and negative sample pairs with multi-mode time sequence enhancement and optimizing contrast loss; the multi-modal level recognition performance under single-mode input is realized through semantic alignment and knowledge migration mechanisms by multi-to-one cross-modal prediction. After the pre-training combined optimization, training and reasoning can be completed only by single-mode data, so that the data dependence is remarkably reduced while the multi-mode performance advantage is maintained. The technical scheme of the invention effectively improves the accuracy and the robustness in the identification of the movement intention of the lower limb in the brain-computer interaction field, and provides an innovative technical solution for the movement auxiliary rehabilitation system.

Inventors

ZHAO YUE
SHI JIANKAI
FENG HAILIN
LIU TONGCUN
LI YANE
Pang Jirong

Assignees

浙江农林大学

Dates

Publication Date: 20260505
Application Date: 20260113

Claims (6)

1. The motion intention recognition method based on the multi-mode enhanced contrast learning is characterized by comprising the following steps of: Acquiring electroencephalogram and electromyogram data samples, preprocessing, and then reconstructing the electroencephalogram and electromyogram data samples into an electroencephalogram 4D representation and an electromyogram 4D representation which comprise time, space and frequency dimensions respectively; The electroencephalogram 4D characterization and the electromyogram 4D characterization are spliced to obtain a joint characterization, the electroencephalogram 4D characterization, the electromyogram 4D characterization and the joint characterization are respectively subjected to respective encoders to obtain electroencephalogram specific characteristics, electromyogram specific characteristics and multi-modal fusion characteristics, then a prediction label is obtained through a corresponding classification decoder, and cross entropy loss is calculated; splicing the multi-modal fusion features and the historical multi-modal fusion features to form a positive sample with time sequence consistency, splicing the multi-modal fusion features and the historical cross-modal average specificity features to form a negative sample with cross-modal movement intention deviation, and performing supervised contrast learning calculation on the positive and negative samples to sense contrast loss; Respectively passing the electroencephalogram specific feature, the electromyogram specific feature and the multi-modal fusion feature through an adapter to generate respective corresponding enhancement features, and then predicting respective corresponding future time sequence characterization based on the respective enhancement features to calculate multi-to-one-cross modal prediction loss; Calculating total loss including cross entropy loss, intention perception contrast loss and multi-to-one cross-modal prediction loss, and updating parameters of the encoder; And inputting the electroencephalogram 4D representation or/and the electromyogram 4D representation of the target to be predicted into a trained encoder, and obtaining a movement intention recognition result through a classifier.
2. The method for identifying exercise intentions based on multi-modal reinforcement contrast learning according to claim 1, wherein the cross-modal average specificity characteristic is an average of electroencephalogram specificity characteristic and electromyogram specificity characteristic.
3. The method for identifying exercise intent based on multimodal enhanced contrast learning of claim 1, wherein the multi-to-one cross-modal prediction penalty includes an electroencephalogram to electroencephalogram prediction penalty, an electromyogram to electromyogram prediction penalty, an electroencephalogram to electromyogram prediction penalty, an electromyogram to electroencephalogram prediction penalty, an electroencephalogram to multimodal fusion prediction penalty, and an electromyogram to multimodal fusion prediction penalty.
4. The method for identifying motion intention based on multi-modal reinforcement contrast learning as claimed in claim 1, wherein the encoder adopts a transducer encoder.
5. The method of claim 1, wherein the adapter comprises a linear layer, a normalization layer, an activation layer, and a linear layer.
6. The method for identifying the movement intention based on the multi-mode enhancement contrast learning according to claim 1, wherein the prediction of the future time sequence characterization based on each enhancement feature is realized by an autoregressive encoder constructed by a plurality of layers of LSTM.

Description

Motion intention recognition method based on multi-mode enhanced contrast learning Technical Field The application belongs to the technical field of lower limb exercise assisting rehabilitation based on brain-computer interaction, and particularly relates to an exercise intention recognition method based on multi-mode enhanced contrast learning. Background It is counted that about 70% of stroke patients develop limb movement disorders to varying degrees, especially with impaired lower limb movement and walking ability most common, severely limiting the patients' activities of daily living and social engagement. During the rehabilitation phase, patients often face multiple challenges such as reduced muscle strength, reduced balance function, abnormal gait, etc. Although a patient with limb dyskinesia can only complete unbalanced, twisted or incomplete movements, the cerebral motor cortex can still generate clear motor intention signals, and the neuromotor function is expected to be partially or completely restored through systematic rehabilitation training. In recent years, the brain-computer interaction technology provides a new approach for rehabilitation of movement functions, can directly decode movement intention in the brain, and opens up a new intervention idea for rehabilitation of movement control. The change in the motion of the human body typically causes a synchronous response of a variety of physiological signals. Currently, the main method of motion intent detection relies on bioelectric signals or mechanical and positional information. Among them, electroencephalogram (EEG) and Electromyogram (EMG) are widely used for lower limb movement classification tasks because of their direct reflection of neuromuscular activity, particularly for the resolution of movement intent. Since EMG signals can directly characterize muscle contraction status, they generally perform better in classification performance. However, the signal is easily interfered by factors such as muscle fatigue, insufficient residual myoelectric activity, electrode positioning accuracy and the like, so that the acquisition difficulty is high in practice. In contrast, EEG signals, while indirectly measured, are highly stable and easier to collect. Given the advantages of EEG and EMG, the fusion of multimodal physiological signals has become a dominant strategy to improve motor intent decoding performance. By integrating the change information of various physiological signals, the motion state of the subject can be more comprehensively and accurately identified, which is also an important reason why the multi-modal model is generally superior to the single-modal model. Application of a method based on contrast learning in the field of multi-modal motion intention recognition is becoming an important technical path for solving key challenges such as multi-modal data difference and model generalization. The contrast learning is used for measuring the similarity between different modal characteristics, pulling the positive sample pair distance with consistent semantics, and pushing away the irrelevant negative sample pair, so that the encoder parameters are optimized by means of contrast loss functions, and the characterization quality of the multi-modal characteristics is improved. However, in practical application, challenges of incomplete modal collection are often faced, and existing researches mostly adopt a cross-modal contrast learning method to try to infer complete multi-modal characterization by using partial modalities. However, due to the significant distribution difference among the modes, the multi-mode characteristics are difficult to fully learn only by a voucher mode, so that the adaptability of the mode deletion is insufficient. In addition, the existing contrast learning method is insufficient in capturing the dynamic characteristics of continuous time sequence signals, and is difficult to process state transition and time sequence dependency in motion intention recognition. Disclosure of Invention The application aims to provide a motion intention recognition method based on multi-mode enhancement contrast learning, which aims to solve the problems of insufficient adaptability of the prior art mode loss, insufficient dynamic characteristic capture of a continuous time sequence signal by the contrast learning method and the like. In order to achieve the above purpose, the technical scheme of the application is as follows: A motion intention recognition method based on multi-mode enhanced contrast learning comprises the following steps: Acquiring electroencephalogram and electromyogram data samples, preprocessing, and then reconstructing the electroencephalogram and electromyogram data samples into an electroencephalogram 4D representation and an electromyogram 4D representation which comprise time, space and frequency dimensions respectively; The electroencephalogram 4D characterization and the electromyogram 4D chara