Search

CN-122018674-A - Cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-machine interaction system and method

CN122018674ACN 122018674 ACN122018674 ACN 122018674ACN-122018674-A

Abstract

The invention provides a cognition rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction system and a cognition rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction method, wherein the system comprises a flexible array electrode, a hardware circuit, an upper computer interface and an interaction module, wherein the flexible array electrode is attached to a wrist Qu Wanji group and a extensor wrist group by adopting a 2X 16-channel serpentine wiring structure and is used for collecting surface myoelectricity signals; the hardware circuit integrates an electrophysiological amplifying and collecting module, a main control module, a power management module and a wireless communication module, and realizes the amplification, filtering, analog-to-digital conversion and wireless transmission of electromyographic signals. The method comprises the steps of analyzing, processing and displaying the received multichannel myoelectricity data by the upper computer, completing gesture recognition based on the activation detection and the multiscale residual error attention network model, and mapping the recognition result into an interaction control event to drive a virtual rehabilitation scene.

Inventors

  • CHENG XIANJIE
  • ZHENG YUE
  • Bao Benkun
  • CUI WEIDONG

Assignees

  • 中国科学院苏州生物医学工程技术研究所

Dates

Publication Date
20260512
Application Date
20251223

Claims (10)

  1. 1. A cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction system is characterized by comprising: the wireless wrist-worn 32-channel high-density surface myoelectricity acquisition device is used for acquiring wrist surface myoelectricity signals; the upper computer interface is used for establishing communication connection with the acquisition device, analyzing, displaying and storing myoelectricity data, and deploying a gesture recognition model to output a gesture recognition result; The interaction module is used for receiving the gesture recognition result and mapping the gesture recognition result into a virtual scene control event so as to drive a cognitive rehabilitation training task; The wireless wrist-worn 32-channel high-density surface myoelectricity acquisition device is composed of a flexible array electrode (1) and a hardware circuit (2), wherein the flexible array electrode (1) is of a flexible array structure of 2 multiplied by 16 channels so as to be attached to a wrist curved surface, and the hardware circuit (2) comprises a main control module (2-8), a power management module (2-2), an FPC connector (2-3), an electrophysiological amplifying and acquisition chip (2-9) and a Wi-Fi module (2-6).
  2. 2. The system according to claim 1, wherein the flexible array electrode (1) is electrically connected using a serpentine wiring structure (1-1) and forms a2 x 16 channel electrode array (1-2) to ensure spatial coverage and attachment compliance of signal acquisition.
  3. 3. The system according to claim 1, characterized in that the flexible array electrode (1) covers the flexor and extensor carpi muscle groups of the wrist area and employs a medical pressure sensitive adhesive tape to achieve stable contact and low impedance matching of the electrode with the skin surface.
  4. 4. The cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction system is characterized in that the hardware circuit (2) is provided with an integrated protective shell, the protective shell is of a three-layer structure of an upper shell (2-1), a middle shell (2-7) and a lower shell (2-4), and the middle shell (2-7) is used for isolating a supporting circuit from a power supply lithium battery (2-5) in an up-down layering mode.
  5. 5. The cognition rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction system is characterized in that a data transmission link of the acquisition device is that myoelectricity signals acquired by the flexible array electrode (1) are amplified, band-pass filtered and analog-to-digital converted through the electrophysiological amplification and acquisition chip (2-9) to obtain digital sampling data, the digital sampling data are transmitted to the main control module (2-8) through the serial peripheral interface SPI2 and written into the annular buffer for temporary storage, the main control module (2-8) packages frames and then transmits the packaged myoelectricity data to the Wi-Fi module (2-6) through the SPI1, and the Wi-Fi module (2-6) wirelessly transmits the packaged myoelectricity data to the upper computer interface based on the transmission control protocol TCP.
  6. 6. A gesture recognition and interaction control method based on the system of any one of claims 1 to 5, comprising: The acquisition step comprises the steps of acquiring electromyographic signals on the surface of a wrist 32 channel through a flexible array electrode (1); the electromyographic signals are amplified, filtered, analog-to-digital converted and framing packaged by a hardware circuit (2), and then are transmitted to an upper computer interface by Wi-Fi wireless; the upper computer interface carries out protocol analysis and filtering treatment on the received data and forms an input segment for recognition; The identification step comprises the steps of calling a multi-scale residual error attention network model deployed on an upper computer to carry out gesture classification on input fragments and outputting gesture labels; and the interaction step is to map the gesture label into a control event and send the control event to an interaction module so as to drive the role behavior response of the virtual rehabilitation scene.
  7. 7. The method of claim 6, wherein the upper computer interface is implemented by using QT DESIGNER to perform interface layout and using python+pyqt5, and at least comprises a TCP connection module (3-1), a data analysis and filtering module (3-2), an arbitrary channel waveform display module (3-3), a data storage module (3-4), a single channel spectrogram module (3-5), and a model deployment and real-time reasoning module (3-6).
  8. 8. The method of claim 6, wherein the real-time activation extraction is performed by a time sliding window mechanism and an absolute average-root mean square dual-threshold activation segment screening method, wherein the time window length is 100ms and the overlapping rate is 90%, 4 core channels with optimal signal-to-noise ratio distribution are screened from 32 channels, namely channels 27-30 respectively, and marked as activation segments when the envelope intensity exceeds the fluctuation threshold alpha more than or equal to 0.4 for the first time, and the activation starting point is determined and myoelectric segments containing all 32 channels are intercepted for subsequent identification when the intensity threshold beta=1.0 is exceeded for the second time in a subsequent observation window.
  9. 9. The method of claim 6, wherein the multi-scale residual attention network model is MSE-Net, three sequentially stacked multi-scale convolution blocks are adopted, each convolution block comprises four parallel convolution branches, branch outputs are fused and connected into a residual attention mechanism through 1 x 1 convolution after channel dimension splicing, and branch core lengths and expansion rate parameters of the three-layer convolution blocks are as follows in sequence: the first layer is (3, 1), (5, 2), (7, 4), (9, 4); the second layer is (3, 1) (7, 2), (11, 4), (15, 8); the third layer is (3, 1), (9, 2), (15, 4), (21,8).
  10. 10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 6 to 9.

Description

Cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-machine interaction system and method Technical Field The invention relates to the technical field of man-machine interaction and rehabilitation engineering, in particular to a cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture man-machine interaction system and method. Background Gesture recognition is used as the most intuitive and flexible natural interface in the field of man-machine interaction, can effectively bridge human intention and machine response, and has infinite potential in the aspects of expanding human capability boundaries and assisting rehabilitation training. However, the existing gesture interaction mainly relies on traditional large-amplitude actions of five-finger coordination, and the mode not only lacks social cautiousness, but also is extremely easy to cause muscle fatigue after long-time operation, and is difficult to meet the requirements of the next-generation AR/VR and metauniverse application on immersive and low-load experience. In contrast, thumb-centered micro-actuation (joint rotation <5 °) has extremely high minuteness and anti-fatigue properties, conforming to the intuitive control concept. However, the interaction aiming at the thumb is still limited to a simple direction instruction, so that the precise identification of progressive fine-grained actions such as amplitude, speed and the like is difficult to realize, and the diversity and depth of the interaction are limited. In the signal capturing and analyzing method, the traditional visual imaging is easily influenced by light shielding and space constraint, and the mechanical sensing has insufficient precision when capturing weak motions. Surface electromyographic signal technology has become the preferred solution in this field due to its advantages of non-invasiveness, environmental independence, and direct recording of neuromuscular electrical activity. However, the gesture interaction system based on the surface electromyographic signals still faces three major core bottlenecks, namely limited identification capability, less interaction category and high misclassification rate caused by the highly similar electromyographic modes particularly in a single-hand mode, insufficient fineness, difficulty in effectively decoding micro-motions containing gradual changes due to the fact that the conventional algorithm only processes end-to-end full-amplitude actions, and difficulty in expanding multi-user cooperation or complex multi-task environments due to the fact that the conventional algorithm is single in interaction paradigm and lacks a multi-variable synchronous control mechanism. In summary, in order to overcome the defects of the prior art in terms of micro-motion capturing precision, interaction diversity and cooperation mode, it is very urgent to develop a fully integrated wireless wrist-worn high-density surface electromyographic signal system. The high-fidelity collection of the progress type thumb micro-actions is realized by adopting a rigid-flexible hybrid architecture, an integrated low-noise amplification chip (such as RHD 2132) and a high-density flexible dry electrode array (such as 32 channels), so that a novel man-machine mode with multiple scenes, multiple instructions and double user interactions is constructed, and the novel man-machine mode becomes a key direction to be broken through in the current man-machine interaction technology. Therefore, a system and a method for real-time gesture human-machine interaction of 32-channel surface myoelectricity for cognitive rehabilitation are urgently needed to solve the above problems. Disclosure of Invention The invention aims to solve the technical problems in the background technology, and provides a cognitive rehabilitation-oriented 32-channel surface myoelectricity real-time gesture human-computer interaction system, which comprises: the wireless wrist-worn 32-channel high-density surface myoelectricity acquisition device is used for acquiring wrist surface myoelectricity signals; the upper computer interface is used for establishing communication connection with the acquisition device, analyzing, displaying and storing myoelectricity data, and deploying a gesture recognition model to output a gesture recognition result; The interaction module is used for receiving the gesture recognition result and mapping the gesture recognition result into a virtual scene control event so as to drive a cognitive rehabilitation training task; the wireless wrist-worn 32-channel high-density surface myoelectricity acquisition device is composed of a flexible array electrode and a hardware circuit, wherein the flexible array electrode is of a flexible array structure of 2 multiplied by 16 channels so as to be attached to a wrist curved surface, and the hardware circuit comprises a main control module, a power management module, an FPC connec