CN-121996070-A - Self-adaptive mode switching method, system and storage medium based on SSMVEP-MI fusion brain-computer interface system

CN121996070ACN 121996070 ACN121996070 ACN 121996070ACN-121996070-A

Abstract

The invention provides a self-adaptive mode switching method, a self-adaptive mode switching system and a storage medium based on SSMVEP-MI fusion brain-computer interface system, and belongs to the technical field of brain-computer interfaces. The method is characterized in that by monitoring SSMVEP signal quality, MI signal quality and eye tracking quality in real time and comparing the signal quality, MI signal quality and eye tracking quality with preset thresholds, a driving system is intelligently switched among three modes of bimodal fusion, SSMVEP leading mode and MI leading mode. And in the fusion mode, dynamically distributing the fusion weight according to the real-time quality score. The invention solves the problems of poor robustness and easy control interruption of the existing fusion brain-computer interface in signal interference or single-mode failure, realizes fault sensing, autonomous fault tolerance and fine decision, remarkably improves the control continuity and reliability of the system in a complex environment, and is particularly suitable for long-time and high-requirement application scenes such as nerve rehabilitation.

Inventors

LIU YAXIN
HE YONGZHENG
WU SHUTING
YANG TIANXING
CAO JIAYI
WANG PENGGANG

Assignees

河南翔宇医疗设备股份有限公司

Dates

Publication Date: 20260508
Application Date: 20260119

Claims (10)

1. An adaptive mode switching method based on SSMVEP-MI fusion brain-computer interface system is characterized by at least comprising the following steps: a signal quality real-time monitoring step of calculating and outputting a normalized first score representing SSMVEP signal quality, a second score representing MI signal quality and a third score representing eye movement tracking quality in real time; the self-adaptive decision step is that the first score, the second score and the third score are respectively compared with a preset first threshold value, a preset second threshold value and a preset third threshold value; The mode switching execution step, based on the comparison result, the control system switches among a bimodal fusion mode, a SSMVEP dominant mode and an MI dominant mode; Wherein when the third score is below the third threshold and the first score is below the first threshold, triggering a switch to the MI dominant mode; and triggering switching to the SSMVEP dominant mode when the second score is below the second threshold.
2. The adaptive mode switching method based on SSMVEP-MI fusion brain-computer interface system according to claim 1, wherein in the bimodal fusion mode, the system dynamically calculates the fusion weight of SSMVEP mode weight and MI mode according to the first score, the second score and the third score, and performs weighted fusion on decoding results of two modes according to the two weights to generate a final control instruction.
3. The adaptive mode switching method based on SSMVEP-MI fusion brain-computer interface system according to claim 2, wherein the dynamically calculated fusion weights adopt a normalization strategy based on a linear score, and the weight w ssmvep of SSMVEP mode and the weight w mi of MI mode are calculated by the following formulas: wherein snr_ ssmvep is the first score, GSS is the second score, conf_mi is the third score, α, β, γ is a preset non-negative weighting coefficient; the values of the weighting coefficients alpha, beta and gamma are respectively alpha epsilon [0.4, 1.0], beta epsilon [0.1, 0.6], gamma epsilon [0.5, 1.2], and alpha+beta+gamma=C, wherein C is a constant.
4. The adaptive mode switching method based on SSMVEP-MI fusion brain-computer interface system according to claim 1, wherein the first score is a SSMVEP signal-to-noise ratio score normalized to a [0-1] interval, calculated based on a ratio of signal power to background noise power at a stimulus frequency, the second score is an MI classification confidence score normalized to the [0-1] interval, calculated based on a maximum posterior probability value output by a classifier, and the third score is an eye movement stability score normalized to the [0-1] interval, calculated based on a standard deviation of gaze point coordinates.
5. The method of claim 1, wherein the third score is obtained by an eye tracking module, the eye tracking module being configured to acquire an eye image of the user in real time and calculate a gaze point parameter.
6. The method for adaptively switching a mode based on SSMVEP-MI fusion brain-computer interface system according to any one of claims 1 to 5, wherein the values of the first threshold, the second threshold, and the third threshold are all in the range of 0.1 to 0.6.
7. The method of adaptive mode switching based on a SSMVEP-MI fusion brain-computer interface system according to any one of claims 1 to 5, wherein said mode switching performing step further comprises a restoration mechanism: Triggering a switch back to the bimodal fusion mode if the third score is monitored to be above the third threshold and the first score is monitored to be above the first threshold when the system is in the MI dominant mode; when the system is in the SSMVEP dominant mode, if the second score is monitored to be higher than the second threshold, a switch back to the bimodal fusion mode is triggered.
8. The method for adaptively switching modes based on SSMVEP-MI fusion brain-computer interface system according to any one of claims 1 to 5, further comprising a mode initialization step of entering the bimodal fusion mode by default after system startup, and providing training labels for MI brain-electrical data acquired synchronously by using SSMVEP decoding results in the bimodal fusion mode.
9. A SSMVEP-MI fusion brain-computer interface system, comprising at least: the electroencephalogram acquisition module is used for acquiring electroencephalogram signals of a user; the eye movement tracking module is used for acquiring eye movement signals of a user in real time; the stimulation and prompt presentation module is used for presenting SSMVEP stimulation and motor imagery prompts to a user; The signal processing module is used for extracting and decoding SSMVEP features and MI features from the electroencephalogram signals; The signal quality monitoring module is used for calculating and outputting a normalized first score representing SSMVEP signal quality, a second score representing MI signal quality and a third score representing eye movement tracking quality in real time; The self-adaptive mode switching module is connected to the signal quality monitoring module and is used for comparing the first score, the second score and the third score with preset thresholds respectively and outputting a system working mode instruction based on a comparison result, wherein the working mode comprises a bimodal fusion mode, a SSMVEP dominant mode and a MI dominant mode, wherein when the third score is lower than the third threshold and the first score is lower than the first threshold, an instruction for switching to the MI dominant mode is output, and when the second score is lower than the second threshold, an instruction for switching to the SSMVEP dominant mode is output; and the fusion decoding and control module is connected with the signal processing module and the adaptive mode switching module and is used for selecting a corresponding decoding strategy to generate a control command according to the system working mode instruction.
10. The SSMVEP-MI fusion brain-computer interface system according to claim 9, wherein a state machine is provided in the adaptive mode switching module, and states of the state machine include a bimodal fusion state, a SSMVEP dominant state, and an MI dominant state, and state transitions are driven by comparison logic of scores output by the signal quality monitoring module and preset thresholds.

Description

Self-adaptive mode switching method, system and storage medium based on SSMVEP-MI fusion brain-computer interface system Technical Field The invention relates to the technical field of brain-computer interfaces, in particular to a self-adaptive mode switching method, a self-adaptive mode switching system and a storage medium based on SSMVEP-MI fusion brain-computer interface system Background Brain-computer interface technology provides a revolutionary means for nerve function rehabilitation and replacement control. At present, two technical paths mainly exist, namely a brain-computer interface based on steady-state visual evoked potential (ssmvep/SSMVEP), which has the advantages of strong signals and less training, but is easy to cause visual fatigue and interference due to full-screen stimulation, and a brain-computer interface based on Motor Imagery (MI), which can actively activate motor cortex to promote nerve plasticity, but has the problems of low initial recognition rate and long training time. To combine the advantages of both, the prior art attempts to fuse ssmvep with MI. However, most existing fusion methods are static or simple weighted fusion, and lack self-adaptive capability to environmental changes and physiological states of users. The main defects of the method include: 1. static fusion strategies are poor in robustness, and when the signal quality of a certain mode (such as SSMVEP due to blinking, line of sight deviation or MI due to distraction) is temporarily reduced, noise is introduced by a fusion mode with fixed weight, so that overall recognition performance is suddenly reduced and even control is interrupted. 2. The existing system cannot sense the signal quality degradation event in real time, and when a key signal (such as an eye tracking signal or SSMVEP signal) is completely invalid, the system cannot autonomously switch to an available backup mode, so that the user experience is interrupted. 3. And the fusion decision is extensive, namely when the bimodal signals are available, the refined and dynamic weight distribution cannot be performed according to the real-time 'credibility' (such as signal-to-noise ratio and classification confidence) of the signals, so that the further improvement of the fusion precision is limited. Therefore, a fusion mechanism capable of sensing the quality of multi-mode signals in real time and performing intelligent decision and adaptive switching according to the quality is urgently needed, so as to solve the problems of insufficient robustness and discontinuous control in the prior art. Disclosure of Invention The invention aims to provide a self-adaptive mode switching method, a self-adaptive mode switching system and a storage medium based on SSMVEP-MI fusion brain-computer interface system, which can at least solve one of the existing problems. The technical scheme of the invention is as follows: according to a first aspect of the present invention, an adaptive mode switching method based on SSMVEP-MI fusion brain-computer interface system is disclosed, at least comprising: a signal quality real-time monitoring step of calculating and outputting a normalized first score representing SSMVEP signal quality, a second score representing MI signal quality and a third score representing eye movement tracking quality in real time; The self-adaptive decision step is that the first score, the second score and the third score are respectively compared with a preset first threshold value, a preset second threshold value and a preset third threshold value; The mode switching execution step, based on the comparison result, the control system switches among a bimodal fusion mode, a SSMVEP dominant mode and an MI dominant mode; wherein when the third score is below the third threshold and the first score is below the first threshold, triggering a switch to MI dominant mode; when the second score is below the second threshold, a switch to SSMVEP dominant mode is triggered. In some embodiments, in the bimodal fusion mode, the system dynamically calculates a fusion weight of the SSMVEP mode weight and the MI mode according to the first score, the second score and the third score, and performs weighted fusion on decoding results of the two modes according to the two weights to generate a final control instruction. In some embodiments, the dynamically calculated fusion weights employ a linear score based normalization strategy, and the weights w ssmvep for SSMVEP and w mi for MI modalities are calculated by the following formulas, respectively: wherein snr_ ssmvep is a first score, GSS is a second score, conf_mi is a third score, and α, β, γ are preset non-negative weighting coefficients; The weighting coefficients alpha, beta and gamma have the values of alpha epsilon [0.4, 1.0], beta epsilon [0.1, 0.6], gamma epsilon [0.5, 1.2], and alpha+beta+gamma=C, wherein C is a constant. In some embodiments, the first score is a SSMVEP signal-to-noise ratio score normal