CN-122004897-A - Brain signal basic processing system and processing method for multi-modal nerve electrical signals

CN122004897ACN 122004897 ACN122004897 ACN 122004897ACN-122004897-A

Abstract

The invention relates to a brain signal basic processing system and a brain signal basic processing method for a multi-modal nerve electric signal, comprising a nerve electric signal abnormality detection module, an intra-brain and external brain unified spatial position coding module, a dynamic brain region spatial convolution module and a compound self-supervision learning module, wherein the nerve electric signal abnormality detection module is configured to output qualified multi-modal nerve electric signals, the brain internal and external brain unified spatial position coding module is configured to output multi-modal nerve electric signals with position codes, the dynamic brain region spatial convolution module is configured to carry out self-adaptive spatial weighting on the multi-modal nerve electric signals and convert heterogeneous multi-channel nerve electric signals into homogeneous single-channel feature sequences, and the compound self-supervision learning module is configured to execute self-supervision training combining mask prediction and autoregressive prediction on the homogeneous single-channel feature sequences and output a pre-trained brain signal basic model so as to complete downstream task assessment based on the pre-trained brain signal basic model. The invention improves the performance of multidimensional downstream tasks in the field of brain-computer interfaces and reduces the requirement of the downstream tasks on the data scale.

Inventors

Xiao Qinfan
QIAO YU
ZHOU BOWEN
ZHANG CHAO
Zhai Zhonghao
WANG ZHENJIE
ZHANG CHI
WU WEN
LI BAOXIANG
LI YUANNING
BAI LEI

Assignees

上海人工智能创新中心

Dates

Publication Date: 20260512
Application Date: 20260407

Claims (10)

1. A brain signal base processing system for a multi-modal neural electrical signal, comprising: the nerve electric signal abnormality detection module is configured to perform bad trace labeling, artifact cleaning and separating and reject unqualified signal fragments on the original multi-mode nerve electric signal and output qualified multi-mode nerve electric signals; The brain-in-brain-out unified spatial position coding module is configured to map the multimode sensor corresponding to the qualified multimode nerve electric signal to a unified three-dimensional coordinate system, fuse the spatial coordinates and physical properties of the sensor to generate a position code capable of being learned, and output the multimode nerve electric signal with the position code; A dynamic brain region spatial convolution module configured to generate a dynamic spatial convolution kernel based on the multi-modal neural electrical signal with position encoding, adaptively spatially weight the multi-modal neural electrical signal, convert the heterogeneous multi-channel neural electrical signal into a homogeneous single-channel feature sequence, and And the compound self-supervision learning module is configured to perform self-supervision training of combining mask prediction and autoregressive prediction on the isomorphic single-channel feature sequence, and output a pre-trained brain signal basic model to process multi-mode nerve electric signals related to downstream tasks based on the pre-trained brain signal basic model.
2. The brain signal basic processing system for the multi-modal nerve electric signals according to claim 1, wherein the nerve electric signal abnormality detection module performs sliding window detection on the original multi-modal nerve electric signals on a time domain and a frequency domain based on a residual one-dimensional convolution network, automatically identifies bad tracks and severe artifact interference, and rejects disqualified signal fragments by scoring signal quality.
3. The system of claim 1, wherein the multi-modal electrical nerve signals include electroencephalogram, magnetoencephalography, and invasive electroencephalogram.
4. The brain signal basic processing system for multi-modal nerve electrical signals according to claim 1, wherein the multi-modal sensor comprises scalp electroencephalogram electrodes, brain magnetic sensors and invasive brain electrical contacts in the brain-in-brain-out unified spatial position coding module.
5. The multi-modal neural signal oriented brain signal base processing system of claim 4, wherein in the intra-brain and extra-brain unified spatial location coding module, generating the location code comprises the steps of: Mapping the multi-modal sensor to a unified MNI standard brain three-dimensional coordinate system; encoding the spatial coordinates of the sensor using a linear layer and converting the type of sensor to a learning-capable encoding, and After the position and physical attribute characteristics of the electrodes are fused, the multilayer perceptron is used for carrying out characteristic fusion, and the final position code is obtained after normalization.
6. The multi-modal neural signal oriented brain signal base processing system of claim 1, wherein the dynamic brain region spatial convolution module comprises: Electrode position coding, and And a learnable inversion matrix which is used for carrying out coding operation with the electrode position to obtain the dynamic space convolution kernel.
7. The system of claim 6, wherein the dynamic brain region spatial convolution module multiplies the inversion matrix by a transpose of the electrode position codes and converts the result to an attention score using an exponential normalization operation to obtain dynamic spatial convolution kernels, the dynamic spatial convolution kernels are a plurality of dynamic spatial filters, and each dynamic spatial convolution kernel learns different weighted filtering modes for sensors located in different brain regions to convert heterogeneous multichannel nerve signals into homogeneous single-channel feature sequences.
8. The system of claim 1, wherein the mask prediction is performed by splitting the brain signal sequence into front, middle and rear segments and recombining the sequences, predicting discrete labels of the masked portion based on context by causal attention, and the autoregressive prediction is performed by predicting probability distribution of discrete labels corresponding to the next position by each position after causal attention calculation of the feature sequence.
9. The multimodal nerve electrical signal oriented brain signal base processing system of claim 1, wherein the downstream tasks include coarse-granularity discrimination tasks including disease-assisted diagnosis, emotional state recognition, sleep staging, motor imagery classification, and fine-granularity generation tasks including language decoding, visual reconstruction, and auditory decoding.
10. The brain signal basic processing method for the multi-mode nerve electric signal is characterized by comprising the following steps of: s1, inputting original data of the multi-mode nerve electric signals into a nerve electric signal abnormality detection module, completing automatic identification and cleaning of bad tracks and artifacts, removing unqualified signal fragments, and outputting qualified multi-mode nerve electric signals; S2, inputting the qualified multi-modal nerve electric signals into an intra-brain and external brain unified space coding module, completing three-dimensional coordinate alignment of the multi-modal sensor, fusing the sensor space coordinates and physical properties to generate a learnable position code, and outputting the multi-modal nerve electric signals with the position code; S3, inputting the multi-mode nerve electric signal with the position codes into a dynamic brain region space convolution module to generate a dynamic space convolution kernel, carrying out self-adaptive space weighting on the heterogeneous multi-channel nerve electric signal, and converting the heterogeneous multi-channel nerve electric signal into a isomorphic single-channel feature sequence; S4, inputting the isomorphic single-channel feature sequence into a composite self-supervision learning module, executing self-supervision training combining mask prediction and autoregressive prediction to complete model pre-training and outputting a brain signal basic model after pre-training, and S5, processing the multi-modal nerve electric signals related to the downstream task by using the brain signal basic model which is pre-trained.

Description

Brain signal basic processing system and processing method for multi-modal nerve electrical signals Technical Field The invention relates to the technical field of intersection of brain science and artificial intelligence, in particular to a brain signal basic processing system and a brain signal basic processing method for multi-mode nerve electrical signals. Background With the development of artificial intelligence, the deep learning technology is widely applied in the field of brain science, and a feature extraction method which is superior to the original method which relies on a large number of human priori designs is shown. The brain science is used as a core field for exploring human cognition and neural activity, and the research result has important application value in the fields of brain-computer interfaces, neural disease diagnosis, human-computer intelligent interaction and the like. The nerve electrical signal is a core carrier reflecting the brain nerve activity, and mainly comprises a plurality of modes such as scalp electroencephalogram (EEG), brain Magnet (MEG), invasive electroencephalogram (iEEG) and the like, and the nerve activity state of the brain is characterized by different mode signals from macroscopic dimensions, microscopic dimensions and the like, and the nerve electrical signal has technical advantages and application scenes. However, research and development and application of the brain signal basic model still face a plurality of technical bottlenecks, the labeled data set in the brain science field is small in scale and high in acquisition cost, meanwhile, the corresponding difference of nerves among people is large, the data heterogeneity of different modal signals of different equipment is obvious, and the brain signal basic model with large enough capacity and strong generalization capability is difficult to train. The prior art has the following defects that 1) the prior art is mainly concentrated on an electroencephalogram single mode, 2) the pre-training paradigm lacks exploration, the prior art mainly uses mask reconstruction means to unsupervised the endophytic knowledge of learning data, for brain signals with lower signal-to-noise ratio, the original signals with higher noise level are used as the final learning target to potentially constrain the upper performance limit of a model, 3) the model generalization capability is weak, the prior brain signal basic model still depends on full fine tuning when being applied at the downstream, the problems of catastrophic forgetting of knowledge and the like can be brought, and the prior model is not enough to generate the characteristic with strong expressive force are described, 4) the model time precision is low, the model windowing time is too long to capture the nerve activity with the rapid change of tens of milliseconds, 5) the downstream task application coverage is narrow, the downstream is mainly focused on the sequence classification tasks such as disease classification, emotion recognition, imagination action classification and the like, the granularity basic model is relatively coarse, and the performance of fine granularity tasks such as language decoding and vision reconstruction is not focused sufficiently. How to expand the data scale and model size, and to learn the internal knowledge structure of various brain signals with high efficiency by using a self-supervision method, thereby enabling the development of downstream tasks of brain-computer interfaces, which is a current urgent problem to be solved. Disclosure of Invention The invention provides a brain signal basic processing system and a brain signal basic processing method for multi-mode nerve electrical signals, and aims to construct a nerve characterization system with a strong generalization capability through multi-mode integration and dual-target learning so as to establish a brain basic model for covering various nerve electrical signals such as brain electricity, brain magnetism, invasive brain electricity and the like. The invention mainly solves the following core challenges existing in the current brain signal basic model when processing multi-modal nerve electric signals: 1) Modality integration and heterogeneous equipment adaptation bottlenecks, namely, although nerve electric signals are excited by common nerve activities, the existing model is limited to a single modality or specific equipment, and cannot effectively integrate various nerve electric signals including EEG, MEG and iEEG, and cannot adapt to heterogeneous acquisition scenes with frequent fluctuation of the number, types and positions of sensors; 2) The data quality and the artifact interference are that a large number of unlabeled bad tracks and artifacts are mixed in open source data, and a high-quality automatic cleaning algorithm is lacked, so that a pre-training model is fitted to false features; 3) The limitation of the pre-training paradigm is that the existing m