CN-121765441-B - Cognitive saturation degree recognition method, system and model training method

CN121765441BCN 121765441 BCN121765441 BCN 121765441BCN-121765441-B

Abstract

The invention provides a cognitive saturation degree recognition method, a system and a model training method, which can be applied to the technical field of cognitive saturation degree recognition. The method comprises the steps of performing spectrum analysis on electroencephalogram data of an object to be identified to obtain frequency domain electroencephalogram characteristic data, obtaining graph domain electroencephalogram characteristic data according to intra-channel visibility relations among a plurality of electroencephalogram sampling values in electroencephalogram time sequence data corresponding to at least one electroencephalogram channel, obtaining graph domain physiological characteristic data according to inter-channel visibility relations among a plurality of physiological sampling values in physiological time sequence data corresponding to a plurality of physiological channels, which are included in physiological signal data, fusing the frequency domain electroencephalogram characteristic data, the graph domain electroencephalogram characteristic data and the graph domain physiological characteristic data to obtain multi-mode characteristic fusion data, and determining the cognitive saturation level of the object to be identified according to the multi-mode characteristic fusion data.

Inventors

GAO ZHONGKE
LI HAOYU
HAO YUSHI
CUI XIAONAN

Assignees

天津大学

Dates

Publication Date: 20260505
Application Date: 20260228

Claims (10)

1. A method for identifying cognitive saturation, comprising: Performing spectrum analysis on electroencephalogram data of an object to be identified to obtain frequency domain electroencephalogram characteristic data, wherein the electroencephalogram data comprises electroencephalogram time sequence data corresponding to at least one electroencephalogram channel respectively, and the frequency domain electroencephalogram characteristic data represents energy distribution of the electroencephalogram data in a plurality of preset physiological frequency bands; For any one of the electroencephalogram channels, obtaining channel domain electroencephalogram characteristic data corresponding to the electroencephalogram channel according to the intra-channel visibility relation among a plurality of electroencephalogram sampling values in electroencephalogram time sequence data corresponding to the electroencephalogram channel, and obtaining domain electroencephalogram characteristic data according to the channel domain electroencephalogram characteristic data corresponding to at least one electroencephalogram channel, wherein the channel domain electroencephalogram characteristic data represents the connectivity of the electroencephalogram sampling moments corresponding to the electroencephalogram sampling values, and the connectivity of the electroencephalogram sampling moments is the number of connectable other electroencephalogram sampling moments determined according to the intra-channel visibility relation; Obtaining domain physiological characteristic data according to cross-channel visibility relations among a plurality of physiological sampling values in physiological time sequence data corresponding to a plurality of physiological channels, wherein the physiological characteristic data of the domain represent the connectivity of each of the physiological sampling values, and the connectivity of the physiological sampling values is the number of connectable other physiological sampling values determined according to the cross-channel visibility relations; fusing the frequency domain brain electrical characteristic data, the map domain brain electrical characteristic data and the map domain physiological characteristic data to obtain multi-mode characteristic fusion data; And determining the cognitive saturation level of the object to be identified according to the multi-mode feature fusion data.
2. The method of claim 1, wherein the fusing the frequency domain electroencephalogram feature data, the map domain electroencephalogram feature data, and the map domain physiological feature data to obtain multi-modal feature fusion data comprises: channel attention fusion is carried out on the frequency domain brain electrical characteristic data and the map domain brain electrical characteristic data, so that brain electrical characteristic fusion data are obtained; And carrying out weighted fusion on the characteristic dimension data in the electroencephalogram characteristic fusion data and the characteristic dimension data in the map domain physiological characteristic data to obtain the multi-mode characteristic fusion data.
3. The method according to claim 2, wherein the performing channel attention fusion on the frequency domain electroencephalogram feature data and the map domain electroencephalogram feature data to obtain electroencephalogram feature fusion data includes: convolving the frequency domain electroencephalogram characteristic data and the map domain electroencephalogram characteristic data to obtain middle frequency domain electroencephalogram characteristic data and middle map domain electroencephalogram characteristic data, wherein the characteristic dimension of the middle map domain electroencephalogram characteristic data is the same as the characteristic dimension of the middle frequency domain electroencephalogram characteristic data; And fusing the intermediate frequency domain electroencephalogram characteristic data and the intermediate map domain electroencephalogram characteristic data according to the characteristic dimension of the intermediate frequency domain electroencephalogram characteristic data and the characteristic dimension of the intermediate map domain electroencephalogram characteristic data to obtain the electroencephalogram characteristic fusion data.
4. The method according to claim 3, wherein the fusing the intermediate frequency domain electroencephalogram feature data and the intermediate map domain electroencephalogram feature data according to the feature dimension of the intermediate frequency domain electroencephalogram feature data and the feature dimension of the intermediate map domain electroencephalogram feature data to obtain the electroencephalogram feature fusion data includes: Determining an electroencephalogram feature fusion dimension weight according to the intermediate frequency domain electroencephalogram feature data and the associated data of the intermediate graph domain electroencephalogram feature data in a feature dimension; And carrying out feature fusion on the intermediate frequency domain electroencephalogram feature data and the intermediate graph domain electroencephalogram feature data according to the electroencephalogram feature fusion dimension weight to obtain the electroencephalogram feature fusion data.
5. The method according to any one of claims 2 to 4, wherein the performing weighted fusion on the feature dimension data in the electroencephalogram feature fusion data and the feature dimension data in the domain physiological feature data to obtain the multi-modal feature fusion data includes: carrying out convolution processing on the domain physiological characteristic data to obtain middle domain physiological characteristic data; carrying out weighted fusion on the characteristic dimension data in the electroencephalogram characteristic fusion data and the characteristic dimension data in the middle map domain physiological characteristic data to obtain electroencephalogram physiological characteristic connection data; and carrying out edge attention convolution on the electroencephalogram physiological characteristic connection data to obtain the multi-mode characteristic fusion data.
6. The method according to any one of claims 1 to 4, wherein the obtaining, according to the intra-channel visibility relationship between a plurality of electroencephalogram sampling values in the electroencephalogram time sequence data corresponding to the electroencephalogram channel, channel map domain electroencephalogram feature data corresponding to the electroencephalogram channel includes: Determining an intra-channel visibility relationship between a plurality of electroencephalogram sampling values in electroencephalogram time sequence data corresponding to the electroencephalogram channel according to a preset electroencephalogram penetration parameter, wherein the electroencephalogram penetration parameter characterizes the number of the electroencephalogram blocking sampling moments which are used for blocking connection of the electroencephalogram sampling moment and other electroencephalogram sampling moments; And obtaining the channel map domain brain electrical characteristic data corresponding to the brain electrical channel according to the intra-channel visibility relation and the brain electrical connectivity sequence length among a plurality of brain electrical sampling values in brain electrical time sequence data corresponding to the brain electrical channel, wherein the brain electrical connectivity sequence length represents the sequence length of connectivity at the brain electrical sampling time.
7. The method according to any one of claims 1 to 4, wherein the obtaining domain physiological characteristic data according to a cross-channel visibility relationship between a plurality of physiological sampling values in physiological time series data corresponding to a plurality of physiological channels included in the physiological signal data includes: And obtaining the domain physiological characteristic data according to a cross-channel visibility relation and a physiological connectivity sequence length between a plurality of physiological sampling values in physiological time sequence data which are respectively corresponding to a plurality of physiological channels and are included in the physiological signal data, wherein the physiological connectivity sequence represents the sequence length of connectivity at physiological sampling time.
8. A method of model training, comprising: Performing spectrum analysis on sample brain electrical signal data of a sample identification object to obtain sample frequency domain brain electrical characteristic data, wherein the sample brain electrical signal data comprises sample brain electrical time sequence data corresponding to at least one sample brain electrical channel, and the sample frequency domain brain electrical characteristic data represents energy distribution of the sample brain electrical signal data in a plurality of preset physiological frequency bands; For any one of at least one sample electroencephalogram channel, obtaining sample channel domain electroencephalogram feature data corresponding to the sample electroencephalogram channel according to a visibility relationship in the sample channel between a plurality of sample electroencephalogram sampling values in sample electroencephalogram time sequence data corresponding to the sample electroencephalogram channel, and obtaining sample domain electroencephalogram feature data according to sample channel domain electroencephalogram feature data corresponding to at least one sample electroencephalogram channel, wherein the sample channel domain electroencephalogram feature data represents the connectivity of the sample electroencephalogram sampling moments corresponding to a plurality of sample electroencephalogram sampling values, and the connectivity of the sample electroencephalogram sampling moments is the number of connectable other sample electroencephalogram sampling moments determined according to the visibility relationship in the sample channel; Obtaining sample domain physiological characteristic data according to sample cross-channel visibility relations among a plurality of sample physiological sampling values in sample physiological time sequence data corresponding to a plurality of physiological channels, wherein the sample domain physiological characteristic data represent respective connectivity of the physiological sampling values, and the connectivity of the sample physiological sampling values is the number of connectable other sample physiological sampling values determined according to the sample cross-channel visibility relations; fusing the sample frequency domain electroencephalogram characteristic data, the sample map domain electroencephalogram characteristic data and the sample map domain physiological characteristic data to obtain sample multi-mode characteristic fusion data; Determining a sample cognitive saturation level of the sample identification object according to the sample multi-mode feature fusion data; Based on the loss function, training a deep learning model according to the sample cognitive saturation level and the sample label.
9. The method of claim 8, wherein determining a sample cognitive saturation level of the sample recognition object from the sample multi-modal feature fusion data comprises: Deleting fusion sub-data to be deleted of a target frame number from the sample multi-mode feature fusion data to obtain sample multi-mode feature fusion updating data; and determining the sample cognitive saturation level of the sample identification object according to the sample multi-mode feature fusion updating data.
10. A cognitive saturation recognition system, comprising: The physiological signal data acquisition module is configured to acquire physiological signal data of an object to be identified, wherein the physiological signal data comprises physiological time sequence data corresponding to a plurality of physiological channels respectively; The electroencephalogram data acquisition module is configured to acquire electroencephalogram data of the object to be identified, wherein the electroencephalogram data comprises electroencephalogram time sequence data corresponding to at least one electroencephalogram channel respectively; An electronic device, comprising: One or more processors; A memory for storing one or more programs, Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9; and the electric stimulation module is configured to generate an electric stimulation signal according to the cognitive saturation level output by the electronic equipment.

Description

Cognitive saturation degree recognition method, system and model training method Technical Field The invention relates to the technical field of cognitive saturation recognition, in particular to a cognitive saturation recognition method, a cognitive saturation recognition system and a model training method. Background The cognitive saturation (Cognitive Saturation, CS) represents the external appearance of the subject resulting from dynamic integration of fatigue status, attentiveness resource attenuation, cognitive overload, etc. under a continuous cognitive task. In the scenes of repetitive operations (such as assembly line assembly and precision element detection), long-term persistent operations (such as central control room monitoring and aviation control) and the like, the cognitive load of an operation object is in a continuous high-load calling state, if the cognitive demand exceeds the recovery and compensation threshold value of the object resource for a long time, the cognitive saturation degree is accelerated to increase, the object is prolonged in reaction on the action level, the error rate is increased, the working memory capacity is reduced, the decision quality is deteriorated, and the subjective experience is often accompanied by listlessness, the motivation weakening and the emotion regulating capability reduction. Thus, accurate identification of the subject's cognitive saturation contributes to maintaining the subject's health, however, current methods of identifying cognitive saturation have low accuracy. Disclosure of Invention In view of the above, the embodiment of the invention provides a cognitive saturation degree identification method, a system and a model training method. An aspect of the embodiment of the invention provides a cognitive saturation degree identification method, which comprises the steps of performing frequency spectrum analysis on electroencephalogram data of an object to be identified to obtain frequency domain electroencephalogram characteristic data, wherein the electroencephalogram data comprises electroencephalogram time sequence data corresponding to at least one electroencephalogram channel respectively, and the frequency domain electroencephalogram characteristic data represents energy distribution of the electroencephalogram data in a plurality of preset physiological frequency bands; obtaining a channel domain electroencephalogram characteristic data corresponding to the electroencephalogram channel according to the intra-channel visibility relation among a plurality of electroencephalogram sampling values in electroencephalogram time sequence data corresponding to the electroencephalogram channel, obtaining a map domain electroencephalogram characteristic data according to the channel domain electroencephalogram characteristic data corresponding to at least one electroencephalogram channel respectively, wherein the channel domain electroencephalogram characteristic data represents the connectivity of the electroencephalogram sampling moment corresponding to the plurality of electroencephalogram sampling values respectively, the connectivity of the electroencephalogram sampling moment is the number of connectable other electroencephalogram sampling moments determined according to the intra-channel visibility relation, obtaining a map domain physiological characteristic data according to the inter-channel visibility relation among a plurality of physiological sampling values in the physiological time sequence data corresponding to the plurality of physiological channels included in the physiological signal data, wherein the map domain physiological characteristic data represents the respective connectivity of the physiological sampling values, the connectivity of the map domain electroencephalogram characteristic data represents the connectivity of the physiological sampling values respectively, the connectivity of the electroencephalogram sampling values is the inter-channel visible relation is determined according to the inter-channel visibility relation, the inter-physiological sampling value is the inter-channel visible relation determined according to the inter-physiological sampling value, and determining the cognitive saturation level of the object to be identified according to the multi-modal feature fusion data. According to another aspect of the embodiment of the invention, a model training method is provided, which comprises the steps of carrying out spectrum analysis on sample electroencephalogram data of a sample identification object to obtain sample frequency domain electroencephalogram feature data, wherein the sample electroencephalogram data comprises sample electroencephalogram time sequence data corresponding to at least one sample electroencephalogram channel, the sample frequency domain electroencephalogram feature data represents energy distribution of the sample electroencephalogram data in a plurality of preset physiological fr