CN-121997171-A - Multi-mode analysis-based anesthesia awakening period patient demand identification method and system

CN121997171ACN 121997171 ACN121997171 ACN 121997171ACN-121997171-A

Abstract

The invention discloses a method and a system for identifying the needs of patients in anesthesia and recovery periods based on multi-modal analysis, which relate to the field of big data and comprise the steps of collecting multi-modal data of the patients in real time, and performing cleaning, standardization and alignment treatment to form a standardized data set; extracting single-mode characteristics of various data, integrating the single-mode characteristics into a fusion characteristic set through a hierarchical multi-mode characteristic fusion model, constructing a deep learning requirement identification model, training and optimizing through a labeling data set, inputting the processed fusion characteristics into a trained model, outputting a patient requirement type and confidence level, preliminarily marking the requirement through a set threshold, carrying out multi-mode data cross-validation on suspected requirements, namely correcting the requirement type if the validation is passed, otherwise, re-acquiring the data and re-identifying. The invention has the advantages that the comprehensive and accurate identification of the patient needs in the anesthesia recovery period is realized by integrating multi-mode data, strengthening core information through hierarchical feature fusion and collocating a confidence judgment and cross-validation mechanism.

Inventors

ZHENG QINGYU
YUAN LINHUI
Zhao Qiaoshu
WANG YUQI
ZHU LING
TU PING

Assignees

南昌大学第二附属医院

Dates

Publication Date: 20260508
Application Date: 20260121

Claims (9)

1. The method for identifying the patient needs in the anesthesia recovery period based on the multi-mode analysis is characterized by comprising the following steps of: the method comprises the steps of collecting multi-mode data of a patient in an anesthesia awakening period in real time, wherein the multi-mode data comprise physiological signal data, behavior image data, voice interaction data and environment perception data; cleaning, standardizing and aligning various collected original data to obtain a standardized multi-mode data set; Based on the standardized multi-mode data set, respectively extracting the characteristics of various mode data to obtain Shan Motai characteristic sets; Constructing a hierarchical multi-mode feature fusion model, and carrying out fusion processing on each single-mode feature subset to obtain a fusion feature set; Constructing a deep learning demand recognition model based on the acquired fusion feature set, and performing model optimization training by taking the standardized multi-modal data set after labeling as a training sample; Inputting the fusion characteristics acquired and processed in real time into a trained demand recognition model, acquiring the demand type and confidence coefficient of a patient, judging the demand type by setting a confidence coefficient threshold value, and adding a label; Based on the marked suspected demand, performing secondary confirmation in a multi-mode data cross-validation mode, correcting the demand type if the cross-validation is passed, and collecting data again to perform demand identification if the cross-validation is not passed.
2. The method for identifying the needs of the patient in the anesthesia and recovery period based on the multi-modal analysis according to claim 1, wherein the real-time collection of the multi-modal data of the patient in the anesthesia and recovery period comprises the following specific steps: Physiological signal data are acquired through noninvasive sensing equipment attached to the body surface of a patient, and acquisition parameters comprise heart rate, respiratory rate, blood pressure, blood oxygen saturation and body temperature; the behavior image data are collected through a high-definition camera of a wake-up room, the collection range covers the head, trunk and upper limb movement areas of a patient, and facial expression, limb movement and body position changes of the patient are recorded; The voice interaction data are collected through a microphone, a region of 1-3 meters around a patient is collected, and voice signals of the patient are recorded after environmental noise is filtered; the environmental perception data are acquired through an environmental sensor, and the acquisition parameters comprise the temperature, humidity, illumination intensity and noise decibel value in a wake-up room.
3. The method for identifying the needs of a patient in an anesthesia recovery period based on multi-modal analysis according to claim 1, wherein the steps of cleaning, normalizing and aligning the collected raw data of various types, and obtaining the normalized multi-modal dataset specifically include: removing abnormal fluctuation data by adopting a sliding window method aiming at physiological signal data, supplementing missing data by adopting a linear interpolation method, and mapping the data to a [0,1] interval by adopting min-max standardization; Noise reduction processing is carried out on the behavior image data by adopting Gaussian filtering, a patient area is extracted through a target detection algorithm, background interference is removed, size normalization processing is carried out on image frames, 1920 x 1080 pixels are unified in resolution, and time sequence ordering of the image frames is completed based on time stamps; extracting voice characteristics of voice interaction data by adopting a Mel frequency cepstrum coefficient method, eliminating environmental noise through frequency spectrum subtraction, cutting invalid fragments without voice signals and converting the invalid fragments into a standardized audio format; Removing abnormal data beyond a normal range aiming at the environment sensing data, and smoothing the data by adopting a moving average method; and (3) performing time sequence alignment on the processed physiological signal data, the behavior image data, the voice interaction data and the environment perception data based on the unified time stamp to obtain a time-synchronized standardized multi-modal data set.
4. The method for identifying the needs of the patient in the anesthesia and wake-up period based on the multi-modal analysis according to claim 1, wherein the feature extraction is performed on each type of modal data based on the standardized multi-modal data set, and the feature set obtaining Shan Motai specifically comprises: extracting physiological signal data features by combining time domain analysis and frequency domain analysis, wherein the time domain features comprise a mean value, a variance, a peak value, a valley value and a waveform factor of signals, and the frequency domain features comprise a main frequency, frequency band energy and frequency spectrum entropy of the signals to form a physiological signal feature subset; Extracting behavior image data features through a deep learning algorithm, extracting facial expression features through a pre-trained convolutional neural network model, extracting limb action features through a gesture estimation algorithm, and forming a behavior image feature subset; Extracting time domain features and frequency domain features of the voice interaction data, wherein the time domain features comprise short-time energy, zero crossing rate and fundamental frequency of a voice signal, and the frequency domain features comprise spectrum centers, spectrum bandwidths and spectrum fluxes to form a voice interaction feature subset; and extracting change trend characteristics of the environment sensing data, including parameter mean values, change rates and fluctuation amplitudes, and forming an environment sensing characteristic subset.
5. The method for identifying the needs of the patient in the anesthesia and wake-up period based on the multi-modal analysis according to claim 1, wherein the constructing a hierarchical multi-modal feature fusion model, performing fusion processing on each single-modal feature subset, and obtaining the fusion feature set specifically comprises: constructing a hierarchical multi-mode feature fusion model, adopting an attention mechanism to distribute weights of all the single-mode feature subsets, and dynamically adjusting feature weights according to the correlation between different mode data and patient requirements; The dimensions of the weighted single-mode features are unified, the dimension reduction processing is carried out on the high-dimensional features by adopting a principal component analysis method, and principal component features with the accumulated contribution rate not lower than 90% are reserved; And combining the feature after dimension reduction into a fusion feature vector through feature splicing, and inputting the fusion feature vector into a bidirectional gating circulation unit to perform cross-modal information interaction to generate a fusion feature set.
6. The method for identifying the needs of the patient in the anesthesia and wake-up period based on the multi-modal analysis according to claim 1, wherein the step of constructing the deep learning need identification model based on the acquired fusion feature set, and the step of performing model optimization training by using the standardized multi-modal data set after labeling as a training sample specifically comprises the steps of: Based on the obtained fusion feature set, constructing a deep learning requirement identification model, wherein the model takes the fusion feature set as input and the requirement type of a patient in an anesthesia recovery period as output, and the requirement type comprises pain requirement, respiratory requirement, drinking water requirement, excretion requirement, posture adjustment requirement and environment adaptation requirement; Taking the marked standardized multi-modal data set as a training sample, and dividing the training sample into a training set, a verification set and a test set according to the proportion of 7:2:1; Calculating a prediction error by using a cross entropy loss function, updating model parameters through back propagation, recording a verification set loss value 1 time per iteration, calculating the accuracy of a test set when the loss value does not drop in 10 continuous periods, stopping training if the loss value is more than or equal to 85%, and storing optimal model parameters.
7. The method for identifying the needs of the patient in the anesthesia recovery period based on the multi-modal analysis according to claim 1, wherein the step of inputting the fusion characteristics acquired and processed in real time into the trained need identification model to obtain the needs type and the confidence level of the patient, and the step of judging the needs type by setting the confidence level threshold value and adding the label specifically comprises the following steps: Inputting the processed fusion characteristics acquired in real time into a trained demand recognition model to acquire the demand type and the corresponding confidence of the patient; Setting a confidence coefficient threshold value to be 0.7, directly judging the model to be a corresponding requirement type when the confidence coefficient output by the model is not lower than 0.7, generating a preliminary judgment result and associating corresponding characteristic data; if the confidence coefficient of all the demand types is less than 0.7, marking the demand type as a suspected demand, recording the current fusion characteristics and the original data fragment, and triggering a secondary verification process.
8. The method for identifying patient needs during an anesthesia awakening period based on multi-modal analysis according to claim 1, wherein the method for identifying the patient needs during the anesthesia awakening period based on the marker is characterized in that the method for identifying the patient needs secondarily by adopting a multi-modal data cross-validation mode, wherein if the cross-validation is passed, the type of the needs is corrected, and if the cross-validation is not passed, the method for identifying the needs by collecting the data again specifically comprises the following steps: For the marked suspected demand, a full original multi-mode data and preprocessing and feature extraction process data of 30 seconds before and after a corresponding period are called based on the time stamp; Screening and removing invalid data fragments, reserving four effective data of physiological signals, behavioral images, voices and environments, and sorting the effective data into a verification data set according to mode classification; And carrying out mode-by-mode feature matching on the verification data set based on the clinical anesthesia awakening period requirement judgment standard, correcting the requirement type and the confidence coefficient if the cross verification is passed, and triggering the data re-acquisition instruction to reenter the identification flow if the cross verification is not passed.
9. A multi-modal analysis based anesthesia recovery phase patient demand identification system for implementing a multi-modal analysis based anesthesia recovery phase patient demand identification method as set forth in any one of claims 1-8, comprising: the multi-data acquisition module acquires physiological signals, behavioral images, voice interaction and environment perception data of a patient in real time through the noninvasive sensing equipment, the high-definition camera, the microphone and the environment sensor; the data preprocessing module is used for cleaning, normalizing and time sequence aligning the original data to generate a normalized multi-mode data set; The feature extraction module is used for respectively extracting time domain features, frequency domain features, deep learning visual features and environmental trend features from physiological signals, behavioral images, voices and environmental data to form a single-mode feature subset; The multi-mode fusion module dynamically weights and fuses the mode characteristics by adopting an attention mechanism, generates fusion characteristic vectors through the dimension reduction and cross-mode interaction, and enhances the expression capability of the related characteristics of the requirements; The demand recognition model module is used for constructing a deep learning model based on the fusion feature set, training and optimizing model parameters through labeling data, and outputting the demand type and the confidence of the patient; the decision and verification module is used for judging the type of the demand according to the confidence threshold value, triggering multi-mode data cross verification on the low confidence result, and correcting or re-collecting data by combining clinical standards; the processor is used for processing the calculation process of each formula and the construction calculation process of each model.

Description

Multi-mode analysis-based anesthesia awakening period patient demand identification method and system Technical Field The invention relates to the field of big data, in particular to a method and a system for identifying the needs of patients in anesthesia and wake-up period based on multi-mode analysis. Background The anesthesia recovery period is a key stage of the patient to recover consciousness from the anesthesia state, and complications may be caused by unstable physiological functions, pain, agitation, cognitive dysfunction and the like during the period, so that life safety is threatened. The traditional identification method mainly depends on subjective observation and limited vital sign monitoring of medical staff, and is difficult to comprehensively and timely capture the dynamic demands of patients, especially special crowds who cannot effectively communicate. The current method for identifying the needs of the patients in the anesthesia and recovery period in the market mostly depends on single-mode data or simple multi-mode splicing, is difficult to comprehensively capture complex needs related information such as fuzzy pronunciation, unstructured limb actions and the like when the consciousness of the patients is not completely recovered, is lack of a system standardization flow in data processing, is easy to be interfered by environmental noise, unconscious body movements and the like, is difficult to ensure data quality, is rough in a characteristic fusion mode, does not combine with the dynamic distribution of modal weights of the needs, is insufficient in core feature extraction, is lack of a scientific confidence judgment and multi-mode cross-validation mechanism, is higher in static threshold value or manual judgment, is misjudged and leakage judged, is not fully attached to the clinical needs law, is poor in adaptability to individual differences of the patients, and has limited generalization capability and practical clinical application value. Disclosure of Invention In order to perfect the existing method and system, the method and system for identifying the needs of the patient in the anesthesia and recovery period are provided, and the method integrates multi-mode data, strengthens core information through standardized processing and hierarchical feature fusion, is matched with a confidence judgment and cross verification mechanism, realizes comprehensive and accurate identification of the needs of the patient in the anesthesia and recovery period, and is suitable for clinical scenes and high in practicability. In order to achieve the above purpose, the invention adopts the following technical scheme: a multi-modal analysis-based anesthesia awakening period patient demand identification method comprises the following steps: the method comprises the steps of collecting multi-mode data of a patient in an anesthesia awakening period in real time, wherein the multi-mode data comprise physiological signal data, behavior image data, voice interaction data and environment perception data; cleaning, standardizing and aligning various collected original data to obtain a standardized multi-mode data set; Based on the standardized multi-mode data set, respectively extracting the characteristics of various mode data to obtain Shan Motai characteristic sets; Constructing a hierarchical multi-mode feature fusion model, and carrying out fusion processing on each single-mode feature subset to obtain a fusion feature set; Constructing a deep learning demand recognition model based on the acquired fusion feature set, and performing model optimization training by taking the standardized multi-modal data set after labeling as a training sample; Inputting the fusion characteristics acquired and processed in real time into a trained demand recognition model, acquiring the demand type and confidence coefficient of a patient, judging the demand type by setting a confidence coefficient threshold value, and adding a label; Based on the marked suspected demand, performing secondary confirmation in a multi-mode data cross-validation mode, correcting the demand type if the cross-validation is passed, and collecting data again to perform demand identification if the cross-validation is not passed. Preferably, the collecting, in real time, the multi-mode data of the patient in the anesthesia wake-up period, including physiological signal data, behavior image data, voice interaction data and environment perception data specifically includes: Physiological signal data are acquired through noninvasive sensing equipment attached to the body surface of a patient, and acquisition parameters comprise heart rate, respiratory rate, blood pressure, blood oxygen saturation and body temperature; the behavior image data are collected through a high-definition camera of a wake-up room, the collection range covers the head, trunk and upper limb movement areas of a patient, and facial expression, limb movement and body positi