CN-121641461-B - Neonate health monitoring system based on multi-mode data fusion

CN121641461BCN 121641461 BCN121641461 BCN 121641461BCN-121641461-B

Abstract

The application discloses a neonate health monitoring system based on multi-mode data fusion, and belongs to the technical field of medical monitoring. The system aims at the problem of misjudgment caused by complex background interference and neonatal development difference, a background-aware characteristic decoupling enhancement network is constructed, a dual-channel architecture is utilized to match with an anti-loss function, foreground skin and background characteristics are forcedly separated, and weak pathological characteristics are enhanced by combining a color space attention mechanism. Meanwhile, by using a development stage self-adaptive classification network, the gestational age is used as priori information to generate development codes, and the fusion characteristics are dynamically modulated and weighted estimated by a premature infant and term infant double-branch classifier. The application effectively eliminates the influence of environmental noise and realizes differential accurate health state identification and risk early warning.

Inventors

DING LIWEN
YANG HAIYING
ZHANG HUAPING

Assignees

成都慧安云信息科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260204

Claims (8)

1. A neonatal health monitoring system based on multimodal data fusion, comprising: A data acquisition unit (1) for acquiring physiological signal data and video frame image data of a neonate, and attaching a time stamp to the data; the feature extraction unit (2) is used for extracting the time domain features of the physiological signal data based on time domain analysis to construct a time domain feature vector, extracting a feature map of the video frame image data through a background channel and a skin channel based on a background sensing feature decoupling enhancement network, and enabling skin features to be independent of background features by utilizing a background sensing anti-loss function to generate a high-dimensional image feature vector; the multi-mode fusion sensing unit (3) is used for mapping the time domain feature vector and the high-dimensional image feature vector to the same dimension and splicing the same, and generating fused feature data through a multi-layer sensing machine; The health state identification and risk assessment unit (4) is used for classifying the current health state of the neonate based on a development stage self-adaptive classification network by utilizing the fused characteristic data and calculating a current health risk score by utilizing a Bayesian network; the development stage self-adaptive classification network introduces neonatal gestational age as priori information to construct premature infant branches and term infant branches, and specifically performs the following steps: Converting the input neonatal gestational age into a three-dimensional development coding vector by a soft distribution coder based on a Gaussian function, wherein the three-dimensional development coding vector is constructed based on similarity probability distribution from the current gestational age to a preset premature central point, a transitional period central point and a term central point; Splicing the three-dimensional development coding vector with the normalized fused characteristic data to form an enhanced input vector; generating a weight vector based on the prior information, and dynamically modulating the fused characteristic data by using the weight vector, wherein the dynamic modulation comprises the step of calibrating characteristic distribution offset by combining a basic weight coefficient and a modulation weight coefficient; The development stage self-adaptive classification network processes premature infant branches connected with premature infant exclusive classification heads and term infant branches connected with term infant exclusive classification heads, calculates normalized weights of current gestational age relative to premature infant and term infant reference gestational age by means of Gaussian kernel functions, respectively carries out weighted summation on probability vectors output by the premature infant exclusive classification heads and probability vectors output by the term infant exclusive classification heads, and completes smooth weighted fusion of output results of the double branches to judge health states.
2. The neonatal healthcare system based on multimodal data fusion as claimed in claim 1, wherein said feature extraction unit (2) comprises an image feature extraction module (22), said image feature extraction module (22) inputs preprocessed video frame image data into said context-aware feature decoupling enhancement network, performing the steps of: The method comprises the steps of dividing video frame image data by a lightweight dividing network to generate a foreground mask corresponding to a neonate skin area and a background mask corresponding to a background area, initializing a background feature library, judging the consistency of the background area based on a frame difference method to update the background feature library, extracting background features by global average pooling through a background channel, extracting foreground features by a spatial attention mechanism through the skin channel, introducing a background discriminator, carrying out feature enhancement on the background features by the background perception countermeasures, constructing a color space attention mechanism based on different pathological symptoms to carry out feature enhancement on the foreground features, carrying out global average pooling and splicing on the enhanced background features and foreground feature images, and outputting the high-dimensional image feature vector.
3. The neonatal health monitoring system based on multimodal data fusion of claim 2, wherein the background perceived contrast loss function comprises two parts: The first part is used for maximizing the identification capacity of the background discriminator on the real background features extracted from the background feature library; the second part is used for minimizing the probability that the background discriminator misjudges the extracted foreground feature as the background feature, the separation of the foreground feature and the background feature is realized through countermeasure training, the enhanced background feature is obtained by iterative optimization based on the combination of the background perception countermeasure loss function and a regularization term, and the regularization term is used for restricting the degree of the enhanced feature deviating from the initial background feature.
4. The neonatal healthcare system based on multimodal data fusion of claim 2, wherein the color space attention mechanism designs enhancement strategies for different pathological symptoms, including: Converting an image into a YCbCr color space, extracting Cb channel information, and constructing a channel attention weight to enhance the color contrast of a jaundice area; Converting the image into Lab color space, extracting b channel image, calculating edge gradient characteristic by Sobel operator to highlight abnormal distribution edge of purple blue region; For pale, a color ratio vector is constructed under an RGB space, a low-color difference area is enhanced by identifying brightness reduction and color saturation weakening characteristics of a skin area, and an enhanced foreground characteristic image is obtained by respectively weighting and summing an original foreground characteristic with an attention weight image aiming at jaundice, an edge gradient characteristic image aiming at cyanosis and an attention weight image aiming at pale.
5. The neonatal health monitoring system based on multimodal data fusion as set forth in claim 1, wherein the training loss function of the development stage adaptive classification network employs a dual supervised cross entropy loss function consisting of a main classification loss, a premature infant branch assist loss, and a term infant branch assist loss weighting, wherein the weight of the premature infant branch assist loss increases as the age of the fetus decreases and the weight of the term infant branch assist loss increases as the age of the fetus increases to ensure that the network obtains corresponding supervision signals at different development stages.
6. The neonatal health monitoring system based on multimodal data fusion as claimed in claim 1, wherein the health status identification and risk assessment unit (4) further comprises a health risk assessment module, which calculates a current health risk score based on the bayesian network, comprising the specific steps of: collecting neonatal individual characteristic data, mapping the neonatal individual characteristic data and health state probability distribution output by the development stage self-adaptive classification network to a structured risk factor space to form multi-dimensional factor nodes, inputting the multi-dimensional factor nodes as observation values based on causal relation and conditional probability tables among nodes learned in advance in the Bayesian network, performing forward reasoning to obtain posterior probability estimated values of various potential risk events, and calculating to obtain a comprehensive risk score through weighted summation according to the posterior probability estimated values of the potential risk events and corresponding risk severity weights.
7. The neonatal health monitoring system based on multi-modal data fusion as set forth in claim 6, wherein the structure of the bayesian network is a directed acyclic graph learned from historical data by a structure learning algorithm that progressively determines parent-child connection relationships between nodes by maximizing a scoring function given a priori ordering of variables, and wherein the conditional probability table is constructed by counting historical data frequency distribution or fitting a conditional probability density function using a Gao Sibei phyllos network.
8. The neonatal health monitoring system based on multimodal data fusion as set forth in claim 6, wherein the health risk assessment module further classifies risk levels according to the composite risk score, wherein the risk is determined to be low if the score is less than a first threshold, medium risk is determined to trigger primary medical intervention advice if the score is greater than or equal to the first threshold and less than a second threshold, high risk is determined to trigger early warning if the score is greater than or equal to the second threshold and less than a third threshold, and extremely high risk is determined to trigger alert pushing if the score is greater than or equal to the third threshold.

Description

Neonate health monitoring system based on multi-mode data fusion Technical Field The invention relates to the technical field of medical monitoring, in particular to a neonate health monitoring system based on multi-mode data fusion. Background With the advancement of medical monitoring technology, multi-modal data fusion technology is gradually applied to Neonatal Intensive Care Units (NICU), and comprehensive perception of neonatal health status is expected to be achieved by combining physiological signals with video image data. Physiological characteristics (such as heart rate and respiration) and appearance characteristics (such as skin color change) of the neonate are key indexes for judging the health condition of the neonate. However, existing neonatal healthcare systems face serious technical challenges in practical applications. Firstly, the NICU environment is complex, and the surrounding of a neonate is often accompanied with bedsheets, clothes and various medical pipeline devices with different colors, and the complex background noise is extremely easy to cause serious interference to the identification of skin pathological features (such as jaundice and cyanosis) based on video images, so that the feature extraction is impure and the identification accuracy is reduced. Secondly, when a neonate is in a rapid development stage, a premature infant and a term infant have significant differences on physiological sign references (for example, the fluctuation range of the heart rate of the premature infant is normal within a specific range, and the same fluctuation can mean pathological risk for the term infant), the prior art usually adopts a uniform classification model for evaluation, and a self-adaptive adjustment mechanism for different gestational age development stages is lacking, so that false alarm or false alarm is easy to be caused. Therefore, there is a need for a healthcare system that effectively eliminates environmental background interference and adaptively adjusts the evaluation criteria according to the neonatal developmental stage. Disclosure of Invention The invention mainly aims to provide a neonate health monitoring system based on multi-mode data fusion so as to solve the problems in the related art. To achieve the above object, according to one aspect of the present invention, there is provided a neonatal health monitoring system based on multi-modal data fusion, comprising The data acquisition unit is used for acquiring physiological signal data and video frame image data of the neonate and adding a time stamp to the data; The feature extraction unit is used for extracting the time domain features of the physiological signal data based on time domain analysis to construct a time domain feature vector, extracting a feature map of the video frame image data through a background channel and a skin channel based on a background perceived feature decoupling enhancement network, and enabling skin features to be independent of background features by utilizing a background perceived contrast loss function to generate a high-dimensional image feature vector; The multi-mode fusion sensing unit is used for mapping the time domain feature vector and the high-dimensional image feature vector to the same dimension and splicing the time domain feature vector and the high-dimensional image feature vector, and generating fused feature data through a multi-layer sensing machine; the health state identification and risk assessment unit is used for classifying the current health state of the neonate based on the development stage self-adaptive classification network by utilizing the fused characteristic data, and calculating the current health risk score by utilizing a Bayesian network; The development stage self-adaptive classification network introduces neonatal gestational age as priori information, constructs premature infant branches and term infant branches, dynamically modulates the fused characteristic data according to the gestational age information, and smoothly weights and fuses output results of the double branches to judge the health state. Further, the feature extraction unit includes an image feature extraction module, and the image feature extraction module inputs the preprocessed video frame image data into the background-aware feature decoupling enhancement network, and performs the following steps: The method comprises the steps of dividing video frame image data by a lightweight dividing network to generate a foreground mask corresponding to a neonate skin area and a background mask corresponding to a background area, initializing a background feature library, judging the consistency of the background area based on a frame difference method to update the background feature library, extracting background features by global average pooling through a background channel, extracting foreground features by a spatial attention mechanism through the skin channel, introducing a backgr