CN-121746388-B - Multi-mode image fusion hand pollution assessment method, system and storage medium

CN121746388BCN 121746388 BCN121746388 BCN 121746388BCN-121746388-B

Abstract

The application relates to the field of health monitoring, and discloses a hand pollution assessment method, a hand pollution assessment system and a storage medium integrating multi-mode images. The method comprises the steps of synchronously collecting visible light, fluorescence and thermal imaging data of hands, carrying out denoising treatment, extracting hand outline and fluorescence intensity distribution to generate an initial pollution distribution characteristic diagram, calculating pollution concentration gradients, grading the pollution intensity of hand areas, carrying out spatial registration and weighted fusion on the thermal imaging data and the fluorescence intensity to obtain a temperature weighted pollution distribution diagram, carrying out correction and normalization treatment on the thermal imaging data and the fluorescence intensity based on the pollution proportions of high-temperature areas, carrying out temperature dependent diffusion simulation and spatial mapping on pollution degree distribution results to generate a hand pollution comprehensive distribution diagram, and carrying out area comparison analysis and evaluation report generation based on the diagram. The method solves the problems that the accuracy is insufficient and the dynamic risk cannot be reflected when the single imaging mode evaluates the hand pollution, and improves the comprehensiveness and accuracy of the evaluation.

Inventors

CHEN LINGLING
CHEN HANGFENG
CHEN BINGCAI

Assignees

杭州春芽环保科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260226

Claims (10)

1. The hand pollution assessment method integrating the multi-mode images is characterized by comprising the following steps of: S101, synchronously acquiring visible light images, microbial fluorescent signals and thermal imaging data on the surface of a hand to form an original multi-modal data set, and denoising the original multi-modal data set to obtain hand multi-modal data; step S102, extracting hand contour features and fluorescence signal intensity distribution from the hand multi-mode data, and finely dividing the hand contour features through edge detection and gradient calculation to obtain an initial hand pollution distribution feature map; Step S103, calculating pollution concentration gradients according to the initial hand pollution distribution feature diagram, and carrying out grading evaluation on pollution intensities of different hand areas by combining the fluorescence signal intensity distribution to determine pollution degree distribution results; Step S104, establishing a space coordinate mapping relation between thermal imaging data and the pollution degree distribution result, performing space registration on the thermal imaging data, calibrating the fluorescence signal intensity of the corresponding region based on the pollution intensity classification of each region in the pollution degree distribution result, performing pixel-level superposition on the thermal imaging data after space registration and the calibrated fluorescence signal intensity data, and obtaining a temperature weighted hand pollution distribution map through temperature field distribution weight calculation; Step 105, marking a region with the temperature higher than 37 ℃ in the temperature weighted hand pollution distribution map as a high temperature region, judging whether the pollution proportion of the high temperature region exceeds a preset proportion threshold value of 0.6, if so, constructing temperature gradient distribution according to the temperature difference value between the temperature value of each pixel point in the high temperature region and the average hand temperature, calculating a temperature correction coefficient through linear interpolation, reducing the pollution intensity weight of the high temperature region through the temperature correction coefficient, carrying out normalization processing on the fluorescence signal intensity, and determining a corrected pollution distribution characteristic map; Step S106, adopting a diffusion coefficient formula according to the corrected pollution distribution characteristic diagram and the pollution degree distribution result Performing temperature-dependent diffusion simulation, processing diffusion simulation results by spatial mapping technology and interpolation, and generating a hand pollution comprehensive distribution map, wherein Indicating that each region corresponds to temperature data, Representing the diffusion coefficient; and S107, carrying out inter-area pollution comparison analysis according to the hand pollution comprehensive distribution diagram, judging the sanitary state of a specific area of the hand, and generating a targeted evaluation report.
2. The method according to claim 1, wherein the step S101 includes: Synchronously acquiring visible light images, microbial fluorescent signals and thermal imaging data of the hand surface through a visible light sensor, a fluorescent imaging device and a thermal imaging device; performing time stamp alignment on the acquired visible light image, the microbial fluorescent signal and the thermal imaging data to generate an original multi-modal data set; And aiming at the original multi-mode data set, performing median filtering processing on the visible light image, performing threshold segmentation processing on the microbial fluorescence signal, and performing Gaussian filtering processing on the thermal imaging data to obtain hand multi-mode data.
3. The method according to claim 2, wherein the step S102 includes: Processing visible light images in the hand multi-mode data through an edge detection algorithm, and identifying hand edge lines to form hand contour features; Performing intensity mapping on the microbial fluorescence signal data in the hand multi-mode data, and quantifying the fluorescence value distribution of each pixel point to obtain fluorescence signal intensity distribution; calculating gradients of the hand contour features in the horizontal and vertical directions to obtain gradient amplitude and direction of each contour point; If the gradient amplitude of any contour point exceeds a preset gradient threshold value, marking the contour point as a boundary turning point, and dividing boundary sub-areas of the contour according to the boundary turning point; Mapping the fluorescence signal intensity distribution to the divided boundary subareas, and generating an initial hand pollution distribution characteristic diagram.
4. The method according to claim 1, wherein the step S103 includes: based on the initial hand pollution distribution characteristic diagram, calculating gradient values of fluorescent signal intensity of each pixel point in the horizontal direction and the vertical direction respectively; calculating the pollution concentration gradient of each pixel point based on the gradient values in the horizontal direction and the vertical direction to form a gradient distribution matrix; dividing the hands into different areas according to the anatomical structure, extracting fluorescence signal intensity data of each area in the gradient distribution matrix, and calculating the mean value and variance of the fluorescence signal intensity in each area to obtain pollution quantification indexes of each area; comparing the pollution quantitative index of each region with a preset pollution intensity grading standard, and dividing the pollution intensity of each region into low, medium and high levels by combining the sanitary risk weights of different regions of the hand; And integrating the pollution intensity dividing results of all the areas to generate a pollution degree distribution result.
5. The method according to claim 1, wherein the step S104 includes: Establishing a space coordinate mapping relation between the thermal imaging data and the pollution degree distribution result, and performing space registration on the thermal imaging data through space transformation to enable the thermal imaging data and a hand region in the pollution degree distribution result to realize space registration; Determining a fluorescent signal intensity calibration coefficient of a corresponding region based on the pollution intensity classification of each region in the pollution degree distribution result, and calibrating the fluorescent signal intensity of the corresponding region by using the calibration coefficient; performing pixel-level superposition processing on the thermal imaging data after spatial registration and the fluorescence signal intensity data after calibration to generate a primary fusion image; extracting temperature field distribution information from thermal imaging data after spatial registration, and converting a temperature value into a corresponding weight coefficient through a weight mapping function; Multiplying the fluorescent signal intensity value of each pixel point in the primary fusion image with the corresponding weight coefficient, and integrating the calculation results of all the pixel points to obtain a temperature weighted hand pollution distribution map.
6. The method according to claim 1, wherein the step S105 includes: identifying a high-temperature region in the temperature weighted hand pollution distribution map, and calculating the coverage proportion of pollution signals in the high-temperature region; Judging whether the pollution coverage proportion exceeds a preset proportion threshold value, if so, extracting the temperature value of each pixel point in a high-temperature area, calculating the temperature difference value with the average temperature of the hands, and constructing temperature gradient distribution based on the temperature difference value; Setting gradient intervals according to the temperature gradient distribution, distributing corresponding correction coefficient reference values for each gradient interval, and calculating the temperature correction coefficient of each pixel point through linear interpolation; reducing the pollution intensity weight of the high-temperature area according to the temperature correction coefficient; extracting fluorescence signal intensity data in the temperature weighted hand pollution distribution map, and carrying out normalization treatment on the fluorescence signal intensity data; and optimizing the pollution distribution presentation by combining the adjusted pollution intensity weight and the normalized fluorescence signal intensity data, and determining a corrected pollution distribution characteristic diagram.
7. The method according to claim 1, wherein the step S106 includes: Extracting initial pollution concentration and corresponding temperature data of each region in the corrected pollution distribution characteristic diagram, and simulating a diffusion path of pollutants along with temperature change by using a finite difference method and a thermodynamic principle; distributing pollution weights according to the pollution degree of each region in the pollution degree distribution result, and adjusting the pollution contribution degree of each region in the diffusion simulation process through weighted average; mapping the diffusion simulation result to a two-dimensional hand image coordinate system by adopting a space mapping technology, and smoothly transiting the pollution concentration gradient through interpolation treatment to form a continuous concentration distribution mapping result; And fusing the diffusion data and concentration distribution mapping result after the regional pollution weight adjustment, and quantizing the pollution concentration level by adopting a color coding mode to generate a hand pollution comprehensive distribution map.
8. The method according to claim 1, wherein the step S107 includes: Extracting pollution concentration quantization indexes of all areas in the hand pollution comprehensive distribution map, and calculating the difference degree of the pollution concentration quantization indexes between adjacent areas; Based on the comparison result of the difference degree and a preset difference threshold, judging the hygienic state of a specific hand area, particularly a hidden area such as a seam and the like, and marking a high-risk pollution area by combining the pollution concentration grade of each area; Summarizing the sanitary state judgment result and the high risk pollution area information, focusing the pollution risk of the hidden area, and quantitatively analyzing pollution distribution characteristics and potential diffusion hidden dangers; and generating a targeted evaluation report according to the quantitative analysis result, and defining the sanitary standard reaching condition, the pollution risk level and the targeted cleaning suggestion of each specific area.
9. A multi-modal image fused hand pollution assessment system for implementing the multi-modal image fused hand pollution assessment method as defined in any one of claims 1 to 8, wherein the multi-modal image fused hand pollution assessment system comprises: The data acquisition module is used for synchronously acquiring visible light images, microbial fluorescent signals and thermal imaging data on the surface of the hand to form an original multi-mode data set, and denoising the original multi-mode data set to obtain hand multi-mode data; the contour segmentation module is used for extracting hand contour features and fluorescence signal intensity distribution from the hand multi-mode data, and finely dividing the hand contour features through edge detection and gradient calculation to obtain an initial hand pollution distribution feature map; the pollution grading module is used for calculating pollution concentration gradients according to the initial hand pollution distribution characteristic diagram, grading and evaluating the pollution intensities of different hand areas by combining the fluorescence signal intensity distribution, and determining pollution degree distribution results; The weighting fusion module is used for establishing a space coordinate mapping relation between the thermal imaging data and the pollution degree distribution result, carrying out space registration on the thermal imaging data, calibrating the fluorescence signal intensity of the corresponding region based on the pollution intensity classification of each region in the pollution degree distribution result, carrying out pixel-level superposition on the thermal imaging data after space registration and the calibrated fluorescence signal intensity data, and obtaining a temperature weighted hand pollution distribution map through temperature field distribution weight calculation; The correction optimization module is used for marking a region with the temperature higher than 37 ℃ in the temperature weighted hand pollution distribution diagram as a high-temperature region, judging whether the pollution proportion of the high-temperature region exceeds a preset proportion threshold value of 0.6, if so, constructing temperature gradient distribution according to the temperature difference value between the temperature value of each pixel point in the high-temperature region and the average hand temperature, calculating a temperature correction coefficient through linear interpolation, reducing the pollution intensity weight of the high-temperature region through the temperature correction coefficient, carrying out normalization processing on the fluorescence signal intensity, and determining a corrected pollution distribution characteristic diagram; the visual presentation module is used for adopting a diffusion coefficient formula according to the corrected pollution distribution characteristic diagram and the pollution degree distribution result Performing temperature-dependent diffusion simulation, processing diffusion simulation results by spatial mapping technology and interpolation, and generating a hand pollution comprehensive distribution map, wherein Indicating that each region corresponds to temperature data, Representing the diffusion coefficient; And the report generation module is used for carrying out inter-area pollution comparison analysis according to the hand pollution comprehensive distribution map, judging the sanitation state of a specific area of the hand and generating a targeted evaluation report.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, causes the processor to perform the hand contamination assessment method of fusing multi-modal images according to any one of claims 1 to 8.

Description

Multi-mode image fusion hand pollution assessment method, system and storage medium Technical Field The application relates to the field of health monitoring, in particular to a hand pollution assessment method, a hand pollution assessment system and a storage medium integrating multi-mode images. Background Hand hygiene assessment is a key link in preventing nosocomial infections and food-borne diseases, especially in high hygiene standard industries such as medical treatment, food processing, and the like. The field mainly adopts a non-contact imaging technology to objectively detect the cleanliness of hands. Current common technical routes include analysis of hand morphology and contaminant residuals using visible light images, or excitation and capture of hand surface microbiological marker signals by fluorescence imaging to achieve quantitative assessment of hand washing effects. The existing evaluation method has three main limitations. First, relying on a single imaging modality results in an insufficient information dimension. For example, although the method based on visible light image only can identify hand outline and macroscopic stains, it can not detect microorganism contamination, while the method relying solely on fluorescence imaging is easily interfered by background fluorescence, hand fold shadow and uneven application of fluorescent agent, and it is difficult to distinguish living microorganisms from nonspecific fluorescent substances. Secondly, the method ignores the combined impact of key physiological and environmental parameters. The hand surface temperature distribution has significant variability, and high temperature areas (such as palm and sulcus) may enhance microbial activity and promote pollution migration, but the existing methods do not incorporate thermal imaging data into the analysis system, and cannot correct the potential influence of temperature on fluorescence signal intensity and pollution diffusion behavior, so that the evaluation result may deviate from real risk. Finally, the assessment result is static and lacks dynamic risk early warning capability. Most methods only output the pollution distribution state at the current moment, and the non-combined temperature field simulates the potential diffusion trend of pollution in hidden areas such as finger joints, wrists and the like, so that prospective guidance cannot be provided for continuous sanitary monitoring and preventive cleaning. The defects limit the accuracy, the robustness and the practical application value of the prior art in complex real scenes. Aiming at the defects, the method and the device solve the problems of single-mode evaluation, neglect of temperature interference and lack of dynamic early warning by synchronously acquiring and fusing the visible light, fluorescence and thermal imaging multi-mode data and combining a temperature weighting correction mechanism and temperature dependent diffusion dynamic simulation, and improve the comprehensiveness, accuracy and risk prediction capability of hand pollution evaluation. Disclosure of Invention The application provides a hand pollution assessment method, a hand pollution assessment system and a storage medium which are integrated with a multi-mode image, solves the problems of single-mode assessment, neglecting temperature interference and lacking dynamic early warning, and improves the comprehensiveness, accuracy and risk prediction capability of hand pollution assessment. In a first aspect, the present application provides a method for assessing hand contamination by fusing multi-modal images, the method comprising: S101, synchronously acquiring visible light images, microbial fluorescent signals and thermal imaging data on the surface of a hand to form an original multi-modal data set, and denoising the original multi-modal data set to obtain hand multi-modal data; step S102, extracting hand contour features and fluorescence signal intensity distribution from the hand multi-mode data, and finely dividing the hand contour features to obtain an initial hand pollution distribution feature map; Step S103, calculating pollution concentration gradients according to the initial hand pollution distribution feature diagram, and carrying out grading evaluation on pollution intensities of different hand areas by combining the fluorescence signal intensity distribution to determine pollution degree distribution results; step S104, performing thermal imaging data space registration and fluorescence signal intensity calibration according to the pollution degree distribution result, then performing superposition processing, and obtaining a temperature weighted hand pollution distribution map through temperature field distribution weight calculation; step 105, judging whether the pollution proportion of the high-temperature area in the temperature weighted hand pollution distribution diagram exceeds a preset proportion threshold, if so, reducing the pollution