CN-122004819-A - Heart rate fusion method and device and electronic equipment

CN122004819ACN 122004819 ACN122004819 ACN 122004819ACN-122004819-A

Abstract

The application is suitable for the field of remote physiological parameter monitoring and provides a heart rate fusion method, a heart rate fusion device and electronic equipment, which comprise the steps of adopting at least two interest region templates to detect interest regions of a video stream comprising the face of a user to be detected, and obtaining corresponding interest regions; determining heart rates corresponding to the regions of interest; the method comprises the steps of determining signal quality scores of all the interested areas based on quality indexes, determining weight values of the corresponding interested areas according to the signal quality scores of all the interested areas, and determining heart rates of users to be detected according to the weight values and heart rates corresponding to all the interested areas. By the method, more accurate weight values can be determined according to the signal quality scores of all the regions of interest, so that the influence of all the regions of interest on the heart rate of the user to be detected can be accurately adjusted.

Inventors

YAN CHENG
WANG LE
ZHANG WEI

Assignees

中广核核电运营有限公司

Dates

Publication Date: 20260512
Application Date: 20260205

Claims (15)

1. A method of fusing heart rate, comprising: acquiring a video stream comprising the face of a user to be detected; Detecting the region of interest of the video stream by adopting at least two region of interest templates to obtain corresponding regions of interest; Determining heart rates corresponding to the regions of interest; Determining a signal quality score of each region of interest based on a quality index, wherein the quality index comprises at least one of signal-to-noise ratio, comprehensive peak significance, time domain stability, motion artifact, illumination quality, spectral entropy and signal periodicity index; Determining a weight value of the corresponding region of interest according to the signal quality scores of the regions of interest; and determining the heart rate of the user to be detected according to the weight value and the heart rate corresponding to each region of interest.
2. The method of claim 1, wherein the quality indicator comprises the signal-to-noise ratio, and wherein determining the signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: determining peak frequency in a heart rate frequency band according to the time-frequency analysis result of the region of interest; calculating the signal energy of the peak frequency; calculating noise energy within the heart rate frequency; And determining the signal-to-noise ratio of the region of interest according to the signal energy and the noise energy, and determining the signal quality score of the region of interest according to the signal-to-noise ratio.
3. The method of claim 1, wherein the quality indicator comprises the signal-to-noise ratio, and wherein determining the signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: Detecting all heart beat peaks of the independent component signals related to heart rate in the region of interest; calculating signal energy according to all the heartbeat peaks; determining a non-heartbeat signal area according to the heartbeat peak value, and calculating noise energy of the non-heartbeat signal area; A signal quality score for the region of interest is determined from the signal energy and the noise energy.
4. The method of claim 1, wherein the quality indicator comprises the integrated peak significance, and wherein determining the signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: Determining peak frequency in a heart rate frequency band according to the time-frequency analysis result of the region of interest; Calculating a power spectrum intensity value and valley values on the left side and the right side of the peak frequency according to the peak frequency; Calculating peak significance according to the power spectrum intensity value and the valley values at two sides of the peak frequency; calculating the width ratio of the full width at half maximum in the heart rate frequency band according to the power spectrum intensity value; determining the integrated peak significance from the peak significance and the width ratio; and determining a signal quality score of the region of interest according to the integrated peak significance.
5. The method of fusion of heart rate of claim 1, wherein the quality indicator comprises the temporal stability, and wherein determining a signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: determining a change rate of the heart rate sequence according to the heart rate sequence of the region of interest, wherein the heart rate sequence of the region of interest comprises heart rates determined by the region of interest at a plurality of continuous moments; determining the time domain stability of the change rate of the heart rate sequence according to the change rate of the heart rate sequence and a preset stability threshold; and determining a signal quality score of the region of interest according to the time domain stability of the rate of change of the heart rate sequence.
6. The method of claim 1, wherein the quality indicator comprises the motion artifact, and wherein determining a signal quality score for each of the regions of interest based on the quality indicator comprises: performing motion detection on the video stream, and estimating global motion parameters between frames of the video stream; determining corresponding motion artifacts according to the global motion parameters between frames; and determining a signal quality score of each region of interest of the corresponding image frame according to the motion artifact.
7. The fusion method of heart rates of claim 1, wherein the quality indicator comprises the illumination quality, and wherein determining a signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: Calculating the average brightness of the region of interest and/or detecting whether shadows exist in the region of interest; And determining a signal quality score of the corresponding region of interest according to the average brightness and/or whether the region of interest has shadows.
8. The method of claim 1, wherein the quality indicator comprises the spectral entropy, and wherein determining the signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: performing spectrum analysis on independent component signals related to heart rate of the region of interest to obtain power spectrum density; normalizing the power spectrum density to obtain a normalized power spectrum; Calculating the spectrum entropy according to the normalized power spectrum; and determining a signal quality score of the region of interest according to the spectrum entropy.
9. The method of claim 1, wherein the quality indicator comprises the signal periodicity indicator, and wherein the determining a signal quality score for each of the regions of interest based on the quality indicator comprises: For any of the regions of interest: performing autocorrelation analysis on independent component signals related to heart rate of the region of interest to obtain an autocorrelation function; Detecting a periodic peak value corresponding to a heartbeat period in the autocorrelation function; calculating the signal periodicity index according to the peak height and/or the peak regularity of the periodicity peak; And determining a signal quality score of the region of interest according to the signal periodicity index.
10. The fusion method of heart rates of claim 1, wherein the determining the heart rate for each of the regions of interest comprises: for each of the regions of interest: determining a time sequence signal corresponding to the region of interest, and performing blind source separation on the time sequence signal by adopting an independent component algorithm to obtain an independent component signal related to the heart rate; Performing time-frequency analysis on the independent component signals related to the heart rate by adopting at least two time-frequency analysis methods to obtain corresponding time-frequency analysis results, wherein the principle of each time-frequency analysis method is different when performing time-frequency analysis; And calculating the heart rate according to each time-frequency analysis result to obtain the heart rate corresponding to the region of interest.
11. The method of claim 10, wherein the quality index further comprises a composite consistency of heart rates determined by the respective time-frequency analysis method, the determining a signal quality score for the respective region of interest based on the quality index comprising: For any of the regions of interest: Calculating heart rate differences of heart rates determined by the time-frequency analysis methods; determining the comprehensive consistency of the heart rate determined by each time-frequency analysis method according to the heart rate difference and a preset first change rate threshold; and determining a signal quality score of the region of interest according to the comprehensive consistency.
12. A heart rate fusion device, comprising: The video stream acquisition module is used for acquiring a video stream comprising the face of the user to be detected; the interest region detection module is used for detecting the interest region of the video stream by adopting at least two interest region templates to obtain a corresponding interest region; The heart rate calculation module of the interested areas is used for determining heart rates corresponding to the interested areas; a signal quality score determining module, configured to determine a signal quality score of each region of interest based on quality indicators, where the quality indicators include at least one of signal-to-noise ratio, integrated peak significance, temporal stability, motion artifacts, illumination quality, spectral entropy, and signal periodicity indicators; The weight value determining module of the region of interest is used for determining the weight value of the corresponding region of interest according to the signal quality scores of the respective regions of interest; And the heart rate determining module of the user to be detected is used for determining the heart rate of the user to be detected according to the weight value and the heart rate corresponding to each region of interest.
13. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 11 when executing the computer program.
14. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method according to any one of claims 1 to 11.
15. A computer program product comprising a computer program which, when run, causes an electronic device to perform the method of any one of claims 1 to 11.

Description

Heart rate fusion method and device and electronic equipment Technical Field The application belongs to the technical field of heart rate monitoring, and particularly relates to a heart rate fusion method, device electronic equipment, a computer readable storage medium and a computer program product. Background The independent component analysis (INDEPENDENT COMPONENT ANALYSIS, ICA) algorithm is mainly used to separate a clean heart rate signal from a blind source in a mixed physiological signal. When heart rate detection is performed according to the ICA algorithm, the process is generally as follows, a region of interest is determined from a sequence of continuous video images containing the face of the subject, and time-varying Red (Red, R), green (Green, G), blue (Blue, B) timing signals are determined from the region of interest, where the RGB timing signals contain weak light intensity variation information caused by blood flow variation. And performing blind source separation on the RGB time sequence signals by using an ICA algorithm to obtain independent component signals related to heart rate, analyzing the independent component signals, and determining corresponding heart rate (or heart rate value) according to analysis results. But the heart rate determined by the above method is less accurate. Disclosure of Invention The embodiment of the application provides a heart rate fusion method, a heart rate fusion device and electronic equipment, which can solve the problem that the accuracy of the heart rate of a user to be detected calculated by the existing method is low. In a first aspect, an embodiment of the present application provides a method for fusing heart rates, including: acquiring a video stream comprising the face of a user to be detected; Detecting the region of interest of the video stream by adopting at least two region of interest templates to obtain corresponding regions of interest; Determining heart rates corresponding to the regions of interest; Determining a signal quality score of each region of interest based on a quality index, wherein the quality index comprises at least one of signal-to-noise ratio, comprehensive peak significance, time domain stability, motion artifact, illumination quality, spectral entropy and signal periodicity index; Determining a weight value of the corresponding region of interest according to the signal quality scores of the regions of interest; and determining the heart rate of the user to be detected according to the weight value and the heart rate corresponding to each region of interest. Compared with the prior art, the embodiment of the application has the beneficial effects that: In the embodiment of the application, because local movement or uneven illumination of a single region of interest is difficult to influence all the regions of interest simultaneously, different regions of interest can provide spatially diversified physiological signal samples, and the fusion process can automatically inhibit local abnormal disturbance, when the heart rate of a user to be detected is obtained by fusion according to the heart rates of different regions of interest of the face of the user to be detected, the heart rate estimation of the user to be detected can be more resistant to local disturbance, and the accuracy is systematically improved. Meanwhile, the quality indexes comprise at least one of signal-to-noise ratio, comprehensive peak significance, time domain stability, motion artifact, illumination quality, spectrum entropy and signal periodicity indexes, and the quality indexes reflect the accuracy of the heart rate of the calculated region of interest, so that signal quality scores are determined according to the quality indexes, and then weight values of the region of interest are determined according to the signal quality scores, thereby being beneficial to accurately adjusting the influence degree of the heart rates of different regions of interest on the heart rate of the finally fused user to be tested, and further being beneficial to improving the accuracy of the heart rate of the obtained user to be tested. In a second aspect, an embodiment of the present application provides a heart rate fusion device, including: The video stream acquisition module is used for acquiring a video stream comprising the face of the user to be detected; the interest region detection module is used for detecting the interest region of the video stream by adopting at least two interest region templates to obtain a corresponding interest region; The heart rate calculation module of the interested areas is used for determining heart rates corresponding to the interested areas; a signal quality score determining module, configured to determine a signal quality score of each region of interest based on quality indicators, where the quality indicators include at least one of signal-to-noise ratio, integrated peak significance, temporal stability, motion artifacts,