CN-122004752-A - Sleep scene-oriented bimodal fusion radar dynamic gesture recognition method and device

CN122004752ACN 122004752 ACN122004752 ACN 122004752ACN-122004752-A

Abstract

The invention discloses a method and a device for identifying dynamic postures of a bimodal fusion radar facing a sleep scene, wherein the method comprises the following steps of S1, collecting and demodulating radar original data; S2, constructing dual window features of a speed mode and a structural mode, S3, carrying out dual-mode time alignment and feature re-weighting on the speed guide, and S4, carrying out dynamic sleeping gesture classification output. The invention breaks through the limitation of the existing single-mode radar processing method in the aspects of weak action visibility, bedding shielding robustness and directivity discrimination capability, and provides a new implementation mode for full-flow dynamic gesture monitoring of complex sleep behaviors.

Inventors

SHI JIAJIA
HUANG YITING
SHI QUAN
CHU LIU
XU ZHIHUO
CHEN FENG
SHEN TONG

Assignees

南通大学

Dates

Publication Date: 20260512
Application Date: 20251219

Claims (10)

1. A method for identifying dynamic postures of a bimodal fusion radar facing a sleep scene is characterized by comprising the following steps: s1, acquiring and demodulating radar original data; Demodulating and de-interleaving the I/Q echo signals in the sleep process by utilizing a millimeter wave radar in a fixed frame period to obtain a multichannel original data sequence arranged according to frames; s2, constructing double window features of a speed mode and a structural mode, and respectively constructing a short-time window speed mode and a long-time window structural mode; S3, carrying out dual-mode time alignment and characteristic re-weighting on speed guidance, determining a plurality of time anchor points based on the positions of speed peaks and energy mutation in a short-time window speed mode, and positioning a structure fragment sequence corresponding to each time anchor point on a unified time axis; Calculating the re-weighting coefficient of each structural segment according to the speed variation amplitude, the energy of the structural segment and the time distance between the speed variation amplitude and the energy of the structural segment, selectively enhancing or inhibiting the structural segment to obtain time-aligned structural enhancement features, and carrying out residual fusion on the structural enhancement features and the speed modal features to obtain bimodal enhancement features for classifying subsequent sleeping positions; In the fusion process, a speed peak value detected in a speed mode is used as a time anchor point, and the time position of a structural fragment sequence is aligned and weighted; s4, classifying and outputting the dynamic sleeping posture, inputting the bimodal enhanced features into a classification network, and outputting the probability of the dynamic sleeping posture state to the continuous frames in a sliding window mode to realize continuous posture change recognition overnight.
2. The method of claim 1, wherein in S1, the millimeter wave radar is disposed at a distance of 0.2-3 meters from the bed side to cover a dynamic transition region upon posture switching.
3. The method according to claim 1, wherein S2 comprises the specific steps of: S21, constructing a short-time window speed mode: Constructing a weighting factor for the speed signal of the neighborhood of the main reflection distance unit according to the combination of the amplitude mean value, the energy trend factor and the distance Gaussian weight, and carrying out weighted fusion on the neighborhood speed signal; Sequentially executing direct current removal, trend suppression and robust filtering on the fusion result, and generating speed modal characteristics by adopting short-time Fourier transform with short time window and high overlapping rate, wherein the short time window is used for capturing micro-motion speed change with the duration of tens of milliseconds; s22, constructing a long-time window structural mode: Flattening radar amplitude vectors of continuous frames to form a structure vector sequence, slicing the structure vector according to a long-time window covering 50-80 frames according to the statistical characteristic of sleep turn-over action duration time of 1.8-2.5 seconds, and sliding adjacent slices at an overlapping rate of 80% -90%; and inputting each structural slice into a multi-layer perceptron for coding to obtain a structural segment mark with uniform dimension, and using a long-time window for representing a slow change stage of the gesture.
4. The method of claim 3, wherein the weighting factors in S21 include at least two combinations of an amplitude mean, an energy trend factor, and a distance gaussian weight.
5. The method of claim 4, wherein the length of the structural slice in S22 is set to cover 50-80 frames according to the duration of the sleep turn-over action to completely cover a turn-over period lasting about 1.8-2.5 seconds, such that each sequence of structural segments can represent a stable posture period.
6. The method of claim 5, wherein the overlapping rate of adjacent structural slices is set to 80% -90% for maintaining structural mode continuity in the time dimension and reducing gesture phase jumps.
7. The method of claim 6 wherein step S3 employs a speed information guided bimodal time alignment and feature re-weighting mechanism to implement matching and semantic consistency of correspondence between speed modes and structural modes in multiple subspaces by aligning and weighting structural features on different time slices with speed variation peaks as time anchor points.
8. The method of claim 7, wherein the sliding window in step S4 is used to output pose predictions for consecutive frames and probability smoothing between different windows to improve stability of overnight monitoring.
9. A sleep scene-oriented bimodal fusion radar dynamic gesture recognition device is characterized in that the device is used for executing the steps in the sleep scene-oriented bimodal fusion radar dynamic gesture recognition method according to any one of claims 1-8.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the steps in the sleep scene oriented bimodal fusion radar dynamic gesture recognition method according to any one of claims 1 to 8.

Description

Sleep scene-oriented bimodal fusion radar dynamic gesture recognition method and device Technical Field The invention belongs to the technical field of human body gesture recognition and sleep monitoring, and particularly relates to a method and a device for recognizing a dynamic gesture of a bimodal fusion radar facing a sleep scene. Background The existing sleep monitoring technology depends on wearable equipment, a camera or a pressure sensing array, but the scheme has non-negligible technical limitations. Although the millimeter wave radar has the advantages of non-contact and penetrability of bedding, the conventional data processing method based on the single-mode radar signal can not solve the key pain point under the complex sleeping behavior. Typical problems include: (1) Bedding shielding causes distortion of the reflecting structure, and micro Doppler energy is further attenuated; (2) The sleeping posture change usually has slow continuous multi-stage evolution, and the traditional model based on single frame characteristics or single sequence cannot completely describe the directivity change; (3) Weak motions (e.g., slow turn over, fine tuning gestures) are easily ignored due to weak energy, resulting in identifying faults; (4) The existing single-mode method cannot process speed change, structure change and shielding compensation at the same time, and the accuracy rate in the turn-over direction judgment and shielding scene is obviously reduced. Therefore, how to realize high-speed dynamic change (micro-motion) and low-speed structural change (turning-over structure) in a sleeping scene can be seen at the same time, and the direction discrimination stability is kept under the shielding condition, which is a key pain point in the prior art. The existing method based on single-mode micro Doppler cannot judge the turning direction, the existing method based on single-sequence structural vectors fails in weak action or shielding scenes, and the existing multi-mode fusion does not involve time scale mismatch, so that attention weight dislocation is caused. Disclosure of Invention Aiming at the problems, the invention provides a method and a device for recognizing the dynamic gesture of a bimodal fusion radar facing a sleep scene. The invention constructs a collaborative fusion mechanism consisting of a speed mode (micro Doppler characteristic) and a structural mode (distance structural characteristic), so that the system can acquire speed change and structural change information simultaneously in real sleeping behaviors such as bedding shielding, weak motion energy, slow posture change, large turning and the like, and continuous, stable and accurate identification of posture direction change is realized. The invention breaks through the limitation of the existing single-mode radar processing method in the aspects of weak action visibility, bedding shielding robustness and directivity discrimination capability, and provides a new technical approach and implementation mode for full-flow dynamic gesture monitoring of complex sleep behaviors. In order to solve at least one of the above technical problems, according to an aspect of the present invention, there is provided a method for identifying dynamic gestures of a bimodal fusion radar for a sleep scene, including the steps of: s1, radar original data acquisition and demodulation: the method comprises the steps of continuously acquiring I/Q echo signals in a sleep process by utilizing a millimeter wave radar in a fixed frame period, demodulating and de-interleaving the signals to obtain a multi-channel original data sequence arranged according to frames, and determining a human body main reflection distance unit and a neighborhood thereof based on distance maintenance energy distribution of a first frame or a plurality of frames. S2, constructing double window features of a speed mode and a structural mode: s21, constructing a short window of a speed mode: Constructing a weighting factor for the speed signal of the neighborhood of the main reflection distance unit according to the combination of the amplitude mean value, the energy trend factor and the distance Gaussian weight, and carrying out weighted fusion on the neighborhood speed signal; and sequentially executing direct current removal, trend suppression and robust filtering on the fusion result, and generating speed modal characteristics by adopting short-time Fourier transform with short time window and high overlapping rate, wherein the short time window is used for capturing micro-motion speed change with the duration of tens of milliseconds. S22, constructing a long window of a structural mode: Flattening radar amplitude vectors of continuous frames to form a structure vector sequence, slicing the structure vector according to a long-time window covering 50-80 frames according to the statistical characteristic of sleep turn-over action duration time of 1.8-2.5 seconds, and sliding adjacent