Search

CN-121987204-A - Depression risk assessment method based on multi-mode gait feature fusion

CN121987204ACN 121987204 ACN121987204 ACN 121987204ACN-121987204-A

Abstract

The invention discloses a depression risk assessment method based on multi-modal gait feature fusion, which comprises the steps of collecting multi-modal gait data of a subject, recording a plurality of complete gait cycles, extracting gait event anchor points, constructing a unified gait phase sequence, completing multi-modal alignment, improving XFeat models, extracting local features, constructing a cross-frame connecting chain, forming a local response embryo body sequence, applying forward and reverse rhythm perturbation, extracting phase deviation correction, starting delay, overshoot, damping and recovery steps, constructing a bidirectional debt transfer closed loop, forming a bidirectional debt loop closed tensor, constructing a gait debt hysteresis loop, extracting area, drift and residual and outputting risk grades. The invention realizes high-sensitivity identification and risk assessment of depression-related psychomotor hysteresis characteristics through subthreshold rhythm perturbation and multi-mode gait characteristic fusion modeling.

Inventors

  • ZHENG HUA

Assignees

  • 佳木斯大学

Dates

Publication Date
20260508
Application Date
20260327

Claims (8)

  1. 1. A depression risk assessment method based on multi-modal gait feature fusion, comprising: Acquiring multi-mode gait data of a subject in a continuous natural walking process, uniformly marking the time stamp of each mode data, and recording a plurality of complete gait cycles; Extracting gait event anchor points from multi-mode gait data, and carrying out phase division and alignment processing on each mode data based on the gait event anchor points to construct a unified gait phase sequence; Inputting a gait video sequence in the multi-mode gait data into an improved XFeat model, extracting local key points, constructing a cross-frame characteristic connecting chain, carrying out phase mapping based on a unified gait phase sequence, constructing a gait local response embryo body sequence, and forming a multi-mode local response characterization set; Applying rhythm perturbation at a unified gait phase position corresponding to a gait event anchor point, extracting phase deviation rectifying starting delay quantity, phase overshoot quantity, backswing damping quantity and recovery closing step number based on a multimode local response representation set and a unified gait phase sequence, and generating a phase plasticity liability sequence; Based on the phase plasticity debt sequence and the unified gait phase sequence, a bidirectional debt transfer closed loop is constructed, and debt transfer residual quantity, a phase reversal point, a phase mismatch section and compensation offset are calculated under the corresponding gait phase to form a bidirectional debt circulation closed tensor; Under the conditions of forward perturbation and reverse perturbation, acquiring a phase plasticity liability sequence and an evolution track of a bidirectional liability circulation closed tensor based on a unified gait phase sequence, constructing a gait liability hysteresis loop, extracting a loop area value, a center drift value and a closed residual value, fusing to generate a depression risk score and outputting a risk grade.
  2. 2. The method for assessing risk of depression based on multi-modal gait feature fusion of claim 1, wherein the multi-modal gait data comprises a gait video sequence, an inertial sensing sequence and a plantar pressure distribution sequence.
  3. 3. The method for assessing risk of depression based on multi-modal gait feature fusion of claim 1, wherein the constructing a unified gait phase sequence comprises: Reading continuous time sampling data from the gait video sequence, the inertia sensing sequence and the plantar pressure distribution sequence, and aligning all mode data according to a unified time sequence to form a multi-mode synchronous time sequence; Identifying a heel strike event, a full plantar contact event, a heel lift event, and a toe lift event based on a pressure change process in the plantar pressure distribution sequence; Identifying the beginning and the end of the double support phase based on the acceleration change trend and the angular velocity change trend in the inertial sensing sequence; Dividing a continuous gait process into a plurality of complete gait cycles according to a heel strike event, a full sole contact event, a heel strike event, a toe end strike event and double support start and end events, dividing a support stage and a swing stage according to an event occurrence sequence in each gait cycle, and determining the position range of each stage in a time axis; and mapping the time course in the gait cycle into a unified phase course by taking the initial event of each gait cycle as a reference, enabling each sampling moment to correspond to one phase position, unifying the phase courses of the asynchronous state cycles into the same scale range, and constructing a unified gait phase sequence.
  4. 4. A method of assessing risk of depression based on multimodal gait feature fusion as claimed in claim 3, wherein the identifying a heel strike event, a plantar full contact event, a heel lift event and a toe lift event comprises: Wherein heel strike event is determined when plantar pressure transitions from a non-contact state to a contact state, plantar full contact event is determined when plantar pressure reaches a steady contact state, heel lift event is determined when plantar pressure transitions from a contact state to a reduced state, and toe lift event is determined when plantar pressure approaches a non-contact state.
  5. 5. The depression risk assessment method based on multi-modal gait feature fusion according to claim 1, wherein the performing phase mapping based on the unified gait phase sequence, constructing a gait local response embryo body sequence, forming a multi-modal local response characterization set, comprises: Inputting a gait video sequence into an improved XFeat-P model frame by frame, wherein the improved XFeat-P model is formed by sequentially connecting an input normalization layer, a shallow texture extraction layer, a multi-level feature pyramid trunk, a phase gating enhancement layer, a double-branch descriptor generation layer and a cross-frame consistency correction layer, the phase gating enhancement layer is arranged between a third feature layer and a fourth feature layer of the multi-level feature pyramid trunk, and the double-branch descriptor generation layer and the cross-frame consistency correction layer are sequentially connected to the output end of the multi-level feature pyramid trunk; performing size unification, brightness equalization and human body region interception processing on each frame of image by an input normalization layer, extracting edge texture and local angular point information by a shallow texture extraction layer, and extracting low-layer contour features, middle-layer joint neighborhood features and high-layer trunk posture features by a multi-level feature pyramid trunk step by step; The unified gait phase sequence is input into a phase gating enhancement layer, weight enhancement is respectively carried out on a lower limb swing area, a pelvis linkage area and a trunk stabilization area according to the current phase position, a phase enhancement feature map is output, the phase enhancement feature map is processed in parallel by a double-branch descriptor generation layer, wherein micro-branches extract lower limb local displacement detail features, macro-branches extract pelvis and trunk integral linkage features, and two branches are output and spliced to form a local key point descriptor; Establishing initial matching pairs between adjacent frames based on local key point position adjacent relation and descriptor consistent relation, connecting the initial matching pairs which are positioned in the same area and keep consistent motion direction in continuous frames end to end according to time sequence to form candidate frame-crossing characteristic connecting chains, and executing breakpoint repairing, short chain eliminating and jump cutting processing on each candidate frame-crossing characteristic connecting chain by a frame-crossing consistency correction layer to obtain frame-crossing characteristic connecting chains; Mapping the frame-crossing characteristic connecting chains to a unified gait phase sequence, and segmenting each frame-crossing characteristic connecting chain according to a heel strike phase, a support transition phase, a heel lift phase and a swing recovery phase to obtain a local response segment corresponding to the asynchronous phase; The method comprises the steps of connecting local response fragments adjacently arranged according to a phase sequence in series in the same area, sequentially recording the displacement change state, the phase stagnation state and the cross-phase transmission state of each local response fragment, merging the adjacent local response fragments into the same gait local response embryo when the adjacent local response fragments are consistent in the area attribution, the movement direction and the phase propulsion sequence, and arranging a plurality of gait local response embryo continuously formed in the same area according to the phase sequence to construct a gait local response embryo sequence; And synchronously associating the gait local response embryo body sequences with the inertia sensing sequences and the plantar pressure distribution sequences according to the phase indexes, so that each gait local response embryo body corresponds to an acceleration change segment, an angular velocity change segment and a pressure center migration segment in the same phase interval to form a multi-mode local response representation set.
  6. 6. A method of assessing risk of depression based on multimodal gait feature fusion as claimed in claim 1, wherein the generating a phase plasticity liability sequence comprises: Based on the unified gait phase sequence, selecting a preset phase interval as a rhythm perturbation triggering interval, generating a forward perturbation signal and a reverse perturbation signal, and applying the perturbation signal in a continuous preset step number range; After perturbation is applied, tracking the phase change process of each gait local response embryo body according to a unified gait phase sequence based on a multi-mode local response characterization set, and determining the phase position corresponding to the perturbation triggering moment and the phase position of each local response embryo body with obvious displacement change for the first time; Extracting a phase deviation correcting start delay amount according to the phase difference between the phase position at the perturbation triggering moment and the phase position at which the obvious displacement change appears for the first time; in the gait response process, determining the maximum deviation degree exceeding the target phase position based on the displacement variation amplitude of each local response embryo in the unified gait phase sequence, taking the maximum deviation degree as the phase overshoot, tracking the variation condition that the deviation process gradually decreases along with the phase propulsion, and determining the backswing damping quantity according to the attenuation process from large to small of the displacement variation amplitude; After the perturbation is finished, continuously tracking the phase change state of each local response embryo body, and when the phase change of the local response embryo body is restored to be consistent with the phase change range corresponding to the steady state gait before the perturbation, counting the number of gait cycles which are undergone from the perturbation triggering to the restoration as the number of restoration closing steps, and combining the phase deviation correcting starting delay amount, the phase overshoot amount, the backswing damping amount and the number of restoration closing steps according to the phase sequence to generate a phase plasticity liability sequence.
  7. 7. The method for assessing risk of depression based on multi-modal gait feature fusion according to claim 1, wherein the constructing a bidirectional liability transfer closed loop, calculating liability transfer residuals, phase reversal points, phase mismatch segments and compensation offsets in corresponding gait phases, forming a bidirectional liability circulation closed tensor, comprises: Traversing the gait local response embryo body sequence based on the phase plasticity debt sequence and the unified gait phase sequence, and connecting gait local response embryo bodies which have continuous transmission relation in adjacent phase sections and are adjacent in space areas to form a candidate debt transmission chain; closing detection is carried out on the candidate liability transfer chains according to the space region and the phase propulsion sequence, when the same candidate liability transfer chain returns to the initial region again in the phase propulsion process and maintains the continuous transfer relationship, the one-way liability transfer closed loop is determined, and the initial phase position and the return phase position of the closed loop are recorded; repeatedly executing candidate liability transfer chain construction and closure detection processes under the forward perturbation condition and the reverse perturbation condition respectively to obtain two groups of unidirectional liability transfer closed loops, and correspondingly matching the two groups of closed loops based on spatial region attribution, initial phase position and phase propulsion path of the loops to form a bidirectional corresponding closed loop pair; Carrying out sectional analysis on each group of closed loop pairs according to the debt transfer process of the unified gait phase sequence in each phase section, extracting the debt change state in each phase section, wherein the debt change state comprises a debt accumulation trend, a transfer direction change position in phase propulsion, a transfer interruption section in a continuous phase section and a compensation offset condition generated under a reverse perturbation condition, and forming a loop-level bidirectional debt characteristic sequence; Rearranging the loop-level bidirectional liability feature sequences according to the sequence of the phase sections, and establishing a continuous connection relationship between adjacent phase sections to obtain a bidirectional liability circulation sequence; And carrying out multidimensional organization on the bidirectional liability loop sequence according to the closed loop, the phase section, the perturbation direction, the modal source and the gait cycle sequence to form a bidirectional liability loop closed tensor.
  8. 8. The method for assessing risk of depression based on multi-modal gait feature fusion according to claim 1, wherein the steps of constructing a gait debt hysteresis loop, extracting a loop area value, a center drift value and a closed residual value, fusing to generate a depression risk score and outputting a risk level include: Under the forward perturbation condition and the reverse perturbation condition, based on the unified gait phase sequence, synchronously traversing the phase plasticity debt sequence and the bidirectional debt loop closure tensor, extracting a start delay state, an overshoot state, a damping state, a recovery state and a loop transfer state corresponding to each phase section, and forming a forward debt state sequence and a reverse debt state sequence; The forward debt state sequence and the reverse debt state sequence are matched in phase according to the unified gait phase sequence, each phase section corresponds to one group of forward debt state and one group of reverse debt state, and state components from different modes in each group of debt states are uniformly ordered and marked in a leading state to form a phase pair debt point column; The method comprises the steps of connecting a phase pair liability point column according to a phase propelling sequence to form a forward liability trip Cheng Zhilu, connecting reverse liability states in the phase pair liability point column according to a phase reverse sequence to form a reverse liability return branch, and constructing a gait liability hysteresis loop by taking corresponding liability states of an initial phase section and a termination phase section as head-to-tail closing points; Performing span continuity correction on adjacent phase sections in the gait debt hysteresis loop, inserting a phase locking compensation section into the corresponding phase section when state fracture, dominant mode switching disorder or closing point offset exists between the adjacent phase sections of the forward debt Cheng Zhilu and the reverse debt return branch, and updating the gait debt hysteresis loop based on the compensated phase connection relationship; Based on the updated gait liability hysteresis loop, extracting a loop area value corresponding to a region surrounded by a forward liability trip Cheng Zhilu and a reverse liability return branch, a center drift value between a forward liability trip Cheng Zhilu and an integral distribution center of the reverse liability return branch and a closing residual value between head and tail closing points; and (3) carrying out alignment fusion on the loop area value, the center drift value and the closed residual value, the phase plasticity debt sequence and the bidirectional debt circulation closed tensor to generate a depression risk score, and outputting a corresponding risk grade according to a preset risk grading rule.

Description

Depression risk assessment method based on multi-mode gait feature fusion Technical Field The invention relates to the technical field of computer vision, in particular to a depression risk assessment method based on multi-mode gait feature fusion. Background With the development of computer vision technology, wearable sensing technology and artificial intelligence technology, human behavior recognition and health state assessment methods based on gait analysis are gradually applied. In the field of depression risk assessment, researchers usually extract characteristics such as gait speed, stride, rhythm stability and the like by acquiring gait videos, inertial sensing data or plantar pressure data, and realize recognition of depression state by combining machine learning or deep learning models. The method has the characteristics of non-invasiveness and strong continuous monitoring capability, and becomes an important supplementary means for evaluating the traditional scale. However, most of the prior art is based on a passive acquisition mode of natural gait, only the apparent characteristics in the steady walking process are analyzed, and the description of the dynamic adjustment capability of the gait under the external disturbance or state switching condition is lacking, so that the common psychomotor hysteresis characteristics in depressed people are difficult to effectively reflect. Most methods adopt a single-mode or simple multi-mode characteristic splicing mode, lack of system modeling on the internal association relationship among video, inertial sensing and plantar pressure data, and cannot accurately describe the transmission and coupling processes between different body parts and different modes, so that the sensitivity and stability of an evaluation result are limited. In the existing method, in the gait abnormality judging process, the unidirectional characteristic change or the static classification result is mostly used as the basis, the analysis of the direction difference of the gait adjusting process is lacking, and particularly, the asymmetric response characteristic which is shown under the condition of rhythm enhancement and rhythm weakening is difficult to identify, and the short-time fluctuation and the continuous abnormal state are difficult to distinguish. In early, mild or implicit depression risk assessment scenarios, the prior art still has the problems of insufficient recognition accuracy, insufficient feature expression and insufficient modeling capability of dynamic mechanisms, and needs to be further improved. Therefore, how to provide a depression risk assessment method based on multi-modal gait feature fusion is a problem that needs to be solved by those skilled in the art. Disclosure of Invention According to the depression risk assessment method based on multi-modal gait feature fusion, a subthreshold rhythm perturbation mechanism is introduced, potential adjustment anomalies are actively excited in a natural gait process, local key features in a gait video are extracted and cross-frame modeling is conducted by combining an improved XFeat model, inertial sensing data and plantar pressure data are fused, a gait local response embryo, a multi-modal debt loop closure tensor and a gait debt hysteresis loop are constructed, dynamic responses, cross-modal transfer relations and forward and reverse direction adjustment differences of the gait under different phases are systematically described, fine recognition and quantitative assessment of mental movement hysteresis features are achieved, and the method has the advantages of being high in recognition sensitivity, strong in response capability to weak abnormal change, good in feature expression integrity and stable and reliable in assessment results. According to the embodiment of the invention, the depression risk assessment method based on multi-mode gait feature fusion comprises the following steps: Acquiring multi-mode gait data of a subject in a continuous natural walking process, uniformly marking the time stamp of each mode data, and recording a plurality of complete gait cycles; Extracting gait event anchor points from multi-mode gait data, and carrying out phase division and alignment processing on each mode data based on the gait event anchor points to construct a unified gait phase sequence; Inputting a gait video sequence in the multi-mode gait data into an improved XFeat model, extracting local key points, constructing a cross-frame characteristic connecting chain, carrying out phase mapping based on a unified gait phase sequence, constructing a gait local response embryo body sequence, and forming a multi-mode local response characterization set; Applying rhythm perturbation at a unified gait phase position corresponding to a gait event anchor point, extracting phase deviation rectifying starting delay quantity, phase overshoot quantity, backswing damping quantity and recovery closing step number based on a mul