CN-121370151-B - Gait parameter extraction method based on combination of cyclic neural network and foot pressure sensor

CN121370151BCN 121370151 BCN121370151 BCN 121370151BCN-121370151-B

Abstract

The application provides a gait parameter extraction method based on a cyclic neural network combined with a foot pressure sensor, which comprises the steps of obtaining a walking video of a subject, inputting the walking video into a human body key point detection model to obtain a human body key point feature vector sequence, obtaining plantar pressure data of the subject, determining a time point of at least one gait event according to the plantar pressure data, and obtaining the gait parameter according to the time point of the gait event and/or the human body key point feature vector sequence. The gait parameter extraction method based on the cyclic neural network combined with the foot pressure sensor is convenient to operate, low in cost and high in precision.

Inventors

ZHU JIALU
CHEN KAI
LIU TINGTING
YANG YING

Assignees

杭州电子科技大学

Dates

Publication Date: 20260508
Application Date: 20251226

Claims (7)

1. The gait parameter extraction method based on the foot pressure sensor is characterized by comprising the following steps of: Acquiring a walking video of a subject; Inputting the walking video into a human body key point detection model to obtain a human body key point feature vector sequence, wherein the human body key point feature vector sequence comprises coordinates of human body key points at different time points, and the human body key points comprise ankle joint key points, knee joint key points and hip joint key points; Acquiring raw heel pressure data of a subject; reserving zero values in the original heel pressure data, and performing Gaussian filtering on non-zero values in the original heel pressure data to obtain plantar pressure data, wherein the plantar pressure data are synchronous with the walking video in time; determining a time point of at least one gait event from the plantar pressure data; obtaining the gait parameters according to the time points of the gait events and/or the human key point feature vector sequences; the gait event includes a heel strike event; the determining a point in time of a heel strike event from the heel pressure data comprises: performing differential processing on the heel pressure data to obtain a heel pressure change rate; selecting a time point at which the heel pressure change rate is greater than or equal to a first threshold as a primary candidate time point for the heel strike event; Calculating the motion speed of the ankle joint key point corresponding to the primary candidate time point of the heel strike event in the vertical direction according to the human body key point feature vector sequence; Selecting a time point at which the movement speed is less than or equal to a second threshold value as a secondary candidate time point of the heel strike event; acquiring coordinates of the ankle joint key point, the knee joint key point and the hip joint key point corresponding to a secondary candidate time point of the heel strike event according to the human body key point feature vector sequence; Calculating a knee joint flexion angle corresponding to a secondary candidate time point of the heel strike event according to coordinates of the ankle joint key point, the knee joint key point, and the hip joint key point; the knee joint flexion angle is calculated using the following formula: ; In the middle of , , Represents the three-side length of the triangle formed by the hip joint, the knee joint and the ankle joint, , , Represent the first Calculating an angle intermediate variable at the nth frame of the joint, wherein In the form of a hip joint, In order to be a knee joint, Is an ankle joint, and is provided with a joint body, A knee joint angle at the nth frame; Selecting the secondary candidate time point corresponding to the maximum knee joint buckling angle as the time point of the heel strike event; the gait parameters include a gait cycle; The step of obtaining the gait parameters according to the time points of the gait events and/or the human key point feature vector sequences comprises the step of calculating the time difference of two adjacent heel strike events positioned on the same side of a human body to obtain the gait cycle.
2. The foot pressure sensor based gait parameter extraction method of claim 1, wherein the gait parameter comprises a stride, the gait event comprises a heel strike event, the gait parameter extraction method further comprising: Determining a point in time of a heel strike event from the heel pressure data; acquiring coordinates of the ankle joint key points corresponding to the time points of the heel strike event in the horizontal direction according to the human body key point feature vector sequence; And calculating the coordinate difference of the key points of the ankle joints of the two adjacent heel strike events on the same side of the human body in the horizontal direction to obtain the stride.
3. The foot pressure sensor based gait parameter extraction method of claim 1, wherein the gait parameter comprises a support phase time, the plantar pressure data comprises big toe pressure data, the gait event comprises a toe off event, the gait parameter extraction method further comprising: determining a point in time of a toe-off event from the big toe pressure data; And calculating the time difference between the toe-off event and the heel-strike event to obtain the support phase time.
4. The foot pressure sensor-based gait parameter extraction method of claim 3, wherein said determining a point in time of a toe-off event from said big toe pressure data comprises: performing differential processing on the big toe pressure data to obtain a big toe pressure change rate; selecting a time point with the big toe pressure change rate being greater than or equal to a third threshold value as a primary candidate time point of the toe-off event; Calculating the motion speed of the ankle joint key point corresponding to the primary candidate time point of the toe-off event in the vertical direction according to the human body key point feature vector sequence; And selecting the time point when the movement speed reaches a fourth threshold value for the first time as the time point of the toe-off event.
5. The method for extracting gait parameters based on the foot pressure sensor according to claim 3, wherein the gait parameters include swing phase time, and the method for extracting gait parameters includes calculating a difference between the gait cycle and the support phase time to obtain the swing phase time.
6. The gait recognition method based on the cyclic neural network is characterized by comprising the following steps of: Acquiring a human body key point feature vector sequence and gait parameters by adopting the gait parameter extraction method of any one of claims 1-5; constructing a cyclic neural network, wherein the input of the cyclic neural network is the human body key point feature vector sequence, and the output of the cyclic neural network is the gait parameters and the disease categories; Training the circulating neural network by taking the human body key point feature vector sequence in the same gait cycle as input and taking the gait parameters of the subject as labels to obtain a trained gait recognition network; and inputting the human body key point feature vector sequence to be identified into the trained gait recognition network to obtain disease types.
7. The cyclic neural network-based gait recognition method of claim 6, wherein the cyclic neural network is a long-short term memory network.

Description

Gait parameter extraction method based on combination of cyclic neural network and foot pressure sensor Technical Field The application relates to the technical field of gait recognition, in particular to a gait parameter extraction method based on a cyclic neural network combined with a foot pressure sensor. Background Human gait is one of the important biological features reflecting the health status of individuals, and is closely related to various physiological and pathological states. The gait parameter analysis can quantitatively and objectively describe the movement mode in the walking process, has important significance in clinical medicine, and has important value for diagnosis and rehabilitation evaluation of nervous system diseases such as parkinsonism, apoplexy sequelae and the like. Currently, the acquisition of gait information is primarily dependent on gait assistance devices, which generally comprise a sensor assembly, an exoskeleton structure and a data processing unit. Human motion data are collected through the sensor and analyzed by combining with a preset algorithm, gait parameters representing the action posture and the behavior characteristics of the user can be extracted. According to different data acquisition modes, the existing gait parameter extraction methods are mainly divided into three types, namely a labeling method, a wearable sensor method and a labeling-free method. The marking method is to paste reflecting markers at key joints or skeleton mark points of human body and to record the spatial track of the markers with motion capturing system comprising several high speed cameras to calculate gait parameters. The method has high precision, is widely applied to scientific research and clinical research, but has obvious limitations that a subject needs to wear close-fitting clothes and carry out complicated labeling operation, has long preparation time, and simultaneously, the system needs larger experimental space and expensive equipment support, and has high cost, complicated use and difficult popularization. The wearable sensor method is to fix an inertial measurement unit (such as an accelerometer and a gyroscope) or a pressure sensor on a specific part (such as a foot and a lower limb) of a human body, and calculate gait parameters by acquiring kinematic or mechanical signals. The method is high in portability, suitable for daily monitoring, still needs to wear equipment, can influence the natural walking state, and is poor in long-term wearing comfort. The marking-free method is based on the computer vision technology, utilizes a depth camera or a common camera to automatically identify key points of a human body, does not need additional marks or wearing equipment, and has the advantages of simplicity and convenience in operation, low cost, small requirements on places and the like. However, due to the lack of direct physical contact feedback, it is susceptible to occlusion, illumination changes, and clothing differences in the temporal decisions of critical events (e.g., heel strike, toe off), resulting in relatively low accuracy in gait parameters extraction. In addition, conventional contour-based methods (e.g., average difference imaging) generate a gait energy pattern (GEI) and use it for recognition by performing centroid alignment, difference computation, and principal component analysis on the human body contours in the video frames. Although the method has higher calculation efficiency, the method is seriously dependent on the consistency of the outline of the human body, when a subject wears different clothes, carries a knapsack or changes the hairstyle, the outline is obviously changed, the recognition accuracy is reduced, and the robustness is insufficient. In summary, the prior art has difficulty in combining high-precision and convenience, namely, a high-precision method is often complicated in equipment and inconvenient to use, and a low-cost and easily-deployed unmarked method has the problem of low precision. Therefore, on the premise of not increasing the burden of a user, the gait parameter extraction precision of the label-free system is effectively improved, and the method becomes an important direction of current research. Therefore, a gait analysis method that combines multi-source information and has high accuracy and practicality is needed. Disclosure of Invention In view of the above-mentioned drawbacks of the prior art, the present application aims to provide a gait parameter extraction method based on a cyclic neural network combined with a foot pressure sensor, which aims to construct a conveniently operated, high-precision and quantifiable gait parameter extraction model, and improve the accuracy of gait recognition. In a first aspect, the present application provides a gait parameter extraction method based on a foot pressure sensor, the gait parameter extraction method comprising: Acquiring a walking video of a subject; inputting the wal