CN-122004838-A - Joint moment prediction method and system based on graph neural network

CN122004838ACN 122004838 ACN122004838 ACN 122004838ACN-122004838-A

Abstract

The invention provides a joint moment prediction method and a joint moment prediction system based on a graph neural network in the technical field of biomechanics sensing and artificial intelligence, wherein the method comprises the steps of S1, acquiring IMU data acquired by IMU sensors distributed on lower limbs of a human body, preprocessing each IMU data at least comprising data integration, filtering denoising and sliding window segmentation, S2, inputting the preprocessed IMU data into a double-branch graph module to construct a static graph and a dynamic graph so as to extract space characteristics, constructing the static graph based on a fixed space layout relation of the IMU sensors, constructing the dynamic graph based on an attention mechanism in a self-adaptive manner, S3, modeling the space characteristics through a gating circulation unit in a time sequence so as to extract time-dependent characteristics, and S4, mapping the time-dependent characteristics through a full-connection layer, and outputting a regression prediction result of joint moment. The method has the advantages that the accuracy, the instantaneity, the generalization capability and the robustness of joint moment prediction are greatly improved.

Inventors

XIONG BAOPING
Deng Haizhi
ZHANG JILIN

Assignees

福建理工大学

Dates

Publication Date: 20260512
Application Date: 20251218

Claims (10)

1. A joint moment prediction method based on a graph neural network is characterized by comprising the following steps: step S1, acquiring IMU data acquired by each IMU sensor arranged on the lower limb of a human body, and preprocessing each IMU data at least comprising data integration, filtering denoising and sliding window segmentation; s2, inputting the preprocessed IMU data into a double-branch graph module to construct a static graph and a dynamic graph so as to extract spatial features, wherein the static graph is constructed based on a fixed spatial layout relation of the IMU sensors, and the dynamic graph is constructed based on an attention mechanism in a self-adaptive way; s3, carrying out time sequence modeling on the spatial features through a gating circulating unit so as to extract time-dependent features; And S4, mapping the time-dependent features through a full-connection layer, and outputting a regression prediction result of the joint moment.
2. The method for predicting joint moment based on a graph neural network of claim 1, wherein in the step S1, the filtering denoising is specifically performed by filtering denoising the IMU data at a filtering frequency of 2Hz through a high pass filter.
3. The method for predicting joint moment based on graph neural network as set forth in claim 1, wherein in the step S2, the static graph construction process specifically includes: based on the fixed space layout relation of each IMU sensor, a graph structure is defined Wherein, the method comprises the steps of, Representing a static graph, V representing a set of nodes of the IMU sensor, E representing a set of edges of the IMU sensor, IMU data representing IMU sensor inputs at time t, Representing an adjacency matrix of IMU sensors in a static diagram, adjacency matrix Elements of (2) The calculation formula of (2) is as follows: ; Wherein, the Representing adjacency matrix The elements of the ith row and the jth column of the system have values of 0 or 1 and are used for representing nodes Sum node Whether a connecting edge exists between the two parts; Representing nodes Is a neighbor set of (a); The node characteristics of the static diagram comprise triaxial acceleration, triaxial angular velocity, L2 acceleration and L2 angular velocity, and the calculation formulas of the L2 acceleration and the L2 angular velocity are as follows: ; ; Wherein, the Represents L2 acceleration; represents the L2 angular velocity; representing the triaxial acceleration; Representing the triaxial angular velocity; The construction process of the dynamic graph specifically comprises the following steps: Defining a dynamic graph structure based on dynamic association relation between attention mechanism self-adaptive learning IMU sensors Wherein, the method comprises the steps of, A dynamic diagram is shown in which, The adjacency matrix of the IMU sensor in the dynamic diagram is represented by the following calculation formula: ; Wherein, the N represents the number of IMU sensors, i.e., the number of nodes; The extraction process of the spatial features comprises the following steps: using graph-annotating force networks, for nodes in static and dynamic graphs, respectively Extracting attention features : ; ; Wherein σ represents the activation function; Representing nodes Opposite node L represents the number of neighbor nodes; Representing a learnable parameter; Representing characteristic stitching; attention features based on the static graph Building static graph feature sets Based on the attention features of the dynamic graph Building dynamic graph feature sets For the static diagram feature set And dynamic graph feature set Weighting and fusing to obtain space characteristics : ; Where b represents an adjustable weight coefficient.
4. The method for predicting joint moment based on a graph neural network according to claim 1, wherein in the step S3, the formula for extracting the time-dependent feature by the gating loop unit is as follows: ; ; ; ; Wherein, the Representing a reset gate; 、、、、、 All represent a learnable parameter; representing the spatial characteristics at m time; Representing the hidden state of the previous moment, namely the time dependent characteristic of m-1 moment; Representing a time dependent characteristic of the moment m; Representing an update gate; Representing candidate hidden states at m time; representing the element-by-element product.
5. The method for predicting joint moment based on the graph neural network of claim 1, wherein the step S4 is specifically: And after flattening the time-dependent characteristics, inputting the flattened time-dependent characteristics into a full-connection layer for nonlinear transformation, and mapping the flattened time-dependent characteristics into regression prediction results of joint moment.
6. The joint moment prediction system based on the graph neural network is characterized by comprising the following modules: The IMU data preprocessing module is used for acquiring IMU data acquired by each IMU sensor arranged on the lower limb of the human body and preprocessing each IMU data at least comprising data integration, filtering denoising and sliding window segmentation; The system comprises a space feature extraction module, a static image acquisition module, a dynamic image acquisition module and a dynamic image acquisition module, wherein the space feature extraction module is used for inputting the preprocessed IMU data into the double-branch image module to construct a static image and a dynamic image so as to extract space features; The time-dependent feature extraction module is used for carrying out time sequence modeling on the spatial features through the gating circulating unit so as to extract time-dependent features; And the joint moment prediction module is used for mapping the time-dependent characteristics through the full-connection layer and outputting a regression prediction result of the joint moment.
7. The joint moment prediction system based on the graph neural network, as set forth in claim 6, wherein the IMU data preprocessing module performs filtering denoising on the IMU data at a filtering frequency of 2Hz by using a high-pass filter.
8. The joint moment prediction system based on a graph neural network as set forth in claim 6, wherein in the spatial feature extraction module, the static graph construction process specifically includes: based on the fixed space layout relation of each IMU sensor, a graph structure is defined Wherein, the method comprises the steps of, Representing a static graph, V representing a set of nodes of the IMU sensor, E representing a set of edges of the IMU sensor, IMU data representing IMU sensor inputs at time t, Representing an adjacency matrix of IMU sensors in a static diagram, adjacency matrix Elements of (2) The calculation formula of (2) is as follows: ; Wherein, the Representing adjacency matrix The elements of the ith row and the jth column of the system have values of 0 or 1 and are used for representing nodes Sum node Whether a connecting edge exists between the two parts; Representing nodes Is a neighbor set of (a); The node characteristics of the static diagram comprise triaxial acceleration, triaxial angular velocity, L2 acceleration and L2 angular velocity, and the calculation formulas of the L2 acceleration and the L2 angular velocity are as follows: ; ; Wherein, the Represents L2 acceleration; represents the L2 angular velocity; representing the triaxial acceleration; Representing the triaxial angular velocity; The construction process of the dynamic graph specifically comprises the following steps: Defining a dynamic graph structure based on dynamic association relation between attention mechanism self-adaptive learning IMU sensors Wherein, the method comprises the steps of, A dynamic diagram is shown in which, The adjacency matrix of the IMU sensor in the dynamic diagram is represented by the following calculation formula: ; Wherein, the N represents the number of IMU sensors, i.e., the number of nodes; The extraction process of the spatial features comprises the following steps: using graph-annotating force networks, for nodes in static and dynamic graphs, respectively Extracting attention features : ; ; Wherein σ represents the activation function; Representing nodes Opposite node L represents the number of neighbor nodes; Representing a learnable parameter; Representing characteristic stitching; attention features based on the static graph Building static graph feature sets Based on the attention features of the dynamic graph Building dynamic graph feature sets For the static diagram feature set And dynamic graph feature set Weighting and fusing to obtain space characteristics : ; Where b represents an adjustable weight coefficient.
9. The joint moment prediction system based on a graph neural network, as set forth in claim 6, wherein the formula for extracting the time-dependent feature by the gating loop unit is: ; ; ; ; Wherein, the Representing a reset gate; 、、、、、 All represent a learnable parameter; representing the spatial characteristics at m time; Representing the hidden state of the previous moment, namely the time dependent characteristic of m-1 moment; Representing a time dependent characteristic of the moment m; Representing an update gate; Representing candidate hidden states at m time; representing the element-by-element product.
10. The joint moment prediction system based on the graph neural network, as set forth in claim 6, wherein the joint moment prediction module is specifically configured to: And after flattening the time-dependent characteristics, inputting the flattened time-dependent characteristics into a full-connection layer for nonlinear transformation, and mapping the flattened time-dependent characteristics into regression prediction results of joint moment.

Description

Joint moment prediction method and system based on graph neural network Technical Field The invention relates to the technical field of biomechanics sensing and artificial intelligence intersection, in particular to a joint moment prediction method and a joint moment prediction system based on a graph neural network. Background The joint moment is a key kinetic parameter for evaluating the motion capacity and the joint health state of a human body, and has important application value in the fields of sports medicine, rehabilitation engineering, intelligent auxiliary equipment control and the like. The traditional joint moment measuring method generally depends on inverse dynamics calculation combined by an optical motion capturing system and a force measuring platform, and has the defects of high equipment cost, complex system, dependence on a fixed laboratory environment and the like, so that the method is difficult to be suitable for real-time and mobile application scenes such as outdoor rehabilitation and daily activity monitoring. With the development of wearable Inertial Measurement Unit (IMU) technology, IMU-based data driving methods are becoming a research hotspot for joint moment prediction. The deep learning model can learn a complex nonlinear mapping relation between the deep learning model and joint moment from IMU data, and has good performance in tasks such as gait analysis. However, existing deep learning methods focus on time series modeling (e.g., using a recurrent or convolutional neural network to extract timing dependent features), while ignoring spatial correlations between multiple sensors. In practical application, the IMU sensors are usually arranged on a plurality of segments of the lower limb of the human body, and inherent spatial topological relation and physiological constraint exist between signals of the IMU sensors, but the existing method cannot effectively utilize the spatial structural characteristics, so that prediction accuracy and model generalization capability under a complex action mode are limited. In recent years, a Graph Neural Network (GNN) as an emerging deep learning technology exhibits excellent temporal-spatial feature fusion capability in the fields of traffic flow prediction, social network analysis, and the like. GNNs are naturally suitable for modeling IMU sensors as graph structures, thereby synchronously characterizing the static spatial associations and dynamic interactions between the sensors. Although GNN theoretically provides a new idea for IMU data processing, its application in joint moment prediction is still in the preliminary exploration phase. The existing method is not provided with a high-efficiency space-time joint modeling mechanism, the feature extraction of the coupling of the static anatomical structure and the dynamic function is difficult to balance, and the synergy between the time modeling and the space fusion is insufficient, so that the instantaneity and the accuracy of joint moment prediction are restricted. Therefore, how to provide a joint moment prediction method and system based on a graph neural network, so as to improve the precision, instantaneity, generalization capability and robustness of joint moment prediction, is a technical problem to be solved urgently. Disclosure of Invention The invention aims to solve the technical problem of providing a joint moment prediction method and a joint moment prediction system based on a graph neural network, which can improve the precision, instantaneity, generalization capability and robustness of joint moment prediction. In a first aspect, the present invention provides a joint moment prediction method based on a graph neural network, including the following steps: step S1, acquiring IMU data acquired by each IMU sensor arranged on the lower limb of a human body, and preprocessing each IMU data at least comprising data integration, filtering denoising and sliding window segmentation; s2, inputting the preprocessed IMU data into a double-branch graph module to construct a static graph and a dynamic graph so as to extract spatial features, wherein the static graph is constructed based on a fixed spatial layout relation of the IMU sensors, and the dynamic graph is constructed based on an attention mechanism in a self-adaptive way; s3, carrying out time sequence modeling on the spatial features through a gating circulating unit so as to extract time-dependent features; And S4, mapping the time-dependent features through a full-connection layer, and outputting a regression prediction result of the joint moment. Further, in the step S1, the filtering denoising is specifically performed by filtering and denoising the IMU data with a filtering frequency of 2Hz through a high-pass filter. Further, in the step S2, the construction process of the static diagram specifically includes: based on the fixed space layout relation of each IMU sensor, a graph structure is defined Wherein, the metho