CN-121997246-A - Skiing behavior recognition method and system based on multi-position multi-branch network

CN121997246ACN 121997246 ACN121997246 ACN 121997246ACN-121997246-A

Abstract

The invention belongs to the technical field of behavior recognition, and provides a skiing behavior recognition method and system based on a multi-position multi-branch network; the method comprises the steps of processing raw data acquired by different body position sensors to obtain characteristic data blocks corresponding to each position, constructing a multi-branch network model, inputting a characteristic data block corresponding to each position sensor into each sub-network, outputting characteristic representation of the position, on one hand, for executing a position recognition task and on the other hand for characteristic fusion, fusing the characteristic representations output by the sub-networks in a time dimension and a space dimension to obtain time characteristic data and space characteristic data, inputting the time characteristic data and the space characteristic data into a behavior recognition network to execute a behavior recognition task, and outputting a skiing behavior recognition result. The invention solves the problem that the prior art cannot fully utilize the contribution of different position information to the recognition task.

Inventors

LIU HAO
Sun Canhang
MA LELE
PENG CHENG
WANG WENRUI
CHEN WEIKANG
WANG ZHICHUANG

Assignees

观云(山东)智能科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251222

Claims (10)

1. A method for identifying skiing behavior based on a multi-location multi-branch network, comprising: acquiring raw data of sensors fixed to at least two different body positions of a skier; Processing the original data to obtain a characteristic data block corresponding to each position; constructing a multi-branch network model, wherein the model comprises sub-networks corresponding to all position sensors, each sub-network respectively inputs a characteristic data block corresponding to a position and outputs a characteristic representation of the position, and each sub-network is provided with a position identification task, and the characteristic representation is used as input for executing the position identification task on one hand and for fusing the characteristics on the other hand; fusing the characteristic representations output by each sub-network in the time dimension and the space dimension respectively to obtain time characteristic data and space characteristic data; And inputting the time characteristic data and the space characteristic data into a behavior recognition network to execute a behavior recognition task, fusing the time characteristic data and the space characteristic data based on an attention mechanism before a classification layer, and outputting a skiing behavior recognition result through the classification layer.
2. The skiing behavior recognition method based on a multi-location multi-branch network according to claim 1, wherein the processing the raw data to obtain the feature data block corresponding to each location specifically comprises: carrying out gesture calculation on the original data of the sensor to extract gesture characteristics, wherein the original data of the sensor comprises triaxial acceleration, triaxial angular velocity and triaxial magnetic field data; Carrying out standardized treatment on the attitude characteristics; And dividing the standardized data by using a sliding window to form a plurality of characteristic data blocks with equal length.
3. The ski behavior recognition method based on a multi-location multi-branch network according to claim 1, wherein the sub-network comprises a Bi-LSTM network layer, a Relu +dropout layer and a full connection layer, which are sequentially connected.
4. A method of identifying skiing behaviour on the basis of a multi-location multi-branch network as claimed in claim 1, wherein said behaviour identification network comprises: the first branch comprises a one-dimensional convolution layer, an attention mechanism layer, a two-way long-short-term memory network layer, a Relu +Dropout layer and a full-connection layer; A second branch comprising at least one two-dimensional convolution layer, at least one BN + Relu layer, and one full connection layer.
5. The method for identifying skiing behavior based on multi-position multi-branch network as claimed in claim 4, wherein the output characteristics of two branches are input into a classification layer after being fused based on an attention mechanism, and the classification layer comprises two layers of full-connection layers, so as to obtain final behavior identification network output for behavior identification tasks, and specifically, the spatial characteristics/time characteristics are respectively taken as Q in the attention mechanism, and the temporal characteristics/spatial characteristics are correspondingly taken as K, V. Q is query, K is key, V is value, Q and K are multiplied to obtain similarity scores of each query vector and all key vectors, the similarity is different weights, normalization is carried out on the similarity scores, and then the similarity scores are multiplied with V to obtain characteristic representation with weights.
6. A method of identifying skiing behavior based on a multi-location multi-branch network as defined in claim 1, further comprising data enhancement prior to entering a feature data block into said multi-branch network model: and carrying out sequential random adjustment on the characteristic data blocks which are from sensors at different positions in the same batch and correspond to each other in time one by one, so that the adjusted single characteristic data blocks keep a corresponding relation in time sequence and simultaneously contain data from different positions.
7. The skiing behavior recognition method based on the multi-position multi-branch network according to claim 1, wherein the cross fusion of time dimensions is characterized in that feature representations output by all sub-networks are spliced according to time to form fusion time feature data fused with different position information, and the cross fusion of space dimensions is characterized in that feature representations output by all sub-networks are spliced according to positions to form fusion space feature data fused with different position information.
8. A ski behavior recognition system based on a multi-location multi-branch network, comprising: a data acquisition module configured to acquire raw data of sensors fixed to at least two different body positions of a skier; the data processing module is configured to process the original data to obtain characteristic data blocks corresponding to each position; A data acquisition module configured to construct a multi-branch network model including sub-networks corresponding to the position sensors; The multi-branch network model building module is configured to input a characteristic data block corresponding to a position in each sub-network and output a characteristic representation of the position; The feature fusion module is configured to fuse the feature representations output by each sub-network in the time dimension and the space dimension respectively to obtain time feature data and space feature data; The behavior recognition module is configured to input the time feature data and the space feature data into a behavior recognition network to execute a behavior recognition task, fuse the time feature data and the space feature data based on an attention mechanism before a classification layer, and output a skiing behavior recognition result through the classification layer.
9. A computer readable storage medium having stored thereon a program, which when executed by a processor performs the steps of a method of identifying skiing behavior based on a multi-location multi-branch network as claimed in any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor implements the steps of a multi-location multi-branch network based skiing behavior recognition method according to any one of claims 1-7 when the program is executed by the processor.

Description

Skiing behavior recognition method and system based on multi-position multi-branch network Technical Field The invention belongs to the technical field of behavior recognition, and particularly relates to a skiing behavior recognition method and system based on a multi-position multi-branch network. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. With the improvement of the living standard of people, the masses begin to pursue rich mental lives. The sports can bring the competitive interests to people, is beneficial to enhancing physical quality, and becomes an important choice in leisure. Among them, skiing is a sport mode, and in recent years, is popular with more and more people, and attracts a plurality of lovers to challenge themselves continuously, thereby improving skills. Skiing is divided into different projects, each type of the projects comprises different behaviors such as turning, jumping, sliding and the like, each type of the behaviors can be subdivided according to skills such as plow-type sliding, parallel-type sliding and the like, and different skiing behaviors play different roles in different projects and even can determine the completion level of the skiing projects. Therefore, the identification of these key behaviors helps to locate and analyze the behaviors, to evaluate the mastery of the behaviors, and the like, and to optimize the skiing level. Existing skiing behavior recognition studies can be largely classified into two types based on visual information and based on sensor information according to data types. The research based on visual information depends on a camera, is complex to deploy and high in cost, is easily influenced by environment, and is not beneficial to wide-range application and popularization. The research based on the sensor information can be conveniently integrated into the wearable equipment by virtue of the advantages of easy installation, low cost and the like, and hardly causes uncomfortable feeling to skiers, so that the research result is more hopeful to be used for practical popularization. Most of the researches focus on single-point information and basically select to be installed on an upper body, neglect the contribution of other position information to behavior recognition, and some of the researches install sensors at a plurality of positions, but basically select simple splicing when information is fused, so that the contribution of different position information to tasks cannot be fully utilized. Disclosure of Invention The invention aims to provide a skiing behavior recognition method and system based on a multi-position multi-branch network, which are used for constructing a multi-branch network model, extracting features with higher semantics and then fusing so as to solve the technical problem that the contribution of different position information to recognition tasks cannot be fully utilized in the prior art. In order to achieve the above purpose, the invention adopts the following technical scheme: the first aspect of the invention provides a skiing behavior recognition method based on a multi-position multi-branch network, which comprises the following steps: acquiring raw data of sensors fixed to at least two different body positions of a skier; Processing the original data to obtain a characteristic data block corresponding to each position; constructing a multi-branch network model, wherein the model comprises sub-networks corresponding to all position sensors, each sub-network respectively inputs a characteristic data block corresponding to a position and outputs a characteristic representation of the position, and each sub-network is provided with a position identification task, and the characteristic representation is used as input for executing the position identification task on one hand and for fusing the characteristics on the other hand; fusing the characteristic representations output by each sub-network in the time dimension and the space dimension respectively to obtain time characteristic data and space characteristic data; And inputting the time characteristic data and the space characteristic data into a behavior recognition network to execute a behavior recognition task, fusing the time characteristic data and the space characteristic data based on an attention mechanism before a classification layer, and outputting a skiing behavior recognition result through the classification layer. As a further technical solution, the processing the raw data to obtain the feature data block corresponding to each position specifically includes: carrying out gesture calculation on the original data of the sensor to extract gesture characteristics, wherein the original data of the sensor comprises triaxial acceleration, triaxial angular velocity and triaxial magnetic field data; Carrying out standardized treatment on the a