CN-120783387-B - Limb motion disassembly and capture system based on multi-modal analysis

CN120783387BCN 120783387 BCN120783387 BCN 120783387BCN-120783387-B

Abstract

The invention discloses a limb motion dismantling and capturing system based on multi-modal analysis, which relates to the technical field of limb motion dismantling and capturing, and solves the technical problem that in the prior art, the limb motion dismantling and capturing cannot be selected through multi-modal collection analysis so as to improve the capturing efficiency of limb motions; the method comprises the steps of obtaining a multi-modal analysis period, performing action execution main body analysis on the multi-modal analysis period by an action disassembly and identification unit, obtaining action execution tracks according to the analysis, sending the action execution tracks to an action track distinguishing unit after finishing action track setting, and distinguishing the action tracks of the action execution main body after the action track distinguishing unit receives the action tracks.

Inventors

WU XUBIAO
WANG MIN
ZHANG CHENGJIN

Assignees

无锡千帆竞桨科技有限公司

Dates

Publication Date: 20260508
Application Date: 20250624

Claims (6)

1. The limb motion disassembly and capture system based on multi-modal analysis is characterized by comprising a motion capture platform, wherein the motion capture platform is in communication connection with a multi-modal analysis module, and the multi-modal analysis platform is connected with a motion disassembly and recognition unit and a motion track distinguishing unit; the multi-modal analysis module records the whole limb actions and performs action selection through multi-modal analysis to obtain multi-modal analysis time periods, wherein the multi-modal analysis module comprises the following processes: Determining an action execution main body, acquiring action videos of the action execution main body through a multi-mode technology, performing action execution main body section selection according to analysis of the acquired videos, taking the section selected videos as action dismantling captured videos, acquiring the acquired videos, performing recognition analysis on point positions and backgrounds of the action execution main body of the acquired videos, and dismantling the captured actions according to the corresponding needs of the action execution main body to obtain scenes required by action execution; firstly determining the execution position of an action execution main body according to a scene required by action execution, and determining action execution main body point positions corresponding to the execution positions and corresponding to the action execution main body point positions one by one; Acquiring the overlapping time of the execution position and the action execution main point, acquiring the overlapping quantity accumulated value of the execution position and the action execution main point corresponding to the overlapping time, and simultaneously acquiring the point quantity floating frequency of the overlapping state of the adjacent overlapping time after the execution position and the action execution main point are overlapped; If the accumulated value of the overlapping quantity of the corresponding execution position and the action execution main point of the overlapping moment exceeds the accumulated value threshold, and the floating frequency of the point quantity of the overlapping state of the adjacent overlapping moment after the overlapping of the execution position and the action execution main point does not exceed the floating frequency threshold, marking the current overlapping moment as a preset node selection moment; If the accumulated value of the overlapping quantity of the corresponding execution position and the action execution main point at the overlapping moment does not exceed the accumulated value threshold, or the floating frequency of the point quantity of the overlapping state of the adjacent overlapping moment after the execution position and the action execution main point are overlapped exceeds the floating frequency threshold, marking the current overlapping moment as a non-preset node selection moment; After the preset node selection time is obtained, monitoring the overlapping point positions of the adjacent time after the preset node selection time, if the overlapping point positions are continuously increased and the adjacent time of the overlapped point positions are constantly overlapped, marking the current preset node time as the set node time; After the set node time is obtained, comparing the background of the scene where the action execution main point position corresponding to the overlapping time is located, if the background parameter is consistent with the action execution main point position execution background parameter, performing acquisition video section selection at the current set node time, and taking the moment when the number of the overlapping point positions of the action execution main point position and the execution position is rapidly reduced as acquisition video section selection ending time to obtain a multi-mode analysis period; after the multi-modal analysis period is obtained, the action disassembly and identification unit performs action execution subject analysis on the multi-modal analysis period; After finishing the setting of the action track, the action track is sent to the action track distinguishing unit, and after the action track distinguishing unit receives the action track, the action executing main body is distinguished into action tracks.
2. The limb motion disassembly capture system based on multi-modal analysis according to claim 1, wherein the process of the motion disassembly recognition unit is as follows: According to the multi-modal analysis period, carrying out point position track analysis on the action execution main body, marking the point position of the point position in which the point position is integrally moved as a main joint point position, and marking the point position of the point position corresponding to the change of the point position angle of the main body point position as a secondary joint point position; Acquiring azimuth change states of the main joint point position moving position and the relative point position in the multi-modal analysis period, if the azimuth change states of the main joint point position moving position and the relative point position are non-fixed azimuth angle changes, deducing main joint point type setting deviation, generating a main joint point position setting deviation signal, sending the main joint point position setting deviation signal to the motion capture platform, and setting the current point position as a main joint point position, and if the azimuth change states of the main joint point position moving position and the relative point position are non-fixed azimuth angle changes, deducing that the main joint point type setting is normal.
3. The limb motion disassembly capturing system based on multi-modal analysis according to claim 2 is characterized in that the number of non-intersection angles of the real-time moving angle range of the secondary joint point position and the moving angle range of the secondary joint point position of the corresponding type in the multi-modal analysis period is obtained, if the number of non-intersection angles exceeds a threshold value of the number of non-intersection angles, deviation of the setting of the current secondary joint point position is deduced, a secondary joint point position setting deviation signal is generated and sent to a motion capturing platform, the current point position is not set as the secondary joint point position, and if the number of non-intersection angles does not exceed the threshold value of the number of the non-intersection angles, the setting of the current secondary joint point position is deduced to be normal.
4. The limb motion disassembly capturing system based on multi-modal analysis according to claim 3, wherein a motion model of a motion execution body is constructed according to distribution of main joint points and auxiliary joint points, various types of point positions of the motion execution body in the motion model construct and execute the motion, point position movement tracks of the main joint points are set as position movement tracks of the motion execution body, and point position movement tracks of the auxiliary joint points are set as motion execution tracks of various points of the motion execution body.
5. The limb motion disassembly capturing system based on multi-modal analysis according to claim 4, wherein the process of the motion trajectory differentiating unit is as follows: The method comprises the steps of monitoring and analyzing an action track of an action execution main body, collecting each position according to the point position of the action execution main body in the action track, periodically detecting the point position according to the stay time of the corresponding point position of each position, determining the execution period of the current periodic action according to the execution action of the current action execution main body, and periodically repeating the point position sporadic offset and the point position corresponding to the adjacent time in the execution period, deducing that the integral change action of the action execution main body caused by the point position offset at the current point sporadic offset time is an unnecessary action, marking the current action, and monitoring the corresponding point position of the action execution main body so as to avoid reappearance.
6. The limb motion disassembly capturing system based on multi-modal analysis according to claim 5, wherein unnecessary motions in the motion trajectories are removed, and reserved motions are disassembled step by step, namely, the positions of all the points of the motion execution body and the position change are combined and analyzed to form the motion trajectories to execute slow motions, and the motion execution body trajectories among the points are disassembled and sent to the motion capturing platform.

Description

Limb motion disassembly and capture system based on multi-modal analysis Technical Field The invention relates to the technical field of limb movement disassembly and capture, in particular to a limb movement disassembly and capture system based on multi-modal analysis. Background The multi-modal analysis is an information processing technology integrating multiple data modes (such as text, images, audio, video and sensor data) and is used for realizing more comprehensive and accurate cognitive understanding by cooperatively analyzing complementary and associated characteristics of different mode data, the limb motion disassembly and capture system is an intelligent system integrating multiple sensor data (such as vision, an Inertial Measurement Unit (IMU), a pressure sensor and the like) and is used for realizing high-precision motion decomposition and real-time capture by a deep learning and sensor fusion technology, and the core aim is to accurately analyze the space-time characteristics of complex motions by cooperative analysis of the multi-dimensional data and provide scientific basis and technical support for the fields of sports training, medical rehabilitation, virtual reality and the like. However, in the prior art, the limb movement cannot be selected through multi-mode acquisition analysis during the disassembly and capture of the limb movement, so that the capturing efficiency of the limb movement is improved, the point position analysis and division cannot be performed according to the multi-mode analysis time period, the accuracy of the movement capturing cannot be ensured, in addition, the movement track cannot be distinguished, the high efficiency of the movement processing cannot be ensured, and the capturing working strength of unnecessary movements cannot be reduced. In view of the above technical drawbacks, a solution is now proposed. Disclosure of Invention The present invention is directed to solving the above-mentioned problems, and provides a limb motion disassembly and capture system based on multi-modal analysis. The aim of the invention can be achieved by the following technical scheme: the limb motion disassembly and capture system based on multi-modal analysis comprises a motion capture platform, wherein the motion capture platform is in communication connection with a multi-modal analysis module, and the multi-modal analysis platform is connected with a motion disassembly and recognition unit and a motion track distinguishing unit; The multi-modal analysis module records the whole limb actions and performs action selection through multi-modal analysis so as to obtain multi-modal analysis time periods; after the multi-modal analysis period is obtained, the action disassembly and identification unit performs action execution subject analysis on the multi-modal analysis period; After finishing the setting of the action track, the action track is sent to the action track distinguishing unit, and after the action track distinguishing unit receives the action track, the action executing main body is distinguished into action tracks. As a preferred embodiment of the present invention, the process of the multi-modal analysis module is as follows: Determining an action execution main body, acquiring action videos of the action execution main body through a multi-mode technology, performing action execution main body section selection according to analysis of the acquired videos, taking the section selected videos as action dismantling captured videos, acquiring the acquired videos, performing recognition analysis on point positions and backgrounds of the action execution main body of the acquired videos, and dismantling the captured actions according to the corresponding needs of the action execution main body to obtain scenes required by action execution; according to the scene required by the action execution, the execution position of the action execution main body is firstly determined, and the action execution main body point positions corresponding to the execution positions are determined and correspond to each other one by one. As a preferred embodiment of the present invention, the overlapping time of the execution position and the action execution main point is obtained, the cumulative value of the overlapping number of the execution position and the action execution main point corresponding to the overlapping time is obtained, and the floating frequency of the point number of the overlapping state of the adjacent overlapping time after the execution position and the action execution main point are overlapped is obtained; If the accumulated value of the overlapping quantity of the corresponding execution position and the action execution main point of the overlapping moment exceeds the accumulated value threshold, and the floating frequency of the point quantity of the overlapping state of the adjacent overlapping moment after the overlapping of the execution position and the