CN-117095457-B - Digital person reconstruction motion scoring method, system, equipment and medium

CN117095457BCN 117095457 BCN117095457 BCN 117095457BCN-117095457-B

Abstract

The invention provides a digital person reconstruction motion scoring method, a system, equipment and a medium, wherein the method comprises the steps of respectively acquiring user skeleton key points and standard skeleton key points from a user video and a standard video according to a target detection algorithm, wherein the user video is a user motion gesture video, and the standard video is a standard motion gesture video; calculating the first included angle of the key point connecting line of the user skeleton and the second included angle of the key point connecting line of the standard skeleton respectively, calculating the score of each key point according to the difference value between the first included angle and the second included angle, multiplying the score of each key point by the preset weight corresponding to each key point and summing to obtain the total score of the movement. The invention can carry out fine evaluation on the movement condition of each body part of the user.

Inventors

HUANG YUXIN
YUAN YIWEI
XIE LING
ZENG XIANGYU
ZHAO BAOQUAN

Assignees

中山大学

Dates

Publication Date: 20260508
Application Date: 20230803

Claims (9)

1. A digital human reconstructive motion scoring method, the method comprising: Acquiring user skeleton key points and standard skeleton key points from a user video and a standard video respectively according to a target detection algorithm, wherein the user video is a user motion gesture video, and the standard video is a standard motion gesture video; Calculating a first included angle of a user skeleton key point connecting line and a second included angle of a standard skeleton key point connecting line respectively, and calculating the score of each key point according to the difference value between the first included angle and the second included angle, wherein the calculation of the score of each key point is expressed as: Wherein, the For each of the key point scores, Is the difference between the first angle and the second angle, The opening and closing degree of the joints of the limbs of the human body; multiplying the scores of the key points by preset weights corresponding to the key points and summing to obtain a total score of the motion; wherein after the total score of the exercise is obtained, the method further comprises: extracting audio from the user video according to an audio extraction algorithm; performing audio track analysis on the audio through a deep neural network, and acquiring a re-shooting moment from the audio; Respectively calculating a first included angle of a line connecting key points of a user skeleton at the moment of re-shooting and a second included angle of a line connecting key points of a standard skeleton at the moment of re-shooting, and calculating the score of each key point at the moment of re-shooting according to the difference value between the first included angle and the second included angle; Multiplying the score of each key point at the moment of re-shooting with the preset weight corresponding to each key point and summing to obtain the rhythm score.
2. The method for scoring a digital human reconstructed motion according to claim 1, wherein the step of obtaining the user skeleton key point and the standard skeleton key point from the user video and the standard video according to the object detection algorithm respectively comprises: Respectively identifying human body areas in the user video and the standard video according to Yolov algorithm; And respectively acquiring user skeleton key points and standard skeleton key points from the human body region according to Openpose algorithm.
3. The method for scoring a digital human reconstructed motion according to claim 1, further comprising, after said obtaining a total score for motion: performing human body reconstruction on the standard video through vibe algorithm to obtain a standard two-dimensional digital human video; And superposing the standard two-dimensional digital human video and the user video to obtain a two-dimensional fusion video.
4. The method for scoring a digital human reconstructed motion according to claim 1, further comprising, after said obtaining a total score for motion: Human body reconstruction is respectively carried out on the user video and the standard video through vibe algorithm, and a user three-dimensional digital human model and a standard three-dimensional digital human model are established; and fusing the user three-dimensional digital human model and the standard three-dimensional digital human model to obtain a dynamic fusion model.
5. The method for scoring a digital human reconstruction motion according to claim 4, wherein the establishing a user three-dimensional digital human model and a standard three-dimensional digital human model according to the user video and the standard video by vibe algorithm respectively comprises: and adjusting the height and body type parameters of the SMPL model according to the user video and the standard video.
6. The method for scoring a digital human reconstructed motion according to claim 4, wherein said obtaining a dynamic fusion model comprises: Sequentially arranging the dynamic fusion models at each moment to obtain a time sequence model; And fusing the time sequence models of the plurality of users in one scene to obtain action models of the plurality of users.
7. A digital human reconstructive motion scoring system, the system comprising: the key point acquisition module is used for acquiring user skeleton key points and standard skeleton key points from a user video and a standard video respectively according to a target detection algorithm, wherein the user video is a user motion gesture video, and the standard video is a standard motion gesture video; The key point score calculation module is used for calculating a first included angle of a key point connecting line of the user skeleton and a second included angle of a key point connecting line of the standard skeleton respectively, and calculating the score of each key point according to the difference value between the first included angle and the second included angle, wherein the calculation of the score of each key point is expressed as follows: Wherein, the For each of the key point scores, Is the difference between the first angle and the second angle, The opening and closing degree of the joints of the limbs of the human body; The motion scoring module is used for multiplying the scores of the key points by preset weights corresponding to the key points and summing the multiplied scores to obtain a total motion score; wherein after the total score of the exercise is obtained, the method further comprises: extracting audio from the user video according to an audio extraction algorithm; performing audio track analysis on the audio through a deep neural network, and acquiring a re-shooting moment from the audio; Respectively calculating a first included angle of a line connecting key points of a user skeleton at the moment of re-shooting and a second included angle of a line connecting key points of a standard skeleton at the moment of re-shooting, and calculating the score of each key point at the moment of re-shooting according to the difference value between the first included angle and the second included angle; Multiplying the score of each key point at the moment of re-shooting with the preset weight corresponding to each key point and summing to obtain the rhythm score.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 6 when the computer program is executed by the processor.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Description

Digital person reconstruction motion scoring method, system, equipment and medium Technical Field The invention relates to the technical field of motion attitude evaluation, in particular to a digital human reconstruction motion scoring method, a system, equipment and a medium. Background The traditional human body motion gesture scoring method mainly depends on expensive motion capture equipment, and needs to monitor a subject in real time and acquire data, acquire information such as acceleration, angular speed and the like of a human body, and then analyze and process the data through an algorithm so as to evaluate the gesture of the human body. The method has high cost and low efficiency, and the use of the motion capture equipment such as the sensor can cause certain interference to the activity of the human body, thereby affecting the accuracy of measurement. The installation and debugging of the device requires a skilled technician to perform the process, which is complicated and increases the use cost. The method has the advantages that the requirements on space are high, the appearance and the motion trail of the human body are required to be scanned and reconstructed, the method has the advantages that the appearance and the motion trail of the human body are required to be scanned and reconstructed, the required computing resources and time are more, and the requirements on hardware are high. In addition, the traditional scoring method often depends on human eyes to observe and judge the motion gesture of the virtual person, and lacks objectivity and accuracy. The motion gesture evaluation is an important health detection means and has wide application prospect. Currently, there are many prior art techniques similar to the present technology, the most common of which is sensor-based motion profile assessment techniques, laser-based motion profile assessment techniques. Referring to domestic patent libraries and related academic researches, the prior art similar to the technology of the invention is selected from (1) equipment and a method for capturing human motion gestures, (2) an ankle joint motion gesture collector, (3) a video-based motion gesture recognition method and system, (4) a motion recognition method and wearable equipment, and (5) a laser human body sensor. For the brief description of the prior art, we can know that the prior art realizes the capture of human body posture, but does not notice the following problems that (1) the device needs to be clung to the surface of the body, which can cause a certain interference to the activity of the human body and affect the accuracy of measurement. (2) The sensor is complex to install and debug, and needs professional technicians to operate, so that the use cost is increased. (3) No score for the pose standard was given for each part of the human body. (4) No visualized dummy is generated for the user to make action gesture standard contrast. (5) The scanning and reconstruction are carried out under specific environments, and the requirement on space is high. Disclosure of Invention The invention aims to provide a digital human reconstruction motion scoring method, a system, equipment and a medium, so as to finely evaluate the motion condition of each body part of a user. To achieve the above object, in a first aspect, an embodiment of the present invention provides a digital human reconstruction motion scoring method, including: Acquiring user skeleton key points and standard skeleton key points from a user video and a standard video respectively according to a target detection algorithm, wherein the user video is a user motion gesture video, and the standard video is a standard motion gesture video; respectively calculating a first included angle of a user skeleton key point connecting line and a second included angle of a standard skeleton key point connecting line, and calculating the score of each key point according to the difference value between the first included angle and the second included angle; and multiplying the scores of the key points by preset weights corresponding to the key points and summing to obtain a total score of the motion. Further, after the exercise total score is obtained, the method further comprises: extracting audio from the user video according to an audio extraction algorithm; performing audio track analysis on the audio through a deep neural network, and acquiring a re-shooting moment from the audio; Respectively calculating a first included angle of a line connecting key points of a user skeleton at the moment of re-shooting and a second included angle of a line connecting key points of a standard skeleton at the moment of re-shooting, and calculating the score of each key point at the moment of re-shooting according to the difference value between the first included angle and the second included angle; Multiplying the score of each key point at the moment of re-shooting with the preset weight correspo