Search

CN-116489453-B - Display device, terminal device and screen-connected action scoring display method

CN116489453BCN 116489453 BCN116489453 BCN 116489453BCN-116489453-B

Abstract

The embodiment of the application provides a display device, a terminal device and a screen-connection action score display method, which respond to a starting instruction, user action data collected by the terminal device is received, the user action data comprising a number of action frames with frame time stamps. After the delay time length, capturing the played standard action video frames according to the acquisition frame rate so as to obtain standard action data. The delay time length is obtained through calculation according to the first system time of the current display device and the second system time of the terminal device, and the acquisition frame rate is the sampling frequency when the terminal device acquires the user action data. And then searching the target frame according to the frame time stamp, and generating and displaying an action score according to the similarity of the action frame and the target frame. According to the embodiment of the application, the delay time is calculated according to the display equipment and the terminal equipment, so that the time references of the display equipment and the terminal equipment are synchronized through the delay time, the display equipment and the terminal equipment are compared with pictures with the same time stamp, and the accuracy of action scoring display is improved.

Inventors

  • ZHAO YANG
  • PANG XIUJUAN

Assignees

  • 海信视像科技股份有限公司

Dates

Publication Date
20260512
Application Date
20230331

Claims (9)

  1. 1. A display device, characterized by comprising: A display; The communicator is configured to establish communication connection with the terminal equipment, and the terminal equipment is used for collecting user action data; A controller configured to: responding to a starting instruction, receiving user action data acquired by terminal equipment, wherein the user action data comprises a plurality of action frames with frame time stamps; transmitting a synchronization instruction to the terminal equipment, wherein the synchronization instruction comprises a first system time of the current display equipment; acquiring synchronous data fed back by the terminal equipment aiming at the synchronous instruction, wherein the synchronous data comprises second system time of the terminal equipment and acquisition frame rate of the user action data, and the acquisition frame rate is sampling frequency when the terminal equipment acquires the user action data; calculating delay time length according to the first system time and the second system time of the current display equipment and the second system time of the terminal equipment; After the delay time length, capturing a played standard action video frame corresponding to the user action data according to the acquisition frame rate so as to obtain standard action data; Searching a target frame in the standard action data according to the frame time stamp; Generating an action score according to the similarity between the action frame and the target frame, and controlling the display to display the action score, wherein the similarity is obtained by calculating according to the coordinate deviation of a skeleton point in the action frame and the coordinate deviation of a skeleton point in the target frame and the angle deviation between a skeleton point connecting line in the action frame and a skeleton point connecting line in the target frame, the coordinate deviation is used for indicating that the action amplitude in the user action data is larger than or smaller than a standard action, and the angle deviation is used for indicating the angle deviation between the action direction in the user action data and the action direction of the standard action.
  2. 2. The display device of claim 1, wherein after the controller performs capturing the played standard motion video frames according to the capture frame rate, the controller is further configured to: Calculating skeleton point coordinates of the standard action video frame according to a joint feature recognition algorithm; and acquiring the generation time of the bone point coordinates, and storing the generation time and the bone point coordinates as a node bone point array.
  3. 3. The display device of claim 2, wherein the controller performing the lookup of the target frame in the standard motion data according to the frame timestamp is further configured to: Searching a target node bone number group according to the frame time stamp, wherein the target node bone number group is a node bone number group with the generation time closest to the frame time stamp; And marking the standard action video frame corresponding to the target node bone number group as a target frame.
  4. 4. The display device of claim 3, wherein the controller executing the generating of the action score based on the similarity of the action frame and the target frame is further configured to: calculating skeleton point coordinates of the action frame according to a joint feature recognition algorithm; extracting skeleton point coordinates of a target frame from the target node skeleton point number group; calculating coordinate deviation of the skeleton point coordinates of the action frame and the skeleton point coordinates of the target frame; and calculating the similarity according to the coordinate deviation.
  5. 5. The display device of claim 4, wherein the controller performing calculating the coordinate deviation of the skeletal point coordinates of the action frame and the skeletal point coordinates of the target frame is further configured to: determining local articulation points according to the limb parts corresponding to the skeleton point coordinates of the action frame and the limb parts corresponding to the skeleton point coordinates of the target frame; calculating local coordinate deviation according to the local joint points; and carrying out average value calculation on the local coordinate deviation according to the number of the local articulation points so as to obtain the coordinate deviation.
  6. 6. The display device of claim 5, wherein the controller is further configured to: Setting a time interval for clearing data; Acquiring the memory allowance of the display equipment; determining a failure time according to the first system time and the time interval; And deleting redundant data if the memory margin is smaller than a memory quantity threshold value, wherein the redundant data comprises an action frame with a frame time stamp positioned before the failure time and/or a node bone point array with the generation time positioned before the failure time.
  7. 7. The display device of claim 1, further comprising a network programming port, the controller further configured to: establishing a data channel and an instruction channel through the network programming port; and receiving the standard action data through the data channel, and receiving the user action data and the starting instruction through the instruction channel.
  8. 8. A terminal device, comprising: a camera configured to collect user action data; a communication port configured to establish a communication connection with the display device of any one of claims 1-7; A controller configured to: Responding to an acquisition signal sent by the display equipment, controlling the camera to acquire user action data, wherein the user action data comprises a plurality of action frames with frame time stamps; Responding to a synchronous instruction sent by the display equipment, sending synchronous data to the display equipment through the communication port so as to enable the display equipment to calculate delay time according to the synchronous data, wherein the synchronous instruction comprises first system time when the display equipment sends the synchronous instruction, the synchronous data comprises second system time of the terminal equipment and acquisition frame rate of the user action data, the acquisition frame rate is sampling frequency when the terminal equipment acquires the user action data, and the delay time is calculated according to the first system time and the second system time; And sending the user action data to the display device through the communication port, so that the display device searches a target frame in standard action data according to the frame time stamp after the delay time, generates an action score according to the similarity between the action frame and the target frame, and controls a display to display the action score, wherein the similarity is obtained by calculating according to the coordinate deviation of a skeleton point in the action frame and the coordinate deviation of a skeleton point in the target frame and the angle deviation between a skeleton point connecting line in the action frame and a skeleton point connecting line in the target frame, the coordinate deviation is used for indicating that the action amplitude in the user action data is larger than or smaller than the standard action, and the angle deviation is used for indicating the angle deviation between the action direction in the user action data and the action direction of the standard action.
  9. 9. A method for displaying a score of a screen-connected action, which is applied to a display device, wherein the display device comprises a display, a communicator and a controller, the communicator is configured to establish a communication connection with a terminal device, and the terminal device is used for collecting user action data, and the method comprises the following steps: responding to a starting instruction, receiving user action data acquired by terminal equipment, wherein the user action data comprises a plurality of action frames with frame time stamps; transmitting a synchronization instruction to the terminal equipment, wherein the synchronization instruction comprises a first system time of the current display equipment; acquiring synchronous data fed back by the terminal equipment aiming at the synchronous instruction, wherein the synchronous data comprises second system time of the terminal equipment and acquisition frame rate of the user action data, and the acquisition frame rate is sampling frequency when the terminal equipment acquires the user action data; calculating delay time length according to the first system time and the second system time of the current display equipment and the second system time of the terminal equipment; After the delay time length, capturing a played standard action video frame corresponding to the user action data according to the acquisition frame rate so as to obtain standard action data; Searching a target frame in the standard action data according to the frame time stamp; Generating an action score according to the similarity between the action frame and the target frame, and controlling the display to display the action score, wherein the similarity is obtained by calculating according to the coordinate deviation of a skeleton point in the action frame and the coordinate deviation of a skeleton point in the target frame and the angle deviation between a skeleton point connecting line in the action frame and a skeleton point connecting line in the target frame, the coordinate deviation is used for indicating that the action amplitude in the user action data is larger than or smaller than a standard action, and the angle deviation is used for indicating the angle deviation between the action direction in the user action data and the action direction of the standard action.

Description

Display device, terminal device and screen-connected action scoring display method Technical Field The application relates to the technical field of terminal interconnection, in particular to display equipment, terminal equipment and a screen connection action scoring display method. Background The display device refers to a terminal device capable of outputting a specific display screen, and may be a terminal device such as a smart television, a communication terminal, a smart advertisement screen, and a projector. Taking intelligent electricity as an example, the intelligent television is based on the Internet application technology, has an open operating system and a chip, has an open application platform, can realize a bidirectional man-machine interaction function, and is a television product integrating multiple functions of video, entertainment, data and the like, and the intelligent television is used for meeting the diversified and personalized requirements of users. Based on the man-machine interaction function, the user can also assist in exercise through the display device, for example, correct exercise through the display device, so as to achieve the aim of building up the body. The display equipment can acquire the body-building action video of the user by establishing a data transmission channel with the image acquisition device, compares and scores the body-building action video with the teaching video played by the display equipment, and corrects the body-building action of the user according to the scoring result. However, in the process that the image acquisition device transmits the body-building action video to the display device, certain time consumption exists, so that the display device can compare the action made by the user at the current time point with the action made by the user at the last time point in the teaching video, the deviation of the body-building action and the teaching action of the user is large, the grading is inaccurate, the displayed screen-connecting action is inaccurate in grading, and the user experience is reduced. Disclosure of Invention The application provides display equipment, terminal equipment and a screen-connection action scoring display method, which are used for solving the problem that the screen-connection action scoring display is inaccurate in the process of transmitting body-building action videos by an image acquisition device. In a first aspect, some embodiments of the present application provide a display device comprising a display, a communicator configured to establish a communication connection with a terminal device for collecting user action data, and a controller configured to: responding to a starting instruction, receiving user action data acquired by terminal equipment, wherein the user action data comprises a plurality of action frames with frame time stamps; After a delay time length, capturing a played standard action video frame corresponding to the user action data according to an acquisition frame rate to obtain standard action data, wherein the delay time length is obtained by calculating according to a first system time of current display equipment and a second system time of terminal equipment; Searching a target frame in the standard action data according to the frame time stamp; And calculating the similarity of the action frame and the target frame, generating an action score according to the similarity, and calculating the similarity according to the deviation of the skeleton point coordinates in the action frame and the skeleton point coordinates in the target frame. In a second aspect, some embodiments of the present application provide a terminal device, including a camera, a communication port, and a controller, where the camera is configured to collect user action data, the communication port is configured to establish a communication connection with the display device of any one of the first aspects, and the controller is configured to: Responding to an acquisition signal sent by the display equipment, controlling the camera to acquire user action data, wherein the user action data comprises a plurality of action frames with frame time stamps; And sending the user action data to the display equipment through the communication port, so that the display equipment searches a target frame in standard action data according to the frame time stamp, and generates an action score according to the similarity between the action frame and the target frame, wherein the similarity is obtained by calculating the deviation between the skeleton point coordinates in the action frame and the skeleton point coordinates in the target frame. In a third aspect, some embodiments of the present application provide a method for displaying a score of a screen-connected action, where the method is applied to a display device, where the display device includes a display, a communicator, and a controller, where the communicator is configur