US-12623154-B2 - System for recognizing player behavior and game situation in sports game video

US12623154B2US 12623154 B2US12623154 B2US 12623154B2US-12623154-B2

Abstract

Provided is a system for recognizing a player behavior and a game situation in a sports game video. The system for recognizing a player behavior and a game situation in a sports game video according to an embodiment includes at least one processor and a memory configured to store a program that is executed by the at least one processor, wherein the processor is configured to generate a video clip by extracting a game video every predetermined time, generate a game situation information by analyzing the video clip through a trained analysis model, and generate a game situation service information through the game situation information and provide the game situation service information to a user, and wherein the game video is a video of a sport game being played between a first team and a second team on a court.

Inventors

Jinwook Kim
Seong Geun YOO
Kyung-Ryoul MUN
Donghoon Kang

Assignees

KOREA INSTITUTE OF SCIENCE AND TECHNOLOGY

Dates

Publication Date: 20260512
Application Date: 20221222
Priority Date: 20220704

Claims (3)

1 . A system for recognizing a player behavior and a game situation in a sports game video, comprising: at least one processor; and a memory configured to store a program that is executed by the at least one processor, wherein the processor is configured to: generate a video clip by extracting a game video every predetermined time, generate a game situation information by analyzing the video clip through a trained analysis model, and generate a game situation service information through the game situation information and provide the game situation service information to a user, and wherein the game video is a video of a sport game being played between a first team and a second team on a court, wherein the processor is further configured to: generate a court recognition information by recognizing a court area by analyzing the game video using semantic segmentation, generate an object information by recognizing players and referees participating in the game in the recognized court area and equipment used in the game, generate a group information by recognizing a behavior of a group by inputting the court recognition information and the object information to a first prediction model, generate a team information by recognizing the first team and the second team based on the group information, and generate the game situation information by inputting the court recognition information, the object information, the team information and the group information to a second prediction model, wherein the first prediction model is pre-trained with a pre-annotated label by encoding the court recognition information and the object information and extracting a specific vector, and wherein the second prediction model is pre-built by training with a game situation information label by encoding the court recognition information, the object information, the team information and the group information and extracting a specific vector.
2 . The system for recognizing a player behavior and a game situation in a sports game video according to claim 1 , wherein the game video is a multi-view video including a plurality of game sub videos of the sports game recorded at a plurality of different angles, and wherein the processor is further configured to: synchronize the plurality of game sub videos on a time axis, and extract the video clip every predetermined time in each of the plurality of synchronized game sub videos.
3 . The system for recognizing a player behavior and a game situation in a sports game video according to claim 1 , wherein the processor is further configured to: record events occurring in the game, and generate in a suitable form for game broadcasting, generate a heatmap by accumulating accumulated locations of the players and a ball possession situation, generate game statistics, and automatically generate situation information in a text form for use in game commentary based on the recognition of the game video.

Description

DESCRIPTION OF GOVERNMENT-FUNDED RESEARCH AND DEVELOPMENT This research is conducted under the support of Ministry of Culture, Sports and Tourism, Building Innovation Infrastructure in Sports Industry (R&D), [Project Name: Data management in training games of players and AI based athletic performance improvement solution technology development, Project Number: 1375027374, Project Serial Number: S202101-07-08]. CROSS-REFERENCE TO RELATED APPLICATION This application claims priority to Korean Patent Application No. 10-2022-0081685, filed on Jul. 4, 2022, and all the benefits accruing therefrom under 35 U.S.C. § 119, the contents of which in its entirety are herein incorporated by reference. BACKGROUND 1. Field The present disclosure relates to a system for recognizing a player behavior and a game situation in a sports game video, and more particularly, to a system which recognizes the behaviors of individual players and teams in a game, recognizes a more complicated game situation and provides to a user. 2. Description of the Related Art Video based activity classification and recognition is necessary in various industrial applications. The video based activity recognition has used the traditional method including extracting a human region in a video, extracting the features of the extracted human, and training a classifier with the calculated feature values to recognize activities. Additionally, methods which extract a human pose, and train features based on the pose to recognize activities have been devised. Recently, attention is directed to end-to-end activity recognition by training activity labels in videos using a deep learning model such as convolutional neural network (CNN) and long short-term memory (LSTM). However, the proposed methods have been greatly developed in the task of recognizing simple activities such as walking and running for a uniform length of time, but in sports games with game rules and teams, they are not suitable to recognize a game situation made up of complex activities by hierarchically recognizing many simple activities. Accordingly, the present disclosure proposes a system and device for recognizing the behaviors of individual players and teams in a game and recognizing a more complex game situation. SUMMARY The present disclosure is designed to solve the above-described problem, and specifically, the present disclosure is directed to providing a system for recognizing a player behavior and a game situation in a sports game video in which in a sports game with game rules and teams, the system recognizes a game situation made up of complex activities by hierarchically recognize many simple activities, analyzes the behaviors of individual players and teams and the game situation together and provides the analysis results to a user. A system for recognizing a player behavior and a game situation in a sports game video includes at least one processor and a memory configured to store a program that is executed by the at least one processor, wherein the processor is configured to generate a video clip by extracting a game video every predetermined time, generate game situation information by analyzing the video clip through a trained analysis model, and generate a game situation service information through the game situation information and provide the game situation service information to a user, and the game video is a video of a sport game being played between a first team and a second team on a court. The game video may be a multi-view video including a plurality of game sub videos of the sports game recorded at a plurality of different angles, and the processor may be further configured to synchronize the plurality of game sub videos on a time axis, and extract the video clip every predetermined time in each of the plurality of synchronized game sub videos. The processor may be further configured to generate court recognition information by recognizing a court area by analyzing the game video using semantic segmentation, generate an object information by recognizing players and referees participating in the game in the recognized court area and equipment used in the game, generate group information by recognizing a behavior of a group by inputting the court recognition information and the object information to a first prediction model, generate team information by recognizing the first team and the second team based on the group information, and generate the game situation information by inputting the court recognition information, the object information, the team information and the group information to a second prediction model, the first prediction model may be pre-trained with a pre-annotated label by encoding the court recognition information and the object information and extracting a specific vector, and the second prediction model may be pre-built by training with a game situation information label by encoding the court recognition information, the object information,