US-12626373-B2 - Smartphone motion tracking system

US12626373B2US 12626373 B2US12626373 B2US 12626373B2US-12626373-B2

Abstract

A system for processing video of a sporting activity is disclosed. The system comprises: a plurality of portable electronic devices, each of which comprises at least one camera for capturing video of the sporting activity; and one or more aggregated data processing devices. Each portable electronic device is configured to: generate, using its camera(s), video data representative of the sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the aggregated data processing device(s). The aggregated data processing device(s) are configured to perform processing of the video analysis data, so as to generate activity analysis data. Apparatus for holding a portable electronic device is also disclosed, which comprises a housing, within which the portable electronic device can be mounted and enclosed, and an active cooling system for cooling the portable electronic device.

Inventors

Kevin William King
Rajiv Tharmeswaran Maheswaran
Tracey Chui Ping Ho
Yu-Han Chang
Alexa Chen Lan CHANG
Jeffrey Wayne Su

Assignees

GENIUS SPORTS SS, LLC

Dates

Publication Date: 20260512
Application Date: 20230712

Claims (18)

1 . A system for processing video of a sporting activity, the system comprising: a plurality of portable electronic devices, each of which comprises at least one camera for capturing video of the sporting activity from a viewpoint of the electronic device; and one or more aggregated data processing devices, which are, in use, in data communication with the plurality of portable electronic devices, wherein each portable electronic device is configured to: generate, using the at least one camera, video data representative of the sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the one or more aggregated data processing devices, wherein the one or more aggregated data processing devices are configured to perform processing of the video analysis data from the plurality of portable electronic devices, so as to generate activity analysis data, and wherein the system is configured, in use, to dynamically shift the processing of the video analysis data to one or more of the plurality of portable electronic devices that are not presently capturing video of the sporting activity so that the one or more portable electronic devices that are not presently capturing video of the sporting activity are used as one or more of the one or more aggregated data processing devices by: receiving video analysis data from one or more of the plurality of portable electronic devices that are presently capturing video of the sporting activity; and processing part or all of the video analysis data.
2 . The system of claim 1 , wherein the processing of the video data by each portable electronic device comprises using at least one neural network.
3 . The system of claim 1 , wherein the processing of the video data by each portable electronic device comprises using an object identification algorithm.
4 . The system of claim 1 , wherein the processing of the video data by each portable electronic device comprises using a spatiotemporal pattern recognition algorithm.
5 . The system of claim 1 , wherein the processing of the video data by each portable electronic device comprises using a feature extraction algorithm.
6 . The system of claim 1 , wherein the video analysis data generated by each portable electronic device comprises motion data representative of the motion, as viewed from the portable electronic device, of at least one of: one or more sporting objects used by participants in the sporting activity; one or more participants in the sporting activity; and one or more body parts of participants in the sporting activity.
7 . The system of claim 6 , wherein the processing of the video analysis data comprises determining three-dimensional motion data based on the motion data received from the plurality of portable electronic devices.
8 . The system of claim 7 , wherein determining the three-dimensional motion data comprises using a triangulation algorithm.
9 . The system of claim 1 , wherein the video analysis data comprises participant identification data, which indicates the identity of one or more participants in the sporting activity.
10 . The system of claim 1 , wherein each portable electronic device is additionally configured to transfer to the one or more aggregated data processing devices reduced bitrate video data, which has a lower bitrate than the video data representative of the sporting activity generated by the at least one camera.
11 . The system of claim 1 , wherein the plurality of portable electronic devices are configured such that they do not transmit any data that is representative of video of the sporting activity to the one or more aggregated data processing devices.
12 . The system of claim 1 , wherein at least some of the one or more aggregated data processing devices are servers.
13 . The system of claim 1 , wherein at least some of the portable devices are smartphones.
14 . The system of claim 1 , wherein at least some of the portable electronic devices receive power, and are in data communication with the one or more aggregated data processing devices, using respective Power over Ethernet connections.
15 . The system of claim 1 , wherein at least some of the portable devices are in data communication with the one or more aggregated data processing devices using 5G.
16 . The system of claim 1 , wherein the plurality of portable electronic devices comprises at least 5 portable electronic devices.
17 . A method for processing video of a sporting activity, the method being performed by a system comprising: a plurality of portable electronic devices, each of which comprises at least one camera for capturing video of the sporting activity from a viewpoint of the electronic device; and one or more aggregated data processing devices, which are, in use, in data communication with the plurality of portable electronic devices, the method comprising: performing, by each portable electronic device: generating, using the at least one camera, video data representative of the sporting activity; performing processing of the video data to generate video analysis data; and transferring the video analysis data to the one or more aggregated data processing devices; processing, by the one or more aggregated data processing devices, the video analysis data from the plurality of portable electronic devices, so as to generate activity analysis data; and dynamically shifting the processing of the video analysis data to one or more of the plurality of portable electronic devices that are not presently capturing video of the sporting activity so that the one or more portable electronic devices that are not presently capturing video of the sporting activity are used as one or more of the one or more aggregated data processing devices by: receiving video analysis data from one or more of the plurality of portable electronic devices that are presently capturing video of the sporting activity; and processing part or all of the video analysis data.
18 . A first portable electronic device for use as one of a plurality of portable electronic devices in a system for processing video of a sporting activity, the system comprising: the plurality of portable electronic devices, each portable electronic device comprising at least one camera for capturing video of the sporting activity from a viewpoint of the electronic device; and one or more aggregated data processing devices, which are, in use, in data communication with the plurality of portable electronic devices, wherein each portable electronic device is configured to: generate, using the at least one camera, video data representative of the sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the one or more aggregated data processing devices, wherein the one or more aggregated data processing devices are configured to perform processing of the video analysis data from the plurality of portable electronic devices, so as to generate activity analysis data, and wherein the system is configured to dynamically shift the processing of the video analysis data to one or more of the plurality of portable electronic devices that are not presently capturing video of the sporting activity so that the one or more portable electronic devices that are not presently capturing video of the sporting activity are used as one or more of the one or more aggregated data processing devices by: receiving video analysis data from one or more of the plurality of portable electronic devices that are presently capturing video of the sporting activity; and processing part or all of the video analysis data, the first portable electronic device being configured to be used as one of the one or more aggregated data processing devices.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application claims the benefit of U.S. Patent Application Ser. No. 63/388,495, filed Jul. 12, 2022, and entitled “SMARTPHONE MOTION TRACKING SYSTEM”. The content of the foregoing application is hereby incorporated by reference in its entirety for all purposes. BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to methods and systems for processing video, in particular video of a sporting activity, such as a sporting event or a practice session. In embodiments disclosed herein, video of the sporting activity is taken from multiple viewpoints. Description of the Related Technology Sports serve as a vital source of entertainment and leisure for millions of people worldwide. The use of video in sports has revolutionized the way teams and athletes analyze, strategize, and improve their performance. Video analysis allows for detailed review of gameplay, technique, and tactics, providing valuable insights that can lead to enhanced performance on the field. Coaches can study their team's strengths and weaknesses, identify patterns, and make data-driven decisions to optimize strategies. Players can analyze their own performances, identify areas for improvement, and refine their skills with precision. Video technology has also transformed the fan experience, allowing spectators to relive and dissect pivotal moments, gain deeper understanding of the game, and engage in discussions and debates with other fans. SUMMARY In accordance with a first aspect of the present disclosure there is provided a system for processing video of a sporting activity, the system comprising: a plurality of portable electronic devices, each of which comprises at least one camera for capturing video of the sporting activity from a viewpoint of the electronic device; and one or more aggregated data processing devices, which are in data communication with the plurality of portable electronic devices, wherein each portable electronic device is configured to: generate, using the at least one camera, video data representative of the sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the one or more aggregated data processing devices, and wherein the one or more aggregated data processing devices are configured to: perform processing of the video analysis data from the plurality of portable electronic devices, so as to generate activity analysis data. In examples, at least some of the portable electronic devices are mobile devices. In addition, or instead, at least some of the portable electronic devices may be smartphones or tablet computing devices. In accordance with a further aspect of the present disclosure there is provided a system for analyzing data relating to a sporting activity, the system comprising one or more aggregated data processing devices, which are configured to: receive, from each of a plurality of portable electronic devices, video analysis data that has been generated by the portable electronic device in question processing video data generated by at least one camera of the portable electronic device; and perform processing of the video analysis data from the plurality of portable electronic devices, so as to generate activity analysis data. In accordance with yet a further aspect of the present disclosure there is provided a portable electronic device, comprising: at least one processor; at least one camera; and computer readable storage medium comprising instructions, which, when executed by the at least one processor, cause the portable electronic device to: generate, using the at least one camera, video data representative of a sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the one or more aggregated data processing devices. In accordance with a further aspect of the present disclosure there is provided a computer program product comprising instructions which, when the instructions are executed by at least one processor of a portable electronic device, cause the portable electronic device to: generate, using at least one camera of the portable electronic device, video data representative of a sporting activity; perform processing of the video data to generate video analysis data; and transfer the video analysis data to the one or more aggregated data processing devices. In accordance with a still further aspect of the present disclosure there is provided an apparatus for holding a portable electronic device that comprises at least one camera, the apparatus comprising: a housing, within which the portable electronic device is mountable, such that the housing encloses the portable device within an interior of the housing; and an active cooling system for cooling the portable electronic device. In some examples, the housing sealingly encloses the portable electronic device. In addition,