US-12627765-B2 - Distributed command execution in multi-location studio environments

Abstract

A distributed studio management system includes a computing system having a computer readable memory and a processor. The computing system is associated with a first production facility that is remote to a second production facility. The processor is configured to perform the operations of receiving a set of device commands having commands targeting first and second devices in the first production facility and where the set of device commands are related to the second production facility, enqueuing a first command from the device commands into a first command queue assigned to the first device and a second command from the device commands into a second command queue assigned to the second device according to each device command's synchronized execution times, and causing the first and second devices to execute commands from their respective command queues relative to each other according to the commands' respective synchronized execution times.

Inventors

• Patrick Soon-Shiong
• Gary Marshall
• Keaton HEINRICHS
• John Wiacek
• Nicholas James Witchey

Assignees

• NANTSTUDIOS, LLC
• NANT HOLDINGS IP, LLC

Dates

Publication Date

20260512

Application Date

20251027

Claims (20)

1. 1 . A computer-based content production management system comprising: at least one non-transitory computer-readable memory storing software instructions; at least one processor coupled with the at least one non-transitory computer-readable memory; at least one physical camera deployed in at least one content production volume, the at least one physical camera configured to store captured content in the at least one memory; and a command queue management module that, upon execution of the software instruction by the at least one processor, performs the following operations: generating at least one camera command based on received virtual camera control instructions, wherein the at least one camera command relates to operation of a virtual camera based on the captured content in a production environment of the at least one content production volume; assigning a synchronized execution time to the at least one camera command; enqueueing the at least one camera command into a camera command queue according to the synchronized execution time; and rendering content associated with the virtual camera based on execution of the at least one camera command from the camera command queue according to the synchronized execution time.
2. 2 . The content production management system of claim 1 , wherein the operation of rendering content associated with the virtual camera is performed by a game engine.
3. 3 . The content production management system of claim 2 , wherein the game engine is selected from the group consisting of: an UnReal game engine, a Unity game engine, a Godot game engine, and a Blender renderer.
4. 4 . The content production management system of claim 1 , further comprising a display module configured to display the rendered content associated with the virtual camera.
5. 5 . The content production management system of claim 1 , wherein the at least one camera command comprises a post-production related command.
6. 6 . The content production management system of claim 1 , wherein: the at least one physical camera is deployed at a first studio; and the system further comprises at least one second physical camera deployed at a second studio remote from the first studio, wherein captured content from the at least one physical camera is transmitted to the second studio.
7. 7 . The content production management system of claim 6 , wherein the operations further include: determining a first latency between the first studio and the content production management system; determining a second latency between the second studio and the content production management system; and adjusting the synchronized execution time of the at least one camera command based on the first latency and the second latency.
8. 8 . The content production management system of claim 7 , wherein the first latency and the second latency are measured using at least one technique selected from the group consisting of: GPS sensors, network time protocol (NTP), ping times, round trip times (RTT), and time to first byte (TTFB).
9. 9 . The content production management system of claim 1 , wherein the operation of rendering content associated with the virtual camera includes rendering on an LED wall of the at least one content production volume.
10. 10 . The content production management system of claim 1 , wherein the operation of rendering content associated with the virtual camera includes adjusting the captured content by at least one adjustment selected from the group consisting of: scaling the captured content, rotating the captured content, skewing the captured content, adjusting lighting of the captured content, adjusting color of the captured content, adjusting contrast of the captured content, adjusting brightness of the captured content, and moving the captured content.
11. 11 . The content production management system of claim 1 , wherein the operations further include integrating the captured content from at least two studios via an implementation of at least one computer vision algorithm.
12. 12 . The content production management system of claim 11 , wherein the at least one computer vision algorithm comprises at least one algorithm selected from the group consisting of: scale-invariant feature transform (SIFT), speeded-up robust features (SURF), FAST Features from Accelerated Segment Test, BRIEF Binary Independent Elementary Features, ORB Object FAST and Rotated BRIEF, Canny Operator, and Canny edge detector.
13. 13 . The content production management system of claim 1 , wherein the at least one camera command relates to at least one camera property selected from the group consisting of: camera angle, camera latitudinal position, camera longitudinal position, camera orientation, camera elevation, camera zoom position, camera ISO setting, camera shutter speed, camera aperture setting, camera frame rate, camera white balance setting, and camera focus.
14. 14 . The content production management system of claim 1 , wherein the operation of rendering content associated with the virtual camera includes updating rendered content based on transformation matrices or projection matrices.
15. 15 . The content production management system of claim 1 , wherein the command queue management module automatically generates the at least one camera command in response to movement of the at least one physical camera.
16. 16 . The content production management system of claim 1 , wherein the content production volume is implemented in at least one environment selected from the group consisting of: physical reality (PR), mixed reality (MR), mixed physical reality (MPR), augmented reality (AR), virtual reality (VR), and projected reality (PrR).
17. 17 . The content production management system of claim 1 , further comprising at least one sensor deployed in the content production volume, wherein the command queue management module is configured to generate the at least one camera command based on data received from the at least one sensor.
18. 18 . The content production management system of claim 1 , wherein the at least one virtual camera command includes a triggering criterion in addition to the synchronized execution time.
19. 19 . The content production management system of claim 1 , wherein the at least one virtual command is generated based on a previs.
20. 20 . The content production management system of claim 1 , wherein the operations further include recording information associate with execution of the at least one wherein the camera command on a notarized ledger.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 19/250,446 filed Jun. 26, 2025, which is a continuation of U.S. patent application Ser. No. 19/064,807 filed Feb. 27, 2025 (now U.S. Pat. No. 12,381,993), which is a continuation of U.S. patent application Ser. No. 18/960,035 filed Nov. 26, 2024 (now U.S. Pat. No. 12,267,614), which is a continuation of U.S. patent application Ser. No. 18/787,107 filed Jul. 29, 2024 (now U.S. Pat. No. 12,212,876), which is a continuation of U.S. patent application Ser. No. 18/637,937 filed Apr. 17, 2024 (now U.S. Pat. No. 12,088,949), which is a continuation of U.S. patent application Ser. No. 18/229,867 filed Aug. 3, 2023 (now U.S. Pat. No. 12,010,451), which is a continuation of U.S. patent application Ser. No. 17/874,728 filed Jul. 27, 2022 (now U.S. Pat. No. 11,765,306), which claims the benefit of U.S. Provisional Application No. 63/229,674 filed Aug. 5, 2021. The entire disclosures of the above applications are incorporated herein by reference. FIELD The present disclosure relates to facilitating remote command and control via distributed, synchronized command queues. BACKGROUND Film studios typically employ green screen technologies, blue screen technologies, or more recently LED wall technologies to film content for movies, television, motion capture, or for other purposes. Such technologies allow for enhanced special effects and unique filming backgrounds. If a studio relies on a green or a blue screen technology, a director is unable to view a final composition of the content until post-production work is complete. This may take weeks, if not months, to complete, which in turn generates additional costs or delays in production. As such, necessary reshoots may be delayed. If a studio relies on LED wall technology, a director may be able to view portions (e.g., animated backgrounds, etc.) of a final composition of the content before post-production work is complete. The background description provided here is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure. SUMMARY A content production management system within a distributed studio environment or ecosystem includes at least one computer-readable medium, at least one command interface module, and at least one command queue management module in communication with the command interface module. The at least one command interface module is configured to render or otherwise present at least one user interface for a set of content production entities. Each content production entity from the set of content production entities is associated with a content production volume from a set of content production volumes within the distributed studio environment. The at least one command queue management module, upon execution of software instructions, is configured to perform the operations of receiving a set of commands from the at least one command interface module and assigning an execution time, possibly a synchronized execution time, to each command of the set of commands. Each command of the set of commands targets a target content production entity from the set of the content production entities. The at least one command queue management module, upon execution of software instructions, is further configured to perform the operations of enqueueing each command of the set of commands into a command queue associated with the target content production entity according to its respective synchronized execution time, and enabling the target content production entity to execute each command of the set of commands from the command queue according to its respective synchronized execution time. The command queue is stored in the computer-readable medium. A computerized method of managing a set of commands in a content production management system within a distributed studio environment is also disclosed. The content production management system includes a command interface module and a command queue management module in communication with the command interface module. The command interface module is configured to render or otherwise present at least one user interface for a set of content production entities, and each content production entity from the set of content production entities is associated with a content production volume from a set of content production volumes within the distributed studio environment. The method includes receiving, from the command interface module, a set of commands, and assigning, with the command queue management module, a synchronized execution time to each command of the set of commands. Each command of the set of commands targets a t