EP-4740084-A1 - INTERACTIVELY PLACING OBJECTS ON A VIRTUAL STAGE

Abstract

Tracking a control object on a stage for a virtual production, including: tracking a control object on the stage with a system that tracks at least one camera used in the virtual production; identifying a location in a virtual environment using tracking information of the control object; and placing virtual assets at the identified location in the virtual environment.

Inventors

• ANDERBERG, TOBIAS
• WISE, JOSEPH

Assignees

• Sony Group Corporation
• Sony Pictures Entertainment, Inc.

Dates

Publication Date

20260513

Application Date

20240709

Claims (20)

1. 1. A method for tracking a control object on a stage for a virtual production, the method comprising: tracking a control object on the stage with a system that tracks at least one camera used in the virtual production; identifying a location in a virtual environment using tracking information of the control object; and placing virtual assets at the identified location in the virtual environment.
2. 2. The method of claim 1, wherein the virtual assets include light cards.
3. 3. The method of claim 1, wherein the tracked control object includes one of : finger, hand, arm, markers on a hand, eyeline, voice, or object tags.
4. 4. The method of claim 1, wherein the control object is a pointer to indicate the location.
5. 5. The method of claim 4, wherein the pointer is tracked to create a virtual laser pointer in the virtual environment by casting a virtual ray from a tip of the pointer to create a visible line.
6. 6. The method of claim 5, wherein the virtual ray guides a virtual artist to place the virtual assets at the identified location.
7. 7. The method of claim 5, wherein the tracked control object includes a tablet computer including software that enables a user to pick and place virtual objects in the virtual environment based on the virtual ray from the tracked tablet computer to a scene in the virtual environment.
8. 8. The method of claim 7, wherein the virtual objects include at least one of 3D models, lights, and light cards.
9. 9. The method of claim 7, wherein the user controls various aspects of the virtual objects directly on the tablet computer.
10. 10. The method of claim 9, wherein the various aspects include at least one of color, brightness, scale, and rotation.
11. 11. A system to track a control object on a stage for a virtual production, the system comprising: a control object; a tracking device to track the control object on the stage, wherein the tracking device tracks at least one camera used in the virtual production; and a computer system to identify a location in a virtual environment displayed on an LED wall using tracking information of the control object obtained by the tracking device, wherein the computer system places virtual assets at the identified location in the virtual environment.
12. 12. The system of claim 11, wherein the virtual assets include light cards.
13. 13. The system of claim 11, wherein the tracked control object includes one of : finger, hand, arm, markers on a hand, eyeline, voice, or object tags.
14. 14. The system of claim 11, wherein the control object is a pointer to indicate the location.
15. 15. The system of claim 14, wherein the pointer includes a virtual laser pointer in the virtual environment to cast a virtual ray from a tip of the pointer to create a visible line.
16. 16. The system of claim 15, wherein the tracked control object includes a tablet computer including software that enables a user to pick and place virtual objects in the virtual environment based on the virtual ray from the tracked tablet computer to a scene in the virtual environment.
17. 17. The system of claim 16, wherein the virtual objects include at least one of 3D models, lights, and light cards.
18. 18. A non- transitory computer-readable storage medium storing a computer program to track a control object on a stage for a virtual production, the computer program comprising executable instructions that cause a computer to: track a control object on the stage with a system that tracks at least one camera used in the virtual production; identify a location in a virtual environment using tracking information of the control obj ect ; and place virtual assets at the identif ied location in the virtual environment .
19. 19 . The non- transitory computer- readable storage medium of claim 18 , further comprising executable instructions that cause the computer to create a virtual laser pointer in the virtual environment by casting a virtual ray from a tip of the control obj ect to create a visible line .
20. 20 . The non- transitory computer- readable storage medium of claim 17 , wherein executable instructions that cause the computer to place virtual assets at the identif ied location in the virtual environment include executable instructions that cause the computer to guide a virtual artist to place the virtual assets at the identif ied location using the virtual ray .

Description

INTERACTIVELY PLACING OBJECTS ON A VIRTUAL STAGE BACKGROUND Field [0001] The present disclosure relates to tracking a control object, and more specifically, to tracking the control object on a stage for a virtual production. Background [0002] In a virtual production, photorealistic sets are displayed on large LED walls behind physical sets using the real-time rendering capabilities of real-time 3-D engines. The cameras are synchronized with the 3-D engines for enhanced realism and depth of perspective. During a shoot on the virtual production, communication between virtual artists and traditional set designers (or the director of photography) may be difficult. In the current workflow, if a virtual object needs to be placed in the world, the set designer needs to verbally communicate with the virtual artist to guide them on placement of virtual assets. Furthermore, to use virtual light cards (i.e. , a virtual light placed on the screen to light practical subjects) , the director of photography also needs to verbally communicate with the virtual artist to guide them on placement of the virtual light card. This may be inefficient as well as inaccurate in a scenario where efficiency and accuracy are vital. SUMMARY [0003] The present disclosure implements techniques for tracking a control obj ect on a stage for a virtual production and interactively placing virtual assets at an identif ied location in a virtual environment . [0004] In one implementation, a method for tracking a control obj ect on a stage for a virtual production is disclosed . The method includes : tracking a control obj ect on the stage with a system that tracks at least one camera used in the virtual production ; identifying a location in a virtual environment using tracking information of the control obj ect ; and placing virtual assets at the identif ied location in the virtual environment . [0005] In another implementation, a system to track a control obj ect on a stage for a virtual production is disclosed . The system includes : a control obj ect ; a tracking device to track the control obj ect on the stage , wherein the tracking device tracks at least one camera used in the virtual production ; and a computer system to identify a location in a virtual environment displayed on an LED wall using tracking information of the control obj ect obtained by the tracking device , wherein the computer system places virtual assets at the identif ied location in the virtual environment . [0006] In yet another implementation, a non- transitory computer- readable storage medium storing a computer program to track a control obj ect on a stage for a virtual production includes executable instructions that cause a computer to : track a control obj ect on the stage with a system that tracks at least one camera used in the virtual production ; identify a location in a virtual environment using tracking information of the control obj ect ; and place virtual assets at the identif ied location in the virtual environment . [0007 ] Other features and advantages should be apparent from the present description which illustrates , by way of example , aspects of the disclosure . BRIEF DESCRIPTION OF THE DRAWINGS [0008] The details of the present disclosure , both as to its structure and operation, may be gleaned in part by study of the appended drawings , in which like reference numerals refer to like parts , and in which : [0009] FIG . 1 is a f low diagram of a process for tracking a control obj ect on stage for a virtual production in accordance with one implementation of the present disclosure ; [0010] FIG . 2 is an illustration of a system for tracking a control obj ect on stage for a virtual production in accordance with one implementation of the present disclosure ; [0011] FIG . 3 is a block diagram of a system for tracking a control obj ect on stage for a virtual production in accordance with one implementation of the present disclosure ; and [0012 ] FIG . 4A is a representation of a computer system and a user in accordance with an implementation of the present disclosure ; and [0013 ] FIG . 4B is a functional block diagram illustrating the computer system 400 hosting the tracking application in accordance with an implementation of the present disclosure . DETAILED DESCRIPTION [0014] As described above , in the current workf low of a virtual production, if a virtual obj ect needs to be placed in the world, the set designer needs to verbally communicate with the virtual artist to guide them on placement of virtual assets . This may be inef f icient as well as inaccurate in a scenario where ef f iciency and accuracy are vital . [0015] To address the issues with the conventional way for the set designer to verbally communicate with the virtual artist to guide them on placement of virtual assets , implementations of the present disclosure provide a technique for tracking an obj ect on stage through the same system that tracks the camera . These implementat