CN-116506990-B - Light control method, device, equipment and storage medium in virtual production

Abstract

The application provides a light control method and a light control device in virtual production, wherein the virtual production is used for fusing a real scene and a virtual scene to generate a target video; the method comprises the steps of taking a physical lamp in a real scene as a reference, creating a virtual lamp corresponding to the physical lamp in the virtual scene, obtaining target audio applied to a target video, playing the target audio, controlling the virtual lamp to generate a first light effect corresponding to the target audio played currently in the virtual scene in the process of playing the target audio, and synchronously controlling the physical lamp to generate a second light effect corresponding to the target audio played currently in the real scene, wherein the first light effect and the second light effect are used for virtual production, and the target video comprising the first light effect and the second light effect is generated. According to the application, synchronous control of the virtual lamp and the physical lamp can be realized, and the efficiency of virtual film production is improved.

Inventors

• LIU HUILONG

Assignees

• 腾讯科技（深圳）有限公司

Dates

Publication Date

20260508

Application Date

20220705

Claims (20)

1. 1. A light control method in a virtual production, wherein the virtual production is used for generating a target video by combining a real scene and a virtual scene, the method comprising: creating a virtual lamp corresponding to the physical lamp in the virtual scene by taking the physical lamp in the real scene as a reference; Acquiring target audio applied to the target video and playing the target audio; In the process of playing the target audio, analyzing the sound spectrum of the target audio in real time according to the current playing time of the target audio, decomposing the sound spectrum into a plurality of different amplitude curves, and determining a plurality of different amplitudes of the playing time point from the plurality of different amplitude curves; Distributing the plurality of different amplitudes to channels of a lamp by using a preset distribution strategy, wherein the channels comprise at least one of horizontal axis rotation, vertical axis rotation, brightness and color phase of the lamp, and the lamp comprises the physical lamp and the virtual lamp; Based on the allocation strategy, the plurality of data with different amplitudes are sent to the virtual lamp, so that the virtual lamp generates a light and shade or color change along with the sound rhythm, the virtual lamp is controlled to generate a first light effect corresponding to the currently played target audio in the virtual scene, and meanwhile, Combining the plurality of different amplitudes into data conforming to a digital multiplexing protocol, and sending the data to the physical lamp, wherein the data is used for synchronously controlling the physical lamp to make corresponding reactions so as to generate a second light effect corresponding to the currently played target audio in the real scene; the first light effect and the second light effect are used for performing virtual film making so as to generate the target video containing the first light effect and the second light effect.
2. 2. The method of claim 1, wherein the controlling the virtual light fixture to produce a first light effect in the virtual scene corresponding to a currently playing target audio comprises: acquiring the audio attribute of the target audio in real time in the process of playing the target audio, wherein the audio attribute comprises at least one of sound frequency, audio rhythm and audio content; And in the virtual scene, controlling the virtual lamp to generate a first light effect corresponding to the audio attribute of the currently played target audio.
3. 3. The method of claim 2, wherein the controlling the virtual light fixture to produce a first light effect corresponding to an audio attribute of a currently playing target audio comprises: when the number of the virtual lamps is multiple, obtaining the mapping relation between the light effect of each virtual lamp and the audio attribute; And controlling each virtual lamp to generate a corresponding light effect based on the audio attribute of the currently played target audio and the mapping relation so as to form the first light effect.
4. 4. The method of claim 3, wherein when the number of virtual luminaires is plural, before the obtaining the mapping relationship between the light effect of each virtual luminaire and the audio attribute, the method further comprises: Obtaining a virtual light effect of the virtual lamp corresponding to the target video; The virtual light effect comprises the first light effect in the virtual scene at each playing time point of the target audio, wherein the first light effect is formed by the light effect of each virtual lamp in a cooperative manner; And establishing a mapping relation between the light effect of each virtual lamp and the audio attribute at each playing time point based on the virtual light effect.
5. 5. The method of claim 2, wherein the controlling the virtual light fixture to produce a first light effect corresponding to an audio attribute of a currently playing target audio comprises: when the audio attribute comprises an audio rhythm, acquiring the audio rhythm of the played target audio in real time in the process of playing the target audio; According to the audio rhythm of the target audio played currently, controlling the virtual lamp in real time to generate a light effect corresponding to the audio rhythm; wherein different ones of the audio rhythms correspond to different ones of the light effects.
6. 6. The method of claim 2, wherein the controlling the virtual light fixture to produce a first light effect corresponding to an audio attribute of a currently playing target audio comprises: when the audio attribute comprises audio content, acquiring the audio content of the played target audio in real time in the playing process of the target audio; content classification is carried out on the obtained audio content to obtain the content type of the audio content; Controlling the virtual lamp to generate a first light effect corresponding to the content type of the audio content; Wherein different content types correspond to different light effects.
7. 7. The method of claim 2, wherein the controlling the virtual light fixture to produce a first light effect corresponding to an audio attribute of a currently playing target audio comprises: When the audio attribute comprises sound frequency, acquiring the sound frequency of the played target audio in real time in the playing process of the target audio; determining the frequency range to which the acquired sound frequency belongs; controlling the virtual lamp to generate a first light effect corresponding to a frequency range to which the sound frequency belongs; Wherein different ones of the frequency ranges correspond to different ones of the light effects.
8. 8. The method of claim 1, wherein the controlling the virtual light fixture to produce a first light effect in the virtual scene corresponding to a currently playing target audio comprises: Acquiring an attribute value of at least one target attribute of the virtual lamp corresponding to the first light effect; And assigning the attribute value of each target attribute to the corresponding target attribute of the virtual lamp so as to enable the virtual lamp to generate the first light effect.
9. 9. The method of claim 1, wherein when the number of virtual luminaires is plural and the target audio comprises audio data corresponding to at least two audio tracks, the controlling the virtual luminaires to produce a first light effect in the virtual scene corresponding to the currently playing target audio comprises: acquiring the corresponding relation between the audio track and the virtual lamp; determining the virtual lamp corresponding to each audio track based on the corresponding relation; In the virtual scene, controlling the virtual lamp corresponding to each audio track to generate a virtual light effect corresponding to the audio data of the audio track so as to form the first light effect; The audio data of different audio tracks correspond to different virtual light effects, and the first light effect is formed by cooperation of the virtual light effects of the virtual lamps.
10. 10. The method of claim 1, wherein the synchronously controlling the physical light fixture to produce a second light effect in the real scene corresponding to a currently playing target audio comprises: establishing communication connection with the physical lamps in the real scene; Based on the communication connection, a control instruction is sent to the physical lamp, wherein the control instruction is used for instructing the physical lamp to generate a second light effect corresponding to the currently played target audio.
11. 11. The method of claim 10, wherein prior to the sending control instructions to the physical light fixture, the method further comprises: Acquiring a value of a light parameter of the physical lamp corresponding to the first light effect, wherein the value of the light parameter is used for the physical lamp to generate the second light effect; Based on the communication connection, sending a control instruction to the physical lamp, including: Based on the communication connection, a control instruction carrying the value of the lamplight parameter is sent to the physical lamp, so that the physical lamp can apply the value of the lamplight parameter.
12. 12. The method of claim 10, wherein the control instructions carry light effect indication information for indicating the second light effect, and wherein the sending the control instructions to the physical light fixture based on the communication connection comprises: based on the communication connection, a control instruction carrying the light effect indication information is sent to the physical lamp; the control instruction is configured to determine, based on the light effect indication information, a value of a light parameter corresponding to the second light effect, and apply the value of the light parameter to generate the second light effect.
13. 13. The method of claim 1, wherein the creating a virtual luminaire in the virtual scene that corresponds to the physical luminaire with reference to the physical luminaire in the real scene comprises: acquiring a value of a lamp parameter of a physical lamp in the real scene and an association relationship between the physical lamp and the virtual lamp; determining the value of the lamp parameter of the virtual lamp in the virtual scene based on the value of the lamp parameter of the physical lamp and the association relation; and assigning the determined value of the lamp parameter to the lamp parameter of the virtual lamp in the virtual scene.
14. 14. A light control device in a virtual production, wherein the virtual production is used for generating a target video by combining a real scene and a virtual scene, the device comprising: The creation module is used for creating a virtual lamp corresponding to the physical lamp in the virtual scene by taking the physical lamp in the real scene as a reference; The acquisition module is used for acquiring target audio applied to the target video and playing the target audio; The control module is used for analyzing the sound spectrum of the target audio in real time according to the current playing time of the target audio in the process of playing the target audio, decomposing the sound spectrum into a plurality of different amplitude curves, and determining a plurality of different amplitudes of the playing time point from the plurality of different amplitude curves; Distributing the plurality of different amplitudes to channels of a lamp by using a preset distribution strategy, wherein the channels comprise at least one of horizontal axis rotation, vertical axis rotation, brightness and color phase of the lamp, and the lamp comprises the physical lamp and the virtual lamp; Based on the allocation strategy, the plurality of data with different amplitudes are sent to the virtual lamp, so that the virtual lamp generates a light and shade or color change along with the sound rhythm, the virtual lamp is controlled to generate a first light effect corresponding to the currently played target audio in the virtual scene, and meanwhile, Combining the plurality of different amplitudes into data conforming to a digital multiplexing protocol, and sending the data to the physical lamp, wherein the data is used for synchronously controlling the physical lamp to make corresponding reactions so as to generate a second light effect corresponding to the currently played target audio in the real scene; the first light effect and the second light effect are used for performing virtual film making so as to generate the target video containing the first light effect and the second light effect.
15. 15. The apparatus of claim 14, wherein the device comprises a plurality of sensors, The control module is further configured to obtain, in real time, an audio attribute of the target audio in the process of playing the target audio, where the audio attribute includes at least one of a sound frequency, an audio rhythm, and an audio content; And in the virtual scene, controlling the virtual lamp to generate a first light effect corresponding to the audio attribute of the currently played target audio.
16. 16. The apparatus of claim 15, wherein the device comprises a plurality of sensors, The control module is further configured to obtain a mapping relationship between the light effect of each virtual lamp and the audio attribute when the number of the virtual lamps is multiple; And controlling each virtual lamp to generate a corresponding light effect based on the audio attribute of the currently played target audio and the mapping relation so as to form the first light effect.
17. 17. The apparatus of claim 16, wherein the device comprises a plurality of sensors, The control module is further used for obtaining the virtual light effect of the virtual lamp corresponding to the target video; The virtual light effect comprises the first light effect in the virtual scene at each playing time point of the target audio, wherein the first light effect is formed by the light effect of each virtual lamp in a cooperative manner; And establishing a mapping relation between the light effect of each virtual lamp and the audio attribute at each playing time point based on the virtual light effect.
18. 18. The apparatus of claim 15, wherein the device comprises a plurality of sensors, The control module is further configured to obtain, in real time, an audio tempo of the target audio to be played in the process of playing the target audio when the audio attribute includes the audio tempo; According to the audio rhythm of the target audio played currently, controlling the virtual lamp in real time to generate a light effect corresponding to the audio rhythm; wherein different ones of the audio rhythms correspond to different ones of the light effects.
19. 19. The apparatus of claim 15, wherein the device comprises a plurality of sensors, The control module is further configured to obtain, in real time, audio content of the target audio that is played in the process of playing the target audio when the audio attribute includes audio content; content classification is carried out on the obtained audio content to obtain the content type of the audio content; Controlling the virtual lamp to generate a first light effect corresponding to the content type of the audio content; Wherein different content types correspond to different light effects.
20. 20. The apparatus of claim 15, wherein the device comprises a plurality of sensors, The control module is further configured to obtain, in real time, a sound frequency of the target audio to be played in the process of playing the target audio when the audio attribute includes the sound frequency; determining the frequency range to which the acquired sound frequency belongs; controlling the virtual lamp to generate a first light effect corresponding to a frequency range to which the sound frequency belongs; Wherein different ones of the frequency ranges correspond to different ones of the light effects.

Description

Light control method, device, equipment and storage medium in virtual production Technical Field The present application relates to the field of virtual production, and in particular, to a light control method, apparatus, device, computer readable storage medium and computer program product in virtual production. Background Virtual production is a brand-new production method which fuses virtual computer images with performances of real actors and can visually present special-effect-added pictures in real time at a shooting site. In the virtual film making, a photographer does not need to bear a green screen, wait for the later period of weeks or even months, and all key decision makers cooperate on site to generate a final picture in real time and can change the final picture at any time. The effect of virtual film production depends in large part on the light effect, and in the process of film shooting with a light-emitting diode (LED) screen as a background, it is often necessary to synchronously adjust the sound of virtual lamps, physical lamps, and the scene. In the related art, the effects of synchronously starting light and sound according to the direction of directors are generally achieved by a sound box engineer, a lamplight engineer, a virtual lamp operator and the like, and the physical lamp, the sound and the virtual lamp operator are difficult to achieve complete unification of behaviors in the mode, so that the synchronicity of the virtual lamp, the physical lamp and the sound is poor, and the shooting efficiency of the virtual film making is low. Disclosure of Invention The embodiment of the application provides a lamplight control method, a lamplight control device, lamplight control equipment, a computer readable storage medium and a computer program product in virtual film production, which can realize synchronous control of a virtual lamp and a physical lamp and improve virtual film production efficiency. The technical scheme of the embodiment of the application is realized as follows: The embodiment of the application provides a light control method in virtual production, wherein the virtual production is used for generating a target video by combining a real scene and a virtual scene, and the method comprises the following steps: creating a virtual lamp corresponding to the physical lamp in the virtual scene by taking the physical lamp in the real scene as a reference; Acquiring target audio applied to the target video and playing the target audio; in the process of playing the target audio, controlling the virtual lamp to generate a first light effect corresponding to the currently played target audio in the virtual scene, and synchronously controlling the physical lamp to generate a second light effect corresponding to the currently played target audio in the real scene; the first light effect and the second light effect are used for performing virtual film making so as to generate the target video containing the first light effect and the second light effect. The embodiment of the application provides a light control device in virtual production, which is used for fusing a real scene and a virtual scene to generate a target video, and comprises the following components: The creation module is used for creating a virtual lamp corresponding to the physical lamp in the virtual scene by taking the physical lamp in the real scene as a reference; The acquisition module is used for acquiring target audio applied to the target video and playing the target audio; The control module is used for controlling the virtual lamp to generate a first light effect corresponding to the currently played target audio in the virtual scene and synchronously controlling the physical lamp to generate a second light effect corresponding to the currently played target audio in the real scene in the process of playing the target audio, wherein the first light effect and the second light effect are used for carrying out virtual film making so as to generate the target video comprising the first light effect and the second light effect. In the above scheme, the control module is further configured to obtain, in real time, an audio attribute of the target audio in the process of playing the target audio, where the audio attribute includes at least one of a sound frequency, an audio rhythm, and an audio content; And in the virtual scene, controlling the virtual lamp to generate a first light effect corresponding to the audio attribute of the currently played target audio. In the above scheme, the control module is further configured to obtain, when the number of the virtual lamps is multiple, a mapping relationship between a light effect of each virtual lamp and the audio attribute; And controlling each virtual lamp to generate a corresponding light effect based on the audio attribute of the currently played target audio and the mapping relation so as to form the first light effect. In the above scheme, the control