KR-20260066143-A - System and method for dynamic 3D model lighting and realistic shadow rendering in automotive mixed reality applications

Abstract

A method for generating a virtual rendering includes receiving first image data comprising a plurality of images. Each image of the plurality of images may correspond to an environment of a non-virtual object. The method also includes determining the current position of a light source for the non-virtual object and determining the current intensity value of the light source, and generating a virtual rendering of the non-virtual object using the plurality of images, the current position of the light source for the non-virtual object, and the current intensity value of the light source. The method also includes providing the virtual rendering on a display.

Inventors

• 노와코프스키 크리스토퍼
• 아모스 앤서니
• 마닐라 아누샬락슈미
• 알메이다 제이미
• 이스트 니코데무스
• 네기 소남

Assignees

• 발레오 컴포트 앤드 드라이빙 어시스턴스

Dates

Publication Date

20260512

Application Date

20240905

Priority Date

20230907

Claims (20)

1. Regarding the method of generating virtual rendering, The method comprises the step of receiving first image data including a plurality of images—each of which corresponds to an environment of a non-virtual object—and A step of determining the current position of a light source for the above-mentioned non-virtual object, and A step of determining the current intensity value of the light source, and The step of generating a virtual rendering of a non-virtual object using the plurality of images, the current position of a light source for the non-virtual object, and the current intensity value of the light source, and In a display, including the step of providing the virtual rendering, method.
2. In paragraph 1, The above first image data is captured using at least one image capturing device, method.
3. In paragraph 2, The above at least one image capturing device is positioned on at least one of the external part of the non-virtual object and the internal part of the non-virtual object, method.
4. In paragraph 3, The above non-virtual object includes a vehicle, method.
5. In paragraph 1, The first image data is at least one of a real-time image and a light source captured prior to the current position of the non-virtual object. method.
6. In paragraph 1, The step of determining the current position of a light source for the above-mentioned non-virtual object is based on global positioning system coordinates corresponding to the above-mentioned non-virtual object, method.
7. In paragraph 1, The step of determining the current position of the light source for the above-mentioned non-virtual object is based on the time of day, method.
8. In paragraph 1, The step of determining the current position of a light source for the above-mentioned non-virtual object is based on sunload data received from a sunload sensor of the above-mentioned non-virtual object, method.
9. In paragraph 1, The step of determining the current position of a light source for the above-mentioned non-virtual object is based on second image data received from one or more image capturing devices associated with the above-mentioned non-virtual object, method.
10. In Paragraph 9, The above method further includes the step of providing the second image data to an artificial intelligence engine using at least one machine learning model to provide at least one prediction, and The above at least one prediction represents the predicted current position of a light source for the above non-virtual object, and the step of determining the current position of a light source for the above non-virtual object is based on the above at least one prediction. method.
11. In paragraph 1, The light from the above light source includes sunlight, method.
12. In paragraph 1, The above light source includes streetlights, method.
13. In paragraph 1, The step of determining the current intensity value of the light source is based on global positioning system coordinates corresponding to the non-virtual object, method.
14. In paragraph 1, The step of determining the current intensity value of the light source is based on the time of day, method.
15. In paragraph 1, The step of determining the current intensity value of the light source is based on line load data received from the line load sensor of the non-virtual object, method.
16. In paragraph 1, The above method further comprises the step of selectively adjusting at least one aspect of the virtual rendering based on a change in at least one of the current position of the light source and the current intensity value of the light source for the non-virtual object. method.
17. In Paragraph 16, A change in at least one of the current position of the light source and the current intensity value of the light source for the non-virtual object corresponds to at least one of a change in the position of the non-virtual object and a change in light exposure from the light source on the non-virtual object. method.
18. In paragraph 1, The above method further comprises the step of periodically and selectively adjusting at least one aspect of the virtual rendering based on changes in time during the day, method.
19. In a system for generating virtual rendering, processor and, Includes memory containing instructions, When the above instruction is executed by the processor, the processor, Receiving first image data comprising a plurality of images—each of the plurality of images corresponds to an environment of a non-virtual object—, Determine the current position of the light source for the above non-virtual object, and Determine the current intensity value of the light source, and A virtual rendering of a non-virtual object is generated using the plurality of images, the current position of a light source for the non-virtual object, and the current intensity value of the light source. In a display, enabling the virtual rendering above, System.
20. In a device for generating a virtual rendering of a vehicle, Includes a vehicle controller, The above vehicle controller is: Receiving first image data comprising a plurality of images—each of the plurality of images corresponds to the environment of a vehicle—and Determining the current position of at least one light source for the above vehicle, and Determining the current intensity value of at least one light source, and A virtual rendering of the vehicle is generated using the above plurality of images, and A virtual light source is positioned within the virtual rendering based on the current position of the at least one light source for the vehicle, and Adjusting the intensity value of the virtual light source based on the current intensity value of at least one light source, and In the display of the vehicle above, configured to provide the virtual rendering, device.

Description

System and method for dynamic 3D model lighting and realistic shadow rendering in automotive mixed reality applications The present disclosure relates to three-dimensional mode rendering for vehicles, and in particular to a system and method for dynamic three-dimensional model lighting and realistic shadow rendering in mixed reality applications for vehicles. Vehicles such as automobiles, trucks, sports utility vehicles, crossover vehicles, minivans, all-terrain vehicles, recreational vehicles, marine vehicles, aircraft vehicles, or other suitable vehicles increasingly utilize image capturing devices such as image sensors, cameras, optical detection and distance measuring sensors, and radio detection and distance measuring sensors. These image capturing devices may be positioned outside and/or inside the vehicle and can capture image data of the vehicle's environment (including, for example, an internal environment such as the interior of the vehicle and an external environment that traverses or surrounds the vehicle). Increasingly, this image data is being used, for example, to render mixed reality videos or images for use in vehicle operation. For instance, a vehicle operator can shift the vehicle's transmission to reverse gear to operate the vehicle in reverse mode. While the vehicle is operating in reverse mode, a rendering of the vehicle can be displayed on the vehicle's display within the vehicle's environment. The operator can use the rendering to assist in operating the vehicle in reverse mode. One aspect of the disclosed embodiment includes a method for generating a virtual rendering. The method includes receiving first image data comprising a plurality of images. Each of the plurality of images may correspond to an environment of a non-virtual object. The method also includes determining the current position of a light source for the non-virtual object, determining the current intensity value of the light source, and generating a virtual rendering of the non-virtual object using the plurality of images, the current position of the light source for the non-virtual object, and the current intensity value of the light source. The method also includes providing the virtual rendering on a display. Another aspect of the disclosed embodiment includes a system for generating a virtual rendering. The system includes a processor and memory. The memory includes instructions that, when executed by the processor, cause the processor to: receive first image data comprising a plurality of images—wherein each image of the plurality of images corresponds to an environment of a non-virtual object—determine the current position of a light source for the non-virtual object, determine the current intensity value of the light source, generate a virtual rendering of the non-virtual object using the plurality of images, the current position of the light source for the non-virtual object, and the current intensity value of the light source, and provide the virtual rendering to a display. Another aspect of the disclosed embodiment includes an apparatus for generating a virtual rendering of a vehicle. The apparatus includes a vehicle controller, wherein the vehicle controller receives first image data comprising a plurality of images— wherein each of the plurality of images corresponds to an environment of the vehicle—determines the current position of at least one light source for the vehicle, determines the current intensity value of at least one light source, generates a virtual rendering of the vehicle using the plurality of images, positions a virtual light source in the virtual rendering based on the current position of at least one light source for the vehicle, adjusts the intensity value of the virtual light source based on the current intensity value of at least one light source, and is configured to provide the virtual rendering to a display of the vehicle. These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims, and the accompanying drawings. The present disclosure is best understood from the following detailed description when read in conjunction with the accompanying drawings. In accordance with general practice, it is emphasized that various features of the drawings are not drawn to scale. Rather, the dimensions of various features are enlarged or reduced at will for clarity. FIG. 1 generally illustrates a vehicle according to the principles of the present disclosure. FIG. 2 generally illustrates a vehicle controller according to the principles of the present disclosure. FIG. 3 generally illustrates a rendering of a virtual object located at the position of a corresponding non-virtual object in an environment of a non-virtual object, in accordance with the principles of the present disclosure. FIG. 4 illustrates a flowchart of a method for generating a virtual rendering of at least a non-virtual object in accordance with the