CN-122003856-A - Generation of 3D user interfaces in augmented reality systems

Abstract

An extended reality (XR) system provides a method for cropping virtual content displayed to a user. The XR system uses an XR user interface model to generate an XR user interface with virtual content. The XR system generates cropped virtual content from the virtual content by cropping the virtual content that is outside of the user's stereoscopic field of view, and provides an XR user interface to the user that contains the cropped virtual content.

Inventors

• Russell Douglas Barton
• James Baudley

Assignees

• 斯纳普公司

Dates

Publication Date

20260508

Application Date

20241008

Priority Date

20231010

Claims (20)

1. 1. A computer-implemented method, comprising: Generating, by at least one processor, a 3D XR user interface comprising virtual content; determining, by the at least one processor, one or more portions of the virtual content that are outside of the stereoscopic field of view of the user using the 3D coordinate data of the virtual content; clipping, by the at least one processor, the one or more portions of the virtual content from the XR user interface that are outside of the user's stereoscopic field of view, and The XR user interface is provided to the user by the at least one processor.
2. 2. The computer-implemented method of claim 1, wherein cropping the portion of virtual content comprises: generating a crop mask based on the stereoscopic field of view of the user, and The virtual content that is not within the stereoscopic field of view of the user is cropped using the cropping mask.
3. 3. The computer-implemented method of claim 2, wherein the trim mask is a trim body.
4. 4. The computer-implemented method of claim 1, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a left monocular field of view by cropping the virtual content from a right virtual content rendered image of the virtual content.
5. 5. The computer-implemented method of claim 1, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a right monocular field of view by cropping the virtual content from a left virtual content rendered image of the virtual content.
6. 6. The computer-implemented method of claim 1, wherein cropping the portion of the virtual content comprises cropping the virtual content from a left virtual content rendered image of the virtual content and a right virtual content rendered image of the virtual content.
7. 7. The computer-implemented method of claim 1, wherein the XR user interface is provided to the user using a headset.
8. 8. A machine, comprising: At least one processor, and At least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising: generating a 3D XR user interface comprising virtual content; determining one or more portions of the virtual content that are outside of the user's stereoscopic field of view using the 3D coordinate data of the virtual content; cropping said one or more portions of said virtual content from said XR user interface outside of the user's stereoscopic field of view, and The XR user interface is provided to the user.
9. 9. The machine of claim 8, wherein cropping the portion of virtual content comprises: generating a crop mask based on the stereoscopic field of view of the user, and The virtual content that is not within the stereoscopic field of view of the user is cropped using the cropping mask.
10. 10. The machine of claim 9, wherein the trim mask is a trim body.
11. 11. The machine of claim 8, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a left monocular field of view by cropping the virtual content from a right virtual content rendered image of the virtual content.
12. 12. The machine of claim 8, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a right monocular field of view by cropping the virtual content from a left virtual content rendered image of the virtual content.
13. 13. The machine of claim 8, wherein cropping the portion of the virtual content comprises cropping the virtual content from a left virtual content rendered image of the virtual content and a right virtual content rendered image of the virtual content.
14. 14. The machine of claim 8, wherein the machine comprises a headset.
15. 15. A machine-readable storage medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising: Generating, by at least one processor, a 3D XR user interface comprising virtual content; determining, by the at least one processor, one or more portions of the virtual content that are outside of the stereoscopic field of view of the user using the 3D coordinate data of the virtual content; clipping, by the at least one processor, the one or more portions of the virtual content from the XR user interface that are outside of the user's stereoscopic field of view, and The XR user interface is provided to the user by the at least one processor.
16. 16. The machine-readable storage medium of claim 15, wherein cropping the portion of virtual content comprises: generating a crop mask based on the stereoscopic field of view of the user, and The virtual content that is not within the stereoscopic field of view of the user is cropped using the cropping mask.
17. 17. The machine-readable storage medium of claim 15, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a left monocular field of view by cropping the virtual content from a right virtual content rendered image of the virtual content.
18. 18. The machine-readable storage medium of claim 15, wherein cropping the portion of the virtual content comprises generating a monocular view of the virtual content in a right monocular field of view by cropping the virtual content from a left virtual content rendered image of the virtual content.
19. 19. The machine-readable storage medium of claim 15, wherein cropping the portion of the virtual content comprises cropping the virtual content from a left virtual content rendered image of the virtual content and a right virtual content rendered image of the virtual content.
20. 20. The machine-readable storage medium of claim 15, wherein the XR user interface is provided to the user using a headset.

Description

Generation of 3D user interfaces in augmented reality systems Priority claim The present application claims the benefit of U.S. patent application Ser. No. 18/378,291, filed 10/2023, which is incorporated herein by reference in its entirety. Technical Field The present disclosure relates generally to user interfaces, and more particularly, to user interfaces for augmented or virtual reality. Background The head-mounted device may be equipped with a transparent display or a translucent display through which a user of the head-mounted device may view the surrounding environment. Such headsets enable a user to view the surrounding environment through a transparent display or a semi-transparent display, and also to see virtual content (e.g., a rendering of a two-dimensional (2D) or three-dimensional (3D) graphical model, images, video, text, etc.) that is generated for display to appear as part of and/or superimposed on the surrounding environment. This is commonly referred to as "augmented reality" or "AR". The headset may also completely block the user's view and display the virtual environment in which the user may move or be moved. This is commonly referred to as "virtual reality" or "VR". In a hybrid form, a view of the surrounding environment is captured using a camera device and then displayed to the user along with the enhanced content on a display that obscures the user's eyes. As used herein, the term augmented reality (eXtended Reality, XR) refers to augmented reality, virtual reality, and any hybrid technology of these technologies, unless the context indicates otherwise. Users of the head-mounted devices may access and use computer software applications to perform various tasks or participate in entertainment activities. To use a computer software application, a user interacts with a user interface provided by the headset. Drawings In the drawings (which are not necessarily drawn to scale), like numerals may describe similar components in different views. To facilitate recognition of the discussion of any particular element or act, one or more of the most significant digits in a reference numeral refer to the figure number in which that element was first introduced. Some non-limiting examples are shown in the accompanying drawings, in which: Fig. 1 is an illustration of a viewer viewing a real world scene according to some examples. Fig. 2A is a perspective view of a head mounted device according to some examples. Fig. 2B illustrates another view of the headset of fig. 2A, according to some examples. FIG. 3 is a schematic representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methods discussed herein, according to some examples. Fig. 4A illustrates a co-operation diagram of components of an XR system, according to some examples. Fig. 4B illustrates a depth-aware content rendering method according to some examples. Fig. 4C illustrates a display of virtual content and associated perceived physical objects or surfaces, according to some examples. Fig. 4D illustrates an XR system providing a display of virtual content to a user, wherein a portion of the virtual content is in a stereoscopic field of view of the user and a portion of the virtual content is outside of the stereoscopic field of view of the user, according to some examples. Fig. 4E illustrates an XR system providing a display of virtual content to a user, wherein the virtual content outside of the user's stereoscopic field of view is cropped, according to some examples. Fig. 5 illustrates a system having an XR system of a headset according to some examples. FIG. 6 is a diagrammatic representation of a networking environment in which an XR system may be deployed, according to some examples. FIG. 7 is a diagrammatic representation of a data structure as maintained in a database in accordance with some examples. Fig. 8 is a diagrammatic representation of a messaging system having both client-side and server-side functions in accordance with some examples. Fig. 9 is a block diagram illustrating a software architecture according to some examples. Detailed Description Fig. 1 is an illustration of a viewer viewing a real world scene. When virtual content, such as virtual content 114, including virtual objects or virtual surfaces, in a real world scene is located within the stereoscopic field of view 104 of the viewer, the virtual content 114 appears to have depth to the viewer 102. The stereoscopic field of view 104 is a field of view consisting of the intersection between the right eye field of view and the left eye field of view of the observer 102. Virtual content that does not fall within the stereoscopic field of view 104 (e.g., virtual content 110 and virtual content 116) may not be presented to the viewer 102 as 3D objects. In addition, virtual content (e.g., virtual content 112) may fall on an edge between the stereoscopic field