Search

EP-4479926-B1 - TURNING A TWO-DIMENSIONAL IMAGE INTO A SKYBOX

EP4479926B1EP 4479926 B1EP4479926 B1EP 4479926B1EP-4479926-B1

Inventors

  • CANDIDO, Salvatore
  • CHEUNG, VINCENT CHARLES
  • ZHANG, Jiemin
  • TSENG, HUNG-YU

Dates

Publication Date
20260513
Application Date
20230214

Claims (13)

  1. A computer-implemented method for producing a skybox in an artificial reality, XR, world from a 2D image, the method comprising: splitting (204) the 2D image (100) with a split line (102) along an axis into two portions (104, 106); swapping the positions of the two portions (104, 106); mapping (206) each portion (104, 106) onto a separate interior face of a 3D cube (108), wherein each interior face is different from the top (122) and bottom (124) face of the 3D cube (108), and wherein the portions (104, 106) are mapped so that the edges of the portions (104, 106) at the split line (102) match up with one another; applying (208) a generative adversarial network, GAN, to a flat image representing the 3D cube (108) unfolded, to fill in a space between the two portions (104, 106) in order to create a panoramic image mapped onto the four interior faces of the 3D cube (108); creating a border area of a top face (122) of the 3D cube (108) from the top edge of the four interior faces; applying the GAN to fill in the top face (122) of the 3D cube (108) based on the created border area of the top face (122); and applying (210) the 3D cube (108) as a skybox in the XR world.
  2. A method as claimed in claim 1, further comprising: receiving instructions, from a user, to execution an application configured for the creation of skyboxes.
  3. A method as claimed in claim 2, wherein said instructions are received following detection of an interaction of the user with one or more virtual control elements within an artificial reality environment.
  4. A method as claimed in claim 3, wherein the interaction includes one or more of: the user looking at a said virtual control element; the user pointing at a said virtual control element; the user touching a said virtual control element.
  5. A method as claimed in claim 2, wherein the instructions are received following detection of a verbal command from the user.
  6. A method as claimed in any one of the preceding claims, wherein the two portions (104, 106) of the 2D image are of equal dimensions; or wherein the two portions (104, 106) of the 2D image are of different dimensions.
  7. A method as claimed in any one of the preceding claims, wherein the swapping of positions of the two portions (104, 106) includes providing the space between the two portions (104, 106).
  8. A system (300) arranged to perform a method as claimed in any one of claims 1 to 7.
  9. A system (300) as claimed in claim 8, wherein the system (300) comprises an artificial reality system.
  10. A system (300) as claimed in claim 9, wherein the artificial reality system comprises a head mounted display, HMD, (400) having a display (445) arranged to display the skybox to a user.
  11. A system (300) as claimed in claim 10, wherein the HMD (400) is connected to a host computer; or wherein the HMD (400) is a standalone HMD.
  12. A computer program product comprising instructions which, when executed by a processor, cause the processor to carry out the steps of a method as claimed in any one of claims 1 to 7.
  13. A computer-readable storage medium, storing instructions that, when executed by a computing system, cause the computing system to perform a method as claimed in any one of claims 1 to 7.

Description

TECHNICAL FIELD The present disclosure relates to techniques for building a skybox for an XR world from a user-selected 2D image. BACKGROUND Many people are turning to the promise of artificial reality ("XR"): XR worlds expand users' experiences beyond their real world, allow them to learn and play in new ways, and help them connect with other people. An XR world becomes familiar when its users customize it with particular environments and objects that interact in particular ways among themselves and with the users. As one aspect of this customization, users may choose a familiar environmental setting to anchor their world, a setting called the "skybox." The skybox is the distant background, and it cannot be touched by the user, but in some implementations it may have changing weather, seasons, night and day, and the like. Creating even a static realistic skybox is beyond the abilities of many users. US 2021/192684 A1 describes apparatuses, systems, and techniques to generate panoramas from individual images. One or more generative neural networks are used to generate a spherical panoramic image using features extracted from a single input image. KYUNGHUN KIM ET AL: "Painting Outside as Inside: Edge Guided Image Outpainting via Bidirectional Rearrangement with Progressive Step Learning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 9 November 2020, also describes relevant technical schemes. SUMMARY OF THE DISCLOSURE In accordance with a first aspect of the present disclosure, there is provided a method according to claim 1. In some embodiments, the method may further comprise: receiving instructions, from a user, to execution an application configured for the creation of skyboxes. In some embodiments, said instructions may be received following detection of an interaction of the user with one or more virtual control elements within an artificial reality environment. In some embodiments, the interaction may include one or more of: the user looking at a said virtual control element; the user pointing at a said virtual control element; the user touching a said virtual control element. In some embodiments, the instructions may be received following detection of a verbal command from the user. In some embodiments, the two portions of the 2D image may be of equal dimensions. In some embodiments, the two portions of the 2D image may be of different dimensions. In some embodiments, the swapping of positions of the two portions includes providing the space between the two portions. In accordance with a further aspect of the present disclosure, there is provided a system arranged to perform the method of the first aspect. In some embodiments, the system may comprise an artificial reality system. In some embodiments, the artificial reality system may comprise a head mounted display (HMD) having a display arranged to display the skybox to a user. In some embodiments, the HMD may be connected to a host computer. In some embodiments, the HMD may be a standalone HMD. In accordance with a further aspect of the present disclosure, there is provided a computer program product comprising instructions which, when executed by a processor, cause the processor to carry out the steps of a method of the first aspect. In accordance with a further aspect of the present disclosure, there is provided a computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform the method of the first aspect. It will be appreciated that any features described herein as being suitable for incorporation into one or more aspects or embodiments of the present disclosure are intended to be generalizable across any and all aspects and embodiments of the present disclosure. Other aspects of the present disclosure can be understood by those skilled in the art in light of the description, the claims, and the drawings of the present disclosure. The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1A is a conceptual drawing of a 2D image to be converted into a skybox.Figures 1B through 1F are conceptual drawings illustrating steps in a process according to the present technology for converting a 2D image into a skybox.Figure 1G is a conceptual drawing of a completed skybox.Figure 2 is a flow diagram illustrating a process used in some implementations of the present technology for converting a 2D image into a skybox.Figure 3 is a block diagram illustrating an overview of devices on which some implementations of the present technology can operate.Figure 4A is a wire diagram illustrating a virtual reality headset which can be used in some implementations of the present technology.Figure 4B is a wire diagram illustrating a mixed reality headset which can be used in some implementations of the present technology.Fi