Search

WO-2026096091-A1 - THREE-DIMENSIONAL RECONSTRUCTION (3DR) MESH COMPRESSION FOR EXTENDED REALITY (XR) SYSTEMS

WO2026096091A1WO 2026096091 A1WO2026096091 A1WO 2026096091A1WO-2026096091-A1

Abstract

Systems and techniques are described herein for determining three-dimensional (3D) data. For instance, a method for determining three-dimensional (3D) data is provided. The method may include determining a difference mesh and a triangle map based on a comparison of a first mesh with a second mesh, wherein the difference mesh includes 3D points of the second mesh that are different from 3D points of the first mesh, and wherein the triangle map indicates triangles that are similar in the second mesh and in the first mesh; encoding the difference mesh using a first encoder to generate an encoded difference mesh; encoding the triangle map using a second encoder to generate an encoded triangle map; and transmitting, to a computing device, the encoded difference mesh and the encoded triangle map.

Inventors

  • KRISHNAN, NIKHIL
  • ANAND, ANKIT
  • MELKOTE KRISHNAPRASAD, VINAY
  • REITMAYR, GERHARD
  • AJAYKUMAR, Sriram

Assignees

  • QUALCOMM INCORPORATED

Dates

Publication Date
20260507
Application Date
20250910
Priority Date
20250429

Claims (20)

  1. 1. An apparatus for determining three-dimensional (3D) data, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to: determine a difference mesh and a polygon map based on a comparison of a first mesh with a second mesh, wherein the difference mesh includes 3D points of the second mesh that are different from 3D points of the first mesh, and wherein the polygon map indicates polygons that are similar in the second mesh and in the first mesh; encode the difference mesh using a first encoder to generate an encoded difference mesh; encode the polygon map using a second encoder to generate an encoded polygon map; and transmit, to a computing device, the encoded difference mesh and the encoded polygon map.
  2. 2. The apparatus of claim 1, wherein the at least one processor is configured to: receive, from the computing device, virtual content based on a reconstructed mesh, wherein the reconstructed mesh is based on an instance of the first mesh, the difference mesh, and the polygon map; and display the virtual content.
  3. 3. The apparatus of claim 1. wherein the at least one processor is configured to: decode the encoded difference mesh using a first decoder; generate a reconstructed mesh based on the first mesh, the decoded difference mesh, and the polygon map; and Qualcomm Ref. No. 2500177WO 106 store the reconstructed mesh for use as an instance of a first mesh when an instance of a second mesh is obtained.
  4. 4. The apparatus of claim 3, wherein, to generate the reconstructed mesh, the at least one processor is configured to: add polygons of the first mesh to the decoded difference mesh based on the polygon map.
  5. 5. The apparatus of claim 4, wherein the at least one processor is configured to, when adding polygons of the first mesh to the decoded difference mesh, adjust 3D points of the polygons of the first mesh to correspond to corresponding points of the decoded difference mesh.
  6. 6. The apparatus of claim 1, wherein the difference mesh comprises 3D coordinates of the 3D points of the second mesh that are not in the first mesh.
  7. 7. The apparatus of claim 1, wherein the difference mesh comprises a list of polygons of the second mesh that are not in the first mesh, and each vertex of a polygon of the list of polygons comprises a 3D coordinate.
  8. 8. The apparatus of claim 1, wherein the polygon map comprises a map of polygons that are in the second mesh and in the first mesh.
  9. 9. The apparatus of claim 1, wherein the first encoder comprises a meshcompression encoder and wherein the second encoder comprises a lossless encoder.
  10. 10. The apparatus of claim 1, wherein, to determine the difference mesh and the polygon map, the at least one processor is configured to: compare 3D points of polygons of the second mesh with 3D points of the first mesh to identify matching points: compare polygons of the first mesh with the matching points to identify matching polygons; store the matching polygons in the polygon map; and Qualcomm Ref. No. 2500177WO 107 store non-matching polygons and 3D coordinates of vertices of the non-matching polygons in the difference mesh.
  11. 11. The apparatus of claim 10, wherein the at least one processor is configured to organize 3D points of the first mesh prior to comparing the 3D points of the polygons of the second mesh with the 3D points of the first mesh.
  12. 12. The apparatus of claim 10, wherein, to compare the 3D points of the polygons of the second mesh with 3D points of the first mesh, the at least one processor is configured to compare the 3D points of the polygons of the second mesh with a subset of the 3D points of the first mesh based on an indication that the subset of the 3D points of the first mesh included at least one 3D point that changed between the second mesh and the first mesh.
  13. 13. The apparatus of claim 12, wherein the at least one processor is configured to: divide the first mesh into a plurality of sub-meshes; and determine whether at least one 3D point within each sub-mesh of the plurality' of sub-meshes changed between the second mesh and the first mesh.
  14. 14. The apparatus of claim 1, wherein the at least one processor is configured to at least one of: remove isolated 3D points from the second mesh; remove degenerate polygons from the second mesh; remove duplicate 3D points from the second mesh; or remove duplicate polygons from the second mesh.
  15. 15. The apparatus of claim 1. wherein the at least one processor is configured to: determine, based on a distortion target metric, a search radius to use when comparing 3D points of second mesh with 3D points of the first mesh; and determine, based on the distortion target metric, a quantization parameter for the first encoder to use when encoding the difference mesh. Qualcomm Ref. No. 2500177WO 108
  16. 16. An apparatus for determining three-dimensional (3D) data, the apparatus comprising: at least one memory' ; and at least one processor coupled to the at least one memory and configured to: decode an encoded difference mesh to generate a difference mesh; decode an encoded polygon map to generate a polygon map; and generate a reconstructed mesh based on a first mesh, the difference mesh, and the polygon map.
  17. 17. The apparatus of claim 16, wherein the at least one processor is configured to: generate virtual content based on the reconstructed mesh; and transmit the virtual content to an extended-reality device.
  18. 18. The apparatus of claim 16, wherein, to generate the reconstructed mesh, the at least one processor is configured to: add polygons of the first mesh to the decoded difference mesh based on the polygon map.
  19. 19. The apparatus of claim 18, wherein the at least one processor is configured to, when adding polygons of the first mesh to the decoded difference mesh, adjust 3D points of the polygons of the first mesh to correspond to corresponding points of the decoded difference mesh.
  20. 20. The apparatus of claim 16, further comprising obtaining the encoded difference mesh and the encoded polygon map from an extended-reality device.

Description

Qualcomm Ref. No. 2500177WO 1 THREE-DIMENSIONAL RECONSTRUCTION (3DR) MESH COMPRESSION FOR EXTENDED REALITY (XR) SYSTEMS TECHNICAL FIELD [0001] The present disclosure generally relates to three-dimensional (3D) data. For example, aspects of the present disclosure include systems and techniques for processing (e.g., generating, compressing, encoding, decoding, decompressing, updating, reconstructing, and/or otherwise processing) 3D data. BACKGROUND [0002] Extended reality (XR) technologies can be used to present virtual content to users, and/or can combine real environments from the physical world and virtual environments to provide users with XR experiences. The term XR can encompass virtual reality (VR), augmented reality (AR), mixed reality (MR), and the like. XR systems can allow users to experience XR environments by overlaying virtual content onto a user’s view of a real-world environment. For example, an XR head-mounted device (HMD) may include a display that allows a user to view the user’s real-world environment through a display of the HMD (e.g., a transparent display). The XR HMD may display virtual content at the display in the user’s field of view overlaying the user’s view of their real-world environment. Such an implementation may be referred to as “see-through” XR. As another example, an XR HMD may include a scene-facing camera that may capture images of the user’s real-world environment. The XR HMD may modify or augment the images (e.g., adding virtual content) and display the modified images to the user. Such an implementation may be referred to as “pass through” XR or as “video see through (VST).” The user can generally change their view of the environment interactively, for example by tilting or moving the XR HMD. SUMMARY [0003] The following presents a simplified summary relating to one or more aspects disclosed herein. Thus, the following summary' should not be considered an extensive overview relating to all contemplated aspects, nor should the following summary’ be considered to identify key or critical elements relating to all contemplated aspects or Qualcomm Ref. No. 2500177WO 2 to delineate the scope associated with any particular aspect. Accordingly, the following summary presents certain concepts relating to one or more aspects relating to the mechanisms disclosed herein in a simplified form to precede the detailed description presented below. [0004] Systems and techniques are described for determining three-dimensional (3D) data. According to at least one example, a method is provided for determining three-dimensional (3D) data. The method includes: determining a difference mesh and a triangle map based on a comparison of a first mesh with a second mesh, wherein the difference mesh includes 3D points of the second mesh that are different from 3D points of the first mesh, and wherein the triangle map indicates triangles that are similar in the second mesh and in the first mesh; encoding the difference mesh using a first encoder to generate an encoded difference mesh; encoding the triangle map using a second encoder to generate an encoded triangle map; and transmitting, to a computing device, the encoded difference mesh and the encoded triangle map. [0005] In another example, an apparatus for determining three-dimensional (3D) data is provided that includes at least one memory and at least one processor (e.g., configured in circuitry) coupled to the at least one memory'. The at least one processor configured to: determine a difference mesh and a triangle map based on a comparison of a first mesh with a second mesh, wherein the difference mesh includes 3D points of the second mesh that are different from 3D points of the first mesh, and wherein the triangle map indicates triangles that are similar in the second mesh and in the first mesh; encode the difference mesh using a first encoder to generate an encoded difference mesh; encode the triangle map using a second encoder to generate an encoded triangle map; and cause at least one transmitter to transmit, to a computing device, the encoded difference mesh and the encoded triangle map. [0006] In another example, a non-transitory computer-readable medium is provided that has stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: determine a difference mesh and a triangle map based on a comparison of a first mesh with a second mesh, wherein the difference mesh includes 3D points of the second mesh that are different from 3D points of the first mesh, and wherein the triangle map indicates triangles that are similar in the second mesh and in the first mesh; encode the difference mesh using a first encoder Qualcomm Ref. No. 2500177WO 3 to generate an encoded difference mesh; encode the triangle map using a second encoder to generate an encoded triangle map; and cause at least one transmitter to transmit, to a computing device, the encoded difference mesh and the encoded triangle