US-20260127771-A1 - SPECTRAL COMPRESSION FOR DYNAMIC MESH ENCODING

Abstract

Apparatuses and methods are disclosed for encoding and decoding mesh data. Encoding techniques are disclosed including coding a mesh into a bitstream. The coding includes generating a base mesh from the mesh, obtaining connectivity and geometry data of the base mesh. Then, subdividing the base mesh, obtaining connectivity and geometry data of the subdivided mesh. Coding proceeds by generating GFT coefficients, based on a Graph Fourier Transform (GFT), using displacement data, and then, coding into the bitstream the coefficients and connectivity data of the base mesh. Decoding techniques are disclosed including decoding the mesh from the bitstream. The decoding includes decoding from the bitstream connectivity data of the base mesh and subdividing the base mesh, obtaining connectivity data of the subdivided mesh. Decoding proceeds by decoding from the bitstream the GFT coefficients and reconstructing the mesh based on the decoded connectivity data of the subdivided mesh and the decoded coefficients.

Inventors

• Jean-Eudes Marvie
• Olivier Mocquard
• Maja Krivokuca
• Julien Ricard

Assignees

• INTERDIGITAL CE PATENT HOLDINGS, SAS

Dates

Publication Date

20260507

Application Date

20231004

Priority Date

20221019

Claims (20)

1. 1 . A method for encoding mesh data, comprising: receiving a mesh sequence; and coding a mesh of the sequence into a bitstream, the coding comprises: generating a base mesh from the mesh, obtaining connectivity data and geometry data associated with vertices of the base mesh, subdividing the base mesh, obtaining connectivity data and geometry data associated with vertices of the subdivided base mesh, computing displacement data, representing spatial differences between vertices of the subdivided base mesh and corresponding vertices of the mesh, generating GFT coefficients, based on a Graph Fourier Transform (GFT), using the computed displacement data, the generated GFT coefficients representative of geometry data of the mesh; and coding into the bitstream the GFT coefficients and connectivity data of the base mesh.
2. 2 . The method according to claim 1 , wherein, in a first operational mode, the generating of GFT coefficients comprises: computing, based on the displacement data, geometry data including vertices of the mesh respective of vertices of the subdivided base mesh; and transforming, based on the GFT, the computed geometry data into the GFT coefficients; and further comprising: coding into the bitstream a syntax element, signaling the first operational mode.
3. 3 . The method according to claim 1 , wherein, in a second operational mode, the generating of GFT coefficients comprises: transforming, based on the GFT, the displacement data into the GFT coefficients; and further comprising: coding into the bitstream the geometry data of the base mesh, and coding into the bitstream a syntax element, signaling the second operational mode.
4. 4 . The method according to claim 1 , wherein, in a third operational mode, the generating of GFT coefficients comprises: removing, from the subdivided base mesh, vertices that correspond to vertices of the base mesh generating a reduced mesh, computing, based on the displacement data, geometry data including vertices of the mesh respective of vertices of the reduced mesh, and transforming, based on the GFT, the computed geometry data of the mesh-into the GFT coefficients; and further comprising: coding into the bitstream the geometry data of the base mesh, and coding into the bitstream a syntax element, signaling the third operational mode.
5. 5 . The method according to claim 1 , wherein, in a fourth operational mode, the generating of GFT coefficients comprises: removing, from the subdivided base mesh, vertices that correspond to vertices of the base mesh generating a reduced mesh, transforming, based on the GFT, part of the displacement data that is respective of vertices of the reduced mesh; into the GFT coefficients; and further comprising: coding into the bitstream the geometry data of the base mesh; and coding into the bitstream a syntax element, signaling the fourth operational mode.
6. 6 . (canceled)
7. 7 . The method according to claim 1 , wherein: the coding of the mesh comprises coding mesh patches; the subdividing of the base mesh comprises subdividing respective faces of the base mesh to generate the mesh patches, obtaining respective connectivity data sets and respective geometry data sets associated with vertices of the mesh patches; the generating of GFT coefficients comprises generating GFT coefficient sets, based on respective GFTs, the GFT coefficient sets represent the respective mesh patches; and the coding of GFT coefficients into the bitstream comprises coding the GFT coefficient sets.
8. 8 . The method according to claim 7 , wherein the subdividing of the respective faces of the base mesh to generate the mesh patches comprises: subdividing of the faces of the base mesh according to respective subdivision depths; and coding into the bitstream the respective subdivision depths.
9. 9 . The method according to claim 8 , further comprising: locally adapting the subdivision depth of neighboring patches of the mesh patches so that common edges of the neighboring patches have the same number of vertices.
10. 10 . A method for decoding mesh data, comprising: receiving a bitstream of a coded mesh sequence; and decoding a mesh from the sequence, the decoding comprises: decoding from the bitstream connectivity data associated with vertices of a base mesh, subdividing the base mesh, obtaining connectivity data associated with vertices of a subdivided base mesh, decoding from the bitstream GFT coefficients representative of geometry data of the mesh, the GFT coefficients generated by an encoder based on a GFT using displacement data representing spatial differences between vertices of the subdivided base mesh and corresponding vertices of the mesh, and reconstructing the mesh based on the connectivity data of the subdivided base mesh and the decoded GFT coefficients.
11. 11 . The method according to claim 10 , further comprising: decoding from the bitstream a syntax element, signaling an operational mode; and responsive to the operational mode being a first operational mode: inverse transforming, based on the GFT, the GFT coefficients obtaining geometry data including vertices of the mesh respective of vertices of the subdivided base mesh, and further reconstructing the mesh based on the obtained geometry data.
12. 12 . The method according to claim 10 , further comprising: decoding from the bitstream a syntax element, signaling an operational mode; and responsive to the operational mode being a second operational mode: decoding from the bitstream geometry data of the base mesh, inverse transforming, based on the GFT, the GFT coefficients obtaining displacement data, and further reconstructing the mesh based on the geometry data of the base mesh and the obtained displacement data.
13. 13 . The method according to claim 10 , further comprising: decoding from the bitstream a syntax element, signaling an operational mode; and responsive to the operational mode being a third operational mode: decoding from the bitstream the geometry data of the base mesh, removing from the subdivided base mesh vertices that correspond to vertices of the base mesh generating a reduced mesh, inverse transforming, based on the GFT, the GFT coefficients obtaining geometry data including vertices of the mesh respective of vertices of the reduced mesh, and further reconstructing the mesh based on the obtained geometry data.
14. 14 . The method according to claim 10 , further comprising: decoding from the bitstream a syntax element, signaling an operational mode; and responsive to the operational mode being a fourth operational mode: decoding from the bitstream the geometry data of the base mesh, removing from the subdivided base mesh vertices that corresponds to vertices of the base mesh generating a reduced mesh, inverse transforming, based on the GFT, the GFT coefficients obtaining displacement data that is respective of vertices of the reduced mesh, further reconstructing the mesh based on the geometry data of the base mesh and the obtained displacement data.
15. 15 - 18 . (canceled)
16. 19 . An apparatus for encoding mesh data, comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the apparatus to: receive a mesh sequence, and code a mesh of the sequence into a bitstream, the coding comprises: generating a base mesh from the mesh, obtaining connectivity data and geometry data associated with vertices of the base mesh, subdividing the base mesh, obtaining connectivity data and geometry data associated with vertices of the subdivided base mesh, computing displacement data, representing spatial differences between vertices of the subdivided base mesh and corresponding vertices of the mesh, generating GFT coefficients, based on a Graph Fourier Transform (GFT), using the computed displacement data, the generated GFT coefficients representative of geometry data of the mesh; and coding into the bitstream the GFT coefficients and connectivity data of the base mesh.
17. 20 . The apparatus according to claim 19 , wherein in a first operational mode, the generating of GFT coefficients comprises: compute, based on the displacement data, geometry data including vertices of the mesh respective of vertices of the subdivided base mesh; and transform, based on the GFT, the computed geometry data into the GFT coefficients; and the instructions further cause the apparatus to: code into the bitstream a syntax element, signaling the first operational mode.
18. 21 . The apparatus according to claim 19 , wherein in a second operational mode, the generating of GFT coefficients comprises: transform, based on the GFT, the displacement data into the GFT coefficients; and the instructions further cause the apparatus to: code into the bitstream the geometry data of the base mesh; and code into the bitstream a syntax element, signaling the second operational mode.
19. 22 - 23 . (canceled)
20. 24 . An apparatus for decoding mesh data, comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the apparatus to: receive a bitstream of a coded mesh sequence, and decode a mesh from the sequence, the decoding comprises: decoding from the bitstream connectivity data associated with vertices of a base mesh, subdividing the base mesh, obtaining connectivity data associated with vertices of a subdivided base mesh, decoding from the bitstream GFT coefficients representative of geometry data of the mesh, the GFT coefficients generated by an encoder based on a GFT using displacement data representing spatial differences between vertices of the subdivided base mesh and corresponding vertices of the mesh, and reconstructing the mesh based on the connectivity data of the subdivided base mesh and the decoded GFT coefficients.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of European Application No. 22306582.2, filed on Oct. 19, 2022, which is incorporated herein by reference in its entirety. BACKGROUND A significant amount of data is required for high quality representation and rendering of content modeled by dynamic meshes. Compression techniques are instrumental in distributing such content to consumers. Generally, the computational complexity of encoding and decoding the geometry and topology of dynamic meshes is proportional to the size of these meshes and compression efficiency depends on how well a coding technique reduces spatiotemporal redundancy. The former can be addressed by techniques that are scalable, whereas the latter can be addressed by techniques that take advantage of spatiotemporal correlations that are typically present in dynamic mesh data. SUMMARY Apparatuses and methods are disclosed herein for encoding and decoding time-varying textured meshes. Recently, the MPEG 3D Graphics Coding (MPEG-3DGC) group called for proposals (CfP) for codec technologies relating to the compression of time-varying volumetric meshes (V-Mesh). See, CfP for Dynamic Mesh Coding, ISO/IEC JTC 1/SC 29/WG 7, 2021. In response, the solution proposed by Mammou et al. was selected to become the MPEG V-Mesh Test Model that will be used as a basis for future development of this standard. See, K. Mammou, J. Kim, A. Tourapis, D. Podborski and K. Kolarov, “MPEG input document m59281-v4-[V-CG] Apple's Dynamic Mesh Coding CfP Response,” ISO/IEC JTC 1/SC 29/WG 7, 2022 (“Mammou”). Aspects disclosed herein refine the MPEG V-Mesh Test Model (referred to herein as “the test model”) to extend its compression capabilities. Aspects disclosed in the present disclosure describe methods for encoding mesh data. The methods comprise receiving a mesh sequence and coding a mesh of the sequence into a bitstream. The coding of a mesh includes generating a base mesh from the mesh, obtaining connectivity data and geometry data associated with vertices of the base mesh. Then, subdividing the base mesh, obtaining connectivity data and geometry data associated with vertices of the subdivided mesh. The coding proceeds by computing displacement data, representing spatial differences between vertices of the subdivided mesh and corresponding vertices of the mesh; generating GFT coefficients, based on a Graph Fourier Transform (GFT), using the computed displacement data; and coding into the bitstream the GFT coefficients and connectivity data of the base mesh. Aspects disclosed in the present disclosure also describe methods for decoding mesh data. The methods comprise receiving a bitstream of a coded mesh sequence and decoding a mesh from the sequence. The decoding of a mesh includes decoding from the bitstream connectivity data associated with vertices of a base mesh. Then, subdividing the base mesh, obtaining connectivity data associated with vertices of a subdivided mesh. The decoding proceeds by decoding from the bitstream GFT coefficients, generated by an encoder based on a GFT using displacement data representing spatial differences between vertices of the subdivided mesh and corresponding vertices of the mesh; and reconstructing the mesh based on the connectivity data of the subdivided mesh and the decoded GFT coefficients. Aspects disclosed in the present disclosure describe an apparatus for encoding mesh data. The apparatus comprises at least one processor and memory storing instructions. The instructions, when executed by the at least one processor, cause the apparatus to receive a mesh sequence and to code a mesh of the sequence into a bitstream. The coding of a mesh includes generating a base mesh from the mesh, obtaining connectivity data and geometry data associated with vertices of the base mesh. Then, subdividing the base mesh, obtaining connectivity data and geometry data associated with vertices of the subdivided mesh. The coding proceeds by computing displacement data, representing spatial differences between vertices of the subdivided mesh and corresponding vertices of the mesh; generating GFT coefficients, based on a Graph Fourier Transform (GFT), using the computed displacement data; and coding into the bitstream the GFT coefficients and connectivity data of the base mesh. Aspects disclosed in the present disclosure also describe an apparatus for decoding mesh data. The apparatus comprises at least one processor and memory storing instructions. The instructions, when executed by the at least one processor, cause the apparatus to receive a bitstream of a coded mesh sequence and to decode a mesh from the sequence. The decoding of a mesh includes decoding from the bitstream connectivity data associated with vertices of a base mesh. Then, subdividing the base mesh, obtaining connectivity data associated with vertices of a subdivided mesh. The decoding proceeds by decoding from the bitstream GFT coefficients, generated by a