US-12621484-B2 - Signaling duplicated vertices removal in mesh motion vector coding

US12621484B2US 12621484 B2US12621484 B2US 12621484B2US-12621484-B2

Abstract

A method of decoding a dynamic mesh sequence, the method includes: receiving a coded bitstream comprising the dynamic mesh sequence in a first frame and a second frame that occurs later than the first frame; reconstructing the dynamic mesh in the first frame; determining whether a duplicate vertex removal mode is enabled in which a motion vector of a vertex in the dynamic mesh second that is a duplicate of a vertex in the dynamic mesh sequence in the first frame; based on determining that the duplicate vertex removal mode is enabled, decoding a motion vector for each vertex in the second frame that is not a duplicate of a vertex in the reconstructed first frame; and reconstruct the dynamic mesh sequence in the second frame using the reconstructed dynamic mesh in the first frame and each decoded motion vector.

Inventors

Jun Tian
Shan Liu
Xiaozhong Xu

Assignees

Tencent America LLC

Dates

Publication Date: 20260505
Application Date: 20240312

Claims (15)

1 . A method of decoding a dynamic mesh sequence, the method comprising: receiving a coded bitstream comprising the dynamic mesh sequence in a first frame and a second frame that occurs later than the first frame; reconstructing the dynamic mesh in the first frame; determining, based on a statistical calculation performed to determine a percentage of vertices that are in the reconstructed first frame, whether a duplicate vertex removal mode is enabled in which a motion vector of a vertex in the dynamic mesh second that is a duplicate of a vertex in the dynamic mesh sequence in the first frame; based on determining that the duplicate vertex removal mode is enabled, decoding a motion vector for each vertex in the second frame that is not a duplicate of a vertex in the reconstructed first frame; and reconstruct the dynamic mesh sequence in the second frame using the reconstructed dynamic mesh in the first frame and each decoded motion vector.
2 . The method according to claim 1 , wherein the duplicate vertex removal mode is enabled based on a determination that a percentage of duplicate vertices in the first reference frame is greater than or equal to a threshold.
3 . The method of claim 1 , further comprising: based on determining that the duplicate vertex removal mode is not enabled, decoding a motion vector for each vertex in the second frame regardless of whether the vertex in the second frame is a duplicate of a vertex in the first frame.
4 . The method of claim 1 , wherein a vertex in the second frame is a duplicate of a vertex in the first frame based on a determination that the vertex in the second frame has the same coordinates as the vertex in the first frame.
5 . The method of claim 1 , wherein the coded video bitstream comprises a flag that indicates that the vertex in the second frame is a duplicate of a vertex in the first frame.
6 . A method of encoding a dynamic mesh sequence, the method comprising: receiving a first frame comprising the dynamic mesh sequence and a second frame comprising the dynamic mesh sequence, the second frame occurring after the first frame; determining a motion vector between each vertex of the dynamic mesh sequence in the first frame and a corresponding vertex of the dynamic mesh sequence in the second frame; generating a coded bitstream that includes each determined motion vector, an indication whether a vertex of the dynamic mesh sequence in the second frame is a duplicate of a vertex of the dynamic mesh sequence in the first frame, and an indication whether a duplicate vertex removal mode is enabled in which a motion vector of a vertex in the dynamic mesh sequence of the second frame that is a duplicate of a vertex in the dynamic mesh sequence in the first frame is skipped in a reconstruction of the second frame; and setting a flag indicating whether a duplicate removal mode is enabled based on a statistical calculation that determines a percentage of vertices of the dynamic mesh sequence in the first frame and vertices of the dynamic mesh sequence in the second frame that are duplicates, the flag included in the coded bitstream.
7 . The method of claim 6 , wherein the flag is set to enable the duplicate removal mode based on a determination that the percentage of vertices of the dynamic mesh sequence in the first frame and vertices of the dynamic mesh sequence in the second frame that are duplicates is greater than a threshold.
8 . The method according to claim 6 , wherein a motion vector for each vertex in the second frame is decoded based on a determination the duplicate vertex removal mode is not enabled regardless of whether the vertex in the second frame is a duplicate of a vertex in the first frame.
9 . The method of claim 6 , wherein a vertex in the second frame is a duplicate of a vertex in the first frame based on a determination that the vertex in the second frame has the same coordinates as the vertex in the first frame.
10 . The method according to claim 6 , wherein the coded bitstream includes a flag indicating whether a vertex of the dynamic mesh sequence in the second frame is a duplicate of a vertex of the dynamic mesh sequence in the first frame.
11 . A method of processing a dynamic mesh sequence, the method comprising: reconstructing a dynamic mesh sequence in a second frame that occurs after the first frame based on a coded bitstream that includes the first frame and the second frame, wherein the first frame is reconstructed, wherein the coded video bitstream includes an indication of whether a duplicate vertex removal mode is enabled in which a motion vector of a vertex in the dynamic mesh second that is a duplicate of a vertex in the dynamic mesh sequence in the first frame, wherein determining whether the duplicate vertex removal mode is enabled is based on a statistical calculation performed to determine a percentage of vertices that are in the reconstructed first frame, wherein based on determining that the duplicate vertex removal mode is enabled, a motion vector for each vertex in the second frame that is not a duplicate of a vertex in the reconstructed first frame is decoded, and wherein the dynamic mesh sequence in the second frame is reconstructed using the reconstructed first frame and each decoded motion vector.
12 . The method according to claim 11 , wherein the duplicate vertex removal mode is enabled based on a determination that a percentage of duplicate vertices in the first reference frame is greater than or equal to a threshold.
13 . The method of claim 11 , wherein determining whether the duplicate vertex removal mode is enabled is based on a determination that a flag in the coded bitstream indicates that the duplicate vertex removal mode is enabled.
14 . The method of claim 11 , wherein based on determining that the duplicate vertex removal mode is not enabled, a motion vector for each vertex in the second frame is decoded regardless of whether the vertex in the second frame is a duplicate of a vertex in the first frame.
15 . The method of claim 1 , wherein a vertex in the second frame is a duplicate of a vertex in the first frame based on a determination that the vertex in the second frame has the same coordinates as the vertex in the first frame.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application claims priority from U.S. Provisional Application No. 63/451,888 filed on Mar. 13, 2023, the disclosure of which is incorporated herein by reference in its entirety. FIELD This disclosure is directed to a set of advanced video coding technologies. More specifically, the present disclosure is directed to signaling duplicated vertices removal in mesh motion vector coding. BACKGROUND The advances in 3D capture, modeling, and rendering have promoted the ubiquitous presence of 3D contents across several platforms and devices. Nowadays, it is possible to capture a baby's first step in one continent and allow the grandparents to see (and maybe interact) and enjoy a full immersive experience with the child in another continent. Nevertheless, in order to achieve such realism, models are becoming ever more sophisticated, and a significant amount of data is linked to the creation and consumption of those models. 3D meshes are widely used to represent such immersive contents. A mesh is composed of several polygons that describe the surface of a volumetric object. Each polygon is defined by its vertices in 3D space and the information of how the vertices are connected, referred to as connectivity information. Optionally, vertex attributes, such as colors, normals, etc., could be associated with the mesh vertices. Attributes could also be associated with the surface of the mesh by exploiting mapping information that parameterizes the mesh with 2D attribute maps. Such mapping is usually described by a set of parametric coordinates, referred to as UV coordinates or texture coordinates, associated with the mesh vertices. 2D attribute maps are used to store high resolution attribute information such as texture, normals, displacements etc. Such information could be used for various purposes such as texture mapping and shading. A dynamic mesh sequence may require a large amount of data since it may consist of a significant amount of information changing over time. Therefore, efficient compression technologies are required to store and transmit such contents. Mesh compression standards IC, MESHGRID, FAMC were previously developed by MPEG to address dynamic meshes with constant connectivity and time varying geometry and vertex attributes. However, these standards do not take into account time varying attribute maps and connectivity information. DCC (Digital Content Creation) tools usually generate such dynamic meshes. In counterpart, it is challenging for volumetric acquisition techniques to generate a constant connectivity dynamic mesh, especially under real time constraints. This type of contents is not supported by the existing standards. MPEG is planning to develop a new mesh compression standard to directly handle dynamic meshes with time varying connectivity information and optionally time varying attribute maps. This standard targets lossy, and lossless compression for various applications, such as real-time communications, storage, free viewpoint video, AR and VR. Functionalities such as random access and scalable/progressive coding are also considered. Mesh geometry information consists of vertex connectivity information, 3D coordinates, and 2D texture coordinates, etc. The compression of vertex 3D coordinates, which is also called vertex position, is very important, as in most cases, it takes up significant portion of the entire geometry related data. SUMMARY According to one or more embodiments, a method of decoding a dynamic mesh sequence, comprises: receiving a coded bitstream comprising the dynamic mesh sequence in a first frame and a second frame that occurs later than the first frame; reconstructing the dynamic mesh in the first frame; determining whether a duplicate vertex removal mode is enabled in which a motion vector of a vertex in the dynamic mesh second that is a duplicate of a vertex in the dynamic mesh sequence in the first frame; based on determining that the duplicate vertex removal mode is enabled, decoding a motion vector for each vertex in the second frame that is not a duplicate of a vertex in the reconstructed first frame; and reconstruct the dynamic mesh sequence in the second frame using the reconstructed dynamic mesh in the first frame and each decoded motion vector. According to one or more embodiments, a method of encoding a dynamic mesh comprises: receiving a first frame comprising the dynamic mesh sequence and a second frame comprising the dynamic mesh sequence, the second frame occurring after the first frame; determining a motion vector between each vertex of the dynamic mesh sequence in the first frame and a corresponding vertex of the dynamic mesh sequence in the second frame; generating a coded bitstream that includes each determined motion vector, an indication whether a vertex of the dynamic mesh sequence in the second frame is a duplicate of a vertex of the dynamic mesh sequence in the first frame, and an indication whether a d