US-12620138-B2 - Method and apparatus to quantize in multiple sub-meshes

US12620138B2US 12620138 B2US12620138 B2US 12620138B2US-12620138-B2

Abstract

A method comprises receiving an input mesh comprising a plurality of sub-meshes; determining, for a respective sub-mesh in the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh; determining, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh; determining, for the respective sub-mesh, a first bitdepth based on the coordinate and the sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh; determining a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes; quantizing the respective sub-mesh based on the second bitdepth to generate a respective quantized sub-mesh; and encoding the respective quantized sub-mesh.

Inventors

Thuong NGUYEN CANH
Xiaozhong Xu
Fang-yi CHAO
Shan Liu

Assignees

Tencent America LLC

Dates

Publication Date: 20260505
Application Date: 20240202

Claims (20)

1 . A method performed by at least one processor of an encoder, comprising: receiving an input mesh comprising a plurality of sub-meshes; determining, for a respective sub-mesh in the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh; determining, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh; determining, for the respective sub-mesh, a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh; determining a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes; quantizing the respective sub-mesh based on the second bitdepth to generate a respective quantized sub-mesh; and encoding the respective quantized sub-mesh.
2 . The method according to claim 1 , wherein the first bitdepth for the respective sub-mesh is determined based on a summation of each face of the respective sub-mesh.
3 . The method according to claim 1 , wherein the first bitdepth of the respective sub-mesh is limited to a bitdepth range comprising a predetermined minimum bitdepth and a predetermined maximum bitdepth.
4 . The method according to claim 3 , wherein the bitdepth range is determined based on a difference in size of the sub-mesh bounding box determined for the respective sub-mesh and a sub-mesh bounding box determined for each remaining sub-mesh from the plurality of sub-meshes.
5 . The method according to claim 1 , wherein the plurality of sub-meshes include a first sub-mesh that is connected to a second sub-mesh, and wherein the method further comprises: separating the first sub-mesh from the second sub-mesh, and adding at least one vertex that is included in both the first sub-mesh and the second sub-mesh.
6 . The method according to claim 5 , reducing a first bitdepth of the first sub-mesh based a difference between the second bitdepth and the first bitdepth of the first sub-mesh, the first bitdepth of the first sub-mesh determined according to an area of the first sub-mesh; and performing quantization on the first sub-mesh based on the reduced first bitdepth.
7 . The method according to claim 6 , reducing a third bitdepth of the second sub-mesh based on a difference between the second bitdepth and the third bitdepth, the third bitdepth of the second sub-mesh determined according to an area of the second sub-mesh; and performing quantization on the second sub-mesh based on the reduced third bitdepth.
8 . The method according to claim 1 , wherein the coordinate is a vertex coordinate.
9 . The method according to claim 1 , wherein the coordinate is a texture coordinate.
10 . An encoder comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: receiving code configured to cause the at least one processor to receive an input mesh comprising a plurality of sub-meshes, first determining code configured to cause the at least one processor to determine, for a respective sub-mesh from the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh, second determining code configured to cause the at least one processor to determine, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh, third determining code configured to cause the at least one processor to determine a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh, fourth determining code configured to cause the at least one processor to determine a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes, first quantizing code configured to cause the at least one processor to quantize the respective sub-mesh based on the second bitdepth, and encoding code configured to cause the at least one processor to encode the respective quantized sub-mesh.
11 . The encoder according to claim 10 , wherein the first bitdepth for the respective sub-mesh is determined based on a summation of each face of the respective.
12 . The encoder according to claim 10 , wherein the first bitdepth of the respective sub-mesh is limited to a bitdepth range comprising a predetermined minimum bitdepth and a predetermined maximum bitdepth.
13 . The encoder according to claim 12 , wherein the bitdepth range is determined based on a difference in size of the sub-mesh bounding box determined for the respective sub-mesh and a sub-mesh bounding box determined for each remaining sub-mesh from the plurality of sub-meshes.
14 . The encoder according to claim 10 , wherein the plurality of sub-meshes include a first sub-mesh that is connected to a second sub-mesh, and wherein the program code further includes: separating code configured to cause the at least one processor to separate the first sub-mesh from the second sub-mesh, and first adding code configured to cause the at least one processor to add at least one vertex that is included in both the first sub-mesh and the second sub-mesh.
15 . The encoder according to claim 14 , wherein the program code further includes: first reducing code configured to cause the at least one processor to reduce a first bitdepth of the first sub-mesh based a difference between the second bitdepth and the first bitdepth of the first sub-mesh, the first bitdepth of the first sub-mesh determined according to an area of the first sub-mesh, and second quantizing code configured to cause the at least one processor to configured to perform quantization on the first sub-mesh based on the reduced first bitdepth.
16 . The encoder according to claim 15 , wherein the program code further includes: second reducing code configured to cause the at least one processor to reduce a third bitdepth of the second sub-mesh based on a difference between the second bitdepth and the third bitdepth, the third bitdepth of the second sub-mesh determined according to an area of the second sub-mesh, and third quantizing code configured to cause the at least one processor to perform quantization on the second sub-mesh based on the reduced third bitdepth.
17 . The encoder according to claim 10 , wherein the coordinate is a vertex coordinate.
18 . The encoder according to claim 10 , wherein the coordinate is a texture coordinate.
19 . A non-transitory computer readable medium having instructions stored therein, which when executed method performed by at least one processor of an encoder cause the encoder to execute a method comprising: receiving an input mesh comprising a plurality of sub-meshes; determining, for a respective sub-mesh from the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh; determining, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh; determining, for each sub-mesh, a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh; determining a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes; quantizing the respective sub-mesh based on the second bitdepth; and encoding the respective quantized sub-mesh.
20 . The non-transitory computer readable medium according to claim 19 , wherein the first bitdepth for each sub-mesh is determined based on an area of a respective sub-mesh.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application claims priority from U.S. Provisional Application No. 63/443,666 filed on Feb. 6, 2023, the disclosure of which is incorporated herein by reference in its entirety. FIELD This disclosure is directed to a set of advanced video coding technologies. More specifically, the present disclosure is directed to quantizing multiple sub-meshes. BACKGROUND VMesh is an ongoing MPEG standard to compress meshes. VMesh separates an input mesh into a simplified base mesh and a displacement mesh. The base mesh is encoded with a lossy encoder (e.g., Draco). The remainder mesh is encoded with subdivision surface fitting and displacement encoding to exploit local characteristics. A complex scene may include multiple objects. However, in complex scenes, the largest object may be a background feature, which is often simple and can tolerate a higher quantization step size. In contrast, the main objects are at a smaller scale and suffer huge quantization error. A mesh may be split to multiple non-connected sub-meshes based one or more characteristics. Each sub-mesh may be quantized with a different quantization bitdepth to improve coding efficiency. Quantization error is significant at the boundary of connected meshes due to the difference in quantization. SUMMARY According to one or more embodiments, a method performed by at least one processor of an encoder, comprises receiving an input mesh comprising a plurality of sub-meshes; determining, for a respective sub-mesh in the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh; determining, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh; determining, for the respective sub-mesh, a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh; determining a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes; quantizing the respective sub-mesh based on the second bitdepth to generate a respective quantized sub-mesh; and encoding the respective quantized sub-mesh. According to one or more embodiments, an encoder comprises: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising receiving code configured to cause the at least one processor to receive an input mesh comprising a plurality of sub-meshes, first determining code configured to cause the at least one processor to determine, for a respective sub-mesh from the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh, second determining code configured to cause the at least one processor to determine, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh, third determining code configured to cause the at least one processor to determine a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh, fourth determining code configured to cause the at least one processor to determine a second bitdepth that is a maximum of the first bitdepth for the respective sub-mesh and a first bit depth determined for each of the remaining sub-meshes from the plurality of sub-meshes, first quantizing code configured to cause the at least one processor to quantize the respective sub-mesh based on the second bitdepth, and encoding code configured to cause the at least one processor to encode the respective quantized sub-mesh. According to one or more embodiments, anon-transitory computer readable medium having instructions stored therein, which when executed method performed by at least one processor of an encoder cause the encoder to execute a method comprising: receiving an input mesh comprising a plurality of sub-meshes; determining, for a respective sub-mesh from the plurality of sub-meshes, a sub-mesh bounding box that encloses the respective sub-mesh; determining, for the respective sub-mesh, a coordinate in the respective sub-mesh, wherein the determined coordinate is a minimum coordinate that has a smallest value in the respective sub-mesh; determining, for each sub-mesh, a first bitdepth based on the determined coordinate and the determined sub-mesh bounding box, wherein the first bitdepth is an adaptive bitdepth that varies according to an area of the respective sub-mesh; determining a second bitdepth that is a maximu