US-12626413-B2 - Coding of displacements using hierarchical coding at subdivision level for vertex mesh (V-MESH)

US12626413B2US 12626413 B2US12626413 B2US 12626413B2US-12626413-B2

Abstract

An apparatus includes a communication interface and a processor operably coupled to the communication interface. The processor is configured to subdivide an intermediate mesh-frame, created from a reconstructed first displacement field and a first subdivided mesh-frame, to create a second subdivided mesh-frame. The processor is also configured to generate a second displacement field based on the second subdivided mesh-frame and an original mesh-frame. The processor is further configured to encode the second displacement field into a bitstream.

Inventors

Rajan Laxman Joshi
Madhukar Budagavi

Assignees

SAMSUNG ELECTRONICS CO., LTD.

Dates

Publication Date: 20260512
Application Date: 20231013

Claims (14)

1 . An apparatus comprising: a communication interface; and a processor operably coupled to the communication interface, the processor configured to: subdivide an intermediate mesh-frame, created from a reconstructed first displacement field and a first subdivided mesh-frame, to create a second subdivided mesh-frame; extract a set of vertex positions from the second subdivided mesh-frame and generate a second displacement field based on the second subdivided mesh-frame and an original mesh-frame, wherein the extracted set of vertex positions of the second subdivided mesh-frame comprise only new vertex positions generated by the subdivision of the intermediate mesh-frame; and encode the second displacement field into a bitstream.
2 . The apparatus of claim 1 , wherein, to create the intermediate mesh-frame, the processor is further configured to: decode an encoded first displacement field to obtain the reconstructed first displacement field; and combine the reconstructed first displacement field with the first subdivided mesh-frame to create the intermediate mesh-frame.
3 . The apparatus of claim 2 , wherein the processor is further configured to: subdivide a reconstructed base mesh-frame to create the first subdivided mesh-frame, wherein the reconstructed base mesh-frame is associated with the original mesh-frame; generate a first displacement field based on the first subdivided mesh-frame and the original mesh-frame; and encode the first displacement field to obtain the encoded first displacement field and write the encoded first displacement field into the bitstream.
4 . The apparatus of claim 3 , wherein the reconstructed base mesh-frame is created by decoding an encoding of an original base mesh-frame.
5 . The apparatus of claim 1 , wherein, to generate the second displacement field, the processor is further configured to: determine a difference between the extracted set of vertex positions of the second subdivided mesh-frame and a set of nearest vertex positions of the original mesh-frame.
6 . A method comprising: subdividing an intermediate mesh-frame, created from a reconstructed first displacement field and a first subdivided mesh-frame, to create a second subdivided mesh-frame; extracting a set of vertex positions from the second subdivided mesh-frame and generating a second displacement field based on the second subdivided mesh-frame and an original mesh-frame, wherein the extracted set of vertex positions of the second subdivided mesh-frame comprise only new vertex positions generated by the subdivision of the intermediate mesh-frame; and encoding the second displacement field into a bitstream.
7 . The method of claim 6 , further comprising: decoding an encoded first displacement field to obtain the reconstructed first displacement field; and combining the reconstructed first displacement field with the first subdivided mesh-frame to create the intermediate mesh-frame.
8 . The method of claim 7 , further comprising: subdividing a reconstructed base mesh-frame to create the first subdivided mesh-frame, wherein the reconstructed base mesh-frame is associated with the original mesh-frame; generating a first displacement field based on the first subdivided mesh-frame and the original mesh-frame; and encoding the first displacement field to obtain the encoded first displacement field and writing the encoded first displacement field into the bitstream.
9 . The method of claim 8 , wherein the reconstructed base mesh-frame is created by decoding an encoding of an original base mesh-frame.
10 . The method of claim 6 , wherein, to generate the second displacement field, the method further comprises: determining a difference between the extracted set of vertex positions of the second subdivided mesh-frame and a set of nearest vertex positions of the original mesh-frame.
11 . An apparatus comprising: a communication interface; and a processor operably coupled to the communication interface, the processor configured to: receive a compressed bitstream of a plurality of displacement fields; decode the compressed bitstream and reconstruct the plurality of displacement fields; and combine each one of the reconstructed plurality of displacement fields with one of a plurality of subdivided mesh-frames to reconstruct a mesh-frame, wherein the processor is further configured to create a second subdivided mesh-frame using a first subdivided mesh-frame of the plurality of subdivided mesh-frames and extract a set of vertex positions from the second subdivided mesh-frame, wherein the extracted set of vertex positions of the second subdivided mesh-frame comprise only new vertex positions generated by a subdivision of a first intermediate mesh-frame.
12 . The apparatus of claim 11 , wherein: each one of the plurality of displacement fields is associated with one of a plurality of subdivision levels; and each one of the plurality of subdivided mesh-frames is associated with one of the plurality of subdivision levels.
13 . The apparatus of claim 11 , wherein, to combine each one of the reconstructed plurality of displacement fields with one of the plurality of subdivided mesh-frames to reconstruct the mesh-frame, the processor is further configured to: combine a reconstructed first displacement field of the reconstructed plurality of displacement fields with the first subdivided mesh-frame of the plurality of subdivided mesh-frames to create the first intermediate mesh-frame, wherein the first subdivided mesh-frame is created from a reconstructed base mesh-frame decoded from the compressed bitstream; subdivide the first intermediate mesh-frame to create the second subdivided mesh-frame of the plurality of subdivided mesh-frames; combine a reconstructed second displacement field of the reconstructed plurality of displacement fields with the second subdivided mesh-frame to create a second intermediate mesh-frame; and combine the first intermediate mesh-frame and the second intermediate mesh-frame.
14 . The apparatus of claim 13 , wherein, to combine the reconstructed second displacement field of the reconstructed plurality of displacement fields with the second subdivided mesh-frame to create the second intermediate mesh-frame, the processor is further configured to: add the reconstructed second displacement field to the extracted set of vertex positions of the second subdivided mesh-frame.

Description

CROSS-REFERENCE TO RELATED APPLICATION AND PRIORITY CLAIM This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63/417,602 filed on Oct. 19, 2022, which is hereby incorporated by reference in its entirety. TECHNICAL FIELD This disclosure relates generally to multimedia devices and processes. More specifically, this disclosure relates to improved coding of displacements using hierarchical coding at the subdivision level for vertex mesh (V-MESH). BACKGROUND Three hundred sixty degree (360°) video and three dimensional (3D) volumetric video are emerging as new ways of experiencing immersive content due to the ready availability of powerful handheld devices such as smartphones. While 360° video enables an immersive “real life,” “being-there,” experience for consumers by capturing the 360° outside-in view of the world, 3D volumetric video can provide a complete six degrees of freedom (DoF) experience of being immersed and moving within the content. Users can interactively change their viewpoint and dynamically view any part of the captured scene or object they desire. Display and navigation sensors can track head movement of a user in real-time to determine the region of the 360° video or volumetric content that the user wants to view or interact with. Multimedia data that is 3D in nature, such as point clouds or 3D polygonal meshes, can be used in the immersive environment. SUMMARY This disclosure provides improved coding of displacements using hierarchical coding at the subdivision level for V-MESH. In a first embodiment, an apparatus includes a communication interface and a processor operably coupled to the communication interface The processor is configured to subdivide an intermediate mesh-frame, created from a reconstructed first displacement field and a first subdivided mesh-frame, to create a second subdivided mesh-frame. The processor is also configured to generate a second displacement field based on the second subdivided mesh-frame and an original mesh-frame. The processor is also configured to encode the second displacement field into a bitstream. In a second embodiment, a method includes subdividing an intermediate mesh-frame, created from a reconstructed first displacement field and a first subdivided mesh-frame, to create a second subdivided mesh-frame. The method also includes generating a second displacement field based on the second subdivided mesh-frame and an original mesh-frame. The method also includes encoding the second displacement field into a bitstream. In a third embodiment, an apparatus includes a communication interface and a processor operably coupled to the communication interface. The processor is configured to receive a compressed bitstream of a plurality of displacement fields. The processor is also configured to decode the compressed bitstream and reconstruct the plurality of displacement fields. The processor is also configured to combine each one of the reconstructed plurality of displacement fields with one of a plurality of subdivided mesh-frames to reconstruct a mesh-frame. Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims. Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document. The term “couple” and its derivatives refer to any direct or indirect communication between two or more elements, whether or not those elements are in physical contact with one another. The terms “transmit,” “receive,” and “communicate,” as well as derivatives thereof, encompass both direct and indirect communication. The terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation. The term “or” is inclusive, meaning and/or. The phrase “associated with,” as well as derivatives thereof, means to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, have a relationship to or with, or the like. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware or a combination of hardware and software and/or firmware. The functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. The phrase “at least one of,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and only one item in the list may be needed. For example, “at least one of: A, B, and C” includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C. Moreover, various functions described below can be implemented or supported by one or more computer pr