EP-4738838-A1 - KEYPOINTS AND LANDMARKS IN AVATAR REPRESENTATION

EP4738838A1EP 4738838 A1EP4738838 A1EP 4738838A1EP-4738838-A1

Abstract

Systems, methods, and instrumentalities are disclosed for avatar keypoints and landmarks representation. A file (e.g., an MPEG avatar representation format (MARF) file) may be received. The file may include a plurality of assets. The file may indicate keypoint and landmark component information. An asset may be selected from the plurality of assets in the file based on parsing the file. A container may be determined based on the selected asset and the keypoint and landmark component information. At least one of a keypoint or a landmark may be determined based on the determined container. At least one of a keypoint or a landmark may animate an avatar based on the at least one of the keypoint or landmark.

Inventors

COVA REGATEIRO, João Pedro
GOSSELIN, Philippe Henri
LE CLERC, FRANCOIS

Assignees

InterDigital CE Patent Holdings, SAS

Dates

Publication Date: 20260506
Application Date: 20241031

Claims (15)

A device comprising: a processor configured at least to: receive a file comprising a representation of an avatar, wherein the file comprises a plurality of assets, and wherein the file indicates keypoint and landmark component information; select an asset from the plurality of assets in the File based on parsing the File; determine a container based on the selected asset and the keypoint and landmark component information; decode at least one of a keypoint or a landmark based on the determined container; and animate an avatar based on the at least one of the keypoint or landmark.
The device of claim 1, wherein the file comprises a keypoints property, wherein the keypoints property indicates the keypoint and landmark component information, wherein the keypoint property indicates a list of keypoint sets and indicates at least one of a name associated with each keypoint set in the list of keypoint sets or an anchor associated with each keypoint set in the list of keypoint sets.
The device of claim 1, wherein the file comprises an anchors property, wherein the anchors property indicates the keypoint and landmark component information, wherein the anchors property indicates a list of anchors and indicates at least one of an identification, a name, or a transform associated with each anchor in the list of anchors.
The device of claim 3, wherein the anchors property further indicates a binding list and a binding weight, wherein the binding list indicates a list of anchors identifiers.
The device of claim 3, wherein the anchors property further indicates at least one of a joints indication or a vertices indication, wherein the joints indication indicates a location of a file associated with a list of vertex identifiers, and wherein the vertices indication indicates a location of a file associated with a list of vertex identifiers.
The device of claim 1, wherein the file comprises a landmark set property, wherein the landmark set property indicates the keypoint and landmark component information, wherein the landmark set property indicates a list of landmark sets for a mesh, and wherein the landmark set property indicates at least one of a name, a description, a type, a landmark identifier, or a mesh associated with each landmark set in the list of landmark sets.
The device of claim 1, wherein the file comprises a landmarks property, wherein the landmarks property indicates the keypoint and landmark component information, wherein the landmarks property indicates a list of landmarks, and wherein the landmarks property indicates at least one of an identifier, a name, a description, a type, a vertex, a face, or a weight associated with each landmark in the list of landmarks.
The device of claim 1, wherein the file comprises at least one of a keypoint set property or a landmark set property, wherein the keypoint set property indicates a keypoint set identifier associated with a controller set associated with the asset, and wherein the landmark set property indicates a landmark set identifier associated with a landmark set associated with the asset.
The device of claim 1, wherein the file comprising the representation of the avatar is an MPEG avatar representation format (MARF) file.
A method comprising: receiving a file comprising a representation of an avatar, wherein the file comprises a plurality of assets, and wherein the file indicates keypoint and landmark component information; selecting an asset from the plurality of assets in the File based on parsing the File; determining a container based on the selected asset and the keypoint and landmark component information; decoding at least one of a keypoint or a landmark based on the determined container; and animating an avatar based on the at least one of the keypoint or landmark.
The method of claim 10, wherein the file comprises a keypoints property, wherein the keypoints property indicates the keypoint and landmark component information, wherein the keypoint property indicates a list of keypoint sets and indicates at least one of a name associated with each keypoint set in the list of keypoint sets or an anchor associated with each keypoint set in the list of keypoint sets.
The method of claim 10, wherein the file comprises an anchors property, wherein the anchors property indicates the keypoint and landmark component information, wherein the anchors property indicates a list of anchors and indicates at least one of an identification, a name, or a transform associated with each anchor in the list of anchors, wherein the anchors property further indicates a binding list and a binding weight, wherein the binding list indicates a list of anchors identifiers, and wherein the anchors property further indicates at least one of a joints indication or a vertices indication, wherein the joints indication indicates a location of a file associated with a list of vertex identifiers, and wherein the vertices indication indicates a location of a file associated with a list of vertex identifiers.
The method of claim 10, wherein the file comprises a landmark set property, wherein the landmark set property indicates the keypoint and landmark component information, wherein the landmark set property indicates a list of landmark sets for a mesh, wherein the landmark set property indicates at least one of a name, a description, a type, a landmark identifier, or a mesh associated with each landmark set in the list of landmark sets, wherein the file comprises a landmarks property, wherein the landmarks property indicates the keypoint and landmark component information, wherein the landmarks property indicates a list of landmarks, and wherein the landmarks property indicates at least one of an identifier, a name, a description, a type, a vertex, a face, or a weight associated with each landmark in the list of landmarks..
The method of claim 10, wherein the file comprises at least one of a keypoint set property or a landmark set property, wherein the keypoint set property indicates a keypoint set identifier associated with a controller set associated with the asset, and wherein the landmark set property indicates a landmark set identifier associated with a landmark set associated with the asset.
The method of claim 10, wherein the file comprising the representation of the avatar is an MPEG avatar representation format (MARF) file.

Description

BACKGROUND The present application is related to video coding systems that may be used to compress digital video signals, e.g., to reduce the storage and/or transmission bandwidth needed for such signals. Video coding systems may include, for example, block-based, wavelet-based, and/or object-based systems. BRIEF SUMMARY Systems, methods, and instrumentalities are disclosed for avatar keypoints and landmarks representation. A file (e.g., an MPEG avatar representation format (MARF) file) may be received. The file may include a plurality of assets. The file may indicate keypoint and landmark component information. An asset may be selected from the plurality of assets in the file based on parsing the file. A container may be determined based on the selected asset and the keypoint and landmark component information. At least one of a keypoint or a landmark may be determined based on the determined container. At least one of a keypoint or a landmark may animate an avatar based on the at least one of the keypoint or landmark. In examples, the file may include a keypoints property. The keypoints property may indicate the keypoint and landmark component information. The keypoint property may indicate a list of keypoint sets and/or may indicate at least one of a name associated with each keypoint set in the list of keypoint sets or an anchor associated with each keypoint set in the list of keypoint sets. In examples, the file may include an anchors property. The anchors property may indicate the keypoint and landmark component information. The anchors property may indicate a list of anchors and/or may indicate at least one of an identification, a name, or a transform associated with each anchor in the list of anchors. In examples, the anchors property may indicate a binding list and/or a binding weight. The binding list may indicate a list of anchors identifiers. In examples, the anchors property further may indicate at least one of a joints indication or a vertices indication. The joints indication may indicate a location of a file associated with a list of vertex identifiers. The vertices indication may indicate a location of a file associated with a list of vertex identifiers. In examples, the file may include a landmark set property. The landmark set property may indicate the keypoint and landmark component information. The landmark set property may indicate a list of landmark sets for a mesh. The landmark set property may indicate at least one of a name, a description, a type, a landmark identifier, or a mesh associated with each landmark set in the list of landmark sets. In examples, the file may include a landmarks property. The landmarks property may indicate the keypoint and landmark component information. The landmarks property may indicate a list of landmarks. The landmarks property may indicate at least one of an identifier, a name, a description, a type, a vertex, a face, or a weight associated with each landmark in the list of landmarks. In examples, the file may include at least one of a keypoint set property or a landmark set property. The keypoint set property may indicate a keypoint set identifier associated with a controller set associated with the asset. The landmark set property may indicate a landmark set identifier associated with a landmark set associated with the asset. BRIEF DESCRIPTION OF THE DRAWINGS The following detailed description will be better understood when read in conjunction with the appended drawings, in which there are shown examples of one or more of the multiple embodiments of the present disclosure. It should be understood, however, that the embodiments described herein are not limited to the precise arrangements and instrumentalities shown in the drawings. FIG. 1 shows an example system according to one or more embodiments of the present disclosure.FIG. 2 shows an example video encoder according to one or more embodiments of the present disclosure.FIG. 3 shows an example video decoder according to one or more embodiments of the present disclosure.FIG. 4 shows facial landmarks associated with an avatar model.FIG. 5 shows an example of parsing of an MPEG avatar representation format (MARF) file.FIG. 6 shows an example flowchart for attributing avatar landmarks. DETAILED DESCRIPTION In describing the various embodiments of the present disclosure, certain terminology is used herein for convenience only and should not be considered as limiting such embodiments. In the drawings, the same reference numerals are employed for designating the same elements throughout the several figures and the present description. Referring to the drawings, there is shown in FIG. 1 a block diagram illustrating an example system 100 in which embodiments of the present disclosure can be implemented. The system 100 may be an electronic device including, for example, a personal computer, laptop computer, mobile phone, tablet computer, multimedia set-top box, digital television receiver, personal video