US-12626464-B2 - Progressively generating fine polygon meshes

Abstract

In implementation of techniques for progressively generating fine polygon meshes, a computing device implements a mesh progression system to receive a coarse polygon mesh. The mesh progression system generates a fine polygon mesh that has a higher level of resolution than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model. The mesh progression system then receives additional data describing a residual feature of a polygon mesh. Based on the additional data, the mesh progression system generates an adjusted fine polygon mesh that has a higher level of resolution than the fine polygon mesh.

Inventors

• Vladimir Kim
• Yun-Chun CHEN
• Noam AIGERMAN
• Alec Jacobson

Assignees

• ADOBE INC.

Dates

Publication Date

20260512

Application Date

20230720

Claims (20)

1. 1 . A method comprising: receiving, by a processing device, a coarse polygon mesh; generating, by the processing device, a fine polygon mesh that has a higher level of resolution than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model; receiving, by the processing device, additional data specifying a shape of a three-dimensional feature corresponding to a portion of the fine polygon mesh, one or more dimensions for the three-dimensional feature, and vertex coordinates for positioning the three-dimensional feature at a location in the fine polygon mesh; and generating, by the processing device, an adjusted fine polygon mesh that has a higher level of resolution than the fine polygon mesh by incorporating the three-dimensional feature at the location of the fine polygon mesh, the three-dimensional feature having a size corresponding to the one or more dimensions.
2. 2 . The method of claim 1 , further comprising incorporating the three-dimensional feature into the portion of the fine polygon mesh by using the machine learning model to change polygons of the three-dimensional feature based on the vertex coordinates.
3. 3 . The method of claim 2 , wherein generating the adjusted fine polygon mesh includes replacing the portion of the fine polygon mesh with a residual feature defined by the vertex coordinates.
4. 4 . The method of claim 1 , wherein the additional data specifies a color or a texture of the portion of the fine polygon mesh.
5. 5 . The method of claim 1 , wherein the fine polygon mesh is displayed in a user interface while the adjusted fine polygon mesh is generated.
6. 6 . The method of claim 1 , wherein generating the fine polygon mesh includes subdividing polygons of the coarse polygon mesh into multiple polygons.
7. 7 . The method of claim 1 , wherein the machine learning model is trained using a dataset including three-dimensional facial features represented as polygons.
8. 8 . The method of claim 1 , wherein the machine learning model is trained using a dataset including a different polygon mesh used to generate the coarse polygon mesh.
9. 9 . The method of claim 1 , wherein the coarse polygon mesh is transmitted separately from the additional data, and a level of resolution of the coarse polygon mesh is based on an amount of bandwidth.
10. 10 . The method of claim 1 , wherein the additional data specifies a pose of the three-dimensional feature.
11. 11 . The method of claim 1 , further comprising progressively updating the adjusted fine polygon mesh by incorporating one or more additional three-dimensional features into the adjusted fine polygon mesh.
12. 12 . The method of claim 1 , further comprising blending the three-dimensional feature into the adjusted fine polygon mesh using the machine learning model.
13. 13 . A system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising: receiving a polygon mesh; generating a coarse polygon mesh that has fewer nodes than the polygon mesh by encoding details of the polygon mesh; transmitting the coarse polygon mesh to a client device, the coarse polygon mesh configured to cause the client device to generate a fine polygon mesh that has more nodes than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model; transmitting additional data specifying a shape of a three-dimensional feature corresponding to a portion of the fine polygon mesh, one or more dimensions for the three-dimensional feature, and vertex coordinates for positioning the three-dimensional feature at a location in the fine polygon mesh; and causing generation of an adjusted fine polygon mesh that has a higher level of resolution than the fine polygon mesh by incorporating the three-dimensional feature at the location of the fine polygon mesh, the three-dimensional feature having a size corresponding to the one or more dimensions.
14. 14 . The system of claim 13 , wherein generating the adjusted fine polygon mesh includes replacing the portion of the fine polygon mesh with a residual feature of the polygon mesh indicated by the vertex coordinates.
15. 15 . The system of claim 13 , wherein the fine polygon mesh is displayed in a user interface while the adjusted fine polygon mesh is generated.
16. 16 . The system of claim 13 , wherein the machine learning model is trained using a dataset including the polygon mesh.
17. 17 . A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising: receiving a coarse polygon mesh; generating a fine polygon mesh that has more nodes than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model; receiving additional data specifying a shape of a three-dimensional feature corresponding to a portion of the fine polygon mesh, one or more dimensions for the three-dimensional feature, and vertex coordinates for positioning the three-dimensional feature at a location in the fine polygon mesh; and generating an adjusted fine polygon mesh that has more nodes than the fine polygon mesh by incorporating the three-dimensional feature at the location of the fine polygon mesh, the three-dimensional feature having a size corresponding to the one or more dimensions.
18. 18 . The non-transitory computer-readable storage medium of claim 17 , wherein the vertex coordinates relate to a residual feature of the fine polygon mesh and generating the adjusted fine polygon mesh includes replacing the portion of the fine polygon mesh with the residual feature of the fine polygon mesh.
19. 19 . The non-transitory computer-readable storage medium of claim 17 , wherein the fine polygon mesh is displayed in a user interface while the adjusted fine polygon mesh is generated.
20. 20 . The non-transitory computer-readable storage medium of claim 17 , wherein the machine learning model is trained using a dataset including three-dimensional facial features represented as polygons.

Description

BACKGROUND In computer graphics, a three-dimensional (3D) mesh is a collection of nodes, edges, and faces that define a geometry of a three-dimensional object. The 3D mesh is a visual component used to represent and render 3D objects in various applications, including video games, virtual reality, computer-aided design (CAD), and animation. By combining nodes, edges, and faces, the 3D mesh represents complex shapes and objects. For example, connections between the nodes and the arrangement of faces define a topology and an overall structure of the 3D mesh. However, in conventional techniques a significant amount of data is used to render 3D meshes, which causes errors and results in visual inaccuracies, computational inefficiencies, and increased power consumption in real world scenarios. SUMMARY Techniques and systems for progressively generating fine polygon meshes are described. In an example, a mesh progression system receives a polygon mesh. Based on the polygon mesh, the mesh progression system generates a coarse polygon mesh that has fewer nodes than the polygon mesh by encoding details of the polygon mesh. The mesh progression system then transmits the coarse polygon mesh to a client device. The mesh progression system generates a fine polygon mesh that has a higher level of resolution than the coarse polygon mesh by decoding the coarse polygon mesh using a machine learning model. For example, generating the fine polygon mesh includes subdividing polygons of the coarse polygon mesh into multiple polygons. In some examples, the machine learning model is trained using a dataset including the polygon mesh used to generate the coarse polygon mesh. Additional data describing a residual feature of the polygon mesh is received by the mesh progression system, the additional data transmitted to the client device separately from the coarse polygon mesh. For example, the additional data specifies vertex coordinates of polygons for the residual feature of the polygon mesh. Based on the additional data, the mesh progression system generates an adjusted fine polygon mesh that has a higher level of resolution than the fine polygon mesh. The adjusted fine polygon mesh is output for display in a user interface. In some examples, the fine polygon mesh is displayed in the user interface while the adjusted fine polygon mesh is generated. This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS The detailed description is described with reference to the accompanying figures. Entities represented in the figures are indicative of one or more entities and thus reference is made interchangeably to single or plural forms of the entities in the discussion. FIG. 1 is an illustration of a digital medium environment in an example implementation that is operable to employ techniques and systems for progressively generating fine polygon meshes as described herein. FIG. 2 depicts a system in an example implementation showing operation of a mesh progression module for progressively generating fine polygon meshes. FIG. 3 depicts an example of generating a coarse polygon mesh by encoding a polygon mesh. FIG. 4 depicts an example of generating a fine polygon mesh by decoding the coarse polygon mesh. FIG. 5 depicts an example of generating an adjusted fine polygon mesh based on the fine polygon mesh. FIG. 6 depicts a procedure in an example implementation of progressively generating fine polygon meshes. FIG. 7 depicts a procedure in an additional example implementation of progressively generating fine polygon meshes. FIG. 8 illustrates an example system including various components of an example device that can be implemented as any type of computing device as described and/or utilized with reference to FIGS. 1-7 to implement embodiments of the techniques described herein. DETAILED DESCRIPTION Overview A polygon mesh is a representation of a three-dimensional (3D) object in computer graphics. A polygon mesh is formed by nodes, edges, and faces that define a shape and a structure of the 3D object. Nodes, or vertices, are individual points in 3D space that define positions of the 3D object's corners, edges, and surface points. A node has three coordinates (x, y, z) to represent its position 3D on a cartesian plane. Edges are line segments connecting pairs of nodes. The edges represent boundaries and contours of the 3D object and define the overall shape and structure of the 3D object. Faces are polygons, including triangles or quadrilaterals, formed by connecting three or more vertices with edges. The faces define visible surfaces of the 3D object. Polygon meshes vary in complexity, from simple geometric shapes to highly detailed models. Fo