Search

CN-122023728-A - Hierarchical 3D refrigerator sticker digital model generation method based on single image

CN122023728ACN 122023728 ACN122023728 ACN 122023728ACN-122023728-A

Abstract

The invention relates to the technical field of computer vision and additive manufacturing, and particularly provides a layering 3D refrigerator sticker digital model generation method based on a single image. The method comprises the steps of generating a reference image of a 3D refrigerator main body sculpture according to creative description information, separating main body objects in the reference image from a background to obtain a foreground RGB image, a front Jing Erwei mask image and a background RGB image, constructing a three-dimensional grid model based on the foreground RGB image, the front Jing Erwei mask image and the background RGB image, and performing geometric optimization on the three-dimensional grid model to obtain a layered 3D refrigerator digital model.

Inventors

  • ZHANG YUWEI
  • WANG YONGQI
  • LI CHANGBAO
  • Han Tongju
  • LI YIXING
  • LIU SIYU
  • SHI YANBIN
  • ZHANG LILI
  • FU XIUZHUO

Assignees

  • 齐鲁工业大学(山东省科学院)

Dates

Publication Date
20260512
Application Date
20260414

Claims (8)

  1. 1. A layering 3D refrigerator sticker digital model generation method based on a single image is characterized by comprising the following steps: Step 1, generating a reference image of a main sculpture of a 3D refrigerator sticker according to creative description information; step 2, separating a main object in the reference image from the background to obtain a foreground RGB image, a front Jing Erwei mask image and a background RGB image; step 3, constructing a three-dimensional grid model based on the foreground RGB image, the front Jing Erwei mask image and the background RGB image; And 4, performing geometric optimization on the three-dimensional grid model to obtain a layered 3D refrigerator paster digital model.
  2. 2. The method according to claim 1, wherein the step 1 comprises: Firstly, inputting a creative Prompt word Prompt, wherein the Prompt word comprises information of a main object, style description, material characteristics and three-dimensional structure description so as to enhance the space layering sense and the volume sense of a generated image; calling an AI (advanced technology) graph generation tool to generate an image, and carrying out the following constraint when the prompt word is designed: the method comprises the steps of (1) highlighting a main body, selecting a front view angle, selecting a front display view angle, enabling a main body boundary to be complete, (4) illuminating clearly, enabling scene illumination to be distributed uniformly, and (5) three-dimensionally expressing that keywords with 3D sculpture styles and 3D rendering effects are added into prompt words so that a generated image has three-dimensional morphological characteristics; based on the prompt word constraint, the finally obtained reference image is a main object with three-dimensional modeling characteristics, and the reference image is basic input of subsequent foreground and background separation and three-dimensional modeling.
  3. 3. The method according to claim 2, wherein the step 2 comprises: Firstly, acquiring a main object of a reference image by using an AI image editing tool or an image segmentation model based on deep learning, and acquiring three types of intermediate data by foreground separation processing: (1) A foreground RGB image, namely a color image containing a central main body object, wherein the background area is removed or transparentized, and only main body sculpture content is reserved, and the foreground RGB image is used for generating a three-dimensional grid model; (2) A front Jing Erwei mask map, which is a binary image consistent with the resolution of the original image, wherein the pixel value of a foreground region is 1, and the pixel value of a background region is 0; (3) And (3) background RGB image, namely after the foreground separation is completed, forming a gap in a main area in the original image, and automatically filling the gap area by using an AI image editing tool to obtain a complete background image, wherein the process generates natural continuous background content by using context texture information, so as to obtain the complete background image.
  4. 4. A method according to claim 3, wherein said step 3 comprises: based on the foreground RGB image, the front Jing Erwei mask image and the background RGB image, generating a three-dimensional grid model for subsequent geometric optimization and printing manufacture, wherein the three-dimensional grid model comprises a background plate three-dimensional solid model and a foreground object three-dimensional grid model, and performing scale and position alignment on the background plate three-dimensional solid model and the foreground object three-dimensional grid model in a three-dimensional space, and the flow is as follows: Step 31, generating a background plate three-dimensional entity model; Firstly, taking a background RGB image as input, generating depth information of the background image by a monocular depth estimation method, and carrying out normalization processing on depth values to enable the minimum depth value in the depth image to correspond to a Z=0 plane in a three-dimensional space, thereby establishing a unified space reference standard; then, according to the mapping relation between the depth map and the image pixel coordinates, converting each pixel point into vertex coordinates in a three-dimensional space, and connecting adjacent pixels into a triangular patch according to an image grid structure, so as to generate a triangular grid curved surface on the front surface of the background plate; then, mapping the background RGB image as a texture map onto a triangular mesh curved surface, and realizing one-to-one correspondence of textures and geometric structures through UV coordinate mapping, so that the surface of the background plate keeps the color and detail information of the original image; Finally, in order to form a background plate structure with solid thickness, carrying out materialization treatment on the triangular mesh curved surface, wherein (1) four boundary outlines of the triangular mesh curved surface are extracted and comprise an upper boundary, a lower boundary, a left boundary and a right boundary, (2) the four boundaries are extruded along the negative direction of a Z axis to generate four side wall planes approximately perpendicular to the front surface of the background plate so as to form a side surface structure of the background plate, (3) a triangular mesh plane parallel to the front surface of the background plate is constructed at the bottom of the side wall and is used as the back surface of the background plate; Through the operation, a complete background plate three-dimensional entity model is obtained; step 32, generating a foreground object three-dimensional grid model; The method comprises the steps of generating a background plate three-dimensional solid model, carrying out three-dimensional reconstruction on a foreground main object to obtain a foreground object three-dimensional grid model, taking a foreground RGB image as input, calling an AI tool of an image generation three-dimensional model to generate a three-dimensional grid model of the foreground object, and outputting the three-dimensional grid model in a standard three-dimensional grid data format, wherein the three-dimensional grid model comprises a three-dimensional Vertex coordinate Vertex, a triangular patch topological structure Face, UV texture coordinates and a corresponding texture map image; Through the operation, a foreground object three-dimensional grid model with complete geometric structure and surface texture is obtained, and the foreground object three-dimensional grid model is subjected to space alignment and structure fusion with a background plate in the follow-up process; step 33, aligning the three-dimensional space; automatic scaling and spatial translation of the three-dimensional object is performed to align it with the foreground mask region on a two-dimensional projection.
  5. 5. The method according to claim 4, wherein said step 33 comprises: step 331, calculating the size of the two-dimensional mask; Firstly, carrying out pixel scanning on a front Jing Erwei mask image, and extracting the boundary range of a foreground mask region in an image coordinate system, wherein the pixel scanning comprises the steps of calculating the maximum value and the minimum value U min 、U max 、V min 、V max of foreground pixels in the UV direction, calculating to obtain mask width U mask =U max –U min and mask height V mask =V max –V min , and calculating a two-dimensional center point O of the foreground mask region; Step 332, calculating the size of the three-dimensional object; Traversing three-dimensional coordinates of all vertexes in the three-dimensional mesh model of the foreground object, calculating a space range X max 、X min 、Y max 、Y min of the three-dimensional mesh model in the XY direction to obtain space dimensions X mesh =X max –X min and Y mesh = Y max –Y min of the three-dimensional object; step 333, scale scaling and translational alignment; In order to make the three-dimensional object consistent with the foreground mask region in the two-dimensional projection, calculating the proportional relation between the three-dimensional object and the mask region, firstly calculating a transverse scaling factor S x = U mask /X mesh and a longitudinal scaling factor S y = V mask /Y mesh , then selecting the smaller value of the two as the whole size scaling factor S=min (S x ,S y ), and then performing unified scaling operation (X, Y, Z) to (S) X, S Y, S Z), after the scaling is completed, calculating a two-dimensional translation vector T so as to align the projection position of a geometric center point C of the three-dimensional object on an xy plane with a two-dimensional center point O of a foreground mask area, and executing translation transformation on all vertex coordinates; Through the operation, the three-dimensional object is consistent with the foreground mask area in the two-dimensional projection plane, so that a space position foundation is provided for the subsequent fusion with the background plate; step 334, three-dimensional height compression; Moderately compressing the height of a foreground three-dimensional object comprises multiplying the Z coordinates of all vertices of the three-dimensional object by a compression coefficient α: Z' =α Z, where α ε [0.4, 0.6].
  6. 6. The method according to claim 4, wherein the step 4 comprises: On the premise of keeping the front vision approximation of the foreground object unchanged, constrained global optimization is carried out on the three-dimensional foreground grid in the Z direction so as to enable the back surface of the foreground object to be closely attached to a background plate and ensure the stability of grid topology and surface details, and the process comprises vertex visibility judgment, Z reference correction, anchor point selection, energy model establishment and sparse linear system solving; Assume that the Z-coordinate of the mesh vertices is marked as a vector N is the number of vertexes, geometric optimization is based on reserving local differential coordinates of a grid, and meanwhile, Z-direction target constraint is applied to a plurality of vertexes, and the mathematical form is equivalent to solving the following least square problem with soft constraint: ; wherein L is a discrete Laplacian matrix based on grid topology, delta is a differential coordinate of an original Z coordinate, and each anchor point group With target depth vector And weight of The free points are regarded as constraint groups with weights close to zero; step 41, vertex visibility judgment, namely front or back visibility; Before selecting an anchor point, performing visibility classification on each vertex, namely whether the anchor point is visible from the front or the rear, so as to ensure that the anchor point is only from a visible area on the back of an object; (1) Calculating Z component Nz of each vertex normal N, and if the Nz is consistent with the observation direction, the current vertex is a candidate visible vertex; (2) For each candidate vertex, shifting a minimum distance epsilon along the observation direction to generate a ray, detecting whether the ray intersects other surface patches of the grid by using a ray triangle intersection acceleration structure, and if not, marking the current vertex as visible; (3) Respectively executing detection in the front-back direction and the back-front direction to obtain a front visible point set and a back visible point set, wherein the front visible point set is used for subsequent Z reference correction; Step 42, Z reference correction, i.e., initial alignment; To ensure that the background plate and the foreground three-dimensional grid have uniform Z reference benchmarks, initial Z reference correction is performed: a. Statistics of minimum values of frontal visible vertices ; B. Subtracting the minimum value from all vertex Z coordinates of the whole foreground three-dimensional grid: ; c. Cut off the corrected Z value less than zero if <0, Order =0; Step 43, selecting strong anchor points, weak anchor points or free points; h. A strong anchor point; Constructing an induced sub-graph in the back visible vertex set according to the vertex adjacency relation of the grid, calculating connected components on the induced sub-graph, and arranging the connected components in descending order according to the number of the vertices; screening in candidate strong anchor point region, selecting Z coordinate less than threshold value tau as strong anchor point set Searching corresponding pixels on the depth map of the background plate and reading background depth values of each strong anchor point according to the UV or XY projection positions of the strong anchor points on the grid, and taking the background depth values as absolute target depth of the current vertex Strong anchor point setting weight The aim of forcibly attaching the background plate is fulfilled through strong constraint; i. weak anchor points and free points; vertices of the back visible vertex set that belong to the back and do not belong to the strong anchor are taken as the weak anchor set For weak anchor point, multiplying its original Z coordinate by compression coefficient as relative target And is given weight Target coefficient Between 0.5 and 0.7, the method is characterized in that the method moves to the background direction and only moves to a part of the original depth, the rest vertexes except the strong anchor point and the weak anchor point are defined as free points, and the target weight of the free points 0.001, Allowing smooth transition with neighborhood by mesh topology pulling; step 44, constructing a sparse linear system and solving; Stacking Laplace equation and anchor point soft constraint in line to form linear system ; ,b= ; Wherein the first blocks L and delta are n multiplied by n discrete Laplace matrix and differential coordinates are calculated For each anchor group, construct a sparse matrix , The number of rows is equal to the number of vertices, the number of columns is equal to n, which is at (row, col) The constructed target vector is ; By constructing normal equations And solving z by using a sparse direct solver, and performing non-negative truncation on z: And finally, updating the obtained new z back to the grid vertexes and deriving the new z.
  7. 7. A computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored program, wherein the program, when run, controls a device in which the computer-readable storage medium is located to execute the single image-based hierarchical 3D refrigerator-mount digital model generation method of any one of claims 1to 6.
  8. 8. An electronic device comprising one or more processors, memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs comprising instructions that, when executed by the device, cause the device to perform the single image based hierarchical 3D refrigerator post digital model generation method of any of claims 1-6.

Description

Hierarchical 3D refrigerator sticker digital model generation method based on single image Technical Field The invention relates to the technical field of computer vision and additive manufacturing, in particular to a layering 3D refrigerator sticker digital model generation method based on a single image. Background With the rapid development of additive manufacturing (3D printing) technology and personalized customization market, the demand for low-cost, short-period, high-fidelity, directly printable automated three-dimensional modeling technology for small-sized ornaments of the literature type (such as 3D refrigerator stickers, badge pendants, etc.) is increasing. Conventional refrigerator patch design methods typically rely on manual modeling by artificial three-dimensional modeling software, or stereoscopic effects by converting two-dimensional images into a simple 2.5D laminate or embossed structure. However, the method has the problems of long modeling period, high labor cost, insufficient three-dimensional layering sense, difficulty in guaranteeing the stability of the model structure and the like. In recent years, significant progress has been made in generative artificial intelligence techniques as well as monocular depth estimation techniques, enabling automatic generation of three-dimensional models or depth maps from text or images. However, such general three-dimensional generation methods are generally directed to open scene designs, and the generation results thereof are difficult to directly use for 3D refrigerator patch designs in terms of geometry, topological stability, manufacturability, and the like. Specifically, on the one hand, the general three-dimensional generating model generally does not carry out structural constraint aiming at special requirements of thin structure and back surface lamination of a refrigerator adhesive product, and the generated three-dimensional object is easy to generate problems of deformation, fracture, incapability of being tightly laminated with a background plate and the like when 3D printing or assembly is carried out due to irregular back surface structure. If the commonly generated three-dimensional object is directly spliced with the background plate, due to the lack of fine analysis and optimization on the relationship between the visibility of the grid vertexes and the spatial structure, the phenomena of local self-intersection, mold penetration, geometric distortion and the like are easily generated, so that the visual quality and the structural stability of the final product are affected. Disclosure of Invention In view of the above, the invention provides a layering 3D refrigerator paste digital model generation method based on a single image, which is used for not only keeping the front-view appearance details, but also ensuring the stable lamination of the back surface of the 3D model and a background plate so as to meet the requirement of 3D printing manufacturability. In a first aspect, the invention provides a hierarchical 3D refrigerator sticker digital model generation method based on a single image, which comprises the following steps: Step 1, generating a reference image of a main sculpture of a 3D refrigerator sticker according to creative description information; step 2, separating a main object in the reference image from the background to obtain a foreground RGB image, a front Jing Erwei mask image and a background RGB image; step 3, constructing a three-dimensional grid model based on the foreground RGB image, the front Jing Erwei mask image and the background RGB image; And 4, performing geometric optimization on the three-dimensional grid model to obtain a layered 3D refrigerator paster digital model. Optionally, the step 1 includes: Firstly, inputting a creative Prompt word Prompt, wherein the Prompt word comprises information of a main object, style description, material characteristics and three-dimensional structure description so as to enhance the space layering sense and the volume sense of a generated image; calling an AI (advanced technology) graph generation tool to generate an image, and carrying out the following constraint when the prompt word is designed: the method comprises the steps of (1) highlighting a main body, selecting a front view angle, selecting a front display view angle, enabling a main body boundary to be complete, (4) illuminating clearly, enabling scene illumination to be distributed uniformly, and (5) three-dimensionally expressing that keywords with 3D sculpture styles and 3D rendering effects are added into prompt words so that a generated image has three-dimensional morphological characteristics; based on the prompt word constraint, the finally obtained reference image is a main object with three-dimensional modeling characteristics, and the reference image is basic input of subsequent foreground and background separation and three-dimensional modeling. Optionally, the step 2 includes: Firstl