CN-116128610-B - Data generation method, device, equipment and storage medium

CN116128610BCN 116128610 BCN116128610 BCN 116128610BCN-116128610-B

Abstract

The application discloses a data generation method, a device, equipment and a storage medium, wherein the data generation method comprises the steps of acquiring a first user image, a model image with target clothes and a target posture image; the method includes determining a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image, generating a second user image with a target pose, a second user pose image with the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, and determining a third user image with the target apparel based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image. According to the application, by inputting the information of different target postures, the virtual fitting results of the human body with different postures can be controlled and generated, so that the virtual fitting effect is improved.

Inventors

WU CHANGJIAN
ZHANG DI
CHEN PENG
XUE JUNYIN
LIAN HUANHUAN
Li Huke
HUANG QIU
ZHANG YANG
WANG CONGRONG

Assignees

杭州海康威视数字技术股份有限公司

Dates

Publication Date: 20260505
Application Date: 20230227

Claims (9)

1. A data generation method, characterized in that the data generation method comprises: acquiring a first user image, a model image with target clothes and a target attitude image; determining a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image; Generating a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image; Determining a third user image with target apparel based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image; the step of determining a third user image with target apparel based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image, comprises: Determining third key point coordinates of the second user gesture image and fourth key point coordinates of the model gesture image; determining a keypoint coordinate map of the third keypoint coordinate and the fourth keypoint coordinate; Inputting the second user image, the model image, the second user segmentation image, the model segmentation image and the key point coordinate map to a preset clothing migration model, and migrating the model segmentation image on the model image to the second user segmentation image on the second user image based on the clothing migration model and the key point coordinate map to obtain a third user image with target clothing.
2. The data generating method according to claim 1, wherein the step of generating a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image, comprises: Determining first key point coordinates of the first user gesture image, second key point coordinates of the target gesture image and segmentation migration information pointed by the first user segmentation image; Inputting the first user image, the segmentation migration information, the first key point coordinates and the second key point coordinates to a preset gesture migration model, migrating the first user image from the first key point coordinates to the second key point coordinates based on the gesture migration model and the segmentation migration information to obtain a second user image with a target gesture and a second user gesture image of the second user image, and performing clothing segmentation on the second user image to obtain a second user segmentation image.
3. The data generation method according to claim 1, wherein the step of determining a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image, comprises: performing human body information expansion on the first user image and the model image respectively to obtain an expanded fourth user image and an expanded model image; Performing feature extraction, key point positioning and classification on the fourth user image and the model image subjected to expansion to obtain a first user gesture image and a model gesture image; And performing clothing segmentation on the fourth user image and the model image subjected to expansion to obtain a first user segmentation image and a model segmentation image.
4. The data generating method as claimed in claim 3, wherein the step of performing human body information expansion on the first user image and the model image, respectively, to obtain an expanded fourth user image and an expanded model image comprises: Extracting first characteristic information of the first user image and second characteristic information of the model image; Positioning a first human frame of the first user image and a second human frame of the model image based on the first feature information and the second feature information; And performing external expansion on the first human body frame and the second human body frame to obtain an external expanded fourth user image and an external expanded model image.
5. The data generating method according to claim 2, wherein the steps of migrating the first user image from the first keypoint coordinates to the second keypoint coordinates based on the gesture migration model and the segmentation migration information, obtaining a second user image with a target gesture and a second user gesture image of the second user image, and performing clothing segmentation on the second user image, obtaining a second user segmented image, include: determining a region to be complemented of the second user image after gesture migration; and performing image restoration on the region to be complemented based on a preset image restoration network to obtain a fifth user image after image restoration.
6. The method for generating data according to claim 5, wherein the step of performing image restoration on the region to be complemented based on a preset image restoration network to obtain a fifth user image after image restoration comprises: Downsampling the region to be complemented for a preset number of times and a preset multiple to obtain first characteristic data of corresponding number; up-sampling the first characteristic data for the preset times to obtain second characteristic data with the same number as the first characteristic data, wherein the second characteristic data has the same scale as the to-be-complemented area image; And carrying out feature fusion on the second feature data to obtain a fifth user image after image restoration.
7. A data generation apparatus, characterized in that the data generation apparatus comprises: the acquisition module is used for acquiring a first user image, a model image with target clothes and a target posture image; a determining module configured to determine a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image; a generation module for generating a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image; A migration module for determining a third user image with a target apparel based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image; the step of determining a third user image with target apparel based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image, comprises: Determining third key point coordinates of the second user gesture image and fourth key point coordinates of the model gesture image; determining a keypoint coordinate map of the third keypoint coordinate and the fourth keypoint coordinate; Inputting the second user image, the model image, the second user segmentation image, the model segmentation image and the key point coordinate map to a preset clothing migration model, and migrating the model segmentation image on the model image to the second user segmentation image on the second user image based on the clothing migration model and the key point coordinate map to obtain a third user image with target clothing.
8. A data generating apparatus is characterized by comprising a memory, a processor, and a program stored on the memory for realizing the data generating method, The memory is used for storing a program for realizing the data generation method; The processor is configured to execute a program implementing the data generation method to implement the steps of the data generation method according to any one of claims 1 to 6.
9. A storage medium having stored thereon a program for realizing the data generating method, the program for realizing the data generating method being executed by a processor to realize the steps of the data generating method according to any one of claims 1 to 6.

Description

Data generation method, device, equipment and storage medium Technical Field The present application relates to the field of computer technologies, and in particular, to a data generating method, apparatus, device, and storage medium. Background With the development of shopping on an e-commerce platform, more and more people purchase clothes online. Because the user can only see the appearance of the commodity when buying the clothes, the effect of the clothes after wearing cannot be known, and therefore the user can find that the commodity does not meet the expected requirement for offline goods returning after receiving the commodity online, and the shopping experience of the user is poor. Referring to patent CN111787242a, which discloses a method and apparatus for virtual fitting on-line, the above problems are solved by a virtual fitting method on-line, specifically, by performing human body positioning and surface coordinate analysis on a model picture and a user photo, covering pixels corresponding to clothes of the model picture at corresponding positions in the user photo, and obtaining an effect that the clothes on the model picture are worn on the user photo. However, when the human body posture in the model picture is large in difference with the human body posture in the user picture, the corresponding position relationship between the model picture and the clothes is wrong, so that the virtual fitting effect is poor. Disclosure of Invention The application mainly aims to provide a data generation method, a device, equipment and a storage medium, and aims to solve the technical problem that in the prior art, when the human body posture in a model picture is large in difference with the human body posture in a user picture, the effect of virtual fitting is poor. To achieve the above object, the present application provides a data generation method including: acquiring a first user image, a model image with target clothes and a target attitude image; determining a first user pose image, a model pose image, a first user segmentation image, and a model segmentation image based on the first user image and the model image; Generating a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image; A third user image with a target apparel is determined based on the second user image, the second user pose image, the second user segmentation image, the model pose image, and the model segmentation image. Optionally, the step of generating a second user image with a target pose, a second user pose image of the second user image, and a second user segmentation image based on the first user image, the first user pose image, the first user segmentation image, and the target pose image includes: Determining first key point coordinates of the first user gesture image, second key point coordinates of the target gesture image and segmentation migration information pointed by the first user segmentation image; Inputting the first user image, the segmentation migration information, the first key point coordinates and the second key point coordinates to a preset gesture migration model, migrating the first user image from the first key point coordinates to the second key point coordinates based on the gesture migration model and the segmentation migration information to obtain a second user image with a target gesture and a second user gesture image of the second user image, and performing clothing segmentation on the second user image to obtain a second user segmentation image. Optionally, the step of determining the first user pose image, the model pose image, the first user segmentation image, and the model segmentation image based on the first user image and the model image includes: performing human body information expansion on the first user image and the model image respectively to obtain an expanded fourth user image and an expanded model image; Performing feature extraction, key point positioning and classification on the fourth user image and the model image subjected to expansion to obtain a first user gesture image and a model gesture image; And performing clothing segmentation on the fourth user image and the model image subjected to expansion to obtain a first user segmentation image and a model segmentation image. Optionally, the step of performing human body information expansion on the first user image and the model image to obtain an expanded fourth user image and an expanded model image includes: Extracting first characteristic information of the first user image and second characteristic information of the model image; Positioning a first human frame of the first user image and a second human frame of the model image based on the first feature information and the second feature information; And performing external e