US-12620141-B2 - Image style conversion method and apparatus, electronic device, and storage medium

US12620141B2US 12620141 B2US12620141 B2US 12620141B2US-12620141-B2

Abstract

Embodiments of this application disclose an image style conversion method performed by an electronic device. The method includes: performing quality enhancement on a first target style image to obtain a second target style image; performing feature extraction on the second target style image to obtain a target style feature; performing migration training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model; inputting a full style feature, the target style feature, and a to-be-converted image into the target style conversion model, and performing style conversion on the to-be-converted image using the target style conversion model to obtain a target image conforming to a target style.

Inventors

Yun Cao
Xinyi Zhang
Junwei Zhu
Ying Tai
Mu Zhang
Chengjie Wang
Feiyue Huang

Assignees

TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Dates

Publication Date: 20260505
Application Date: 20230329
Priority Date: 20210723

Claims (17)

1 . An image style conversion method, performed by an electronic device, the method comprising: performing quality enhancement on a first target style image to obtain a second target style image; performing feature extraction on the second target style image to obtain a target style feature; performing transfer training on a preset target style conversion model by using a full style conversion model, the first target style image and the target style feature to obtain a target style conversion model, wherein a model architecture of the full style conversion model is the same as a model architecture of the preset target style conversion model; encoding a to-be-converted image by using the target style conversion model to obtain an intermediate style feature of the to-be-converted image, further including: extracting feature information from the to-be-converted image; performing distribution mapping on the feature information to obtain a distribution feature of the feature information; and generating the intermediate style feature according to the distribution feature; converting the intermediate style feature by using the target image style feature to obtain a converted style feature; and decoding the converted style feature by using the target style conversion model to obtain the target image.
2 . The method according to claim 1 , wherein the converting the intermediate style feature by using the target image style feature to obtain a converted style feature comprises: performing feature fusion on a full style feature and the target style feature to obtain a fused image style feature; and converting the intermediate style feature by using the fused image style feature to obtain the converted style feature.
3 . The method according to claim 2 , wherein the full style feature comprises a plurality of basic style features, and the performing feature fusion on the full style feature and the target style feature to obtain a fused image style feature comprises: performing statistical processing on each basic style feature to obtain a statistical image style feature; and fusing the statistical image style feature and the target style feature to obtain the fused image style feature.
4 . The method according to claim 1 , wherein the method further comprises: adjusting the target style conversion model based on the full style conversion model to obtain an adjusted target style conversion model.
5 . The method according to claim 1 , wherein the performing transfer training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model comprises: initializing the preset target style conversion model by using the full style conversion model to obtain an initialized target style conversion model; and training the initialized target style conversion model by using the target style feature to obtain the target style conversion model.
6 . The method according to claim 5 , wherein the training the initialized target style conversion model by using the target style feature to obtain the target style conversion model comprises: obtaining a training image; performing style conversion on the training image by using the target style feature and the initialized target style conversion model to obtain a style-converted image; calculating loss information between the style-converted image and a preset target style image corresponding to the training image; and adjusting the initialized target style conversion model according to the loss information to obtain the target style conversion model.
7 . An electronic device, comprising a memory and a processor, the memory storing a plurality of instructions that, when executed by the processor, cause the electronic device to perform an image style conversion method including: performing quality enhancement on a first target style image to obtain a second target style image; performing feature extraction on the second target style image to obtain a target style feature; performing transfer training on a preset target style conversion model by using a full style conversion model, the first target style image and the target style feature to obtain a target style conversion model, wherein a model architecture of the full style conversion model is the same as a model architecture of the preset target style conversion model; encoding a to-be-converted image by using the target style conversion model to obtain an intermediate style feature of the to-be-converted image, further including: extracting feature information from the to-be-converted image; performing distribution mapping on the feature information to obtain a distribution feature of the feature information; and generating the intermediate style feature according to the distribution feature; converting the intermediate style feature by using the target image style feature to obtain a converted style feature; and decoding the converted style feature by using the target style conversion model to obtain the target image.
8 . The electronic device according to claim 7 , wherein the converting the intermediate style feature by using the target image style feature to obtain a converted style feature comprises: performing feature fusion on a full style feature and the target style feature to obtain a fused image style feature; and converting the intermediate style feature by using the fused image style feature to obtain the converted style feature.
9 . The electronic device according to claim 8 , wherein the full style feature comprises a plurality of basic style features, and the performing feature fusion on the full style feature and the target style feature to obtain a fused image style feature comprises: performing statistical processing on each basic style feature to obtain a statistical image style feature; and fusing the statistical image style feature and the target style feature to obtain the fused image style feature.
10 . The electronic device according to claim 7 , wherein the method further comprises: adjusting the target style conversion model based on the full style conversion model to obtain an adjusted target style conversion model.
11 . The electronic device according to claim 7 , wherein the performing transfer training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model comprises: initializing the preset target style conversion model by using the full style conversion model to obtain an initialized target style conversion model; and training the initialized target style conversion model by using the target style feature to obtain the target style conversion model.
12 . The electronic device according to claim 11 , wherein the training the initialized target style conversion model by using the target style feature to obtain the target style conversion model comprises: obtaining a training image; performing style conversion on the training image by using the target style feature and the initialized target style conversion model to obtain a style-converted image; calculating loss information between the style-converted image and a preset target style image corresponding to the training image; and adjusting the initialized target style conversion model according to the loss information to obtain the target style conversion model.
13 . A non-transitory computer-readable storage medium, storing a plurality of instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an image style conversion method including: performing quality enhancement on a first target style image to obtain a second target style image; performing feature extraction on the second target style image to obtain a target style feature; performing transfer training on a preset target style conversion model by using a full style conversion model, the first target style image and the target style feature to obtain a target style conversion model, wherein a model architecture of the full style conversion model is the same as a model architecture of the preset target style conversion model; encoding a to-be-converted image by using the target style conversion model to obtain an intermediate style feature of the to-be-converted image, further including: extracting feature information from the to-be-converted image; performing distribution mapping on the feature information to obtain a distribution feature of the feature information; and generating the intermediate style feature according to the distribution feature; converting the intermediate style feature by using the target image style feature to obtain a converted style feature; and decoding the converted style feature by using the target style conversion model to obtain the target image.
14 . The non-transitory computer-readable storage medium according to claim 13 , wherein the converting the intermediate style feature by using the target image style feature to obtain a converted style feature comprises: performing feature fusion on a full style feature and the target style feature to obtain a fused image style feature; and converting the intermediate style feature by using the fused image style feature to obtain the converted style feature.
15 . The non-transitory computer-readable storage medium according to claim 14 , wherein the full style feature comprises a plurality of basic style features, and the performing feature fusion on the full style feature and the target style feature to obtain a fused image style feature comprises: performing statistical processing on each basic style feature to obtain a statistical image style feature; and fusing the statistical image style feature and the target style feature to obtain the fused image style feature.
16 . The non-transitory computer-readable storage medium according to claim 13 , wherein the method further comprises: adjusting the target style conversion model based on the full style conversion model to obtain an adjusted target style conversion model.
17 . The non-transitory computer-readable storage medium according to claim 13 , wherein the performing transfer training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model comprises: initializing the preset target style conversion model by using the full style conversion model to obtain an initialized target style conversion model; and training the initialized target style conversion model by using the target style feature to obtain the target style conversion model.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation application of PCT Patent Application No. PCT/CN2022/099989, entitled “IMAGE STYLE CONVERSION METHOD AND APPARATUS, ELECTRONIC DEVICE, AND STORAGE MEDIUM” filed on Jun. 21, 2022, which claims priority to Chinese Patent Application No. 202110839235.X, entitled “IMAGE STYLE CONVERSION METHOD AND APPARATUS, ELECTRONIC DEVICE, AND STORAGE MEDIUM” and filed with the China National Intellectual Property Administration on Jul. 23, 2021, the entire contents of which are incorporated herein by reference. FIELD OF THE TECHNOLOGY This application relates to the field of computer technologies, and in particular, to an image style conversion method and apparatus, an electronic device, and a storage medium. BACKGROUND OF THE DISCLOSURE With the rapid development of communication and computer technologies, an image processing technology based on computers and communication has also been developed robustly and rapidly and applied to various fields. For example, the image processing technology may be used to convert an image style to obtain an image of a different style. In the process of research and practice of the related art, the inventors of this application found that in the related art, during image style conversion, accuracy of training an image style conversion model is reduced due to large costs of obtaining high-quality training samples. SUMMARY Embodiments of this application provide an image style conversion method and apparatus, an electronic device, and a storage medium, which improve accuracy of training an image style conversion model by using limited and low-quality samples. According to one aspect, an embodiment of this application provides an image style conversion method, performed by an electronic device, and including: performing quality enhancement on a first target style image to obtain a second target style image;performing feature extraction on the second target style image to obtain a target style feature;performing migration training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model; andinputting a full style feature, the target style feature, and a to-be-converted image into the target style conversion model, and performing style conversion on the to-be-converted image using the target style conversion model to obtain a target image conforming to a target style. According to another aspect, an embodiment of this application further provides an image style conversion apparatus, including: an obtaining unit, configured to perform quality enhancement on a first target style image to obtain a second target style image;a feature extraction unit, configured to perform feature extraction on the second target style image to obtain a target style feature;a migration training unit, configured to perform migration training on a preset target style conversion model by using a full style conversion model and the target style feature to obtain a target style conversion model; anda style conversion unit, configured to input a full style feature, the target style feature, and a to-be-converted image into the target style conversion model, and performing style conversion on the to-be-converted image using the target style conversion model to obtain a target image conforming to a target style. According to another aspect, an embodiment of this application further provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, and the computer instructions being stored in a computer-readable storage medium. According to another aspect, an embodiment of this application further provides an electronic device, including a memory and a processor, the memory storing a plurality of instructions that, when executed by the processor, cause the electronic device to perform the image style conversion method described above. According to another aspect, an embodiment of this application further provides a non-transitory computer-readable storage medium, the storage medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the image style conversion method described above. BRIEF DESCRIPTION OF THE DRAWINGS To describe the technical solutions in the embodiments of this application more clearly, the following briefly describes accompanying drawings required for describing the embodiments. Apparently, the accompanying drawings in the following description show merely some embodiments of this application, and a person skilled in the art may still derive other drawings from these accompanying drawings without creative efforts. FIG. 1 is a schematic scenario diagram of an image style conversion method according to an embodiment of this application. FIG. 2 is a schematic flowchart of an image style conversion m