CN-121982135-A - Image processing method, device, electronic equipment and storage medium

CN121982135ACN 121982135 ACN121982135 ACN 121982135ACN-121982135-A

Abstract

The disclosure relates to an image processing method, an image processing device, image processing equipment and a storage medium, and relates to the technical field of image processing. The method comprises the steps of responding to an image modification instruction input by a user, determining an image modification area and an image modification type, wherein the image modification type comprises a first image modification type and a second image modification type, extracting style characteristics of an image to be processed to obtain image style characteristic data, carrying out noise adding processing on the image to be processed to obtain a first image based on the image modification area and the image modification type, and carrying out image generation according to the image to be processed, the image style characteristic data, the first image and the image modification type to obtain a target generated image. The method and the device enable the generated images to keep consistent styles by combining the image styles of the images to be processed, so that the image processing quality of the target generated images is improved, the requirements of users are met, and more intelligent image processing services are provided for the users.

Inventors

LIANG YUNHAO
XIE FUSHI
SHI JINJIN

Assignees

北京小米移动软件有限公司

Dates

Publication Date: 20260505
Application Date: 20241029

Claims (13)

1. An image processing method, comprising: Determining an image modification area and an image modification type in response to a user input of an image modification instruction, wherein the image modification type comprises a first image modification type and a second image modification type; Extracting style characteristics of an image to be processed to obtain image style characteristic data; Based on the image modification area and the image modification type, carrying out noise addition processing on the image to be processed to obtain a first image; and generating an image according to the image to be processed, the image style characteristic data, the first image and the image modification type to obtain a target generated image.
2. The image processing method according to claim 1, wherein performing style feature extraction on the image to be processed to obtain image style feature data, comprises: performing style detection on the image to be processed to obtain image style classification of the image to be processed; and carrying out style feature extraction on the image to be processed based on the image style classification to obtain the image style feature data.
3. The image processing method according to claim 2, characterized by further comprising: and mapping the image style characteristic data to obtain mapped image style characteristic data conforming to the output dimension of the low-rank adaptive model.
4. The image processing method according to claim 3, wherein generating an image based on the image to be processed, the image style feature data, the first image, and the image modification type to obtain a target generated image, comprises: and splicing channel dimensions of the mapped image style characteristic data through the input of the self-attention layer corresponding to the image generation model, wherein the image generation module is obtained by adjusting and training the potential diffusion model.
5. The image processing method according to claim 1, wherein performing noise addition processing on the image to be processed based on the image modification area and the image modification type to obtain a first image, comprises: Extracting an image modification region boundary range from the image to be processed based on the image modification region in response to the image modification type being a first image modification type; and carrying out noise adding processing on the image in the boundary range of the image modification area and the image of the image modification area to obtain the first image.
6. The image processing method according to claim 5, wherein the image modification region boundary range includes a first range and a second range, the image modification region including the first range; the noise adding processing is performed on the image within the boundary range of the image modification area and the image of the image modification area, and the obtaining of the first image includes: Adding random noise to the image in the boundary range of the image modification area and the image of the image modification area to obtain a second image; Randomizing noise in the second image to obtain a third image; reserving image noise characteristics in the first range in the third image, and setting zero for image noise except the first range in the third image to obtain a fourth image; and combining the fourth image and the image to be processed to obtain the first image.
7. The image processing method according to claim 1, wherein performing noise addition processing on the image to be processed based on the image modification area and the image modification type to obtain a first image, comprises: Responding to the image modification type as a second image modification type, and performing image elimination on the image to be processed to obtain a fifth image; Extracting an image modification region boundary range from the fifth image based on the image modification region; and carrying out noise adding processing on the image in the boundary range of the image modification area and the image of the image modification area to obtain the first image.
8. The image processing method according to claim 7, wherein performing noise addition processing on the image within the boundary of the image modification region and the image of the image modification region to obtain the first image, comprises: adding random noise to the image in the boundary range of the image modification area and the image of the image modification area in the fifth image to obtain a sixth image; performing noise filling on the image of the image modification region based on the image noise characteristics in the boundary range of the image modification region in the sixth image to obtain a seventh image; And combining the seventh image and the image to be processed to obtain the first image.
9. The image processing method according to claim 1, wherein generating an image based on the image to be processed, the image style feature data, the first image, and the image modification type to obtain a target generated image, comprises: performing text coding based on the acquired image processing information to obtain image processing text coding characteristics; inputting the image style characteristic data, the first image and the image processing text coding characteristic into an image generation model to generate an output image; determining a first partial image in the image modification area in the output image; Determining a second partial image except the image modification area in the image to be processed; and splicing the first partial image and the second partial image to obtain the target generated image.
10. The image processing method according to claim 9, wherein image generation is performed according to the image to be processed, the image style characteristic data, the first image, the image modification type, to obtain a target generated image, further comprising: The iteration comprises the following steps: inputting the target generated image, the image style feature data and the image processing text coding feature into the image generation model to regenerate the output image; Redefining a first partial image in the image modification area in the output image; and splicing the first partial image and the second partial image to obtain the updated target generated image.
11. An image processing apparatus, comprising: An information determination unit configured to determine an image modification area and an image modification type in response to a user inputting an image modification instruction, the image modification type including a first image modification type and a second image modification type; The style characteristic extraction unit is used for extracting style characteristics of the image to be processed to obtain image style characteristic data; The noise adding unit is used for carrying out noise adding processing on the image to be processed based on the image modification area and the image modification type to obtain a first image; And the image generation unit is used for generating an image according to the image to be processed, the image style characteristic data, the first image and the image modification type to obtain a target generated image.
12. An image processing apparatus, characterized by comprising: A processor; A memory for storing processor-executable instructions; wherein the processor is configured to implement the image processing method of any one of claims 1 to 10.
13. A non-transitory computer readable storage medium, which when executed by a processor of a mobile terminal, causes the mobile terminal to perform an image processing method according to any one of claims 1 to 10.

Description

Image processing method, device, electronic equipment and storage medium Technical Field The disclosure relates to the technical field of image processing, and in particular relates to an image processing method, an image processing device, electronic equipment and a storage medium. Background AI image content elimination techniques refer to the use of artificial intelligence algorithms to remove specified content or objects from an image so that the image appears clearer and cleaner. At present, AI image content elimination techniques have been widely studied and applied. The AI image content local redrawing technology is an image editing method based on artificial intelligence technology, and is mainly used for redrawing a certain part of an image. This technique can be used in many applications, such as removing image noise, repairing damaged images, matting, etc. At present, AI image content elimination technology and AI image content local redrawing technology have become one of the popular research directions in the fields of computer vision and image processing. In the related art, local content elimination or local redrawing can be performed on a real image by using technologies such as deep learning, but as the demands of users increase, for example, it is expected to perform local content elimination or local redrawing on a non-real image such as a drawn cartoon, and higher requirements are put on the image processing technologies of local content elimination or local redrawing. It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art. Disclosure of Invention The disclosure provides an image processing method, an image processing device, an electronic device and a storage medium. According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method including: Determining an image modification area and an image modification type in response to a user input of an image modification instruction, wherein the image modification type comprises a first image modification type and a second image modification type; Extracting style characteristics of an image to be processed to obtain image style characteristic data; Based on the image modification area and the image modification type, carrying out noise addition processing on the image to be processed to obtain a first image; and generating an image according to the image to be processed, the image style characteristic data, the first image and the image modification type to obtain a target generated image. In some embodiments, performing style feature extraction on an image to be processed to obtain image style feature data, including: performing style detection on the image to be processed to obtain image style classification of the image to be processed; and carrying out style feature extraction on the image to be processed based on the image style classification to obtain the image style feature data. In some embodiments, the image processing method provided further includes: and mapping the image style characteristic data to obtain mapped image style characteristic data conforming to the output dimension of the low-rank adaptive model. In some embodiments, generating an image according to the image to be processed, the image style characteristic data, the first image and the image modification type to obtain a target generated image, including: and splicing channel dimensions of the mapped image style characteristic data through the input of the self-attention layer corresponding to the image generation model, wherein the image generation module is obtained by adjusting and training the potential diffusion model. In some embodiments, based on the image modification area and the image modification type, performing noise addition processing on the image to be processed to obtain a first image, including: Extracting an image modification region boundary range from the image to be processed based on the image modification region in response to the image modification type being a first image modification type; and carrying out noise adding processing on the image in the boundary range of the image modification area and the image of the image modification area to obtain the first image. In some embodiments, the image modification region boundary range includes a first range and a second range, the image modification region including the first range; the noise adding processing is performed on the image within the boundary range of the image modification area and the image of the image modification area, and the obtaining of the first image includes: Adding random noise to the image in the boundary range of the image modification area and the image of the image modification area to obtain a second