CN-120543673-B - Image coloring method, device and medium based on iterative optimization

CN120543673BCN 120543673 BCN120543673 BCN 120543673BCN-120543673-B

Abstract

The application discloses an image coloring method, equipment and medium based on iterative optimization, and relates to the technical field of image processing, wherein the method comprises the steps of obtaining a gray image; the method comprises the steps of performing initial coloring on a gray image by using a trained conditional coloring model to obtain an initial coloring result, performing color evaluation on the initial coloring result by using a trained evaluation model to identify areas which do not meet a set standard of color confidence in the initial coloring result to obtain unsatisfactory areas, re-coloring the unsatisfactory areas by using the trained conditional coloring model to obtain updated coloring results, replacing the initial coloring result with the updated coloring results, repeating the re-coloring and evaluation processes to obtain a final coloring result, and decoding the final coloring result to obtain a final color image.

Inventors

Dai longquan
WANG SHAOMENG
ZHANG LIYAN
CHEN YIBING
DU XIAOYU

Assignees

南京派米智能科技有限公司

Dates

Publication Date: 20260508
Application Date: 20250514

Claims (7)

1. An iterative optimization-based image coloring method, characterized in that the iterative optimization-based image coloring method comprises the following steps: Acquiring a gray level image; the gray image is initially colored by using a trained conditional coloring model to obtain an initial coloring result, which specifically comprises the steps of respectively encoding the gray image and a color prompt image to obtain a gray token sequence and a color prompt token sequence, wherein the color prompt image is an image obtained by carrying out color marking on a part of areas in the gray image, the gray token sequence comprises a plurality of gray tokens, the color prompt token sequence comprises a plurality of color prompt tokens, an intermediate token sequence is generated according to the gray token sequence and the color prompt token sequence, the intermediate token sequence and a masking mask sequence are spliced to obtain a splicing mask sequence, the masking mask sequence is a token sequence which is masked by all masks and has the same size as the intermediate token sequence, the splicing mask sequence is input into the trained conditional coloring model to obtain an initial coloring result, the conditional coloring model comprises a visual mask token model and a conditional diffusion model, the visual mask token model is used for processing the splicing token sequence to obtain an intermediate token sequence, and the conditional diffusion model is used for iterating the intermediate token sequence; Performing color evaluation on the initial coloring result by using a trained evaluation model, and identifying areas which do not meet the set standard of the color confidence in the initial coloring result to obtain unsatisfactory areas, wherein the method specifically comprises the following steps of: inputting the initial coloring result and the gray token sequence into a trained evaluation model, evaluating the color confidence in the initial coloring result, and determining an area, which does not accord with a set standard, of the color confidence of the initial coloring result as an unsatisfactory area; masking the unsatisfactory region to obtain an updated masking mask sequence; Re-coloring the unsatisfactory area by using the trained conditional coloring model to obtain an updated coloring result; and replacing the initial coloring result with the updated coloring result, repeating the re-coloring and evaluating processes to obtain a final coloring result, and decoding the final coloring result to obtain a final color image.
2. The iterative optimization-based image coloring method according to claim 1, wherein an intermediate token sequence is generated from the gray token sequence and the color hint token sequence, specifically comprising: Splicing the gray token sequence and the color prompt token sequence to obtain a spliced token sequence; and processing the spliced token sequence by using MLP to obtain an intermediate token sequence.
3. The iterative optimization-based image rendering method of claim 1, wherein the conditional diffusion model comprises a number of residual blocks, each residual block comprising LayerNorm, linear transforms, and SiLU activation function layers.
4. The iterative optimization-based image coloring method according to claim 1, wherein the unsatisfied area is recoloured by using the trained conditional coloring model to obtain updated coloring results, specifically comprising: Replacing the updated shielding mask sequence with the shielding mask sequence, and returning to the step of splicing the intermediate token sequence and the shielding mask sequence to obtain an updated spliced mask sequence; and inputting the updated spliced mask sequence into a trained conditional coloring model to obtain an updated coloring result.
5. The iterative optimization-based image coloring method according to claim 1, wherein the evaluation model comprises a first fully connected layer, a Mixer module and a second fully connected layer which are sequentially connected, and the Mixer module comprises a plurality of Mixer layers.
6. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement the iterative optimization based image rendering method of any one of claims 1-5.
7. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the iterative optimization based image rendering method of any one of claims 1-5.

Description

Image coloring method, device and medium based on iterative optimization Technical Field The present application relates to the field of image processing technologies, and in particular, to an image coloring method, apparatus, and medium based on iterative optimization. Background Many black and white photographs from the past age still exist to date, often requiring the coincidental hands of the artist to add color to them, making them a realistic task. However, such manual coloring processes are not only time consuming, but also laborious. With a single coloring paradigm, many approaches have been proposed that aim to replicate the prior knowledge and intuition of human experts. These methods have made significant progress, providing high quality results. Although some of these related techniques incorporate additional conditions, they generally still lack iterative optimization that human experts typically do, resulting in lower realism and fidelity of image rendering. Disclosure of Invention The application aims to provide an image coloring method, device and medium based on iterative optimization, which can realize higher realism and fidelity in the aspect of image coloring. In order to achieve the above object, the present application provides the following solutions: in a first aspect, the present application provides an image rendering method based on iterative optimization, comprising the following steps. A gray scale image is acquired. And (3) carrying out initial coloring on the gray level image by using the trained conditional coloring model to obtain an initial coloring result. And carrying out color evaluation on the initial coloring result by using the trained evaluation model, and identifying the region which does not accord with the set standard of the color confidence in the initial coloring result to obtain an unsatisfactory region. And re-coloring the unsatisfactory area by using the trained conditional coloring model to obtain an updated coloring result. And replacing the initial coloring result with the updated coloring result, repeating the re-coloring and evaluating processes to obtain a final coloring result, and decoding the final coloring result to obtain a final color image. In a second aspect, the application provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the iterative optimization-based image rendering method described above. In a third aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described iterative optimization-based image rendering method. According to the specific embodiment provided by the application, the application discloses the following technical effects: The application provides an image coloring method, device and medium based on iterative optimization, which are characterized in that a trained evaluation model is used for carrying out color evaluation on an initial coloring result, a region which does not meet a set standard of color confidence in the initial coloring result is identified, an unsatisfactory region is obtained, the unsatisfactory region is subjected to recolouring by using a trained conditional coloring model, and in a coloring stage, the coloring model generates initial color prediction. The evaluation stage uses an evaluation model to identify the region with lower color confidence, and in the recolouring stage, the unsatisfactory region (i.e. the region with lower color confidence) is recoloured, so that the region needing improvement can be purposefully adjusted, the self-adaptive feedback loop can iteratively improve the quality and consistency of the color, higher sense of reality and fidelity are realized in the aspect of coloration, and the image coloring quality is improved. Drawings In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. FIG. 1 is a diagram of an application environment of an image coloring method based on iterative optimization in an embodiment of the application. Fig. 2 is a flowchart of an image coloring method based on iterative optimization according to an embodiment of the present application. FIG. 3 is a schematic diagram of an iterative coloring-evaluation-recolouring process according to an embodiment of the present application. Fig. 4 is a schematic diagram of a training phase and an reasoning phase of image coloring according to an embodiment of the present