CN-119784658-B - Video coloring method based on similar frame assistance

Abstract

A video coloring method based on similar frame assistance aims at remarkably improving video quality, firstly, key frames are extracted from correctly colored video through a preprocessing step and necessary image processing is carried out, meanwhile, reference frames similar to the frames to be colored are generated, then, the frames (black and white, correct coloring and similar coloring) are jointly input into a deep learning network for training, the network can learn and utilize information among the frames to improve the coloring effect, finally, a confidence ranking mechanism is utilized, the similarity between the frames and the correct coloring frames is output through a comparison model, and the prediction result is evaluated and corrected.

Inventors

• JIA ZHAOHONG
• XU WENNA

Assignees

• 天津市兴奇光桦科技有限公司

Dates

Publication Date

20260508

Application Date

20241216

Claims (4)

1. 1. S1, video preprocessing, namely firstly, obtaining a correctly colored video, and carrying out similarity processing on each frame of image to obtain a similar frame; S2, training the three frames together, namely enabling the black-and-white frames, the correct coloring frames and the similar coloring frames to enter a coloring network together for training; S3, ranking the confidence coefficient of the result, namely performing similarity calculation on the result output by the model and a correct coloring frame and a similar coloring frame to obtain the ranking of the correct coloring frame and the similar coloring frame and the predicted result, and correcting the incorrect ranking; the coloring network in the step S2 has the structure that a first Block 1 is: 3 x 3 convolutional layers, each layer having 32 convolutional kernels, each convolutional layer being followed by a GeLU activation function, a 3 x 3 max pooling layer, step size 3, Second Block 2: 35 x 5 convolutional layers, each layer having 64 convolutional kernels, each convolutional layer being followed by a GeLU activation function, a 5 x 5 max pooling layer, step size 5, Third Block 3: Two 7 x 7 convolutional layers, each with 128 convolutional kernels, each followed by a GeLU activation function, a 7 x 7 max pooling layer, step size 7, The 1 st to 3 rd blocks are the front-end feature extraction blocks; Fourth Block 4: Two 7 x 7 convolutional layers, each with 128 convolutional kernels, each followed by a GeLU activation function, a 5 x5 max pooling layer, step size 5, Fifth Block 5: two 5 x5 convolutional layers, each with 128 convolutional kernels, each followed by a GeLU activation function, a 5 x5 max pooling layer, step size 5, The 4 th to 5 th blocks are post feature extraction blocks; The method comprises the steps of respectively processing black and white frames, correctly coloring frames and similar coloring frames to extract respective key features, combining the extracted features together in a weighted superposition mode, wherein each feature obtains different weights according to the importance of the feature, sending the combined features into a post-feature extraction block for further processing and refining to form a feature representation with more recognition, then, using the features to calculate cross entropy loss and evaluate the difference between the prediction and the actual value, finally, entering data into a scoring and ranking module based on the loss, and finally evaluating and ranking to complete the whole image processing flow.
2. 2. The video coloring method based on the similar frame assistance according to claim 1, wherein the similarity process in S1 is an image editing technology, wherein the visual effect is enhanced and the visual style is adapted by performing a series of fine adjustments on the original image, the adjustments including changing the color temperature, which involves adjusting the color temperature of the image to make the image exhibit different hues, the adjustment of the color temperature being used for creating an atmosphere; HSV adjustment, namely changing the color characteristics of the image by adjusting hue, saturation and brightness, wherein the hue adjustment changes the color in the image, the saturation adjustment changes the intensity of the color, and the brightness adjustment changes the overall brightness of the image; Contrast adjustment for changing the difference between the darkest and brightest regions in the image, thereby enhancing the depth and dimension of the image; Brightness adjustment-changing the overall brightness level of the image, brightness adjustment correcting underexposed or overexposed pictures and creating a visual atmosphere.
3. 3. The video coloring method based on similar frame assistance as defined in claim 2, wherein the structure of step S3 is that the scoring and ranking module: a cosine similarity calculation layer, The full connection layer is 1 to 256 dimensions, The full link layer is 2 to 128 dimensions, The full connection layer is 3:2 dimensional, The output of the post feature extraction block firstly enters a marking and ranking module, cosine similarity calculation is carried out in the module to determine the similarity between a black and white frame and a correctly colored frame and a similarly colored frame, the step is based on the principle of cosine similarity to evaluate the similarity degree of the feature vector directions among different frames, and then the features of the correctly colored frame and the similarly colored frame are sent to a full-connection layer for further marking, and in the link, the two-norm loss is calculated.
4. 4. A video coloring method based on similar frame assistance according to claim 3, wherein the two-norm loss is combined with the previously calculated cross entropy loss for weighting, and finally, the combined total loss is used for model back propagation, in which the parameters of the model are updated according to the gradient of the loss function, so as to optimize the model performance.

Description

Video coloring method based on similar frame assistance Technical Field The invention relates to the technical field of video processing, in particular to a video coloring method based on similar frame assistance. Background Video coloring technology, namely, converting original black-and-white video into color video, is an important research topic in the field of digital image processing, development and realization of the technology are significant for repairing historical image data, and a wide application space is provided for modern movie production, advertisement production and personal video editing, the origin of the video coloring technology can be traced back to the beginning of the 20 th century, most of movies at that time are black-and-white films, as the technology is developed, people start to try to color the movies manually, for example, by coloring films or using coloring templates, however, the method is time-consuming and labor-consuming, the effect is limited by the current technology level, the color is as far as the middle of 20 th century, as the color film technology is developed, the interest of people in early black-and-white movies is gradually increased, as the computer technology is developed, the video coloring starts to be digitalized, the initial digital coloring technology depends on the operation, the artist needs to color each frame of image independently, the process is still very time-consuming, however, as the time goes on the automatic and semi-automatic method starts to color quality and quality are greatly improved; The basic principle of video coloring is that a black-and-white image is analyzed through an algorithm, the color value of each pixel is predicted, the process involves a plurality of aspects including color theory, image segmentation, feature extraction, machine learning and the like, understanding how colors are combined and interacted in nature is important for predicting correct colors, the image can be divided into a plurality of areas or objects through image segmentation, each area can be independently colored, enough information such as texture, shape, edges and the like can be extracted from the black-and-white image to help the algorithm understand the image content, and a large amount of color image data is analyzed through a machine learning algorithm, especially deep learning, so that the training model predicts the colors; Modern video coloring technology increasingly depends on deep learning and artificial intelligence, a deep learning model can learn a complex color mode by training a large amount of color image data and automatically colors black and white images, and the methods not only improve the coloring accuracy, but also remarkably improve the processing speed; The application field of video coloring technology is very wide, in the aspect of repairing historical images, repairing historical films, documentaries and old photos, enabling the historical films, documentaries and old photos to reproduce colors, providing more abundant visual information for historical research and education, in movie and advertisement production, video coloring can be used for creating specific visual styles or emotion effects, and for personal entertainment, coloring personal videos and enhancing visual attractiveness and emotion expression of the videos; In the future, with the improvement of computing power and the continuous progress of deep learning technology, video coloring technology will become more efficient and accurate, in addition, with the development of artificial intelligence, we may see more innovative applications, such as real video coloring, personalized color selection and the like, and the development of video coloring technology not only represents the progress of technology, but also provides a brand new way for us to see and understand history, and simultaneously creates more possibilities for modern visual arts; However, the existing video coloring technology is not natural and accurate enough in use, and meanwhile, the overall consistency and the natural sense are not enough, so that the video coloring method based on similar frame assistance is used for solving the problems. Disclosure of Invention According to the technical problems, the invention provides a video coloring method based on similar frame assistance, which is characterized in that a video coloring process is improved by using a similar frame assistance strategy, the method can guide and optimize the coloring process by using color information in the similar frames, and can generate more natural and accurate coloring results by combining the color information extracted from an original frame and the similar frames, and the method further comprises fine adjustment of colors so as to ensure overall consistency and naturalness, and specifically comprises the following steps: S1, an image similarity processing is an image editing technology, and aims to enhance the