CN-121981883-A - Image interpolation method based on double-stage disturbance and DR splitting expansion

CN121981883ACN 121981883 ACN121981883 ACN 121981883ACN-121981883-A

Abstract

The invention belongs to the technical field of image interpolation, and particularly discloses an image interpolation method based on double-stage disturbance and DR splitting expansion. According to the method, the linear interpolation operator is used as an initialization module of the network to conduct initial interpolation, and is mapped into the graph adjacent matrix through the interpolation operator theorem, so that the stability of the training process and the reliability of interpolation results are ensured. In addition, the invention realizes high-precision and interpretable image interpolation by introducing a double-stage optimization model of the directed graph disturbance matrix and the undirected graph disturbance matrix and expanding the model into a learnable neural network structure by utilizing a DR splitting iterative algorithm. The directed graph carries out self-adaptive modeling and correction on the connection relation between the observed pixels and the pixels to be interpolated, and the undirected graph carries out local refinement constraint on the smoothness between the interpolated pixels, so that the high-frequency detail and the overall smoothness of the image are effectively enhanced while the original structural information is maintained, and the accuracy of the image interpolation result is ensured.

Inventors

ZHANG XUE
Hu Bingshuo
LU FAMING
WANG LU
WANG ZHIHUI
YUAN GUIYUAN
LI XIANGJU

Assignees

山东科技大学

Dates

Publication Date: 20260505
Application Date: 20260408

Claims (10)

1. An image interpolation method based on double-stage disturbance and DR splitting expansion is characterized by comprising the following steps: Step 1, acquiring original image data and dividing the original image data into a plurality of image blocks; step 2, initializing the relation between the known pixels and the pixels to be interpolated in each image block, and establishing an initial adjacent matrix Acquiring an initial interpolation image; Step 3, constructing a first disturbance matrix by using the initial interpolation image For a pair of Carrying out disturbance enhancement; Step 4, establishing a MAP optimization model based on GSV based on the new adjacency matrix after disturbance enhancement, and solving the optimization model through a developed BICG iteration algorithm to obtain an interpolation image in the first stage; Step 5, introducing a second disturbance matrix Smoothing and refining the interpolation image in the first stage; Step6, based on And (3) with Establishing a joint optimization model containing graph displacement variation items and graph Laplace regularization items; solving a joint optimization model through DR splitting, and converting a DR iterative optimization process into an end-to-end trainable network through expansion, wherein the network comprises a plurality of iteration layers, and each iteration layer corresponds to one iteration of DR; each iteration executes the following process until the iteration is finished; Construction of input images by means of graph learning And Wherein in the first iteration layer, the interpolation image of the first stage is used as input, and in other iteration layers, the output image of the previous iteration layer is used as input; constructing a sub-problem aiming at a graph displacement variation item and a graph Laplace regularization item, and respectively solving by using an expanded BICG iteration algorithm to obtain a first intermediate variable and a second intermediate variable; updating the pixel to be interpolated by combining the iteration updating formula with the first intermediate variable and the second intermediate variable, and splicing and fusing the pixel to be interpolated with the original known pixel to obtain an output image of the current iteration layer; and 7, splicing and fusing all the optimized image blocks to construct a complete interpolation image.
2. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 1, The first disturbance matrix Corresponds to a directed graph for describing the connection between the known pixel and the pixel to be interpolated and the similarity measure, Corresponds to an undirected graph for describing the connection relationship and smoothness constraint between the pixels to be interpolated.
3. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 1, In the step 2, initializing the relation between the known pixels and the pixels to be interpolated in each image block according to a preset linear interpolation operator, and establishing an initial adjacency matrix And acquiring an initial interpolation image, wherein the process is as follows: The initial adjacency matrix a is defined as a sparse blocking matrix expressed in mathematical form as: ; Wherein the method comprises the steps of Is one × Is provided for the initial contiguous matrix sub-blocks, for describing slave The pixels to be interpolated are connected back Directed edge relationships for individual known pixels; 、、 Respectively is × 、 × 、 × Zero matrix of (a); And (3) with Representing the number of known pixels and the number of pixels to be interpolated in each image block, ; In the known linear interpolation operator Mapping relation between known pixel set and pixel to be interpolated is established based on mapping relation between known pixel set and pixel to be interpolated, and linear interpolation operator is ensured On the premise of being reversible, the device can be used for controlling the temperature of the liquid, Is arranged as Inverse matrix of (a), i.e 。
4. The method for image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 3, In the step 2, a linear interpolation operator For the improved bicubic interpolation operator Bicubic +; wherein Bicubic + utilizes a lightweight multi-layer perceptron MLP network, and the MLP network adaptively predicts 4×4 neighborhood interpolation weight according to the coordinate position difference of the pixel to be interpolated and the neighborhood pixel; the MLP comprises three hidden layers, wherein the dimensionalities of each layer are respectively 32, 64 and 32, the input dimensionality is 2 and represents the relative coordinate difference in the x and y directions, and the output dimensionality is 1 and represents the weight coefficient of the corresponding position; The weight coefficient is processed by a Sigmoid activation function to ensure the positive value of the weight coefficient, so that accurate self-adaptive interpolation weights are generated for 16 pixels in the neighborhood, and the restoration precision of the image edge and texture detail in the initialization stage is improved.
5. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 2, The first disturbance matrix Corresponding directed graph Edge weights of (a) Is signed, a first disturbance matrix Mathematically defined as a sparse partitioned matrix whose structure is expressed as: ; is a learnable disturbance submatrix for describing a slave The pixels to be interpolated are connected back The directional connection and similarity measure for each known pixel, 、、 Zero matrices of m× M, N × M, N ×n, respectively; Wherein the method comprises the steps of And Respectively representing the number of known pixels and the number of pixels to be interpolated in each image block; By introducing the blocking matrix, an initial adjacency matrix Enhanced to a new adjacency matrix ; Learnable perturbation submatrices Is calculated based on pixel features of the initial interpolated image extracted by a shallow CNN that maps each pixel of the image into a high-dimensional feature vector ; For a pair of pixel points, their characteristic distance By the formula Calculating to obtain; where index i represents the pixel to be interpolated, index j represents the known pixel, Is a learnable positive definite metric matrix; Representing a high-dimensional feature vector corresponding to the pixel to be interpolated, Representing a high-dimensional feature vector corresponding to the known pixel; Finally, a first disturbance matrix Is a signed edge weight of (1) By the formula Determining; Wherein the method comprises the steps of Is a predetermined offset as a demarcation point for defining similarity and dissimilarity.
6. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 1, In the step 4, the objective function of the MAP optimization model using GSV as the smoothness constraint prior is defined as: ; Wherein the method comprises the steps of Known pixel data for the original image; Is a new adjacency matrix after enhancement; is a regularization parameter; Is a unit matrix; Representing the complete signal; is a sampling matrix for the signal from the whole Before selecting in (a) -A number of known pixel signals; Representation of Is a matrix of units of (a); Representation of Zero matrix of (a); By relating to the above objective function Deriving and zeroing the pixel to be interpolated The solution of (2) is reduced to the following linear system: ; Wherein the method comprises the steps of Is one × Is provided for the initial contiguous matrix sub-blocks, Is a learnable disturbance submatrix, using And linear interpolation operator Inverse matrix mapping relation of (a) The linear system equivalent transformation is: Wherein Representation of Is a matrix of units of (a); aiming at the linear system, a biconjugate gradient BICG iteration algorithm is adopted to solve, and an iteration process of BICG is unfolded into a neural network layer to realize the predicted value of the pixel to be interpolated in the first stage Is calculated and parameterized; Will then And is combined with the known pixel data y into a complete image, i.e. a first stage interpolated image.
7. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 2, The second disturbance matrix Corresponding undirected graph The method is used for describing the connection relation and smoothness constraint between pixels to be interpolated, and aims to further denoise and refine the interpolation result in the first stage; Defined as a symmetrical and semi-definite matrix of the graph Laplace whose structure is connected only Pixel points to be interpolated; The disturbance matrix Is adaptively generated based on pixel features extracted by a shallow CNN mapping each pixel of the image into a high-dimensional feature vector based on the first-stage interpolation result ; For a pair of pixel points to be interpolated, their characteristic distances By the formula Calculating; where indices i and j both represent the pixels to be interpolated, Is a learnable positive definite metric matrix; representing the high-dimensional feature vector corresponding to the pixel i to be interpolated, Representing a high-dimensional feature vector corresponding to a pixel j to be interpolated; finally, undirected graph Positive edge weight of (2) By the formula And (5) determining.
8. The method of image interpolation based on dual-stage perturbation and DR splitting expansion according to claim 6, The joint optimization model comprising graph displacement variation term and graph Laplace regularization term is defined as: Wherein Is a selection matrix for extracting pixels to be interpolated; representing a zero matrix of N x M; Representation of Is a matrix of units of (a); The combined optimization model The first two items of (a) are MAP optimization models in the step 4, and comprise MAP displacement variation items; the third term of (2) is the Laplace term corresponding to the second disturbance matrix The formed undirected graph smoothes the prior; Solving through expanded DR splitting iterative algorithm The process of (1) is specifically as follows: will first Splitting into map displacement variation terms Regularization term with graph Laplace : ; ; For a pair of And (3) with Respectively deriving and making zero to obtain two linear systems corresponding to near-end mapping, wherein each iteration layer is respectively an h-step linear system and a g-step linear system; The expression of the h-step linear system is as follows: ; the expression of the g-step linear system is as follows: ; Wherein the superscript k is the number of the kth iteration layer; Is a first intermediate variable; Is a step size parameter; And (3) with Interpolation results of the current iteration layer and the previous iteration layer are respectively obtained; Is a regularization parameter; is a second disturbance matrix; Is a second intermediate variable; The DR splitting iterative algorithm solves a first intermediate variable and a second intermediate variable by alternately calling two linear systems, namely an h-step linear system and a g-step linear system, and combining variable updating formulas to respectively and iteratively solve; after the solving of the h step and the g step is completed, the iterative solution of the kth layer is updated through the DR iterative updating formula And (5) performing iterative updating: ; Wherein, the For the updated solution, i.e. the predicted pixel to be interpolated.
9. A computer device comprising a memory and one or more processors, wherein the memory has stored therein executable code that, when executed by the processors, is operable to implement the image interpolation method of any one of claims 1 to 8 based on dual-stage perturbation and DR split expansion.
10. A computer readable storage medium having stored thereon a program which, when executed by a processor, is adapted to implement the image interpolation method based on dual-stage perturbation and DR split expansion of any one of claims 1 to 8.

Description

Image interpolation method based on double-stage disturbance and DR splitting expansion Technical Field The invention belongs to the field of image interpolation, and relates to an image interpolation method based on double-stage disturbance and DR splitting expansion. Background Image interpolation is the basic research direction in the field of image processing and computer vision, and is mainly aimed at estimating and reconstructing missing or low-resolution pixels in an image by using the information of part of known pixel points, so as to obtain a more complete and high-resolution image. In the traditional image interpolation method, algorithms such as bilinear interpolation, bicubic interpolation and spline interpolation have the advantages of simple calculation and convenient implementation, but because the interpolation methods are usually based on local weighted average or polynomial fitting, the interpolation results often appear blurring and structural distortion in complex textures, edges and detail areas. In addition, the conventional interpolation method cannot adaptively adjust the interpolation weight, so that it is difficult to obtain an ideal reconstruction effect in both a high-frequency variation region and a low-texture region. With the development of deep learning, an image interpolation and reconstruction method based on a convolutional neural network (Convolutional Neural Network, CNN) and a visual transducer gradually becomes a research hotspot. The method realizes complex feature transformation from low resolution to high resolution through end-to-end nonlinear mapping, and achieves remarkable improvement on objective evaluation indexes (such as peak signal to noise ratio PSNR and structural similarity SSIM). However, such data-driven models often lack interpretability, have large parameter scales and rely on a large amount of labeling data for training, have high demands on computing resources and video memory, and have poor stability of reconstruction results when input noise or observation loss is serious. In addition, conventional depth networks tend to ignore topological correlations between pixels in the interpolation task, resulting in insufficient modeling of local structure and global consistency. Therefore, how to construct a high-efficiency interpolation method which has mathematical interpretability and can adaptively capture the structural characteristics of an image becomes a key problem of current research. The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. Disclosure of Invention Aiming at the problems of insufficient interpretability, fixed structure and limited interpolation precision in the prior art, the invention provides an image interpolation method based on double-stage disturbance and DR division expansion, wherein a traditional linear interpolation operator is mapped into an initial adjacent matrix, a double-stage optimization model of a directed graph disturbance matrix and an undirected graph disturbance matrix is introduced, and the image interpolation method is expanded into a learnable neural network structure by utilizing a DR division iteration algorithm so as to realize high-precision and interpretable image interpolation. In order to achieve the above purpose, the invention adopts the following technical scheme: an image interpolation method based on double-stage disturbance and DR splitting expansion comprises the following steps: Step 1, acquiring original image data and dividing the original image data into a plurality of image blocks; step 2, initializing the relation between the known pixels and the pixels to be interpolated in each image block, and establishing an initial adjacent matrix Acquiring an initial interpolation image; Step 3, constructing a first disturbance matrix by using the initial interpolation image For a pair ofCarrying out disturbance enhancement; Step 4, establishing a MAP optimization model based on GSV based on the new adjacency matrix after disturbance enhancement, and solving the optimization model through a developed BICG iteration algorithm to obtain an interpolation image in the first stage; Step 5, introducing a second disturbance matrix Smoothing and refining the interpolation image in the first stage; Step6, based on And (3) withEstablishing a joint optimization model containing graph displacement variation items and graph Laplace regularization items; solving a joint optimization model through DR splitting, and converting a DR iterative optimization process into an end-to-end trainable network through expansion, wherein the network comprises a plurality of iteration layers, and each iteration layer corresponds to one iteration of DR; each iteration executes the following process until the iteration is finished; Construction of input images by means of graph learning AndWherein in the first iteration layer, the inte