CN-122023131-A - Full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation

CN122023131ACN 122023131 ACN122023131 ACN 122023131ACN-122023131-A

Abstract

The invention relates to the technical field of imaging processing, in particular to a full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation. The method comprises the steps of 100, carrying out feature interaction on a low-resolution light field image through feature extraction and convolution calculation of different depths to obtain a super-resolution light field image with different depth features, 102, determining a space variable kernel matrix with different depths according to the low-resolution light field image and the super-resolution light field image under the constraint of a parallax image based on depth constraint and combining mask depths, 104, carrying out convolution embedding on the super-resolution light field image with different depth features and the space variable kernel matrix with different depths to obtain a new super-resolution light field image, and repeating 102-104, and optimizing and iterating for a plurality of times to obtain the super-resolution light field image with depth features. The scheme can complete the full-focus super-resolution task.

Inventors

KONG DEQIAN
GUAN LING
LI LIANGSHENG
SUN WANG
YANG MENG
HUANG JIAWEI
ZHOU MAO

Assignees

北京环境特性研究所

Dates

Publication Date: 20260512
Application Date: 20260202

Claims (10)

1. The full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation is characterized by comprising the following steps of: 100, performing feature interaction on the low-resolution light field image through feature extraction and convolution calculation of different depths to obtain a super-resolution light field image with different depth features; 102, determining a space variable kernel matrix with different depths according to the low-resolution light field image and the super-resolution light field image under the constraint of a parallax map based on depth constraint and combining mask depths; 104, respectively carrying out convolution embedding on the super-resolution light field images with different depth characteristics and the space variable kernel matrixes with different depths to obtain new super-resolution light field images; And repeating 102-104, and obtaining the super-resolution light field image with depth characteristics after optimization iteration is performed for a plurality of times.
2. The method according to claim 1, wherein after performing feature interaction on the low-resolution light field image through feature extraction and convolution calculation of different depths, obtaining a super-resolution light field image with different depth features includes: Inputting a low-resolution light field image in an image pyramid structure; Extracting features of the images in the pyramid structure of the input images and reducing the dimension of the image features; Restoring the spatial features and details of the graph through up-sampling and convolution calculation; And dividing the features into different depth features by using a depth mask to form super-resolution light field images with the different depth features.
3. The method of claim 1, wherein determining a spatially variable kernel matrix having different depths in combination with mask depths based on depth constraints from the low-resolution light field image and the super-resolution light field image under the constraint of a disparity map comprises: The super-resolution light field image is connected with the low-resolution light field image in a channel dimension after being subjected to convolution downsampling; Respectively multiplying the super-resolution light field image and the low-resolution light field image pixel by using masks with different depths to obtain a plurality of image features under the same depth, wherein the masks are obtained by performing depth segmentation and binarization on a light field standard parallax image; All image feature channel dimensions are connected, and conditional input and basic input are integrated through residual mapping to enhance features, so that the space variable kernel matrix is obtained.
4. A method according to claim 3, wherein the conditional input is a downsampled feature of the super-resolution light field image and the basic input is a low resolution light field image.
5. A method according to claim 3, characterized in that the residual map comprises a convolution layer and a channel attention layer.
6. The method of claim 1, wherein the super-resolution light field image standard form is a light field sub-aperture image array.
7. The method of claim 1, wherein the convolution calculation relies on a depth convolution module comprising a feature-integration layer for integrating features, a multi-feature extraction module for extracting features, and a window transformer.
8. A fully focused blind super resolution reconstruction apparatus based on depth constrained kernel estimation for implementing the method of any one of claims 1-7, the apparatus comprising: the full-focusing super-resolution module is used for carrying out feature interaction on the low-resolution light field image through feature extraction and convolution calculation of different depths to obtain super-resolution light field images with different depth features, and is also used for carrying out convolution embedding on the super-resolution light field images with different depth features and the space variable kernel matrix with different depths respectively to obtain new super-resolution light field images; And the blind kernel estimation module is used for determining a space variable kernel matrix with different depths according to the low-resolution light field image and the super-resolution light field image under the constraint of the parallax map and combining the mask depths based on depth constraint.
9. An electronic device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the method of any of claims 1-7 when the computer program is executed.
10. A computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any of claims 1-7.

Description

Full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation Technical Field The invention relates to the technical field of imaging processing, in particular to a full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation. Background Light field imaging technology is gaining wide attention because of its special four-dimensional imaging structure. The camera can record the intensity and direction information of light through a four-dimensional structure, wherein the view angle dimension records the direction of the light, and the space dimension records the intensity of the light. Because of the need to save both spatial and perspective information and the limitations in detector size, the spatial resolution of light field images is low compared to conventional imaging techniques. Therefore, improving the spatial resolution of the light field imaging system becomes a key subject in the research field for performance optimization and functional expansion in subsequent applications. Therefore, it is desirable to provide a full focus blind super resolution reconstruction method based on depth constraint kernel estimation. Disclosure of Invention The embodiment of the invention provides a full-focus blind super-resolution reconstruction method, device, electronic equipment and storage medium based on depth constraint kernel estimation, which can explore the three-dimensional implementation of a fuzzy kernel by utilizing depth information in a light field image, develop full-focus blind super-resolution research based on depth learning on the basis, construct an end-to-end full-focus light field blind super-resolution network based on an iterative optimization strategy and finish full-focus super-resolution tasks in one step. In a first aspect, an embodiment of the present invention provides a method for full-focus blind super-resolution reconstruction based on depth constraint kernel estimation, including: 100, performing feature interaction on the low-resolution light field image through feature extraction and convolution calculation of different depths to obtain a super-resolution light field image with different depth features; 102, determining a space variable kernel matrix with different depths according to the low-resolution light field image and the super-resolution light field image under the constraint of a parallax map based on depth constraint and combining mask depths; 104, respectively carrying out convolution embedding on the super-resolution light field images with different depth characteristics and the space variable kernel matrixes with different depths to obtain new super-resolution light field images; And repeating 102-104, and obtaining the super-resolution light field image with depth characteristics after optimization iteration is performed for a plurality of times. In a second aspect, an embodiment of the present invention further provides a device for blind super-resolution reconstruction with full focus based on depth constraint kernel estimation, where the device is configured to implement a method according to any one embodiment of the present specification, and the device includes: the full-focusing super-resolution module is used for carrying out feature interaction on the low-resolution light field image through feature extraction and convolution calculation of different depths to obtain super-resolution light field images with different depth features, and is also used for carrying out convolution embedding on the super-resolution light field images with different depth features and the space variable kernel matrix with different depths respectively to obtain new super-resolution light field images; And the blind kernel estimation module is used for determining a space variable kernel matrix with different depths according to the low-resolution light field image and the super-resolution light field image under the constraint of the parallax map and combining the mask depths based on depth constraint. In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of the present specification is implemented. In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform a method according to any of the embodiments of the present specification. Compared with the prior art, the invention has at least the following beneficial effects: The embodiment of the invention provides a full-focus blind super-resolution reconstruction method based on depth constraint kernel estimation, which introduces a space variable kernel matrix calculation method constrained b