CN-121981898-A - Space-time-based image sequence correction method, device, equipment and medium

CN121981898ACN 121981898 ACN121981898 ACN 121981898ACN-121981898-A

Abstract

The invention discloses a space-time based image sequence correction method, a device, equipment and a medium, which relate to the technical field of image processing, wherein the method comprises the steps of removing blind pixels and noise points of a kth frame of infrared image based on median filtering to obtain an initial expected image; filtering and fusing the initial expected image based on the guide filtering and the side window filtering to obtain a fused expected image, obtaining global difference, local difference and image smoothness between frames according to the k-1 frame infrared image and the k-2 frame infrared image, updating the self-adaptive step length, substituting the self-adaptive step length of the fused expected image and the k-1 frame infrared image into a preset image correction formula for calculation, and obtaining a target expected image corrected by the k frame infrared image. The invention ensures that the edge details of the image are reserved through the joint filtering, and updates the self-adaptive step length based on the time-space sequence information, so that the image correction formula is more accurate, the non-uniformity is removed while the details are reserved, and the 'ghost' phenomenon of the image sequence is effectively avoided.

Inventors

QI SHUXIA
YAN XUN
Xu Xiujin
WANG SHUN
ZHOU HUIXIN
XIANG PEI
LI HUAN
WANG BINGJIAN
SHI JIN
ZHU YONG
LI ZHIBIN

Assignees

西安电子科技大学

Dates

Publication Date: 20260505
Application Date: 20251226

Claims (10)

1. A method for correcting a space-time based image sequence, the method comprising: acquiring a kth frame of infrared image, wherein K epsilon [3, K ] is the total frame number in an infrared image sequence to be corrected; performing blind pixel and noise point identification on the k frame infrared image, and removing the identified blind pixels and noise points based on median filtering to obtain an initial expected image after denoising; filtering and fusing the initial expected image based on the guide filtering and the side window filtering to obtain a fused expected image; Obtaining global differences, local differences and image smoothness between frames according to pixel values of a kth-1 frame infrared image and a kth-2 frame infrared image in the infrared image sequence, and updating self-adaptive step sizes according to the global differences, the local differences and the image smoothness between frames to obtain self-adaptive step sizes of the kth-1 frame infrared image; substituting the self-adaptive step length of the fusion expected image and the k-1 frame infrared image into a preset image correction formula to calculate so as to obtain a target expected image corrected by the k frame infrared image.
2. The method of claim 1, wherein the performing blind pixel and noise point recognition on the kth frame of infrared image, and removing the recognized blind pixel and noise point based on median filtering, to obtain a denoised initial expected image, specifically includes: based on a double-threshold segmentation technology, marking pixels in the kth frame of infrared image, the pixel values of which do not meet the preset pixel threshold condition, as blind pixels or noise points; Determining denoising windows corresponding to all the blind pixels or the noise points on the kth frame of infrared image according to the positions of all the blind pixels or the noise points and the preset first window size; and sequencing the pixel values of all pixels except the blind pixels or the noise points in the denoising window, and replacing the pixel values of the blind pixels or the noise points corresponding to the denoising window with the median value of the sequenced sequence to obtain a denoised initial expected image.
3. The method according to claim 1, wherein the filtering and fusing the initial desired image based on the guided filtering and the side window filtering to obtain a fused desired image specifically includes: conducting guided filtering on the initial expected image to obtain a filtered first expected image; performing side window filtering on the initial expected image to obtain a filtered second expected image; and carrying out weighted fusion calculation according to the first expected image, the second expected image and preset weights to obtain a fusion expected image.
4. A method according to claim 3, wherein the performing guided filtering on the initial desired image to obtain a filtered first desired image comprises: taking the initial expected image as an input image and a guide image, and obtaining a plurality of local windows of the guide image according to the guide image and a preset sliding window; Calculating according to pixel values in all the local windows, pixel values in the input image and a preset coefficient calculation formula to obtain a linear coefficient of the initial expected image; calculating according to the linear coefficient, pixel values of pixels in the local windows and a preset linear formula to obtain a local output image of each local window after linear solution; Calculating the average value of the corresponding pixel values of the target pixel in all the local output images, and determining a first expected pixel value of the target pixel, wherein the target pixel is any pixel in the initial expected image; a first desired image is determined from the first desired pixel values of all of the target pixels.
5. A method according to claim 3, wherein said side window filtering said initial desired image to obtain a filtered second desired image, comprises: Performing edge detection on the initial expected image by using a Sobel operator, and determining an edge in the initial expected image; Searching a center pixel in the initial expected image, and determining a filter window on the initial expected image according to the center pixel and a preset second window size; dividing the region in the filter window into an effective region and an ineffective region according to the edge in the filter window, wherein the boundary between the effective region and the ineffective region is an edge, if the gray scale difference between the central pixel of the filter window and the target region is small, the target region is the effective region, the other regions are the ineffective regions, and the target region is any one region of all regions divided by the edge of the filter window; Moving the center pixel to the boundary or corner of the effective area, and forming a new filter window according to the size of the second window; After a filtering window is determined, carrying out weighted average calculation according to all pixel values of pixels in an effective area of the filtering window to obtain a second expected pixel value of a center pixel corresponding to the filtering window after filtering; And generating a second expected image according to the pixel values of the pixels in the initial expected image and all the second expected pixel values.
6. The method according to claim 1, wherein the obtaining global differences, local differences and image smoothness between frames according to pixel values of the k-1 th frame of infrared image and the k-2 th frame of infrared image in the infrared image sequence, and updating the adaptive step size according to the global differences, the local differences and the image smoothness between frames, and obtaining the adaptive step size of the k-1 th frame of infrared image specifically includes: obtaining global differences and local differences among frames according to pixel values of a kth-1 frame infrared image and a kth-2 frame infrared image in the infrared image sequence and preset global motion factors and local motion factors; determining image smoothness according to pixel values in the k-1 frame infrared image; And obtaining the self-adaptive step length corresponding to the k-1 frame infrared image according to the product of the image smoothness, the global difference and the local difference among frames and the preset basic step length.
7. The method of claim 6, wherein substituting the adaptive step sizes of the fused desired image and the k-1 frame infrared image into a preset image correction formula for calculation to obtain the target desired image corrected by the k frame infrared image specifically comprises: Substituting the self-adaptive step length of the k-1 frame infrared image into a preset correction coefficient updating formula to calculate so as to obtain a correction coefficient corresponding to the k frame infrared image; substituting the correction coefficient corresponding to the kth frame of infrared image and the fusion expected image into a preset image correction formula to obtain a corrected target expected image.
8. A spatio-temporal based image sequence correction apparatus, the apparatus comprising: the image acquisition unit is used for acquiring a kth frame of infrared image, wherein K is E [3, K ], and K is the total frame number in the infrared image sequence to be corrected; The image filtering unit is used for identifying blind pixels and noise points of the k frame infrared image, removing the identified blind pixels and noise points based on median filtering to obtain a denoised initial expected image; The image correction unit is used for obtaining global difference, local difference and image smoothness between frames according to pixel values of a k-1 frame infrared image and a k-2 frame infrared image in the infrared image sequence, updating the self-adaptive step length according to the global difference, the local difference and the image smoothness between frames to obtain the self-adaptive step length of the k-1 frame infrared image, substituting the self-adaptive step length of the fusion expected image and the self-adaptive step length of the k-1 frame infrared image into a preset image correction formula for calculation, and obtaining the target expected image corrected by the k frame infrared image.
9. A computer readable storage medium storing a computer program, which when executed by a processor causes the processor to perform the steps of the method according to any one of claims 1 to 7.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program which, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1 to 7.

Description

Space-time-based image sequence correction method, device, equipment and medium Technical Field The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for correcting an image sequence based on space-time. Background In practical engineering applications, the information of a single image is often insufficient to support efficient non-uniformity correction of the entire image sequence, and most scene-based non-uniformity correction algorithms rely on the image sequence. Multiple scholars find the relation and the difference among multiple frames of images, so that the non-uniformity correction is better realized, and a neural network-based non-uniformity correction algorithm provides clear thinking and iterative modes for the non-uniformity correction of an infrared image sequence. However, the current non-uniformity correction algorithm based on the neural network is poor in image correction effect, so that 'ghosts' appear in the corrected image, and a good correction effect is not achieved. Disclosure of Invention Based on this, it is necessary to provide a space-time based image sequence correction method, apparatus, device and medium to solve the above problems, so as to effectively remove the non-uniformity of the infrared sequence while retaining the details, and effectively avoid the "ghosting" phenomenon of the image sequence. To achieve the above object, a first aspect of the present application provides a space-time based image sequence correction method, the method comprising: acquiring a kth frame of infrared image, wherein K epsilon [3, K ] is the total frame number in an infrared image sequence to be corrected; performing blind pixel and noise point identification on the k frame infrared image, and removing the identified blind pixels and noise points based on median filtering to obtain an initial expected image after denoising; filtering and fusing the initial expected image based on the guide filtering and the side window filtering to obtain a fused expected image; Obtaining global differences, local differences and image smoothness between frames according to pixel values of a kth-1 frame infrared image and a kth-2 frame infrared image in the infrared image sequence, and updating self-adaptive step sizes according to the global differences, the local differences and the image smoothness between frames to obtain self-adaptive step sizes of the kth-1 frame infrared image; substituting the self-adaptive step length of the fusion expected image and the k-1 frame infrared image into a preset image correction formula to calculate so as to obtain a target expected image corrected by the k frame infrared image. Further, the step of performing blind pixel and noise point recognition on the kth frame of infrared image, and removing the recognized blind pixel and noise point based on median filtering to obtain a denoised initial expected image specifically includes: based on a double-threshold segmentation technology, marking pixels in the kth frame of infrared image, the pixel values of which do not meet the preset pixel threshold condition, as blind pixels or noise points; Determining denoising windows corresponding to all the blind pixels or the noise points on the kth frame of infrared image according to the positions of all the blind pixels or the noise points and the preset first window size; and sequencing the pixel values of all pixels except the blind pixels or the noise points in the denoising window, and replacing the pixel values of the blind pixels or the noise points corresponding to the denoising window with the median value of the sequenced sequence to obtain a denoised initial expected image. Further, the filtering and fusing are performed on the initial expected image based on the guiding filtering and the side window filtering to obtain a fused expected image, which specifically includes: conducting guided filtering on the initial expected image to obtain a filtered first expected image; performing side window filtering on the initial expected image to obtain a filtered second expected image; and carrying out weighted fusion calculation according to the first expected image, the second expected image and preset weights to obtain a fusion expected image. Further, the performing guided filtering on the initial expected image to obtain a filtered first expected image specifically includes: taking the initial expected image as an input image and a guide image, and obtaining a plurality of local windows of the guide image according to the guide image and a preset sliding window; Calculating according to pixel values in all the local windows, pixel values in the input image and a preset coefficient calculation formula to obtain a linear coefficient of the initial expected image; calculating according to the linear coefficient, pixel values of pixels in the local windows and a preset linear formula to