KR-20260062576-A - Method and computer program for removing noise from images, Image processing device

KR20260062576AKR 20260062576 AKR20260062576 AKR 20260062576AKR-20260062576-A

Abstract

A method for removing image noise according to an embodiment of the present invention may include: a step of generating a first output frame by removing noise according to a first noise removal method for a first motion area, which is a motion area within a first frame, and removing noise according to a second noise removal method for a first remaining area excluding the first motion area within the first frame, wherein the first motion area is an area determined based on a comparison between the first frame and a second frame, which is a frame preceding the first frame; and a step of generating a second output frame by removing noise according to a third noise removal method for a second motion area, which is a motion area within the first output frame, and removing noise according to a fourth noise removal method for a second remaining area excluding the second motion area within the first output frame, wherein the second motion area is an area determined based on a comparison between the first output frame and a background frame.

Inventors

최은철

Assignees

한화비전 주식회사

Dates

Publication Date: 20260507
Application Date: 20241029

Claims (17)

Regarding video noise removal methods, A step of generating a first output frame by removing noise according to a first noise removal method for a first motion area, which is a motion area within a first frame, and removing noise according to a second noise removal method for a first remaining area excluding the first motion area within the first frame, wherein the first motion area is an area determined based on a comparison between the first frame and a second frame, which is a frame preceding the first frame; and A method for removing image noise, comprising: a step of removing noise according to a third noise removal method for a second motion area, which is a motion area within the first output frame, and removing noise according to a fourth noise removal method for a second remaining area excluding the second motion area within the first output frame, wherein the second motion area is an area determined based on a comparison between the first output frame and a background frame.
In claim 1 The step of generating the first output frame above A step of determining the first movement region and the first remaining region based on a comparison between the first frame and the second frame; A step of removing noise from the first target pixel by referring to the surrounding pixels of the first target pixel on the first frame with respect to the first motion region; and A method for removing image noise, comprising: a step of removing noise of a second target pixel based on a pixel corresponding to a second target pixel on the first frame in each of at least one frame prior to the first frame for the first remaining area.
In claim 2 The step of removing noise from the second target pixel is A step of generating a first accumulated frame by applying weights to the pixels of individual time points from the time point when each pixel belonged to the last movement area prior to the first time point to the first time point for each individual pixel, wherein the first time point is a time point corresponding to the first frame; and Image noise removal method comprising: a step of removing noise from the second target pixel by referring to the first accumulated frame.
In claim 3 The above image noise removal method After the above first output frame generation step, The method further includes the step of generating the background frame based on the first accumulated frame; The step of generating the above background frame is A step of determining a pixel among the pixels constituting the first accumulated frame that has a number of accumulated frames greater than or equal to a predetermined threshold as a background pixel; and A method for removing image noise, comprising the step of generating the background frame based on the background pixels.
In claim 1 The step of generating the second output frame above A step of determining a second motion region and a second remaining region based on a comparison between the first output frame and the background frame; A step of removing noise from the third target pixel by referring to the surrounding pixels of the third target pixel on the first output frame with respect to the second motion region; and A method for removing image noise, comprising the step of removing noise from the fourth target pixel by referring to a pixel corresponding to the fourth target pixel on the first output frame in the background frame for the second remaining area.
In claim 1 The above image noise removal method After the above second output frame generation step, A method for removing image noise, further comprising the step of generating a third output frame by removing noise miscorrection based on the comparison result between the second output frame and the first output frame.
In claim 6 The step of generating the third output frame above A step of comparing the second output frame and the first output frame to determine an outlier region where the difference between the frames is greater than or equal to a threshold difference, and a third remaining region in the second output frame excluding the outlier region; A step of generating an area corresponding to the third remaining area of the third output frame based on the second output frame; and A method for removing image noise, comprising the step of generating an area corresponding to the outlier area of the third output frame based on the first output frame.
In claim 7 In determining the first motion region above, a mask of a first size is used for comparison between two frames, and In determining the second motion region above, a mask of a second size is used for comparison between two frames, and In determining the above outlier region, a mask of a third size is used for comparison between two frames, and Image noise removal method, wherein the first size mask is larger than the second size mask, and the second size mask is larger than the third size mask.
A computer program stored on a medium to execute the method of any one of claims 1 to 8 using a computer.
In an image processing device including a processor, The above processor is, A first output frame is generated by removing noise according to a first noise removal method for a first motion area, which is a motion area within a first frame, and removing noise according to a second noise removal method for a first remaining area excluding the first motion area within the first frame, and the first motion area is an area determined based on a comparison between the first frame and the second frame, which is a frame preceding the first frame. An image processing device that generates a second output frame by removing noise according to a third noise removal method for a second motion area, which is a motion area within the first output frame, and removing noise according to a fourth noise removal method for the remaining second area excluding the second motion area within the first output frame, wherein the second motion area is an area determined based on a comparison between the first output frame and a background frame.
In claim 10 In generating the first output frame, the processor above, Determining the first movement region and the first remaining region based on a comparison between the first frame and the second frame, and For the first motion area, noise of the first target pixel is removed by referring to the surrounding pixels of the first target pixel on the first frame, and An image processing device that removes noise from a second target pixel based on a pixel corresponding to a second target pixel on the first frame in each of at least one frame prior to the first frame for the first remaining area.
In claim 11 The above processor, in removing noise from the second target pixel, For each individual pixel, a first accumulated frame is generated by applying weights to the pixels of the individual time points from the time point when each pixel belonged to the last movement area prior to the first time point to the first time point, and the first time point is a time point corresponding to the first frame. An image processing device that removes noise from the second target pixel by referring to the first accumulated frame.
In claim 12 The above processor The background frame is generated based on the first accumulated frame, An image processing device that determines a pixel among the pixels constituting the first accumulated frame that has a number of accumulated frames greater than or equal to a predetermined threshold as a background pixel, and generates the background frame based on the background pixel.
In claim 10 In generating the second output frame, the processor above, Determining the second motion region and the second remaining region based on the comparison between the first output frame and the background frame, and For the second motion region, noise of the third target pixel is removed by referring to the surrounding pixels of the third target pixel on the first output frame, and An image processing device that removes noise from a fourth target pixel by referencing a pixel corresponding to a fourth target pixel on a first output frame in the background frame for the second remaining area.
In claim 10 The above processor An image processing device that generates a third output frame by removing noise miscorrection based on the comparison result between the second output frame and the first output frame.
In claim 15 In generating the third output frame, the processor above, By comparing the second output frame and the first output frame, an outlier region where the difference between the frames is greater than or equal to a threshold difference and a third remaining region excluding the outlier region in the second output frame are determined, and Based on the second output frame, an area corresponding to the third remaining area of the third output frame is generated, and An image processing device that generates an area corresponding to the outlier area of the third output frame based on the first output frame.
In claim 16 In determining the first motion region above, a mask of a first size is used for comparison between two frames, and In determining the second motion region above, a mask of a second size is used for comparison between two frames, and In determining the above outlier region, a mask of a third size is used for comparison between two frames, and An image processing device in which the first size mask is larger than the second size mask, and the second size mask is larger than the third size mask.

Description

Method and computer program for removing noise from images, Image processing device The present invention relates to a method for removing noise from an image using spatiotemporal information, and more specifically, to a method for removing noise from an image using the correlation between consecutive frames. General Spatio-Temporal Noise Reduction applies Spatial Noise Reduction (SNR) to motion areas and Temporal Noise Reduction (TNR) to non-motion areas, and applies an appropriate mixture of the aforementioned SNR and TNR until TNR is fully applied after motion occurs. While this method can effectively remove noise from video, it has the problem that image quality degradation occurs because it partially uses the SNR results until TNR is fully applied after motion occurs. In particular, after an object passes, the application rate of the SNR increases, leading to image quality degradation such as noise trailing. In environments with sufficient light, the impact of noise is minimal, so the effect of noise trailing on image quality is negligible; however, in environments with high sensor gain, such as low light conditions, significant image degradation due to noise trailing is observed. FIG. 1 is a schematic diagram illustrating the configuration of an image processing device (100) according to one embodiment of the present invention. Figure 2 is a diagram illustrating the structure of an image processed by an image processing device (100). FIG. 3 is a drawing illustrating an exemplary movement area (331). FIG. 4 is a drawing illustrating an exemplary remaining area (341). FIG. 5 is a drawing showing the size of the masks (351, 352, 353) used in comparing two frames in the present invention. FIG. 6 is a drawing for explaining the first noise removal method and the third noise removal method of an image processing device (100) according to one embodiment of the present invention. FIG. 7 is a diagram illustrating a second noise removal method and a fourth noise removal method of an image processing device (100) according to an embodiment of the present invention. FIG. 8 is a diagram illustrating the process of an image processing device (100) according to an embodiment of the present invention generating an accumulated frame (400). FIG. 9 is a flowchart illustrating a method for removing noise from an image using an image processing device (100) according to one embodiment of the present invention. The present invention is capable of various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described below in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense, but for the purpose of distinguishing one component from another. In the following embodiments, singular expressions include plural expressions unless the context clearly indicates otherwise. In the following embodiments, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. In the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and shape of each component shown in the drawings were depicted arbitrarily for convenience of explanation, and therefore the present invention is not necessarily limited to what is depicted. FIG. 1 is a schematic diagram illustrating the configuration of an image processing device (100) according to one embodiment of the present invention. An image processing device (100) according to one embodiment of the present invention can remove noise from an image it has acquired or received. An image processing device (100) according to one embodiment of the present invention may include a processor (110), an ISP (120), a light source (130), a lens group (140), a filter group (150), an image sensor (160), and a motor driver (170) as shown in FIG. 1. A processor (110) according to one embodiment of the present invention can control the components of an image processing device (100). For example, the processor (110) can drive a motor driver (170) according to user operation to move a lens group (140) to an appropriate position. In addition, the processor (110) can perform a series of operation