KR-20260064032-A - Method and apparatus for reducing noise in real-time video data

KR20260064032AKR 20260064032 AKR20260064032 AKR 20260064032AKR-20260064032-A

Abstract

The present disclosure relates to a method and apparatus for reducing noise included in real-time video data. A method for reducing noise included in real-time video data according to one aspect may include: a step of obtaining first output data by inputting at least one frame prior to time t and at least one frame after time t among a plurality of frames included in real-time video data into a neural network model; a step of obtaining second output data by inputting the frame at time t, at least one frame prior to time t, and at least one frame after time t among the plurality of frames into a motion correction network and a recursive filter; a step of training the noise correction model using at least one of the first output data and the second output data; and a step of reducing noise included in the real-time video data using the trained noise correction model.

Inventors

최장환
변규리
전선영

Assignees

이화여자대학교 산학협력단

Dates

Publication Date: 20260507
Application Date: 20241031

Claims (12)

A method for reducing noise included in real-time video data, A step of obtaining first output data by inputting at least one frame prior to time t and at least one frame after time t into a neural network model among a plurality of frames included in the real-time video data; A step of obtaining second output data by inputting the frame at time t, at least one frame prior to time t, and at least one frame after time t among the plurality of frames into a motion correction network and a recursive filter; A step of training the noise correction model using at least one of the first output data and the second output data; A step of reducing noise included in the real-time image data using the above-mentioned learned noise correction model; A method including
In Article 1, The above-mentioned learning step is, A step of obtaining third output data by inputting the frame at time t into the noise correction model; A step of training the noise correction model based on the error between the first output data and the third output data; A method including,
In Article 1, The step of obtaining the first output data above is, A step of extracting features of the above-mentioned input frame; A step of calculating the residual of each frame and, based on the residual, calculating the attention weight for each feature; and A step of obtaining the first output data based on the attention weights above; A method including
In Article 1, The above neural network model is, A method comprising at least one architecture among a convolution filter, a recursive residual convolution, and an attention gate.
In Article 1, The above-mentioned learning step is, A step of obtaining third output data by inputting the frame at time t into the noise correction model; A step of performing motion correction on the frame at time t, at least one frame prior to time t, and at least one frame after time t; A step of obtaining the second output data by inputting the motion-corrected frames into a recursive filter; and A step of training the noise correction model based on the error between the second output data and the third output data; A method including
In Article 5, The step of performing the above motion correction is, A step of calculating an optical flow between a frame at time t, at least one frame prior to time t, and at least one frame after time t, and performing motion correction based on the optical flow; A method including
In Article 5, The step of obtaining the second output data above is, Step of setting weights between the motion-corrected frames; and A step of obtaining the second output data based on the above weights; A method including
In Article 5, A step of strengthening the edges of the second output data and the third output data; and A step of training the noise correction model based on the error between the edge-enhanced second output data and the edge-enhanced third output data; A method that further includes.
In Article 8, The step of reinforcing the above edge is, A step of decomposing the second output data and the third output data into a plurality of sub-bands through wavelet transformation; A step of adjusting the intensity of a sub-band according to a preset condition among the plurality of sub-bands; and A step of strengthening edges by reconstructing the second output data and the third output data through an inverse wavelet transform; A method including
In Article 1, A method in which the above time point t is updated over time.
A computer-readable recording medium having a program for executing the method of claim 1 on a computer.
At least one memory; and It includes at least one processor, The above processor is, Among the plurality of frames included in the real-time video data, at least one frame prior to time t and at least one frame after time t are input into a neural network model to obtain first output data, and Among the plurality of frames, the frame at time t, at least one frame prior to time t, and at least one frame after time t are input into a motion correction network and a recursive filter to obtain second output data, and The noise correction model is trained using at least one of the first output data and the second output data, and Reducing noise included in the real-time video data using the above-mentioned learned noise correction model, A device for reducing noise included in real-time video data.

Description

Method and apparatus for reducing noise in real-time video data The present disclosure relates to a method and apparatus for reducing noise included in real-time video data. Fluoroscopy is a technique that generates and visualizes real-time X-ray images and plays an important role in various medical procedures, such as catheter insertion or monitoring during orthopedic surgery. However, this technique generates radiation, which can pose a potential risk to patients and medical staff. Accordingly, the use of low-dose X-ray fluoroscopy is recommended to reduce radiation, but there is a problem in that the images generated by low-dose X-ray fluoroscopy are prone to increased noise or distortion. To overcome these problems, methods such as restoring images by modeling noise and reducing noise using deep learning/machine learning-based artificial intelligence models are being attempted. However, these approaches must be consistently applicable in clinical environments where various fluoroscopy devices are used, and must be able to guarantee performance despite the problem of data scarcity in clinical settings. The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention. FIG. 1 is a diagram illustrating an example of a system for reducing noise included in real-time image data according to one embodiment. FIG. 2 is a flowchart illustrating a method for reducing noise included in real-time video data according to one embodiment. FIG. 3 is a block diagram illustrating the learning of a noise correction model according to one embodiment. FIG. 4 is a flowchart illustrating a method for obtaining first output data according to one embodiment. FIG. 5 is a schematic diagram illustrating a method for obtaining second output data according to one embodiment. FIG. 6 is a block diagram illustrating a method for training a noise correction model using edge-enhanced second output data and third output data according to one embodiment. FIG. 7 is a block diagram illustrating a method for reinforcing the edges of a frame using an edge reinforcing module according to one embodiment. FIG. 8 is a block diagram of a user device according to one embodiment. The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, detailed descriptions of related known technologies are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention. The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as 'comprising' or 'having' are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Additionally, terms such as 'unit' and 'module' described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software. Additionally, terms including ordinal numbers, such as 'first' or 'second' used in the specification, may be used to describe various components, but said components shall not be limited by said terms. Such terms may be used for the purpose of distinguishing one component from another. Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and/or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in vario