Search

KR-20260064871-A - DEVICE AND METHOD FOR AI-BASED NLOS (NON-LINE-OF-SIGHT) IMAGING RECONSTRUCTION

KR20260064871AKR 20260064871 AKR20260064871 AKR 20260064871AKR-20260064871-A

Abstract

The present invention relates to an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device, comprising an input processing unit that samples a partial image from an original image, a neural network processing unit that generates a frequency-converted image for the partial image and inputs the frequency-converted image into time and frequency domain networks to generate a prediction pacer field, and a scene reconstruction unit that reconstructs a hidden scene based on the prediction pacer field.

Inventors

  • 김선주
  • 조 인

Assignees

  • 연세대학교 산학협력단

Dates

Publication Date
20260508
Application Date
20241030

Claims (9)

  1. Input processing unit that samples partial images from the original image; A neural network processing unit that generates a frequency-converted image for the above-mentioned partial image and inputs the frequency-converted image into time and frequency domain networks to generate a prediction pacer field; and An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device comprising a scene reconstruction unit that reconstructs a hidden scene based on the above-mentioned predicted pacer field.
  2. In paragraph 1, the input processing unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by generating a noise-removed partial image by performing noise removal through sensor noise simulation in the above partial image.
  3. In paragraph 1, the neural network processing unit Input phasor convolution is performed to generate the frequency-transformed image by performing an FFT transform, applying an illumination function, and an IFFT transform on the above partial image, and An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by the above illumination function extracting a frequency band of interest by passing a specific frequency band through the frequency band of the above partial image.
  4. In paragraph 3, the neural network processing unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by generating a temporal information-preserving image by inputting the above frequency-converted image into the above time domain network implemented as a residual block for temporal information processing.
  5. In paragraph 4, the neural network processing unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by generating a frequency information processing image by inputting it into a frequency domain network implemented as a convolution layer for processing the frequency components of the above temporal information preservation image.
  6. In paragraph 5, the neural network processing unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by generating a prediction pacer field that predicts a hidden object by extracting a frequency band of interest from the frequency information processing image through target training.
  7. In paragraph 6, the neural network processing unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by implementing the target training as a loss function for controlling outliers of the hidden object.
  8. In paragraph 1, the scene reconstruction unit An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device characterized by restoring the hidden scene by determining the location and shape of an object through RSD (Rayleigh-Sommerfeld Diffraction) operations on the frequency band constituting the predicted pacer field.
  9. In an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction method performed in an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device, Input processing step for sampling partial images from the original image; A neural network processing step of generating a frequency-converted image for the above-mentioned partial image and inputting the frequency-converted image into time and frequency domain networks to generate a prediction pacer field; and An artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction method comprising a scene reconstruction step for reconstructing a hidden scene based on the above-mentioned predicted pacer field.

Description

Device and Method for AI-Based NLOS (Non-Line-of-Sight) Imaging Reconstruction The present invention relates to an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction technology, and more specifically, to an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction apparatus and method capable of reconstructing a hidden scene based on a prediction pacer field generated by inputting a frequency-converted image of a partial image generated through a neural network into time and frequency domain networks. AI-based image reconstruction technology refers to a technology that utilizes artificial intelligence (AI) and machine learning (ML) algorithms to restore damaged or missing images or to convert low-resolution images into high-resolution ones. This technology is used to fill in missing parts of an image or to transform and reconstruct images of better quality. To this end, deep learning algorithms, particularly generative models and convolutional neural networks (CNNs), are primarily utilized. Korean Published Patent No. 10-2022-0180535 (December 21, 2022) includes the steps of: setting a mask for a plurality of regions based on an original image; outputting a latent code for generating a first image reconstructed for each of the plurality of regions from the original image based on a first artificial intelligence learning model; generating the first images reconstructed from the latent code for each of the plurality of regions based on a second artificial intelligence learning model; and applying the mask to each of the first images generated for each of the plurality of regions, and combining the first images to which the mask is applied to generate a second image reconstructed for the original image. FIG. 1 is a diagram illustrating an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device according to one embodiment of the present invention. Figure 2 is a diagram illustrating the functional configuration of the artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device of Figure 1. Figure 3 is a diagram illustrating the system configuration of the artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction device of Figure 1. FIG. 4 is a flowchart illustrating an artificial intelligence-based NLOS (Non-Line-of-Sight) imaging reconstruction method according to the present invention. Figure 5 is a diagram of the reconstruction result of FK (right) for the illumination function in the frequency domain (left) and wavenumber filtered measurements. Figure 6 is a diagram of the qualitative results for Bike and Dragon in the Stanford real dataset. Figure 7 is a qualitative result diagram related to resolution for non-confocal 16 × 16 sparse sampling in an actual dataset. Figure 8 is a diagram showing the results of qualitative ablation for the denoising criterion. Figure 9 shows the qualitative ablation result for frequency filtering. The description of the present invention is merely an example for structural or functional explanation, and therefore the scope of the present invention should not be interpreted as being limited by the examples described in the text. That is, since the examples are subject to various modifications and may take various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical concept. Furthermore, the objectives or effects presented in the present invention do not imply that a specific example must include all of them or only such effects; therefore, the scope of the present invention should not be understood as being limited by them. Meanwhile, the meaning of the terms described in this application should be understood as follows. Terms such as "first," "second," etc., are intended to distinguish one component from another, and the scope of rights shall not be limited by these terms. For example, the first component may be named the second component, and similarly, the second component may be named the first component. When it is stated that one component is "connected" to another component, it should be understood that it may be directly connected to that other component, or that there may be other components in between. Conversely, when it is stated that one component is "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions describing the relationships between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way. A singular expression should be understood to include a plural expression unless the context clearly indicates otherwise, and terms such as "include" or "have" are intended to specify the existence of the implemented features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood not