KR-102963690-B1 - LOCAL FEATURE EXTRACTION FROM SALIENT REGIONS BY FEATURE MAP TRANSFORMATION

Abstract

A feature point extraction method performed by a feature point extraction system may include: a step of acquiring a respective feature map for each feature extracted from an image pair in a Feature Map Generation (FMG) module; and a step of extracting feature points to minimize region prediction from each acquired feature map using structural information and style information in a Feature Map Transformation (FMT) module.

Inventors

• 이상철
• 정예림

Assignees

• 인하대학교 산학협력단

Dates

Publication Date

20260512

Application Date

20230111

Claims (10)

1. In a feature extraction method performed by a feature extraction system, In the Feature Map Generation (FMG) module, a step of acquiring a respective feature map for each feature extracted from an image pair; and In the Feature Map Transformation Module (FMT), a step of extracting feature points from each of the acquired feature maps using structural information and style information to minimize region prediction. Includes, The above extraction step is, A step of calculating a transformation loss using descriptors to transform and manipulate each of the acquired feature maps using the style and structural information above. A feature point extraction method including
2. In paragraph 1, The above-mentioned acquisition step is, A step of outputting a point extraction feature map including a descriptor, a reliability map, and a repeatability map for each feature extracted from the image pair through a feature map generation network. A feature point extraction method including
3. In paragraph 2, The above-mentioned acquisition step is, A step of calculating loss functions for repeatability and reliability using the above-mentioned output descriptors, reliability map, and repeatability map. A feature point extraction method including
4. In paragraph 2, The above feature map generation network is, A feature point extraction method characterized by each backbone network corresponding to an image pair to extract feature information from an image pair, and depth-separable convolution behind the backbone network to reduce the weight of the feature map generation network.
5. In paragraph 3, The above-mentioned acquisition step is, A step of predicting a reliability score using local descriptors at each pixel of a first image and local descriptors at each pixel of a second image, and calculating a reliability loss optimized to a differentiable approximation using the predicted reliability score. A feature point extraction method including
6. In paragraph 3, The above-mentioned acquisition step is, A step of calculating repeatability loss using peak prediction and similarity between feature pairs of the above image pairs A feature point extraction method including
7. delete
8. In paragraph 1, The above extraction step is, A step of obtaining a first covariance matrix and a second covariance matrix generated by each standardized feature map substituted from the descriptor of the first image and the descriptor of the second image, and calculating the difference between the obtained first covariance matrix and the obtained second covariance matrix to generate a matrix representing the sensitivity of the covariance to the photometric transformation. A feature point extraction method including
9. In paragraph 8, The matrix representing the sensitivity of the covariance to the above-mentioned light intensity transformation is, A feature point extraction method characterized by maintaining style information for elements whose covariance value is greater than or equal to a preset value, and maintaining structural information for elements whose covariance value is less than or equal to a preset value.
10. In a feature point extraction system, A Feature Map Generation (FMG) module that acquires a respective feature map for each feature extracted from an image pair; and A Feature Map Transformation Module (FMT) module that extracts feature points to minimize region prediction from each of the aforementioned acquired feature maps using structural information and style information. Includes, The above Feature Map Transformation Module (FMT) module is, Calculating transformation loss using descriptors to transform and manipulate each of the acquired feature maps using the style and structural information above. A feature point extraction system including

Description

Method and System for Extracting Enhanced Feature Points via Feature Map Transformation Using Deep Learning {LOCAL FEATURE EXTRACTION FROM SALIENT REGIONS BY FEATURE MAP TRANSFORMATION} The following description concerns feature point extraction technology. Extracting and describing local features for matching is essential, particularly in computer vision tasks involving image matching, search, tracking, and 3D reconstruction. Feature matching focuses on three main steps when two similar images are given: feature detection, feature description, and feature matching. The primary goal of feature matching is to optimize matching accuracy while minimizing the memory space of the preceding application. The extracted features must be sparse, repeatable, and precise. Key features of each image, such as corners, are initially identified as points of interest during the detection phase. Then, local descriptors are extracted based on the neighboring regions of these points of interest and used in the matching algorithm. However, conventional techniques for extracting local features have been insufficient in considering the lighting and structural information of the images. For reference, Korean registered patent No. 10-2234311 discloses an image feature extraction method and system for extracting feature points from an image and describing feature information of the image. FIG. 1 is a block diagram illustrating the configuration of a feature point extraction system in one embodiment. FIG. 2 is a flowchart illustrating a feature point extraction method in one embodiment. FIG. 3 is a diagram illustrating the network architecture of a feature map generation module in one embodiment. FIG. 4 is a diagram illustrating the operation of converting a feature map in a feature map conversion module in one embodiment. Hereinafter, embodiments will be described in detail with reference to the attached drawings. FIG. 1 is a block diagram illustrating the configuration of a feature point extraction system in one embodiment, and FIG. 2 is a flowchart illustrating a feature point extraction method in one embodiment. The processor of the feature point extraction system (100) may include a feature map generation module (110) and a feature map conversion module (120). These components of the processor may be representations of different functions performed by the processor according to control commands provided by program code stored in the feature point extraction system. The processor and the components of the processor may control the feature point extraction system to perform steps (210 to 220) included in the feature point extraction method of FIG. 2. At this time, the processor and the components of the processor may be implemented to execute instructions according to the code of an operating system included in memory and the code of at least one program. The processor can load program code stored in a file of a program for a feature point extraction method into memory. For example, when a program is executed in a feature point extraction system, the processor can control the feature point extraction system to load program code from a file of a program into memory under the control of an operating system. At this time, the feature map generation module (110) and the feature map conversion module (120) may each be different functional representations of the processor for executing instructions of corresponding parts of the program code loaded into memory to execute subsequent steps (210 to 220). In step (210), the feature map generation module (110) can obtain a respective feature map for each feature extracted from the image pair. The feature map generation module (110) can output a point extraction feature map containing a descriptor, a reliability map, and a repeatability map for each feature extracted from the image pair through a feature map generation network. The feature map generation module (110) can calculate a loss function for repeatability and reliability using the output descriptor, the reliability map, and the repeatability map. The feature map generation module (110) can predict a reliability score using local descriptors at each pixel of the first image and local descriptors at each pixel of the second image, and calculate a reliability loss optimized to a differentiable approximation using the predicted reliability score. At this time, the second image may refer to an image with different luminosity conditions from the first image. The feature map generation module (110) can calculate a repeatability loss using peak prediction and similarity between feature pairs of the image pair. In step (220), the feature map transformation module (120) can extract feature points to minimize region prediction from each acquired feature map using structural information and style information. The feature map transformation module (120) can calculate a transformation loss using descriptors to transform and manipulate each acquired