CN-116524385-B - Angle constraint-based feature descriptor learning and image processing method and device

Abstract

The invention provides a learning and image processing method and device of a feature descriptor based on angle constraint, wherein the learning method comprises the steps of inputting a sample data set into a learning model to obtain a predicted feature descriptor of each sample image in the sample data set; the method comprises the steps of obtaining a distance loss function of a learning model based on the distance between prediction feature descriptors of any two sample images, obtaining an angle loss function of the learning model based on the space angle between the prediction feature descriptors of each pair of matched sample image pairs, constructing an integral loss function of the learning model based on the distance loss function and the angle loss function, and carrying out iterative optimization on parameters of the learning model based on the integral loss function to obtain an optimal learning model. According to the invention, on the basis of the distance loss function, the angle loss function based on the angle constraint is added in the loss function of the learning model, so that the influence of noise and external points on the learning model can be reduced, and the robustness and accuracy of the feature descriptors are improved.

Inventors

• DONG QIULEI
• LI JIANAN

Assignees

• 中国科学院自动化研究所

Dates

Publication Date

20260505

Application Date

20220119

Claims (9)

1. 1. The learning method of the feature descriptors based on the angle constraint is characterized by comprising the following steps of: Inputting a sample data set into a learning model to obtain a prediction feature descriptor of each sample image in the sample data set, wherein the sample data set comprises a plurality of matched sample image pairs; Acquiring a distance loss function of the learning model based on the distance between the predicted feature descriptors of any two sample images, and acquiring an angle loss function of the learning model based on the space angle between the predicted feature descriptors of each matched sample image pair; constructing an overall loss function of the learning model based on the distance loss function and the angle loss function of the learning model; Based on the integral loss function, carrying out iterative optimization on parameters of the learning model to obtain an optimal learning model; the optimal learning model is used for obtaining a prediction feature descriptor of the image to be processed according to the image to be processed; each pair of matched sample image pairs contains a first sample image and a second sample image; The step of obtaining the angle loss function comprises the following steps: Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the first sample image in each matched sample image pair in a first feature space formed by the predicted feature descriptors of all the first sample images in the sample data set; Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the second sample image in each matched sample image pair in a second feature space formed by the predicted feature descriptors of all the second sample images in the sample data set; Calculating the space angle between the predicted feature descriptors of the first sample image and the second sample image in each matched sample image pair and a plurality of corresponding neighborhood feature descriptors respectively; And acquiring the angle loss function based on the spatial angles corresponding to the first sample image and the second sample image in each matched sample image pair.
2. 2. The method of claim 1, wherein the arbitrary two sample images are a matched sample image pair or a non-matched sample image pair; correspondingly, the obtaining the distance loss function of the learning model based on the distance between the prediction feature descriptors of any two sample images comprises the following steps: And constructing the distance loss function by taking the minimum distance between the predicted feature descriptors of all matched sample image pairs in the sample data set and the maximum distance between the predicted feature descriptors of all unmatched sample image pairs as targets.
3. 3. The method for learning an angle constraint-based feature descriptor according to any one of claims 1 to 2, wherein the constructing an overall loss function of the learning model based on a distance loss function and an angle loss function of the learning model comprises: And carrying out weighted addition on the distance loss function and the angle loss function of the learning model to obtain the integral loss function of the learning model.
4. 4. The method for learning feature descriptors based on angle constraints according to any one of claims 1-2, wherein the inputting the sample dataset into the learning model to obtain the predicted feature descriptors for each sample image in the sample dataset includes: for a current training batch, taking a last sample image obtained by sequentially sampling a last training batch of the current training batch as a starting point, sequentially sampling a first preset number of sample images from the sample data set, and randomly sampling a second preset number of sample images from the rest of sample data sets to obtain a sub-sample data set of the current training batch; Inputting the sub-sample data set of the current training batch into the learning model to obtain a predicted feature descriptor of each sample image in the sub-sample data set of the current training batch; And continuing to sequentially sample the last sample image obtained by the current training batch, and iteratively executing the processes of sequential sampling, random sampling and input until the result of sequential sampling is null as the starting point of the next training batch of the current training batch.
5. 5. The method of learning angle constraint based feature descriptors of any of claims 1-2, wherein said inputting the sample dataset into the learning model comprises: preprocessing each sample image in the sample data set; wherein the preprocessing includes cropping and image enhancement, the image enhancement includes one or more combinations of rotation, flipping, and affine transformation; And inputting the preprocessed sample data set into the learning model.
6. 6. An image processing method of a feature descriptor based on angle constraint, comprising: acquiring an optimal learning model; Inputting an image to be processed into the optimal learning model to obtain a predicted feature descriptor of the image to be processed; Processing the image to be processed according to the predicted feature descriptors of the image to be processed to obtain a processing result of the image to be processed; wherein the processing includes image recognition or image matching; The optimal learning model is trained based on the learning method of the angle constraint-based feature descriptors of any one of claims 1 to 5.
7. 7. A learning device for a feature descriptor based on angular constraint, comprising: The system comprises a first learning module, a second learning module and a first prediction module, wherein the first learning module is used for inputting a sample data set into a learning model to obtain a prediction feature descriptor of each sample image in the sample data set, and the sample data set comprises a plurality of matched sample image pairs; The first acquisition module is used for acquiring a distance loss function of the learning model based on the distance between the predicted feature descriptors of any two sample images, and acquiring an angle loss function of the learning model based on the space angle between the predicted feature descriptors of each matched sample image pair; The construction module is used for constructing the integral loss function of the learning model based on the distance loss function and the angle loss function of the learning model; The training module is used for carrying out iterative optimization on the parameters of the learning model based on the integral loss function to obtain an optimal learning model; the optimal learning model is used for obtaining a prediction feature descriptor of the image to be processed according to the image to be processed; each pair of matched sample image pairs contains a first sample image and a second sample image; The construction module is specifically used for: Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the first sample image in each matched sample image pair in a first feature space formed by the predicted feature descriptors of all the first sample images in the sample data set; Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the second sample image in each matched sample image pair in a second feature space formed by the predicted feature descriptors of all the second sample images in the sample data set; Calculating the space angle between the predicted feature descriptors of the first sample image and the second sample image in each matched sample image pair and a plurality of corresponding neighborhood feature descriptors respectively; And acquiring the angle loss function based on the spatial angles corresponding to the first sample image and the second sample image in each matched sample image pair.
8. 8. An image processing apparatus for a feature descriptor based on an angle constraint, comprising: the second acquisition module is used for acquiring an optimal learning model; the second learning module is used for inputting the image to be processed into the optimal learning model to obtain a predicted feature descriptor of the image to be processed; The processing module is used for processing the image to be processed according to the prediction feature descriptors of the image to be processed to obtain a processing result of the image to be processed; wherein the processing includes image recognition or image matching; The optimal learning model is trained based on the learning method of the angle constraint-based feature descriptors of any one of claims 1 to 5.
9. 9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of learning the angular constraint based feature descriptors of any of claims 1 to 5 or the steps of the method of image processing of the angular constraint based feature descriptors of claim 6 when the program is executed.

Description

Angle constraint-based feature descriptor learning and image processing method and device Technical Field The invention relates to the technical field of image processing, in particular to a method and a device for learning and image processing of feature descriptors based on angle constraint. Background The traditional image feature descriptor learning method mainly utilizes neighborhood pixels of feature points, but the performance of the image feature descriptor generated by the method is easy to be interfered by various factors, such as scale, rotation, illumination, affine, noise and the like, so that larger errors occur in feature matching, and the robustness and the accuracy of the generated image feature descriptor are poor. With the development of deep learning, convolutional neural networks are widely applied to most of computer vision tasks at present, and a learning network of descriptors based on Euclidean distance constraint can learn local feature descriptors so as to properly improve the problem of poor robustness and accuracy of image feature descriptors; However, the method is easily affected by outliers and noise, so that the generated image feature descriptors still have the problems of low robustness and low accuracy. Therefore, how to eliminate the influence of outliers and noise to generate image feature descriptors with high robustness and high accuracy is an important issue to be solved in the industry. Disclosure of Invention The invention provides a method and a device for learning and image processing of a feature descriptor based on angle constraint, which are used for solving the problems that a learning network of the feature descriptor based on Euclidean distance constraint in the prior art is easily influenced by outliers and noise, so that the generated image feature descriptor still has the defects of low robustness and low accuracy, and the robustness and the accuracy of the image feature descriptor are improved. The invention provides a learning method of a feature descriptor based on angle constraint, which comprises the following steps: Inputting a sample data set into a learning model to obtain a prediction feature descriptor of each sample image in the sample data set, wherein the sample data set comprises a plurality of matched sample image pairs; Acquiring a distance loss function of the learning model based on the distance between the predicted feature descriptors of any two sample images, and acquiring an angle loss function of the learning model based on the space angle between the predicted feature descriptors of each matched sample image pair; constructing an overall loss function of the learning model based on the distance loss function and the angle loss function of the learning model; Based on the integral loss function, carrying out iterative optimization on parameters of the learning model to obtain an optimal learning model; the optimal learning model is used for obtaining a prediction feature descriptor of the image to be processed according to the image to be processed. According to the learning method of the feature descriptors based on the angle constraint, each matched sample image pair comprises a first sample image and a second sample image; accordingly, the obtaining the angle loss function of the learning model based on the space angle between the prediction feature descriptors of each pair of matched sample image pairs includes: Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the first sample image in each matched sample image pair in a first feature space formed by the predicted feature descriptors of all the first sample images in the sample data set; Acquiring a plurality of neighborhood feature descriptors of the predicted feature descriptors of the second sample image in each matched sample image pair in a second feature space formed by the predicted feature descriptors of all the second sample images in the sample data set; Calculating the space angle between the predicted feature descriptors of the first sample image and the second sample image in each matched sample image pair and a plurality of corresponding neighborhood feature descriptors respectively; And acquiring the angle loss function based on the spatial angles corresponding to the first sample image and the second sample image in each matched sample image pair. According to the learning method of the feature descriptors based on the angle constraint, any two sample images are matched sample image pairs or unmatched sample image pairs; correspondingly, the obtaining the distance loss function of the learning model based on the distance between the prediction feature descriptors of any two sample images comprises the following steps: And constructing the distance loss function by taking the minimum distance between the predicted feature descriptors of all matched sample image pairs in the sample data set and the maximum distance