CN-122023843-A - Image matching method and device based on different-distribution-diagram topology perception

CN122023843ACN 122023843 ACN122023843 ACN 122023843ACN-122023843-A

Abstract

The invention relates to an image matching method based on heteroleptic image topology perception, which comprises the following steps of obtaining two images to be matched, constructing a heteroleptic image matching network model, executing heteroleptic image matching network model training, inputting a trained heteroleptic image matching network model into a graph structure and an initial node representation corresponding to the two images to be matched, obtaining a similarity matrix between the two images to be matched, and inputting the similarity matrix into a graph matching solver to realize image matching. The present invention balances homoleptic and heteroleptic by a topology aware dynamic weighting mechanism to learn information node representations for graph matching that benefits from multi-hop neighbors and performs dynamic purification of graph topology, and introduces a cross-graph annotation mechanism based on node similarity between source and target graphs. In this way, semantic differences between nodes in different graphs are eliminated and the problem of excessive smoothing caused by a multi-layer graph rolling network for message passing is alleviated.

Inventors

XIE YU
LIANG YILONG
LIANG XINYAN
DU HANGYUAN
WANG ZHIQIANG

Assignees

山西大学

Dates

Publication Date: 20260512
Application Date: 20260120

Claims (9)

1. An image matching method based on heteroleptic graph topology perception is characterized by comprising the following steps: Acquiring two images to be matched, and respectively preprocessing the two images to obtain a graph structure corresponding to the two images; constructing initial node representations of two graph structures; Constructing a heteroleptic graph matching network model; performing heteroleptic graph matching network model training; Inputting the graph structures and initial node representations corresponding to the two images to be matched into a trained heteroleptic graph matching network model to obtain a similarity matrix between the two images to be matched, and inputting the similarity matrix into a graph matching solver to realize image matching; The processing procedure of the heteroleptic graph matching network model comprises the following steps: Respectively carrying out graph topology rewiring on the two graph structures, and constructing homozygosity subgraphs and heterozygosity subgraphs corresponding to the two graph structures; obtaining a weighted adjacent matrix corresponding to the homozygosity subgraph and the heterozygosity subgraph according to the weight of each edge in the homozygosity subgraph and the heterozygosity subgraph in the corresponding graph; k-hop feature interaction is carried out according to the initial node representation and the weighted adjacency matrix, and node embedding corresponding to the homozygosity subgraph and the heterozygosity subgraph is obtained 、 ; The homogametic map is used to generate an aggregate weight alpha, Wherein, the Represents a collection of edges of a homoleptic graph, Representing a neighborhood node set of the node i in the same-node graph, wherein a symbol (three) represents characteristic splicing operation, and kappa ho represents a curvature value; Calculating a node characteristic matrix H of the image based on the aggregation weight alpha, so as to obtain node characteristic matrices H s 、H t of the two images to be matched; wherein, the symbol ++indicates multiplication by element, η is the super parameter of the control weight; According to the node characteristic matrix H s 、H t , calculating through an attention mechanism to obtain the context characteristics C s and C t of the two images to be matched, and then calculating through the following formula to obtain the node final characteristics F s 、F t of the two images to be matched; wherein f θ denotes a learnable linear transformation layer; And obtaining a similarity matrix between the two images to be matched according to the final characteristics F s 、F t of the nodes.
2. The image matching method based on heteroleptic image topology awareness of claim 1, wherein constructing homoleptic subgraphs and heteroleptic subgraphs corresponding to two image structures specifically comprises the following steps: Calculating an edge curvature value through an adjacent matrix of the graph structure, removing the edge with the lowest curvature value of a preset proportion, and obtaining a trimmed graph as a homozygosity subgraph; and calculating cosine similarity of node characteristics in the multi-hop neighborhood, and selecting a node pair with minimum similarity of a preset proportion to construct an edge, wherein the obtained graph is the heteroleptic subgraph.
3. The image matching method based on the heteroleptic graph topology awareness according to claim 1, wherein the weight of each edge in the homoleptic subgraph and the heteroleptic subgraph in the corresponding graph is a mapping ψ (i, j) of the relationship between a node i and a node j: wherein, κ (I, j) represents the original weight of the edge I, phi I and phi j represent the dimension reduction characteristics of the nodes I and j respectively, w represents the weight matrix, b represents the deviation, and sigma () represents the activation function; The weighted adjacency matrix A ho 、A he corresponding to homozygosity subgraph and heterozygosity subgraph is: Where κ ho and κ he represent curvature and cosine similarity, respectively, x i 、x j ∈R 1×d is node feature, E ho 、E he is edge set of homozygote and heterogamote graphs, respectively, and θ ho 、θ he is a trainable parameter.
4. The image matching method based on heteroleptic image topology awareness of claim 1, wherein performing K-hop feature interaction based on the initial node representation H 0 and the weighted adjacency matrix is represented as: Where W ho and W he are independent trainable matrices that are non-parameter shared, concat represents a join operation, 、 The nodes of the homozygosity subgraph and the heterozygosity subgraph are respectively embedded.
5. The image matching method based on the heteroleptic image topology awareness of claim 1, wherein the attention mechanism calculation process can be expressed as: wherein d is a feature dimension; Wherein β and γ represent the attention weights of the first image to the second image and the second image to the first image, respectively; the contextual features C s and C t of the two images to be matched are represented as: 。
6. The image matching method based on heteroleptic image topology awareness according to claim 1, wherein a similarity matrix S between two images to be matched is expressed as: Inputting the similarity matrix S into a graph matching solver to obtain a final replacement matrix X, wherein the element value of the final replacement matrix X is 0 or 1: 。
7. The image matching method based on the heteroleptic graph topology awareness according to claim 1, wherein in the heteroleptic graph matching network model training process, binary cross entropy loss is adopted as a loss function: Wherein Y is a tag matrix.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor performs the steps in the method of image matching based on heteroleptic topology awareness of any of claims 1 to 7.
9. An image matching device based on heteroleptic graph topology awareness, comprising: A memory for storing a software application, A processor for executing the software application program, each program of the software application program correspondingly executing the steps in the image matching method based on the heteroleptic image topology awareness as set forth in any one of claims 1 to 7.

Description

Image matching method and device based on different-distribution-diagram topology perception Technical Field The invention belongs to the technical field of artificial intelligence, and particularly relates to an image matching method and device based on heteroleptic graph topology awareness. Background Graph matching is a method of finding a node correspondence between two or more graph data using structure information of a graph data structure. The graph matching technology shows reliable and efficient performance in establishing the corresponding relation between key points in different graphs, and has wide application, such as scene graph discovery, target tracking, image recognition, target detection, gesture estimation and the like. Existing graph matching methods generally assume that the nodes of an object correspond to a graph of key points of the object, and that their edges are constructed by a fixed algorithm (e.g., delaunay triangulation, full connectivity). These graphs generally follow the homography assumption that nodes tend to connect to other nodes with similar features or labels. Many real world graphs are in a "heteroleptic" setting, resulting in graphs with heteroleptic properties, where nodes tend to connect with other nodes with different features or labels. In the heteroleptic graph scenario, the inherent low-pass filtering of the GCN neighbor aggregation operation results in the node representation becoming too smooth, eroding the discrimination information required for an exact match. Thus, when operating on a heteroleptic structure, features acquired by the GCN-based graph matching model do not contain sufficient feature difference information. However, most existing heteroleptic networks are designed for classification tasks. Simple replacement of the GCN in the visual map matching model with these class-oriented heteroleptic map networks presents several problems. In particular the number of the elements, The classification task forces node embedding to become more similar to its neighbors, while the visual map matching task requires that discrimination information be maintained, which places a higher demand on the discrimination capability of node embedding. This is a requirement that class-oriented designs are generally not satisfactory. Furthermore, in current GCN-based graph matching models, repeated application of smooth convolution tends to cause node embedding to become too similar to neighboring nodes, which hampers the effectiveness of visual graph matching. Disclosure of Invention The invention aims to provide an image matching method and device based on heteroleptic graph topology perception, which can effectively match heteroleptic graphs; In order to achieve the above purpose, the invention adopts the following technical scheme: an image matching method and device based on heteroleptic graph topology perception comprises the following steps: Acquiring two images to be matched, and respectively preprocessing the two images to obtain a graph structure corresponding to the two images; constructing initial node representations of two graph structures; Constructing a heteroleptic graph matching network model; performing heteroleptic graph matching network model training; Inputting the graph structures and initial node representations corresponding to the two images to be matched into a trained heteroleptic graph matching network model to obtain a similarity matrix between the two images to be matched, and inputting the similarity matrix into a graph matching solver to realize image matching; The processing procedure of the heteroleptic graph matching network model comprises the following steps: Respectively carrying out graph topology rewiring on the two graph structures, and constructing homozygosity subgraphs and heterozygosity subgraphs corresponding to the two graph structures; obtaining a weighted adjacent matrix corresponding to the homozygosity subgraph and the heterozygosity subgraph according to the weight of each edge in the homozygosity subgraph and the heterozygosity subgraph in the corresponding graph; k-hop feature interaction is carried out according to the initial node representation and the weighted adjacency matrix, and node embedding corresponding to the homozygosity subgraph and the heterozygosity subgraph is obtained 、; The homography is used to generate aggregate weights alpha, Wherein, the Represents a collection of edges of a homoleptic graph,Representing a neighborhood node set of the node i in the same-node graph, wherein a symbol (three) represents characteristic splicing operation, and kappa ho represents a curvature value; Calculating a node characteristic matrix H of the image based on the aggregation weight alpha, so as to obtain node characteristic matrices H s、Ht of the two images to be matched; wherein, the symbol ++indicates multiplication by element, η is the super parameter of the control weight; According to the node characteristic