CN-121505470-B - Multi-target detection and association method, device and equipment based on space base and space base

CN121505470BCN 121505470 BCN121505470 BCN 121505470BCN-121505470-B

Abstract

The application provides a multi-target detection and association method, device and equipment based on space base and space base, and relates to the technical fields of remote sensing information processing and the like. The method comprises the steps of obtaining a space-based target set and a space-based remote sensing image of space-based detection, inputting the space-based target set and the space-based remote sensing image into a pre-trained target search network, extracting a space-based target association feature set and a space-based data basic feature corresponding to a plurality of space-based targets through a shared backbone network, detecting the space-based data basic feature based on a target detection head in the target search network, determining the space-based target set in the space-based remote sensing image, screening the space-based target association feature set from the space-based data basic feature based on a plurality of space-based targets included in the space-based target set, and associating the plurality of space-based targets with the plurality of space-based targets based on the similarity between the space-based target association feature set and the space-based target association feature set, so that multi-target association efficiency can be improved.

Inventors

LIU YU
ZHANG LINPING
WANG XUEQIAN
MAO YONGQIANG
LI GANG

Assignees

清华大学

Dates

Publication Date: 20260512
Application Date: 20260113

Claims (12)

1. A method for multi-target detection and association based on space and space, comprising: Acquiring a space-based target set of space-based detection and a space-based remote sensing image acquired by a space-based, wherein the space-based target set comprises a plurality of space-based targets; The space-based target set and the space-based remote sensing image are input into a pre-trained target search network, a space-based target associated feature set corresponding to a plurality of space-based targets is extracted through a shared backbone network in the target search network, space-based data basic features in the space-based remote sensing image are extracted, target detection is carried out on the space-based data basic features based on a target detection head in the target search network, the space-based target set in the space-based remote sensing image is determined, and a space-based target associated feature set is screened from the space-based data basic features based on a plurality of space-based targets included in the space-based target set, wherein the target search network is obtained by training an initial search network based on a plurality of sky-based triplet samples, and each sky-based triplet sample comprises a space-based remote sensing image sample, a space-based target positive sample set corresponding to the space-based remote sensing image sample and a space-based target negative sample set; and associating the plurality of space-based targets with the plurality of space-based targets based on the similarity between the space-based target association feature set and the space-based target association feature set.
2. The method of claim 1, wherein the determining the set of space-based targets in the space-based remote sensing image based on target detection heads in the target search network performing target detection on the space-based data base features comprises: performing target detection on the space-based data basic features based on the target detection head to obtain a plurality of prediction candidate frames in the space-based remote sensing image and confidence degrees corresponding to each prediction candidate frame; and determining the space-based target set in the space-based remote sensing image based on the confidence degree corresponding to each prediction candidate frame.
3. The method of claim 1, wherein the associating the plurality of space-based targets with the plurality of space-based targets based on a similarity between the set of space-based target-associated features and the set of space-based target-associated features comprises: Constructing a corresponding association strategy cost augmentation matrix based on the similarity between the space-based target association feature set and the space-based target association feature set, wherein the association strategy cost augmentation matrix is used for representing the possibility of association of the space-based target and the space-based target; Constructing a correlation plan augmentation matrix corresponding to the plurality of space-based targets and the plurality of space-based targets, wherein the correlation plan augmentation matrix is used for carrying out space-time consistency constraint on the plurality of space-based targets and the plurality of space-based targets; and associating the plurality of space-based targets with the plurality of space-based targets based on the association plan augmentation matrix and the association policy cost augmentation matrix.
4. The method of claim 3, wherein the constructing a corresponding association policy cost augmentation matrix based on a similarity between the set of space-based target association features and the set of space-based target association features comprises: constructing a corresponding target similarity matrix based on the similarity between every two targets in the space-based target association feature set and the space-based target association feature set; and constructing a corresponding association strategy cost augmentation matrix based on the target similarity matrix.
5. The method of claim 3, wherein the association plan augmentation matrix is to place space-time consistency constraints on the plurality of space-based objects and the plurality of space-based objects, comprising: Based on Space-time consistency constraint is carried out on the plurality of space-based targets and the plurality of space-based targets; wherein N represents the number of the plurality of space-based targets, M represents the number of the plurality of space-based targets, And representing the degree of association between the space-based targets of the ith row and the space-based targets of the jth column in the association plan augmentation matrix.
6. The method of claim 3, wherein the associating the plurality of space-based objectives with the plurality of space-based objectives based on the association plan augmentation matrix and the association policy cost augmentation matrix comprises: Based on Correlating the plurality of space-based targets with the plurality of space-based targets so as to satisfy Obtaining association results between the plurality of space-based targets and the plurality of space-based targets; Wherein, the Representing the association plan augmentation matrix, N representing the number of the plurality of space-based objects, M representing the number of the plurality of space-based objects, Representing the degree of association between the space-based object of the ith row and the space-based object of the jth column in the association plan augmentation matrix, Representing the associated policy cost augmentation matrix, And representing the possibility that the space-based target of the ith row is associated with the space-based target of the jth column in the association strategy cost augmentation matrix.
7. The method of any of claims 1-6, wherein the training of the initial search network by the target search network based on the plurality of sky-based triplet samples comprises: For each sky-based triplet sample, the following is performed: Determining the similarity between a space-based target sample and a first space-based target sample in a space-based remote sensing image sample and the similarity between the space-based target sample and a second space-based target sample through the initial search network, wherein the first space-based target sample is a space-based target sample corresponding to the space-based target sample in the space-based target positive sample set, and the second space-based target sample is a space-based target sample corresponding to the space-based target sample in the space-based target negative sample set; constructing a target association loss function based on the similarity between the space-based target sample and the first day-based target sample and the similarity between the space-based target sample and the second day-based target sample; constructing a target detection loss function based on a prediction target class to which the space-based target sample belongs and a prediction candidate frame corresponding to the space-based target sample; constructing a multi-task loss function corresponding to the sky-based triplet sample based on the target associated loss function and the target detection loss function; And updating model parameters of the initial search network based on the corresponding multitask loss function of each sky-based triplet sample to obtain the target search network.
8. The method of claim 7, wherein constructing the target detection loss function based on the prediction target class to which the space-based target sample belongs and the prediction candidate box corresponding to the space-based target sample comprises: constructing a class loss function based on a predicted target class to which the space-based target sample belongs and a target class label to which the space-based target sample belongs; constructing a regression loss function based on the prediction candidate frame corresponding to the space-based target sample and the candidate frame label corresponding to the space-based target sample; And constructing the target detection loss function based on the category loss function and the regression loss function.
9. A space-based and space-based multi-target detection and correlation device, comprising: the system comprises an acquisition unit, a space-based detection unit and a space-based remote sensing image acquisition unit, wherein the acquisition unit is used for acquiring a space-based target set for space-based detection and a space-based remote sensing image for space-based acquisition, and the space-based target set comprises a plurality of space-based targets; The processing unit is used for inputting the space-based target set and the space-based remote sensing image into a pre-trained target search network, extracting space-based target associated feature sets corresponding to a plurality of space-based targets through a shared backbone network in the target search network, extracting space-based basic features in the space-based remote sensing image, carrying out target detection on the space-based basic features based on a target detection head in the target search network, determining a space-based target set in the space-based remote sensing image, and screening space-based target associated feature sets from the space-based basic features based on a plurality of space-based targets included in the space-based target set, wherein the target search network is obtained by training an initial search network based on a plurality of sky-based triplet samples, and each sky-based triplet sample comprises a space-based remote sensing image sample, a space-based target positive sample set corresponding to the space-based remote sensing image sample and a space-based target negative sample set; And the association unit is used for associating the plurality of space-based targets with the plurality of space-based targets based on the similarity between the space-based target association feature set and the space-based target association feature set.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the space-based and space-based multi-target detection and correlation method according to any one of claims 1 to 8 when executing the computer program.
11. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the space-based and space-based multi-target detection and correlation method according to any of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the space-based and space-based multi-target detection and correlation method according to any of claims 1 to 8.

Description

Multi-target detection and association method, device and equipment based on space base and space base Technical Field The application relates to the technical fields of remote sensing information processing and the like, in particular to a method, a device and equipment for multi-target detection and association based on space base and air base. Background In a space-based (e.g., low-orbit satellite, high-orbit satellite, etc.) and space-based (e.g., unmanned plane, manned aviation aircraft, etc.) collaborative observation scene, sky-based multi-objective detection and association in the scene are generally required, and key technical support is provided for the fields of national defense safety monitoring, disaster emergency response, marine objective monitoring, etc. At present, when sky-based multi-target detection and association are performed, a separate process of 'detection first and then association' is generally adopted, namely, targets are firstly extracted from the space-based remote sensing image through a target detection algorithm, and then are associated with the targets detected by the sky-based remote sensing image through an independent association algorithm, such as a similarity matching algorithm. However, with the method, the problem of repeated extraction of the detection features and the association features exists, so that the multi-target association efficiency is low. Disclosure of Invention The application provides a multi-target detection and association method, device and equipment based on a space base and a space base, which are used for solving the defect of low association efficiency caused by repeated extraction of detection features and association features in the multi-target detection and association method based on the space base and the space base in the prior art, thereby improving the multi-target association efficiency. The application provides a multi-target detection and association method based on space base and space base, which is characterized by comprising the following steps: Acquiring a space-based target set of space-based detection and a space-based remote sensing image acquired by a space-based, wherein the space-based target set comprises a plurality of space-based targets; The space-based target set and the space-based remote sensing image are input into a pre-trained target search network, a space-based target associated feature set corresponding to a plurality of space-based targets is extracted through a shared backbone network in the target search network, space-based data basic features in the space-based remote sensing image are extracted, target detection is carried out on the space-based data basic features based on a target detection head in the target search network, a space-based target set in the space-based remote sensing image is determined, and a space-based target associated feature set is screened from the space-based data basic features based on a plurality of space-based targets included in the space-based target set, wherein the target search network is obtained by training an initial search network based on a plurality of sky-based triplet samples; and associating the plurality of space-based targets with the plurality of space-based targets based on the similarity between the space-based target association feature set and the space-based target association feature set. According to the multi-target detection and association method based on space base and space base provided by the application, the target detection head in the target search network is used for carrying out target detection on the space base basic characteristics, and the space base target set in the space base remote sensing image is determined, and the method comprises the following steps: performing target detection on the space-based data basic features based on the target detection head to obtain a plurality of prediction candidate frames in the space-based remote sensing image and confidence degrees corresponding to each prediction candidate frame; and determining the space-based target set in the space-based remote sensing image based on the confidence degree corresponding to each prediction candidate frame. According to the method for detecting and associating multiple targets based on space and space, the method for associating the multiple space targets with the multiple space targets based on the similarity between the space target association feature set and the space target association feature set comprises the following steps: Constructing a corresponding association strategy cost augmentation matrix based on the similarity between the space-based target association feature set and the space-based target association feature set, wherein the association strategy cost augmentation matrix is used for representing the possibility of association of the space-based target and the space-based target; Constructing a correlation plan augmentation matrix corres