CN-121982105-A - Spacecraft pose estimation system and method based on progressive heat map refinement

CN121982105ACN 121982105 ACN121982105 ACN 121982105ACN-121982105-A

Abstract

The invention relates to the technical field of spacecraft detection, in particular to a spacecraft pose estimation system and method with progressive heat map refinement, wherein a feature extraction and fusion module extracts multi-scale features from images of a target spacecraft, performs cross-scale fusion on the multi-scale features to obtain multi-scale fusion features, a multi-stage key point prediction module detects key points of the spacecraft by using a key point detection head based on fusion features with corresponding sizes in the key point prediction process of each stage, connects the obtained key point heat map with key point heat map residues obtained in the previous stage to obtain a key point heat map of the current stage, and a key point screening and pose solving module screens two-dimensional key points in the final key point heat map according to response values, finds one corresponding three-dimensional coordinates, analyzes the obtained two-dimensional feature point pair set and obtains a final pose result. The invention maintains stable positioning accuracy under complex illumination and real imaging conditions.

Inventors

LIU PEIXUN
ZENG XIN
WU GUOLIANG

Assignees

中国科学院长春光学精密机械与物理研究所

Dates

Publication Date: 20260505
Application Date: 20260407

Claims (10)

1. A spacecraft pose estimation system with progressive heat map refinement, comprising: The feature extraction and fusion module extracts multi-scale features from the image of the target spacecraft, and performs cross-scale fusion on the multi-scale features through parallel paths to obtain multi-scale fusion features; The multi-stage key point prediction module is used for detecting key points of the spacecraft by using a key point detection head based on fusion characteristics of corresponding sizes in the key point prediction process of each stage, and connecting the obtained key point heat map with the key point heat map residual error obtained in the previous stage to obtain the key point heat map of the current stage; The key point screening and pose solving module screens the two-dimensional key points in the final key point heat map according to the response values of the two-dimensional key points in the final key point heat map, finds three-dimensional coordinates corresponding to the screened two-dimensional key points one by one on the target spacecraft, analyzes the two obtained three-dimensional feature point pairs, inputs the two three-dimensional feature point pairs into the pose computing model, carries out iterative solution on the pose model through a random sampling consistency mechanism, screens an optimal model by taking a reprojection error as an evaluation index, and obtains a rotation matrix and a translation vector of the target to obtain a final pose result.
2. The progressive heat map refined spacecraft pose estimation system of claim 1, wherein the feature extraction and fusion module comprises a feature extraction sub-module and a multi-path feature fusion sub-module, wherein: in the feature extraction submodule, performing step-by-step feature extraction on an input image by adopting a CvT network to obtain features of different stage scales; in the multipath feature fusion submodule, after each feature output by the feature extraction submodule is added with the feature of the previous scale, the feature is spliced with the feature channels of two adjacent scales, and the corresponding fusion feature is obtained.
3. The progressive heat map refined spacecraft pose estimation system of claim 1, wherein in the multi-stage keypoint prediction module: in the key point prediction process of the first stage, inputting the fusion characteristic of the minimum scale into a key point detection head to obtain a key point heat map of the first stage; in the process of predicting the key points from the first stage to the second last stage, performing feature enhancement processing on other fusion features, inputting the feature enhancement processed feature enhancement into a key point detection head, and connecting the obtained key point heat map with the key point heat map residual error obtained in the previous stage to obtain the key point heat map of the current stage; in the final stage of key point prediction, feature enhancement processing is carried out on the fusion features with the maximum size, the feature enhancement is finished, the fusion features are input into a key point detection head, and the obtained key point heat map is connected with the key point heat map residual errors obtained in all stages, so that the final key point heat map is obtained.
4. A spacecraft pose estimation system according to claim 3, characterized in that in the multi-stage keypoint prediction module further comprises predicting depth information of the keypoints, wherein: In the depth information prediction process of the first stage, inputting the fusion characteristic of the minimum scale into a depth detection head to obtain the depth information of the key point of the first stage; In the depth information prediction process from the second stage to the penultimate stage, performing feature enhancement processing on other fusion features, inputting the feature enhancement processed feature enhancement into a depth detection head, and connecting the obtained depth information with the depth information residual error obtained in the previous stage to obtain the depth information of the current stage; in the depth information prediction process of the final stage, carrying out feature enhancement processing on the fusion features with the maximum size, inputting the fusion features into a depth detection head after feature enhancement, and connecting the obtained depth information with depth information residual errors obtained in all stages to obtain final depth information.
5. The progressive heat map refined spacecraft pose estimation system of claim 1, wherein in the keypoint screening and pose solving module: extracting two-dimensional coordinates of key points from the final key point heat map, and calculating response values of the two-dimensional coordinates of each key point; According to the response value, carrying out synchronous descending order sorting on the two-dimensional key points and three-dimensional coordinates corresponding to the two-dimensional key points one by one on the target spacecraft, selecting the first M two-dimensional characteristic point pairs in the sorting result to obtain two-dimensional characteristic point pair sets, wherein M is larger than 3; Inputting the two three-dimensional feature point pair sets and internal references of a camera shooting the target spacecraft into a pose calculation model, carrying out iterative solution on the pose model through a random sampling consistency mechanism, and screening an optimal model by taking a reprojection error as an evaluation index to obtain a rotation matrix and a translation vector of the target as a final pose result.
6. The spacecraft pose estimation method for progressive heat map refinement is characterized by comprising the following steps of: s1, acquiring a spacecraft monocular image data set, and preprocessing brightness and space light and shadow adjustment of the data set to obtain a training set, wherein the spacecraft monocular image data set comprises a spacecraft monocular image and a key point thermal icon corresponding to the spacecraft monocular image; S2, constructing the spacecraft pose estimation system with progressive heat map refinement according to any one of claims 1 to 5; S3, training the spacecraft pose estimation system constructed in the step S2 by taking the training set constructed in the step S1 as input and the corresponding key point heat map label as output to obtain a spacecraft pose estimation model; and S4, inputting the monocular image of the spacecraft to be detected into the spacecraft pose estimation model obtained in the step S3, and obtaining a predicted pose result of the spacecraft.
7. The spacecraft pose estimation method of progressive heat map refinement according to claim 6, wherein the preprocessing of brightness and spatial light shadow adjustment in step S1 comprises: Extracting a bright region in a monocular image of the spacecraft to generate a bright region mask; Carrying out Gaussian blur on the mask in the bright area to form a diffuse high light spot; And overlapping the diffuse high light spots to the monocular image of the spacecraft according to preset intensity, and then executing overall dynamic range compression.
8. The method for estimating pose of a spacecraft with progressive heat map refinement according to claim 6, wherein step S1 further comprises: blurring and sharpening a monocular image of the spacecraft with a preset probability, wherein the monocular image is used for simulating motion blurring caused by relative motion between the spacecraft and a sensor; Adding noise into a monocular image of the spacecraft with preset probability, and simulating degradation caused by sensor noise and environmental scattering in the real in-orbit imaging; Performing structure degradation and enhancement operation on the monocular image of the spacecraft with preset probability, and simulating local shielding and information deletion of the spacecraft; intensity transformation is performed on the spacecraft monocular image with a preset probability for enhancing the brightness distribution diversity of the dataset.
9. The method for estimating pose of a spacecraft with progressive heat map refinement according to claim 6, wherein the training process in step S3 further comprises: acquiring actual monocular images of a plurality of target spacecrafts, and performing preprocessing operation of the step S1 on the actual monocular images; Inputting the preprocessed actual multiple monocular images into a spacecraft pose estimation model obtained through training to obtain a plurality of corresponding estimated pose results; And selecting a plurality of pose results with the best effect from the plurality of estimated pose results as a secondary training label, and performing secondary training on the spacecraft pose estimation model by taking corresponding actual plurality of monocular images as training input to obtain a final spacecraft pose estimation model.
10. The spacecraft pose estimation method of progressive heat map refinement of claim 6, wherein in step S3, the spacecraft pose estimation system is trained by joint optimization loss of the formula: L total =L hm +γ 1 L pnp +γ 2 L 3d ; Wherein L total represents a joint optimization penalty, L pnp represents a projection consistency supervision penalty, L 3d represents a three-dimensional geometric alignment penalty, L hm represents a heat map penalty, and gamma 1 and gamma 2 represent penalty weights; The heat map loss is: ; where S represents the number of stages in the multi-stage keypoint prediction module, C represents the number of keypoints, U V represents the heat map resolution, Representing a predicted heat map output in a j-th stage of the multi-stage keypoint prediction module, H j representing a corresponding real heat map, beta representing a gain controlled parameter, Representing the square of the Frobenius norm; projection consistency supervision loss is: ; Wherein p i represents the two-dimensional truth coordinates of the ith key point, Representing the two-dimensional coordinates of the ith key point in the predictive heat map output in the jth stage, Representing an ith key point two-dimensional coordinate obtained after re-projecting the corresponding three-dimensional point to an image plane under the j-th stage estimated posture; the three-dimensional geometric alignment loss is: ; Wherein, the And (3) representing a pose estimation matrix obtained by performing rigid alignment on the three-dimensional key point set in the predictive heat map output in the j-th stage and the three-dimensional key point set of the spacecraft model, wherein T represents a spacecraft pose truth value matrix.

Description

Spacecraft pose estimation system and method based on progressive heat map refinement Technical Field The invention belongs to the technical field of spacecraft detection, and particularly relates to a spacecraft pose estimation system and method based on progressive heat map refinement. Background With the development of space tasks such as on-orbit service, active fragment removal, formation flight and the like, the service spacecraft is required to perform relative pose sensing on the target spacecraft only by virtue of an onboard sensor under the condition of lacking a target cooperative mark and communication cooperation. Due to the fact that the multi-mode sensor fusion scheme is limited by the power consumption, the volume and the computing power of the satellite-borne platform, the multi-mode sensor fusion scheme is complex to deploy in engineering application, and the monocular vision system becomes an important technical route for estimating the pose of the current non-cooperative target due to the advantages of simple structure, low power consumption, high integration level and the like. However, monocular vision spacecraft pose estimation still faces a number of challenges in engineering applications. Firstly, the conditions of sparse texture, strong reflection, local shielding and the like commonly exist on the surface of a spacecraft, and strong directional illumination, shadow change and background interference are accompanied in a real space environment, so that the distribution difference of image features under different observation conditions is obvious, and an unstable phenomenon is easy to occur in an attitude estimation result. Secondly, the existing deep learning method is trained by relying on synthetic data, and the real on-orbit or hardware has obvious differences with the synthetic image in the aspects of illumination distribution, noise characteristics and imaging blurring degree of the ring-collected image, so that the model is easy to generate obvious performance degradation when the synthetic domain is transferred to the real domain, and the requirement of an actual task on stability is difficult to meet. In addition, in order to improve the cross-domain robustness and the gesture estimation precision, a part of methods introduce a complex network structure or a multi-task joint learning mechanism, and although higher precision is obtained on a specific test set, the model parameter scale and the calculation complexity are obviously increased, real-time reasoning is difficult to realize on a satellite-borne embedded platform, and the engineering deployment feasibility is limited. On the other hand, even if an indirect method of adding geometric solution to the key points is adopted, when the prediction precision of the key points is insufficient or the local noise interference is strong, the geometric solution result is still easily affected by the outlier, so that the attitude estimation error is amplified. Therefore, in the visual navigation task of the non-cooperative spacecraft, how to improve the stability of the model under complex illumination and cross-domain conditions and meet the strict constraint of a satellite-borne platform on computing resources and real-time performance while guaranteeing the pose estimation precision is a key technical problem to be solved. Disclosure of Invention In view of the above, the invention aims to provide a spacecraft pose estimation system and method with progressive heat map refinement, which can maintain stable positioning precision under complex illumination and real imaging conditions by constructing a stable multi-scale feature expression structure, a progressive key point refinement mechanism and a geometric consistency driving method model, and simultaneously control the scale and calculation complexity of the model to realize a visual navigation scheme capable of being deployed in engineering. In order to achieve the above purpose, the technical scheme of the invention is realized as follows: a spacecraft pose estimation system with progressive heat map refinement comprises a feature extraction and fusion module, a multi-stage key point prediction module, a key point detection module and a key point screening and pose solving module, wherein the feature extraction and fusion module extracts multi-scale features from images of a target spacecraft, cross-scale fusion is carried out on the multi-scale features through parallel paths to obtain multi-scale fusion features, the multi-stage key point prediction module detects key points of the spacecraft by using a key point detection head based on fusion features with corresponding sizes in a key point prediction process, the obtained key point heat map is connected with key point heat map residues obtained in the previous stage to obtain a key point heat map in the current stage, the key point screening and pose solving module screens two-dimensional key points in the final key p