CN-122015800-A - Relative pose measurement method based on sequence frame image

CN122015800ACN 122015800 ACN122015800 ACN 122015800ACN-122015800-A

Abstract

The invention provides a relative pose measurement method based on a sequence frame image, which comprises the steps of acquiring an infrared image of a target in real time by using an infrared camera in the process of measuring the relative position and pose of the aircraft, acquiring inertial pose information of the aircraft in real time by using an inertial navigation system, grouping the acquired images in a sliding window mode, acquiring a first homography matrix and a second homography matrix, determining a final homography matrix, acquiring final point line characteristics on a current frame image, acquiring blanking points of intersection of a left edge line of the target and a right edge line of the target on an image coordinate system and shadow eliminating points of an initial line of the target on the image coordinate system, establishing a pose calculation equation, and carrying out pose calculation on the pose calculation equation to obtain pitch angle, heading angle, rolling angle of the aircraft and lateral displacement, forward displacement and vertical displacement of the aircraft relative to the target.

Inventors

WEI YONGSHU
LI ZHI
WANG KANG
SHANG KEJUN
HU GUANGFENG
XU CE
MING LI
ZHAO LIANG
ZHANG WEIJIAN
ZHAO YUFEI
LIU CHONGLIANG

Assignees

北京自动化控制设备研究所

Dates

Publication Date: 20260512
Application Date: 20251205

Claims (6)

1. The method for measuring the relative pose based on the sequence frame images is characterized by comprising the following steps of: In the process of measuring the relative position and attitude of the aircraft, an infrared camera is utilized to acquire an infrared image of a target in real time, and an inertial navigation system is utilized to acquire inertial attitude information of the aircraft in real time; Grouping the acquired images in a sliding window mode, wherein each group comprises n continuous frame images, the previous n-1 frame images are historical frame images, the nth frame image is a current frame image, and identifying each frame image by utilizing a feature identification network to obtain a target point line feature coordinate, wherein the target point line feature comprises a boundary line, a central line and a corner point of a target; The method comprises the steps of carrying out image feature extraction on n frames of images of each group by utilizing a super-point image feature extraction network to obtain n extracted image features, registering the image features extracted by the previous n-1 frames with the image features extracted by the nth frame by utilizing a super-glue feature registration network to obtain a registration relationship between a historical frame image and a current frame image; Judging whether the pixel coordinate distance of the target point line characteristic projection result of the first homography matrix and the second homography matrix is smaller than a first preset value, if so, taking the first homography matrix as a final homography matrix, otherwise, taking the second homography matrix as the final homography matrix; Respectively projecting the target point line characteristics on the previous n-1 frame images to the current frame image based on the target point line characteristic coordinates of each frame image and the final homography matrix to serve as a target point line characteristic extraction result on the current frame image; Establishing a linear equation of three lines of a target left line, a target starting line and a target right line on an image based on final line characteristics on a current frame image, and acquiring vanishing points of the target left line and the target right line intersecting on an image coordinate system and vanishing points of the target starting line on the image coordinate system based on the linear equation; And establishing a pose solving equation based on the vanishing points of the left edge line and the right edge line of the target intersecting on the image coordinate system, the vanishing points of the initial line of the target on the image coordinate system, the camera internal parameter matrix, the rotation matrix from the world coordinate system to the camera coordinate system, the unit direction vector of the edge line of the target and the unit direction vector of the initial line of the target, and carrying out pose solving on the pose solving equation to obtain the pitch angle, the course angle, the roll angle and the lateral displacement, the forward displacement and the vertical displacement of the aircraft relative to the target.
2. The method of claim 1, wherein comprehensively optimizing the extraction result of the target line features on the current frame image to obtain the final line features on the current frame image comprises: acquiring preliminary center points corresponding to n groups of target point line features in a target point line feature extraction result on a current frame image, wherein the target point line feature extraction result on the current frame image comprises n groups of target point line features, each frame image corresponds to one group of target point line features, and each group of target point line features comprises a plurality of target point line features; The preliminary center point is the mean value point of the characteristic coordinates of n groups of target point lines; Acquiring Z scores of each target point line characteristic in each group of target point line characteristics based on the coordinates of each group of target point line characteristics and the coordinates of the preliminary center point; and screening out target point line characteristics corresponding to Z fractions with absolute values smaller than or equal to a second preset value, and taking the preliminary central points of all the screened target point line characteristics as final point line characteristics on the current frame image.
3. The method according to claim 1 or 2, wherein the Z-score of each target point line feature of each set of target point line features is obtained by: where (x α ,y α ) (α=1, 2,., n) is the coordinate of one target point line feature in the α -th group of n-group of target point line features, (x c ,y c ) is the coordinate of the preliminary center point, obtained by calculating the mean value of the n-group of target point line feature coordinates, μ d 、σ d is the distance mean value and the distance standard deviation from the coordinate of the n-group of target point line features to the preliminary center point, respectively, and Z α is the Z fraction of each target point line feature.
4. A method according to any one of claims 1-3, characterized in that the pose solving equation is established by: wherein K is a matrix of parameters in the camera, For a rotation matrix of the world coordinate system to the camera coordinate system, For vanishing points where object edges intersect in the image coordinate system, For the vanishing point of the target initial line on the image coordinate system, d 1 、d 2 is the unit direction vector of the target edge line and the target initial line, and a 1 、a 2 is the vanishing point Is a scale factor of (a).
5. The method of claim 1, wherein the pitch angle, heading angle, roll angle of the aircraft is obtained by: wherein, theta, phi and gamma are pitch angle, course angle and roll angle of the aircraft respectively, Is that The j-th column element of the i-th row corresponding to the matrix, i=1, 2,3, j=1, 2,3.
6. The method of claim 1, wherein the lateral displacement, forward displacement, vertical displacement of the aircraft relative to the target is obtained by: In the formula, The lateral displacement, the forward displacement and the vertical displacement of the aircraft relative to the target are respectively, alpha left is a scale factor of a left line of the target, w is a target width, l is a target length, l lx 、l lz is an x coordinate and a y coordinate of the left line of the target in an image coordinate system respectively, l sx 、l sy is an x coordinate and a y coordinate of an initial line of the target in the image coordinate system respectively, and a ij is The i-th row and j-th column elements corresponding to the matrix, i=1, 2,3, j=1, 2,3, k * are cofactor matrices of the parameter matrix in the camera.

Description

Relative pose measurement method based on sequence frame image Technical Field The invention relates to the technical field of computer vision, in particular to a relative pose measurement method based on a sequence frame image. Background In the process of measuring the relative position and attitude of an aircraft, when the traditional point line features extracted based on single-frame images are used for measuring the relative position and attitude, the factors such as environmental changes and ground feature interference also greatly influence the reliable extraction of the image quality and geometric features, so that the problems of measurement jump, discontinuity and the like exist in the measurement of the relative position and attitude based on the single-frame images, and the reliability of the measurement of the relative position and attitude is seriously influenced. Disclosure of Invention The invention provides a relative pose measurement method based on sequence frame images, which can process the sequence images by utilizing the relativity of the features among the sequence frames so as to improve the robustness and the precision of a relative pose measurement algorithm. The invention provides a relative pose measurement method based on a sequence frame image, which comprises the following steps: In the process of measuring the relative position and attitude of the aircraft, an infrared camera is utilized to acquire an infrared image of a target in real time, and an inertial navigation system is utilized to acquire inertial attitude information of the aircraft in real time; Grouping the acquired images in a sliding window mode, wherein each group comprises n continuous frame images, the previous n-1 frame images are historical frame images, the nth frame image is a current frame image, and identifying each frame image by utilizing a feature identification network to obtain a target point line feature coordinate, wherein the target point line feature comprises a boundary line, a central line and a corner point of a target; The method comprises the steps of carrying out image feature extraction on n frames of images of each group by utilizing a super-point image feature extraction network to obtain n extracted image features, registering the image features extracted by the previous n-1 frames with the image features extracted by the nth frame by utilizing a super-glue feature registration network to obtain a registration relationship between a historical frame image and a current frame image; Judging whether the pixel coordinate distance of the target point line characteristic projection result of the first homography matrix and the second homography matrix is smaller than a first preset value, if so, taking the first homography matrix as a final homography matrix, otherwise, taking the second homography matrix as the final homography matrix; Respectively projecting the target point line characteristics on the previous n-1 frame images to the current frame image based on the target point line characteristic coordinates of each frame image and the final homography matrix to serve as a target point line characteristic extraction result on the current frame image; Establishing a linear equation of three lines of a target left line, a target starting line and a target right line on an image based on final line characteristics on a current frame image, and acquiring vanishing points of the target left line and the target right line intersecting on an image coordinate system and vanishing points of the target starting line on the image coordinate system based on the linear equation; And establishing a pose solving equation based on the vanishing points of the left edge line and the right edge line of the target intersecting on the image coordinate system, the vanishing points of the initial line of the target on the image coordinate system, the camera internal parameter matrix, the rotation matrix from the world coordinate system to the camera coordinate system, the unit direction vector of the edge line of the target and the unit direction vector of the initial line of the target, and carrying out pose solving on the pose solving equation to obtain the pitch angle, the course angle, the roll angle and the lateral displacement, the forward displacement and the vertical displacement of the aircraft relative to the target. Preferably, the step of comprehensively optimizing the extraction result of the target point line feature on the current frame image to obtain the final point line feature on the current frame image includes: acquiring a preliminary center point corresponding to n groups of target point line characteristics in a target point line characteristic extraction result on a current frame image, wherein the target point line characteristic extraction result on the current frame image comprises n groups of target point line characteristics, each frame image corresponds to one group of target