CN-121746472-B - Visual positioning method and device, storage medium and electronic equipment

CN121746472BCN 121746472 BCN121746472 BCN 121746472BCN-121746472-B

Abstract

The application discloses a visual positioning method, a visual positioning device, a storage medium and electronic equipment, and relates to the technical field of information. The method comprises the steps of obtaining a current frame image and a previous frame image corresponding to the current frame image, extracting feature points and feature lines corresponding to the previous frame image, extracting feature points and feature lines corresponding to the current frame image, carrying out feature point matching and feature line matching on the previous frame image and the current frame image to obtain feature point matching pairs and feature line matching pairs, respectively constructing point reprojection errors and line reprojection errors between the previous frame image and the current frame image based on the feature point matching pairs and the feature line matching pairs, constructing a target optimization function according to the point reprojection errors and the line reprojection errors, solving relative pose between the current frame image and the previous frame image, and carrying out visual positioning on a target object according to the relative pose. The method and the device can overcome the limitation of feature point extraction in the weak texture environment, thereby improving the accuracy and the robustness of visual positioning.

Inventors

CHEN JIE
HOU PENGSHUAI
ZHAO HAOYU

Assignees

东北大学

Dates

Publication Date: 20260508
Application Date: 20260302

Claims (8)

1. A method of visual localization comprising: acquiring a current frame image and a previous frame image corresponding to the current frame image; Extracting feature points and feature lines corresponding to the previous frame image, and feature points and feature lines corresponding to the current frame image, carrying out feature point matching and feature line matching on the previous frame image and the current frame image to obtain feature point matching pairs and feature line matching pairs, respectively calculating direction differences, length ratios and midpoint distances between the feature lines corresponding to the previous frame image and the feature lines corresponding to the current frame image when the feature lines are matched, and determining that the feature lines corresponding to the previous frame image are matched with the feature lines corresponding to the current frame image if the direction differences are smaller than a preset angle, the length ratios are larger than a preset length, and the midpoint distances are smaller than a preset midpoint distance; Respectively constructing a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the characteristic point matching pair and the characteristic line matching pair, wherein the line re-projection error comprises a two-dimensional re-projection error and a three-dimensional re-projection error, and when the three-dimensional re-projection error is constructed, back-projecting a second characteristic line into space based on the pixel coordinate of the second characteristic line in the current frame image in any characteristic line pair to obtain the three-dimensional coordinate of the second characteristic line under a world coordinate system; calculating a vertical distance between a first space line segment corresponding to the first characteristic line and a second space line segment corresponding to the second characteristic line and an endpoint distance between the first space line segment and the second space line segment based on a three-dimensional coordinate of the second characteristic line under a world coordinate system and a three-dimensional coordinate of a first characteristic line in any one of the characteristic line pairs under the world coordinate system; constructing an objective optimization function according to the point re-projection error and the line re-projection error; Based on the target optimization function, calculating the relative pose between the current frame image and the previous frame image; According to the relative pose, performing visual positioning on a target object to obtain positioning information of the target object based on the current frame image; wherein after the current frame image and the previous frame image corresponding to the current frame image are obtained, the method further comprises: respectively determining potential dynamic feature points and potential dynamic feature lines in the current frame image, and potential dynamic feature points and potential dynamic feature lines in the previous frame image; after the feature point matching and the feature line matching are performed on the previous frame image and the current frame image, the feature point matching pair and the feature line matching pair are obtained, the method further comprises: determining potential dynamic feature point matching pairs and potential dynamic feature line matching pairs according to the potential dynamic feature points and the potential dynamic feature lines in the current frame image and the potential dynamic feature points and the potential dynamic feature lines in the previous frame image; determining a static feature point matching pair and a static feature line matching pair based on the potential dynamic feature point matching pair and the potential dynamic feature line matching pair; Calculating a basic matrix based on the static feature point matching pairs, and determining polar lines between the current frame image and the previous frame image; Calculating the epipolar distances respectively corresponding to the potential dynamic feature point matching pairs and the potential dynamic feature line matching pairs based on the potential dynamic feature point matching pairs and the potential dynamic feature line matching pairs, and the epipolar lines and the basic matrix; Determining a real static feature point matching pair in the potential dynamic feature point matching pair and a real static feature line matching pair in the potential dynamic feature line matching pair based on the epipolar distance; determining the target feature point matching pair and the target feature line matching pair based on the real static feature point matching pair and the real static feature line matching pair and the static feature line matching pair respectively; The step of respectively constructing a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the characteristic point matching pair and the characteristic line matching pair comprises the following steps: and respectively constructing a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the target characteristic point matching pair and the target characteristic line matching pair.
2. The method of claim 1, wherein constructing a point re-projection error between the previous frame image and the current frame image based on the feature point matching pair comprises: Calculating three-dimensional coordinates of a first feature point in the world coordinate system based on pixel coordinates of the first feature point in the previous frame image in any one feature point matching pair and historical relative pose between the previous frame image and the previous two frame images corresponding to the current frame image; Calculating the projection coordinates of the first feature point in the current frame image based on the three-dimensional coordinates of the first feature point in a world coordinate system, a camera internal reference matrix and a scale coefficient; calculating a re-projection error corresponding to any one of the feature point pairs based on pixel coordinates of a second feature point in the current frame image and projection coordinates of the first feature point in the current frame image; And determining the point re-projection error according to the re-projection error corresponding to any one of the characteristic point pairs.
3. The method of claim 1, wherein constructing a line re-projection error between the previous frame image and the current frame image based on the pair of feature line matches comprises: Performing two-dimensional re-projection calculation on any one of the characteristic line pairs aiming at any one of the characteristic line pairs to obtain a two-dimensional re-projection error corresponding to the any one of the characteristic line pairs; carrying out three-dimensional reprojection calculation on any one of the characteristic line pairs to obtain a three-dimensional reprojection error corresponding to the any one of the characteristic line pairs; and determining the line re-projection error between the previous frame image and the current frame image based on the two-dimensional re-projection error and the three-dimensional re-projection error corresponding to any one characteristic line pair.
4.A method according to claim 3, wherein the performing two-dimensional re-projection calculation on the arbitrary feature line pair to obtain a two-dimensional re-projection error corresponding to the arbitrary feature line pair includes: Determining three-dimensional coordinates of a first characteristic line in the world coordinate system based on pixel coordinates of the first characteristic line in the previous frame image in any characteristic line pair; Determining projection coordinates of the first characteristic line in the current frame image based on three-dimensional coordinates of the first characteristic line in a world coordinate system; Calculating a vertical distance between a projection line segment corresponding to the first characteristic line and a second characteristic line in any characteristic line pair based on pixel coordinates of the second characteristic line in the current frame image and projection coordinates of the first characteristic line in the current frame image; And determining a two-dimensional re-projection error based on the vertical distance between the projection line segments corresponding to the second characteristic line and the first characteristic line.
5. The method of claim 1, wherein the determining potential dynamic feature points and potential dynamic feature lines in the current frame image and potential dynamic feature points and potential dynamic feature lines in the previous frame image, respectively, comprises: respectively carrying out target recognition on the current frame image and the previous frame image to obtain semantic information corresponding to the current frame image and semantic information corresponding to the previous frame image; And respectively determining potential dynamic feature points and potential dynamic feature lines in the current frame image and potential dynamic feature points and potential dynamic feature lines in the previous frame image based on the semantic information corresponding to the current frame image and the semantic information corresponding to the previous frame image.
6. A visual positioning device, comprising: an acquisition unit, configured to acquire a current frame image and a previous frame image corresponding to the current frame image; The extraction unit is used for extracting the characteristic points and the characteristic lines corresponding to the previous frame image, and the characteristic points and the characteristic lines corresponding to the current frame image, carrying out characteristic point matching and characteristic line matching on the previous frame image and the current frame image to obtain characteristic point matching pairs and characteristic line matching pairs, respectively calculating the direction difference, the length ratio and the midpoint distance between the characteristic lines corresponding to the previous frame image and the characteristic lines corresponding to the current frame image when the characteristic lines are matched, and determining that the characteristic lines corresponding to the previous frame image are matched with the characteristic lines corresponding to the current frame image if the direction difference is smaller than a preset angle, the length ratio is larger than a preset length and the midpoint distance is smaller than a preset midpoint distance; A construction unit, configured to respectively construct a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the feature point matching pair and the feature line matching pair, where the line re-projection error includes a two-dimensional re-projection error and a three-dimensional re-projection error, and when constructing the three-dimensional re-projection error, back-project a second feature line in space based on a pixel coordinate of the second feature line in the current frame image in any one feature line pair, to obtain a three-dimensional coordinate of the second feature line in a world coordinate system; calculating a vertical distance between a first space line segment corresponding to the first characteristic line and a second space line segment corresponding to the second characteristic line and an endpoint distance between the first space line segment and the second space line segment based on a three-dimensional coordinate of the second characteristic line under a world coordinate system and a three-dimensional coordinate of a first characteristic line in any one of the characteristic line pairs under the world coordinate system; The construction unit is further used for constructing a target optimization function according to the point re-projection error and the line re-projection error; A resolving unit, configured to resolve a relative pose between the current frame image and the previous frame image based on the objective optimization function; the positioning unit is used for carrying out visual positioning on the target object according to the relative pose to obtain positioning information of the target object based on the current frame image; a determining unit, configured to determine a potential dynamic feature point and a potential dynamic feature line in the current frame image, and a potential dynamic feature point and a potential dynamic feature line in the previous frame image, respectively; The determining unit is further configured to determine a potential dynamic feature point matching pair and a potential dynamic feature line matching pair according to the potential dynamic feature point and the potential dynamic feature line in the current frame image and the potential dynamic feature point and the potential dynamic feature line in the previous frame image, determine a static feature point matching pair and a static feature line matching pair based on the potential dynamic feature point matching pair and the potential dynamic feature line matching pair, calculate a base matrix based on the static feature point matching pair and determine a polar line between the current frame image and the previous frame image, calculate respective polar line distances of the potential dynamic feature point matching pair and the potential dynamic feature line matching pair based on the potential dynamic feature point matching pair and the potential dynamic feature line matching pair, and determine a true static feature point matching pair of the potential dynamic feature point matching pair and the true static feature point matching pair and a true static feature line matching pair based on the polar line distance, and the true static feature point matching pair and the true feature line matching pair, respectively; The construction unit is specifically configured to respectively construct a point reprojection error and a line reprojection error between the previous frame image and the current frame image based on the target feature point matching pair and the target feature line matching pair.
7. A storage medium having stored thereon a computer program, which when executed by a processor, implements the method of any of claims 1 to 5.
8. An electronic device comprising a storage medium, a processor and a computer program stored on the storage medium and executable on the processor, characterized in that the processor implements the method of any one of claims 1 to 5 when executing the computer program.

Description

Visual positioning method and device, storage medium and electronic equipment Technical Field The present application relates to the field of information technologies, and in particular, to a visual positioning method, a visual positioning device, a storage medium, and an electronic device. Background With the continuous evolution of robotics, visual SLAM (Simultaneous Localization AND MAPPING, instant positioning and map building) has become an indispensable positioning navigation core technology in the fields of autopilot, industrial robots, unmanned aerial vehicles, and the like. The technology does not need to rely on traditional positioning means such as GPS and the like, and shows irreplaceable practical value in a scene without satellite signal coverage or high-precision positioning requirement. Although the visual SLAM technology has made breakthrough progress in a plurality of fields, a plurality of technical bottlenecks to be solved still face under complex actual working conditions, and positioning performance is easily affected by special environments. Currently, a visual positioning method based on feature points is generally adopted. However, in a weak texture environment formed by an open corridor, a large-area flat surface or a sparse texture area, stable characteristic points are lacking, and the visual positioning mode based on the characteristic points only in the prior art can influence the positioning accuracy and robustness. Disclosure of Invention In view of the above, the present application provides a visual positioning method, a device, a storage medium and an electronic apparatus, which mainly overcome the limitation of feature point extraction in a weak texture environment, so as to improve the accuracy and robustness of visual positioning. According to a first aspect of the present application there is provided a visual positioning method comprising: acquiring a current frame image and a previous frame image corresponding to the current frame image; extracting feature points and feature lines corresponding to the previous frame image, and feature points and feature lines corresponding to the current frame image, and performing feature point matching and feature line matching on the previous frame image and the current frame image to obtain feature point matching pairs and feature line matching pairs; respectively constructing a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the characteristic point matching pair and the characteristic line matching pair; constructing an objective optimization function according to the point re-projection error and the line re-projection error; Based on the target optimization function, calculating the relative pose between the current frame image and the previous frame image; And according to the relative pose, performing visual positioning on a target object to obtain positioning information of the target object based on the current frame image. According to a second aspect of the present application there is provided a visual positioning device comprising: an acquisition unit, configured to acquire a current frame image and a previous frame image corresponding to the current frame image; The extraction unit is used for extracting the characteristic points and the characteristic lines corresponding to the previous frame image, the characteristic points and the characteristic lines corresponding to the current frame image, and carrying out characteristic point matching and characteristic line matching on the previous frame image and the current frame image to obtain characteristic point matching pairs and characteristic line matching pairs; the construction unit is used for respectively constructing a point re-projection error and a line re-projection error between the previous frame image and the current frame image based on the characteristic point matching pair and the characteristic line matching pair; The construction unit is further used for constructing a target optimization function according to the point re-projection error and the line re-projection error; A resolving unit, configured to resolve a relative pose between the current frame image and the previous frame image based on the objective optimization function; And the positioning unit is used for carrying out visual positioning on the target object according to the relative pose to obtain positioning information of the target object based on the current frame image. According to a third aspect of the present application, there is provided a storage medium having stored thereon a computer program which when executed by a processor implements the above-described visual positioning method. According to a fourth aspect of the present application, there is provided an electronic device comprising a storage medium, a processor and a computer program stored on the storage medium and executable on the process