EP-4738278-A1 - MAP REFINEMENT FOR INSIDE-OUT LOCATION TRACKING AND MAPPING SYSTEM

Abstract

Techniques for map refinement (e.g., tracking data refinement) by an inside-out location tracking system may include computing a transform from a reference point to an epipolar line using an Essential Matrix derived from a reference frame camera motion in a live frame, computing several appearance errors between a feature associated with the reference point and other projected and optimized features, using a perpendicular projection hypothesis and an optimized point generated on the epipolar line, and evaluating the appearance errors using a greedy, ordered optimization. If the appearance error is less than a predetermined quality threshold, the optimized point is retained in tracking data. Otherwise the tracking data may be reset at the reference point location for reinitialization. An updated map and associated map data reflecting updated optimized points may be provided, for example, to a client device, as well as returned to a visual inertial odometry system.

Inventors

• LONG II, JOHN DAVIS

Assignees

• Qwake Technologies, Inc.

Dates

Publication Date

20260506

Application Date

20251007

Claims (18)

1. A method for map refinement by an inside-out location tracking system, the method comprising: computing a transform from a reference point to an epipolar line using an Essential Matrix derived from a reference frame camera motion in a live frame; computing a first appearance error between a first feature associated with the reference point and a second feature; computing a second appearance error between the first feature and a third feature using a perpendicular projection hypothesis; estimating a direction and a step size along the epipolar line using linear systems theory, thereby generating an optimized point on the epipolar line; computing a third appearance error between the first feature and a fourth feature associated with the optimized point; evaluating the first, second, and third appearance errors using a greedy, ordered optimization; and if the appearance error is less than a predetermined quality threshold, retaining the optimized point in tracking data, otherwise resetting the tracking data at a location associated with the reference point for reinitialization.
2. The method in claim 1, wherein the perpendicular projection hypothesis is configured to reduce a geometric error to zero.
3. The method of claim 1, further comprising taking a directional derivative along the epipolar line.
4. The method of claim 1, further comprising solving a linear system.
5. The method in claim 1, further comprising outputting an updated map and associated map data, one or both of the updated map and associated map data including any updated optimized points.
6. The method in claim 5, wherein the updated map and associated map data reflects tracking data that has been optimized for both geometric consistency and appearance consistency.
7. The method of claim 5, further comprising providing the updated map and associated map data to a client device.
8. The method of claim 5, further comprising providing the updated map and associated map data to a sparse mapping backend.
9. The method of claim 5, further comprising providing the updated map and associated map data to an autonomous navigation system.
10. The method of claim 5, further comprising providing the updated map and associated map data to a medical imaging system.
11. The method of claim 5, further comprising providing the updated map and associated map data to a robotics system.
12. A system for map refinement for inside-out location tracking, the system comprising: a memory comprising non-transitory computer-readable storage medium configured to store instructions and data, the data being stored in an associative data structure; and a processor communicatively coupled to the memory, the processor configured to execute instructions stored on the non-transitory computer-readable storage medium to: compute a transform from a reference point to an epipolar line using an Essential Matrix derived from a reference frame camera motion in a live frame; compute a first appearance error between a first feature associated with the reference point and a second feature; compute a second appearance error between the first feature and a third feature using a perpendicular projection hypothesis; estimate a direction and a step size along the epipolar line using linear systems theory, thereby generating an optimized point on the epipolar line; compute a third appearance error between the first feature and a fourth feature associated with the optimized point; evaluate the first, second, and third appearance errors using a greedy, ordered optimization; and if the appearance error is less than a predetermined quality threshold, retain the optimized point in tracking data, otherwise reset the tracking data at a location associated with the reference point for reinitialization.
13. The system of claim 12, wherein the associative data structure comprises a tracking grid configured to update information about camera and scene points.
14. The system of claim 12, wherein the associative data structure comprises a tracking grid configured to eliminate and insert new cameras and scene points.
15. The system of claim 12, wherein the associative data structure comprises a tracking grid configured to evaluate a quality of a tracked scene point.
16. The system in claim 12, wherein the data is associated with the reference frame camera motion in the live frame.
17. The system of claim 12, wherein the data comprises tracking data associated with the live frame.
18. The system of claim 12, wherein the data is associated with predetermined thresholds.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation-in-part to U.S. Patent Application No. 17/685,590 entitled "Estimating Camera Motion Through Visual Tracking In Low Contrast High Motion Single Camera Systems," filed March 3, 2022, which claims the benefit of U.S. Provisional Application No. 63/156,246, filed on March 3, 2021, all of which are hereby incorporated by reference in their entirety. BACKGROUND OF INVENTION In high stress and oftentimes hazardous environments-firefighting, accident scene, search and rescue, disaster relief, oil and gas, fighter pilots, mining, police or military operation, special operations, and the like-workers and other personnel often need to navigate as a team in an environment where it is very difficult, if not impossible, for team members to locate each other through visual or verbal means. Often team members are too dispersed, either due to hazards, obstacles, or size of operating location, to maintain visual or verbal contact. Even where radio contact is available, in many hazardous environments (e.g., fire, military engagement, disaster environments) it may not be possible for a team member to accurately describe their location, particularly relative to others to aid in navigating quickly and efficiently to a desired location. Also, the operating locations might be remote where conventional location tracking technologies (e.g., GPS and cellular) are unreliable (i.e., intermittent or insufficient resolution). Other persons (e.g., jogger, hiker, adventurer) also trek into remote areas and often get lost in locations where conventional location tracking technology is unreliable. While conventional GPS and cellular triangulation methods work well enough within urban environments, they often perform poorly in remote locations or in a disaster situation. Many conventional existing team location tracking and mapping solutions require outside-in location tracking infrastructure, relying on external location services, such as GPS. Outside-in location tracking systems require infrastructure (e.g., GPS satellites, warehouse cameras, emitters, etc.) that is often lacking in these environments. Sparse feature tracking requires high quality images. Known camera-based inside-out team location tracking systems assume high-quality visible light images (i.e., for extracting sparse features, which are used for matching across time in order to estimate camera motion and scene structure). Since the hazardous or disaster environments in which emergency responders and critical workers often need to operate typically do not have access to external location services and cannot accommodate the capture of high-quality visible light images in real time, these conventional solutions are of limited use to them. Thus, there is a need for an improved inside-out location tracking and mapping system. BRIEF SUMMARY The present disclosure provides techniques for map refinement (e.g., tracking data refinement) by an inside-out location tracking and mapping system. A method for map refinement by an inside-out location tracking and mapping system may include: computing a transform from a reference point to an epipolar line using an Essential Matrix derived from a reference frame camera motion in a live frame; computing a first appearance error between a first feature associated with the reference point and a second feature; computing a second appearance error between the first feature and a third feature using a perpendicular projection hypothesis; estimating a direction and a step size along the epipolar line using linear systems theory, thereby generating an optimized point on the epipolar line; computing a third appearance error between the first feature and a fourth feature associated with the optimized point; evaluating the first, second, and third appearance errors using a greedy, ordered optimization; and if the appearance error is less than a predetermined quality threshold, retaining the optimized point in tracking data, otherwise resetting the tracking data at a location associated with the reference point for reinitialization. In some examples, the perpendicular projection hypothesis is configured to reduce a geometric error to zero. In some examples, the method also includes taking a directional derivative along the epipolar line. In some examples, the method also includes solving a linear system. In some examples, the method also includes outputting an updated map and associated map data, one or both of the updated map and associated map data including any updated optimized points. In some examples, the updated map and associated map data reflects tracking data that has been optimized for both geometric consistency and appearance consistency. In some examples, the method also includes providing the updated map and associated map data to a client device. In some examples, the method also includes providing the updated map and associated map data to a sparse mapping backen