EP-4740185-A1 - SYSTEMS AND METHODS FOR USER INTERACTIVE GEOREGISTRATION

Abstract

Systems and methods for georegistration are provided. An example method includes receiving a video stream including a plurality of video frames collected by an image sensor, presenting the video stream via a video player, and receiving user input associated with a first video frame of the plurality of video frames and a reference image. In some examples, the first video frame includes incomplete telemetry data. In some examples, the method further includes determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image, determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates, and generating a georegistration transform based on the determined telemetry data and the reference image.

Inventors

• VAN ANDEL, ETHAN
• DRISCOLL, JOSEPH
• RAMSEY, STEPHEN
• CAMERON, MARY
• BETTEN, MATTHEW
• FEDDERLY, MATTHEW
• HAI, DUYEN LUU
• WING, LUKE
• LYMPEROPOULOS, DIMITRIOS

Assignees

• Palantir Technologies Inc.

Dates

Publication Date

20260513

Application Date

20240627

Claims (20)

1. 1. A method for georegistration, the method comprising: receiving a video stream including a plurality of video frames collected by an image sensor; presenting the video stream via a video player; receiving user input associated with a first video frame of the plurality 7 of video frames and a reference image, the first video frame including incomplete telemetry data; determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image; determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates; and generating a georegistration transform based on the determined telemetry data and the reference image; wherein the method is performed using one or more processors.
2. 2. The method of claim 1, further comprising: applying the georegistration transform to one or more video frames of the plurality of video frames to generate one or more registered video frames: and presenting the one or more registered video frames via the video player.
3. 3. The method of claim 1, wherein the receiving user input associated with the first video frame and the reference image comprises receiving a first frame location on the first video frame and a first image location on the reference image.
4. 4. The method of claim 3, wherein the first frame location is received based on a click on the first video frame.
5. 5. The method of claim 4. wherein the first image location is received based on a click on the reference image.
6. 6. The method of claim 4, wherein the first image location is determined by a machine learning model based on the first frame location and the reference image.
7. 7. The method of claim 1, wherein the receiving user input associated with the first video frame and the reference image comprises receiving a plurality 7 of pairs of user inputs to the first video frame and the reference image, wherein each pair of user inputs includes a user input to the first video frame and a user input to the reference image.
8. 8. The method of claim 7, wherein the plurality 7 of pairs of user inputs comprise three or more pairs of user inputs.
9. 9. The method of claim 1, wherein the determining incomplete telemetry data associated with the first video frame comprises determining the incomplete telemetry 7 data including at least one selected from a group consisting of a geospatial coordinate associated with the first video frame, a pointing angle of the image sensor, and a focal length of the image sensor.
10. 10. The method of claim 1, wherein the receiving user input associated with the first video frame and the reference image comprises receiving user input via a prompt, wherein the determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image comprises extracting a description from the prompt using a large language model.
11. 11. The method of claim 1, wherein the georegistration transform is a first georegistration transform, wherein the method further comprises: identifying an anchor frame as a second video frame associated with the first video frame; determining a second georegistration transform based at least in part on the anchor frame.
12. 12. The method of claim 11, wherein the identifying an anchor frame comprises determining the second video frame meeting one or more selection criteria based on the first video frame.
13. 13. The method of claim 12, wherein the one or more section criteria comprises at least one selected from a group consisting of a criterion on a number of frames between the first video frame and the second video frame, a criterion on a difference between a second sensor characteristic associated with the second video frame and a first sensor characteristic associated with the first video frame.
14. 14. The method of claim 1, wherein the incomplete telemetry data includes at least a piece of missing telemetry data or at least a piece of unreliable telemetry data.
15. 15. The method of claim 1, wherein the determining one or more coordinates associated with the video frame comprises: identifying a first frame location on the video frame; identifying a first image location on the reference image, wherein the first image location is associated with a set of geospatial coordinates; and determining a set of geospatial coordinates associated with the video frame based on the set of geospatial coordinates associated with the first image location.
16. 16. A system for georegistration, the system comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising: receiving a video stream including a plurality of video frames collected by an image sensor; presenting the video stream via a video player; receiving user input associated with a first video frame of the plurality of video frames and a reference image, the first video frame including incomplete telemetry data; determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image; determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates; and generating a georegistration transform based on the determined telemetry data and the reference image.
17. 17. The system of claim 16, wherein the set of operations further comprises: applying the georegistration transform to one or more video frames of the plurality of video frames to generate one or more registered video frames; and presenting the one or more registered video frames via the video player.
18. 18. The system of claim 16, wherein the receiving user input associated with the first video frame and the reference image comprises receiving a first frame location on the first video frame and a first image location on the reference image.
19. 19. The system of claim 16, wherein the determining incomplete telemetry data associated with the first video frame comprises determining the incomplete telemetry data including at least one selected from a group consisting of a geospatial coordinate associated with the first video frame, a pointing angle of the image sensor, and a focal length of the image sensor.
20. 20. A method for georegistration, the method comprising: receiving a video stream including a plurality of video frames collected by an image sensor; presenting the video stream via a video player; receiving user input associated with a first video frame of the plurality of video frames and a reference image, the first video frame including incomplete telemetry data, the incomplete telemetry data including at least a piece of missing telemetry data or at least a piece of unreliable telemetry data; determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image; determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates; generating a georegistration transform based on the determined telemetry data and the reference image; applying the georegistration transform to one or more video frames of the plurality of video frames to generate one or more registered video frames; and presenting the one or more registered video frames via the video player; wherein the method is performed using one or more processors.

Description

SYSTEMS AND METHODS FOR USER INTERACTIVE GEOREGISTRATION CROSS REFERENCE TO RELATED APPLICATIONS [0001] This application claims priority to U.S. Provisional Application No. 63/525.440, entitled “SYSTEMS AND METHODS FOR USER INTERACTIVE GEOREGISTRATION,” and filed on July 7, 2023, which is incorporated by reference herein for all purposes in its entirety. TECHNICAL FIELD [0002] Certain embodiments of the present disclosure relate to georegistration. More particularly, some embodiments of the present disclosure relate to aligning received images, such as satellite images, with a coordinate location. BACKGROUND [0003] Georegistration is a process for aligning two or more images or datasets spatially to a common coordinate system. In some examples, georegistration involves finding a transformation that maps points in one image or dataset to corresponding points in another image or dataset. [0004] Hence, it is desirable to improve techniques for georegistration. SUMMARY [0005] Certain embodiments of the present disclosure relate to georegistration. More particularly, some embodiments of the present disclosure relate to aligning received images, such as satellite images, with a coordinate location. [0006] At least some aspects of the present disclosure are directed to a method for georegistration. In some embodiments, the method includes: receiving a video stream including a plurality of video frames collected by an image sensor, presenting the video stream via a video player, and receiving user input associated with a first video frame of the plurality' of video frames and a reference image. In some embodiments, the first video frame includes incomplete telemetry data. In some embodiments, the method further includes determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image, determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates, and generating a georegistration transform based on the determined telemetry data and the reference image. In some embodiments, the method is performed using one or more processors. [0007] At least some aspects of the present disclosure are directed to a system for georegistration. In some embodiments, the system includes at least one processor and at least one memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations. In some embodiments, the set of operations includes: receiving a video stream including a plurality of video frames collected by an image sensor, presenting the video stream via a video player, and receiving user input associated w ith a first video frame of the plurality of video frames and a reference image. In some embodiments, the first video frame includes incomplete telemetry data. In some embodiments, the set of operations further includes determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image, determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates, and generating a georegistration transform based on the determined telemetry data and the reference image. [0008] At least some aspects of the present disclosure are directed to a method for georegistration. In some embodiments, the method includes: receiving a video stream including a plurality7 of video frames collected by an image sensor, presenting the video stream via a video player, and receiving user input associated with a first video frame of the plurality of video frames and a reference image. In some embodiments, the first video frame includes incomplete telemetry data. In some embodiments, the incomplete telemetry data includes at least a piece of missing telemetry' data or at least a piece of unreliable telemetry' data. In some embodiments, the method further includes determining one or more coordinates associated with the first video frame based on user input associated with the first video frame and the reference image, determining the incomplete telemetry data associated with the first video frame based on the one or more determined coordinates, generating a georegistration transform based on the determined telemetry data and the reference image, applying the georegistration transform to one or more video frames of the plurality of video frames to generate one or more registered video frames, and presenting the one or more registered video frames via the video player. In some embodiments, the method is performed using one or more processors. [0009] Depending upon embodiment, one or more benefits may be achieved. These benefits and various additional objects, features and advantages of the present disclosure can be fully appreciated with reference to the detailed description and accompanying dr