US-20260127763-A1 - CALIBRATION OF PARAMETERS OF A SENSOR

Abstract

Systems and techniques are described herein for image processing. For example, a computing device can determine, from a first image and a second image received from or by a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; apply a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determine a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimate at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and process an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor.

Inventors

• Bjoern Johansson
• Hagen Spies

Assignees

• QUALCOMM INCORPORATED

Dates

Publication Date

20260507

Application Date

20241107

Claims (20)

1. 1 . An apparatus for image processing, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to: determine, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; apply a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determine a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimate at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and process an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor.
2. 2 . The apparatus of claim 1 , wherein the at least one processor is configured to time filter the at least one intrinsic parameter of the sensor.
3. 3 . The apparatus of claim 1 , wherein the at least one processor is configured to time filter the at least one intrinsic parameter of the sensor using at least one of an average filter, a Kalman filter, or a histogram based filter.
4. 4 . The apparatus of claim 1 , wherein the at least one processor is configured to: determine, from a third image and a fourth image received from the sensor, a second plurality of two-dimensional points, wherein the second plurality of two-dimensional points are represented in the third image and correspond to two-dimensional points represented in the fourth image; and apply the non-linear lens distortion function associated with the sensor to the second plurality of two-dimensional points to adjust the second plurality of two-dimensional points; wherein the determination of the plurality of epipolar constraints is additionally based on the adjusted second plurality of two-dimensional points.
5. 5 . The apparatus of claim 1 , wherein the sensor is an image sensor.
6. 6 . The apparatus of claim 5 , wherein the at least one intrinsic parameter includes at least one of a focal length or a principal point of the image sensor.
7. 7 . The apparatus of claim 1 , wherein the at least one processor is configured to: receive temperature data associated with a temperature of the sensor; and determine to update the at least one intrinsic parameter of the sensor based on the temperature.
8. 8 . The apparatus of claim 1 , wherein the at least one processor is configured to: determine to update the at least one intrinsic parameter of the sensor based on a period of elapsed time from a previous update exceeding a predetermined threshold.
9. 9 . The apparatus of claim 1 , wherein the at least one processor is configured to: process the plurality of epipolar constraints to reduce a goal function associated with the at least one intrinsic parameter of the sensor, wherein the at least one estimated intrinsic parameter is based on the plurality of epipolar constraints and the reduced goal function.
10. 10 . The apparatus of claim 9 , wherein the at least one processor is configured to: process the plurality of epipolar constraints to reduce the goal function associated with the at least one intrinsic parameter of the sensor, wherein the goal function is additionally associated with a relative motion of the sensor.
11. 11 . The apparatus of claim 10 , wherein the relative motion of the sensor is represented as a matrix or a vector associated with a change in orientation of the sensor and a translation vector associated with a change in location of the sensor.
12. 12 . The apparatus of claim 11 , wherein the at least one processor is configured to: estimate the relative motion of the sensor based on the reduced goal function.
13. 13 . A method for image processing, the method comprising: determining, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; applying a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determining a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimating at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and processing an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor.
14. 14 . The method of claim 13 , further comprising time filtering the at least one intrinsic parameter of the sensor.
15. 15 . The method of claim 13 , further comprising time filtering the at least one intrinsic parameter of the sensor using at least one of an average filter, a Kalman filter, or a histogram based filter.
16. 16 . The method of claim 13 , further comprising: determining, from a third image and a fourth image received from the sensor, a second plurality of two-dimensional points, wherein the second plurality of points are represented in the third image and correspond to two-dimensional points represented in the fourth image; and applying the non-linear lens distortion function associated with the sensor to the second plurality of two-dimensional points to adjust the second plurality of two-dimensional points; wherein the determination of the plurality of epipolar constraints is additionally based on the adjusted second plurality of two-dimensional points.
17. 17 . The method of claim 13 , wherein the sensor is an image sensor.
18. 18 . The method of claim 17 , wherein the intrinsic parameter includes at least one of a focal length or a principal point of the image sensor.
19. 19 . The method of claim 13 , further comprising: receiving temperature data associated with a temperature of the sensor; and determining to update the at least one intrinsic parameter of the sensor based on the temperature.
20. 20 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to: determine, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; apply a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determine a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimate at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and process an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor.

Description

FIELD The present disclosure generally relates to calibration techniques for sensors. For example, aspects of the present disclosure relate to systems and techniques for calibration of parameters of a sensor. BACKGROUND Sensors are generally designed with an understanding of the accuracy and capabilities of the sensor based on the hardware components used in the sensors. The capabilities of sensors can change over time as hardware of the sensor degrades. Many systems and devices (e.g., autonomous vehicles, such as autonomous and semi-autonomous cars, drones, mobile robots, mobile devices, extended reality (XR) devices, and other suitable systems or devices) include multiple sensors to gather information about the environment. The accuracy of sensor data is important for systems and devices in motion, such as autonomous vehicles, because inaccurate sensor data resulting from hardware degradation can have unintended consequences, such as causing a collision. Calibration of sensors is important to ensure accuracy of sensor data as the capabilities and accuracy of a sensor deviate over time. SUMMARY The following presents a simplified summary relating to one or more aspects disclosed herein. Thus, the following summary should not be considered an extensive overview relating to all contemplated aspects, nor should the following summary be considered to identify key or critical elements relating to all contemplated aspects or to delineate the scope associated with any particular aspect. Accordingly, the following summary has the sole purpose to present certain concepts relating to one or more aspects relating to the mechanisms disclosed herein in a simplified form to precede the detailed description presented below. In some aspects, an apparatus for image processing is provided. The apparatus can include at least one memory and at least one processor coupled to the at least one memory. The processor can be configured to determine, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; apply a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determine a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimate at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and process an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor. In some aspects, a method for image processing is provided. The method can include: determining, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; applying a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determining a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimating at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and processing an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor. In some aspects, a non-transitory computer-readable medium is provided having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to: determine, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the second image; apply a non-linear lens distortion function associated with the sensor to the first plurality of two-dimensional points to adjust the first plurality of two-dimensional points; determine a plurality of epipolar constraints based on the adjusted first plurality of two-dimensional points; estimate at least one intrinsic parameter of the sensor based on the plurality of epipolar constraints; and process an additional image received from the sensor using the at least one estimated intrinsic parameter of the sensor. In some aspects, an apparatus for image processing is provided. The apparatus includes: means for determining, from a first image and a second image received from a sensor, a first plurality of two-dimensional points, wherein the first plurality of two-dimensional points are represented in the first image and correspond to two-dimensional points represented in the secon