Search

US-12626342-B2 - Fisheye lens structure, housing, image capture, and image processing

US12626342B2US 12626342 B2US12626342 B2US 12626342B2US-12626342-B2

Abstract

An image capture device includes a camera that captures video frames of a scene surrounding the camera. The image capture device further includes a lens attachment that directs light rays from the lens of the camera across a wider field of view than the lens of the camera. The image capture device further includes a housing that removably couples the lens attachment to the camera. The image capture device further includes a processor communicatively coupled to the camera. The processor receives a video frame captured by the camera through the lens attachment and generates a two-dimensional projection of the video frame by mapping pixels of the video frame from the three-dimensional coordinate space to a two-dimensional coordinate space.

Inventors

  • Akshay Chandrasekhar
  • Aaron Francis Hawkey
  • Brian James Vernarsky
  • Paul Georg Kefer
  • Abate De Mey
  • Kavodel Ohiomoba

Assignees

  • Baller, Inc.

Dates

Publication Date
20260512
Application Date
20240216

Claims (20)

  1. 1 . An image capture device comprising: a camera configured to capture video frames of a scene surrounding the camera, the camera comprising a first lens with a first field of view; a lens attachment comprising a second lens with a second field of view and configured to direct light rays incident upon the second lens onto the first lens, wherein the second field of view is wider than the first field of view of the lens of the camera; a housing configured to removably couple the lens attachment to camera, wherein the housing secures the camera such that the second lens overlays the first lens such that the first field of view is included within the second field of view; and a processor communicatively coupled to the camera, wherein the processor is configured to: receive a video frame captured by the camera; and generate a modified video frame by mapping each pixel of the captured video frame from a fisheye coordinate space into a perspective projection coordinate space corresponding to the first lens of the camera using a distortion model that characterizes at least a focal length of the second lens.
  2. 2 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a first lens component, the first lens component comprising a first convex surface with a radius of curvature of 64.7009 mm±3.24 mm or 64.7009±1.29 mm and a second concave surface with a radius of curvature of 18.5026 mm±0.93 mm or 18.5026±0.37 mm.
  3. 3 . The image capture device of claim 2 , wherein the first convex surface has an effective surface diameter of 58.8995 mm±2.94 mm or 58.8995 mm±1.18 mm and the second concave surface has an effective surface diameter of 33.423 mm±1.67 mm or 33.423 mm±0.67 mm.
  4. 4 . The image capture device of claim 2 , wherein the lens attachment is configured with optical glass having a refractive index of 1.6968±0.084 or 1.6968±0.034 and an Abbe number of 55.53±2.78 or 55.53±1.11.
  5. 5 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a second lens component, the second lens component comprising a first concave surface with a radius of curvature of 18.0554 mm±0.902 mm or 18.0554 mm±0.36 mm and a second concave surface with a radius of curvature of 13.2123 mm±0.66 mm or 13.2123 mm±0.26 mm.
  6. 6 . The image capture device of claim 5 , wherein the first concave surface has an effective surface diameter of 29.6301±1.48 mm or 29.6301±0.59 mm and the second concave surface has an effective surface diameter of 24.1718±1.21 mm or 24.1718±0.48 mm.
  7. 7 . The image capture device of claim 5 , wherein the lens attachment is configured with optical glass having a refractive index of 1.534611±0.076 or 1.534611±0.0306 and an Abbe number of 56.07±2.803 or 56.07±1.12.
  8. 8 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a third lens component, the third lens component comprising a first concave surface with a radius of curvature of 191.431±5% or 191.431±2% and a second concave surface with a radius of curvature of 11.725±5% or 11.725±2%.
  9. 9 . The image capture device of claim 8 , wherein the first concave surface has an effective surface diameter of 22.9612 mm±1.148 mm or 22.9612 mm±0.459 mm and the second concave surface has an effective surface diameter of 17.6847 mm±0.88 mm or 17.6847 mm±0.35 mm.
  10. 10 . The image capture device of claim 8 , wherein the lens attachment is configured with optical glass having a refractive index of 1.754998±0.0877 or 1.754998±0.035 and an Abbe number of 52.34±2.617 or 52.34±1.0468.
  11. 11 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a fourth lens component, the fourth lens component comprising a first convex surface with a radius of curvature of 14.5015 mm±0.0877 mm or 14.5015 mm±0.29 mm and a second convex surface with a radius of curvature of 31.8867 mm±1.59 mm or 31.8867 mm±0.638 mm.
  12. 12 . The image capture device of claim 11 , wherein the first convex surface has an effective surface diameter of 17.131 mm±0.857 mm or 17.131 mm±0.343 mm and the second convex surface has an effective surface diameter of 17.7131 mm±0.886 mm or 17.7131 mm±0.354 mm.
  13. 13 . The image capture device of claim 11 , wherein the lens attachment is configured with optical glass having a refractive index of 1.761823±0.088 or 1.761823±0.035 and an Abbe number of 26.61±1.33 or 26.61±0.53.
  14. 14 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a fifth lens component, the fifth lens component comprising a first concave surface with a radius of curvature of 33.165 mm±1.658 mm or 33.165 mm±0.663 mm and a second concave surface with a radius of curvature of 11.9193 mm±0.596 mm or 11.9193 mm±0.238 mm.
  15. 15 . The image capture device of claim 14 , wherein the first concave surface has an effective surface diameter of 15.5653 mm±0.778 mm or 15.5653 mm±0.311 mm and the second concave surface has an effective surface diameter of 11.0862 mm±0.554 mm or 11.0862 mm±0.222 mm.
  16. 16 . The image capture device of claim 14 , wherein the lens attachment is configured with optical glass having a refractive index of 2.003307±0.1001 or 2.003307±0.04 and an Abbe number of 28.32±1.416 or 28.32±0.57.
  17. 17 . The image capture device of claim 1 , wherein the second lens of the lens attachment comprises a fifth lens component, the sixth lens component comprising a first convex surface with a radius of curvature of 10.8818 mm±0.544 mm or 10.8818 mm±0.218 mm and a second convex surface with a radius of curvature of 14.4267±0.721 mm or 14.4267±0.289 mm.
  18. 18 . The image capture device of claim 17 , wherein the first convex surface and second convex surface have an effective surface diameter of 11.1089 mm±0.555 mm or 11.1089 mm±0.222 mm.
  19. 19 . The image capture device of claim 17 , wherein the lens attachment is configured with optical glass having a refractive index of 1.613395±0.0806 or 1.613395±0.0322 and an Abbe number of 44.17±2.208 or 44.17±0.88.
  20. 20 . An image capture device comprising: camera configured to capture video frames of a scene surrounding the camera, the camera comprising a first lens with a first field of view; a lens attachment comprising a second lens with a second field of view and configured to direct light rays incident upon the second lens onto the first lens, wherein the second field of view is wider than the first field of view of the lens of the camera; and a processor communicatively coupled to the camera, wherein the processor is configured to: receive a video frame captured by the camera through the lens attachment, wherein the video frame is captured in a three-dimensional coordinate space; and generate a two-dimensional projection of the video frame by mapping pixels of the video frame from the three-dimensional coordinate space to a two-dimensional coordinate space using a distortion model that characterizes at least a focal length of the second lens.

Description

BACKGROUND This disclosure relates generally to methods and devices for projecting an image without distortion and, more specifically, to transforming a visually distorted image into an image projection without distortion. Mobile devices and other client devices are often integrated with a camera that includes a traditional pinhole lens. Such traditional lenses offer only a restricted field of view, which makes it difficult to capture images of expansive areas using the camera of the mobile device. Fisheye lenses offer a much wider field of view. As a result, a camera configured with a fisheye lens offers a wider field of view and may capture larger areas in a single image or frame compared to traditional, pinhole lenses. However, images captured using conventional fisheye lens are subject to various distortions that can be visually distracting to a user. For example, an image of a scene captured by a conventional fisheye lens may stretch the perceived distance between objects in a frame or distort the depth of objects in the frame compared to how the scene would have been perceived by the human eye. SUMMARY In one embodiment, the image capture device includes a camera that captures video frames of image data of a scene surrounding the camera. For example, the lens of the camera is a traditional pinhole lens with a narrow field of view. The image capture device further includes a lens attachment that directs light rays from the lens of the camera across a wider field of view than the lens of the camera. For example, the lens attachment is a fisheye lens with a wider field of view than the lens of the camera. The image capture device further includes a housing that removably couples the lens attachment to the camera. The housing secures the camera such that the lens attachment overlays the lens of the camera. The image capture device also includes a processor communicatively coupled to the camera. The processor receives a video frame captured by the camera through the lens attachment. The video frame captured through the wider view lens attachment is captured in a three-dimensional coordinate space. The processor generates a two-dimensional projection of the video frame by mapping pixels of the video frame from the three-dimensional coordinate space to a two-dimensional coordinate space. In one embodiment, the image capture device accesses an input image captured by a camera. The input image is a two-dimensional image comprising pixels distorted in a radial direction. The image capture device defines a projection of a region within the image corresponding to a set of control signals mimicking adjustments to a field of view of the camera. The projection is a two-dimensional coordinate plane of the region without distortions in the radial direction. The image capture device normalizes each point on the projection onto a three-dimensional coordinate sphere representing a full range of motion of the camera. The image capture device maps each normalized point on the coordinate sphere to a two-dimensional point on the input image. For each mapped point on the input image, the capture system extracts a pixel value from a pixel at the mapped point to project the pixel value at a corresponding position on the projection of the region. BRIEF DESCRIPTION OF DRAWINGS The figures use like reference numerals to identify like elements. A letter after a reference numeral, such as “104A,” indicates that the text refers specifically to the element having that particular reference numeral. A reference numeral in the text without a following letter, such as “104,” refers to any or all of the elements in the figures bearing that reference numeral. FIG. 1 illustrates an example system environment for image processing platform, according to one or more embodiments. FIG. 2 illustrates a cross-section of the lens of the image capture device, according to one or more embodiments. FIG. 3A-F illustrates a side view of the lens component, according to one or more embodiments. FIG. 4A is an example video frame captured using a fisheye lens attachment, according to one or more embodiments. FIG. 4B is an example perspective projection of the region of interest captured in the video frame, according to one or more embodiments. FIG. 5 illustrates an example system architecture for an image processing platform, in accordance with one or more embodiments. FIG. 6 illustrates an inverse mapping of pixels from an output fame to pixels of a source frame, in accordance with one or more embodiments. FIG. 7 illustrates an example system architecture of the warp mapping module, in accordance with one or more embodiments. FIGS. 8A-B illustrate a two-dimensional coordinate plane of a perspective projection relative to a three-dimensional coordinate sphere, in accordance with one or more embodiments. FIG. 9A illustrates a perspective projection 900 near the edge of a fisheye frame, according to one or more embodiments. FIG. 9B illustrates a representatio