Search

US-12621570-B2 - Optical image stabilization in a scanning folded camera

US12621570B2US 12621570 B2US12621570 B2US 12621570B2US-12621570-B2

Abstract

A Tele folded camera operative to compensate for an undesired rotational motion of a handheld electronic device that includes such a camera, wherein the compensation depends on the undesired rotational motion and on a point of view of the Tele folded camera.

Inventors

  • Ephraim Goldenberg
  • Noy Cohen
  • Yiftah Kowal

Assignees

  • COREPHOTONICS LTD.

Dates

Publication Date
20260505
Application Date
20241126

Claims (20)

  1. 1 . A device for optical image stabilization, comprising: a scanning Tele folded camera comprising a first optical path folding element (OPFE) for folding light from a first optical path that forms an angle of less than 90 degrees to a normal of the device, toward a second optical path that is orthogonal to the normal of the device, a lens with a lens optical axis along the second optical path, and an image sensor, wherein the device is a handheld electronic device; a first OPFE actuator, configurable to tilt the first OPFE in one or more directions to direct a point of view (POV) of the scanning Tele folded camera towards a segment of a scene such that coordinate systems of the Tele folded camera and the device are not aligned; a motion sensor for sensing an undesired rotational motion of the device; a second actuator configurable to move at least one component of the scanning Tele folded camera to compensate for the undesired rotational motion of the device, wherein the compensation depends on the undesired rotational motion of the device and on the scanning Tele folded camera POV, wherein a misalignment between the coordinate system of the scanning Tele folded camera and the coordinate system of the device corresponds to the POV; and a Wide camera having a field of view FOVw larger than a field of view FOV T of the Tele camera, wherein the Wide camera provides image data that is used to track an object in the FOVw and wherein information obtained from the tracking is used to direct the POV of the scanning Tele folded camera towards the tracked object for object tracking with the scanning Tele folded camera.
  2. 2 . The device of claim 1 , wherein the movement of the at least one component of the scanning Tele folded camera to compensate for the undesired rotational motion of the device is disabled during the object tracking with the scanning Tele folded camera.
  3. 3 . The device of claim 1 , wherein the Wide camera includes a Wide camera component that is moved to compensate for the undesired rotational motion of the device, and wherein the movement of the Wide camera component is disabled during the object tracking with the scanning Tele folded camera.
  4. 4 . The device of claim 1 , wherein the movement of the at least one component of the scanning Tele folded camera to compensate for the undesired rotational motion of the device is performed at a frequency range different from a frequency range that is used for the object tracking with the scanning Tele folded camera.
  5. 5 . The device of claim 4 , wherein the frequency range for the object tracking is smaller than 30 Hz and wherein the frequency range to compensate for the undesired rotational motion of the device is larger than 30 Hz.
  6. 6 . The device of claim 1 , wherein the scanning Tele folded camera is a double-folded Tele camera comprising a second OPFE in addition to the first OPFE.
  7. 7 . The device of claim 1 , wherein the undesired rotation motion is around the device normal.
  8. 8 . The device of claim 1 , wherein the sensing of the undesired rotational motion includes sensing of the undesired rotational motion in three directions.
  9. 9 . The device of claim 1 , wherein the compensating of the undesired rotational motion includes compensating the undesired rotational motion in three directions.
  10. 10 . The device of claim 1 , wherein the first OPFE actuator is used for moving the first OPFE to compensate for the undesired rotational motion of the device.
  11. 11 . The device of claim 1 , wherein the second actuator includes a lens actuator that moves the lens.
  12. 12 . The device of claim 1 , wherein the second actuator includes a sensor actuator that moves the sensor.
  13. 13 . The device of claim 1 , wherein the handheld electronic device is a smartphone.
  14. 14 . A device for optical image stabilization, comprising: a scanning Tele folded camera comprising an optical path folding element (OPFE) for folding light from a first optical path that forms an angle of less than 90 degrees to a normal of the device toward a second optical path that is orthogonal to the normal of the device, a lens with a lens optical axis along the second optical path, and an image sensor, wherein the device is a handheld electronic device; an OPFE actuator for tilting the OPFE in one or more directions to direct a point of view (POV) of the scanning Tele folded camera towards a segment of a scene, wherein the tilting of the OPFE introduces a rotational POV aberration corresponding to a misalignment between a coordinate system of the scanning Tele folded camera and a coordinate system of the device; and a sensor actuator, wherein the sensor actuator rotates the image sensor around a normal of the image sensor to compensate for the rotational POV aberration.
  15. 15 . The device of claim 14 , wherein the device further comprises a motion sensor for sensing an undesired rotational motion of the device and at least one actuator for moving at least one component of the scanning Tele folded camera to compensate for the undesired rotational motion of the device, and wherein the compensation depends on the undesired rotational motion of the device and on the scanning Tele folded camera POV.
  16. 16 . The device of claim 14 , wherein the handheld electronic device is a smartphone.
  17. 17 . A method for optical image stabilization, comprising: providing a device comprising a scanning Tele folded camera that includes an optical path folding element (OPFE) for folding light from a first optical path that forms an angle of less than 90 degrees to a normal of the device toward a second optical path that is orthogonal to a normal of the device, a lens with a lens optical axis along the second optical path, and an image sensor, wherein the device is a handheld electronic device; providing an OPFE actuator for tilting the OPFE in one or more directions to direct a point of view (POV) of the scanning Tele folded camera towards a segment of a scene, wherein the tilting of the OPFE introduces a rotational POV aberration corresponding to a misalignment between a coordinate system of the scanning Tele folded camera and a coordinate system of the device; and providing a sensor actuator for rotating the image sensor around a normal of the image sensor to compensate for the rotational POV aberration.
  18. 18 . The method of claim 17 , wherein the method further includes providing a motion sensor for sensing an undesired rotational motion of the device and providing at least one actuator for moving at least one component of the scanning Tele folded camera to compensate for the undesired rotational motion of the device, and wherein the compensation depends on the undesired rotational motion of the device and on the scanning Tele folded camera POV.
  19. 19 . The method of claim 17 , wherein the method is performed by a handheld electronic device.
  20. 20 . The method of claim 19 , wherein the handheld electronic device is a smartphone.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/607,757 filed Mar. 18, 2024 (now allowed) which was a continuation of U.S. patent application Ser. No. 17/614,382 filed Nov. 26, 2021 (now U.S. Pat. No. 11,968,453) which was a 371 of international application PCT/IB2021/056617 filed Jul. 22, 2021, and is related to and claims the benefit of priority from U.S. provisional patent application No. 63/064,565 filed Aug. 12, 2020, which is incorporated herein by reference in its entirety. FIELD Examples disclosed herein relate in general to digital cameras and in particular to correction of images obtained with folded digital cameras. BACKGROUND Compact digital cameras having folded optics, also referred to as “folded cameras” are known, see e.g. co-owned international patent application PCT/IB2016/057366. In handheld electronic devices (also referred to herein as “handheld devices”) such as smartphones, tablets etc. a folded Tele camera is often part of a multi-camera and accompanied by one or more additional cameras, e.g. an Ultra-wide camera and a Wide camera. An Ultra-wide camera has a larger field of view (FOVUW) than a Wide camera, which has a larger FOVW than a Tele camera having FOVT (assuming similar image sensor sizes). FIG. 1A shows schematically a folded Tele camera numbered 100 from a perspective view. Camera 100 includes a lens 102 with a lens optical axis 110, an optical path folding element (OPFE) 104 and an image sensor 106. OPFE 104 folds a first optical path along an axis 108 substantially parallel to the X axis from an object, scene or panoramic view section 114 into a second optical path along an axis 110 substantially parallel to the Z axis. Camera 100 is designed to rotate OPFE 104 around axis 110 (the X axis) relative to the image sensor, i.e. in the Y-Z plane, a rotation indicated by an arrow 112. That is, folded Tele camera 100 is a “scanning” Tele camera (“STC”). FIG. 1B shows OPFE 104 after rotation by 30 degrees from the zero position. FIG. 1C shows a handheld device 120 including a STC 100 having lens 102, OPFE 104 and image sensor 106 in a top view. A device normal (“N”) is orthogonal to a screen 116 of device 120 and points towards the observer. The camera's optical axis is parallel to the X axis. In other examples, STC 100 may be included in 120 so that the camera's optical axis is parallel to the Y axis. Images are acquired from a certain point of view (POV) of a camera. The POV is the direction defined by the vector that has the location of a camera's aperture as starting point and an object point at the center of the FOV as end point (see FIG. 3A, with two POV vectors 324 and 328 corresponding to two FOVTs 326 and 332). Instead of POV vector, one may also speak of the FOV center direction vector (FOVCD). As an example, in spherical coordinates (r, θ, φ) defined according to the ISO convention, the POV for a camera at r=0 is defined by (1, θ, φ), with the polar angle θ and azimuthal angle φ defining the location of the object point at the center of the Tele FOV. The length of the POV vector may be 1 (unit vector) or may have some constant length (e.g. EFL) or may have a varying length e.g. so that it comes to lie on a specific plane. As e.g. described in the co-owned PCT Patent Application No. PCT/IB2016/057366 and with reference to FIGS. 1A-1B, rotation of the OPFE may be performed around the X axis and around the Y axis for “scanning” with the FOV in 2 dimensions (2D) in FIGS. 1A-1B. Modern cameras that are included in handheld devices often include optical image stabilization (OIS) for mitigating undesired camera motion caused by a user's hand motion (often referred to as “hand-shake”). For OIS, optical components are moved to reduce movements of imaged objects on a camera's image sensor. The lens module and/or the image sensor and/or the OPFE and/or the entire camera can be moved. An inertial measurement unit (IMU) included in the handheld device provides motion data along 6 degrees of freedom, namely and with reference to FIG. 1C, linear movements in X-Y-Z, roll “tilt about” (or “tilt around”) the Z axis, yaw (tilt around the Y axis) and pitch (tilt around the X axis). Usually, OIS is provided for Pitch and Yaw rotation compensation only, and not for roll rotation, as Pitch and Yaw rotation account for the major share of image deterioration caused by hand-shake. Coordinate systems of the IMU, of a regular (i.e. a non-scanning) camera and of the including handheld device can be aligned and do not evolve in time. For a STC, this is not valid. The relation between a handheld device's coordinate system and that of a STC does change when FOV scanning is performed. Therefore, OIS as known in the art cannot be used for hand motion compensation in a STC. There is a need for and it would be advantageous to have OIS for scanning Tele cameras. SUMMARY Henceforth and for simplicity, the terms “electronic device”, “electron