Search

JP-7855322-B2 - Imaging device, its control method, and program

JP7855322B2JP 7855322 B2JP7855322 B2JP 7855322B2JP-7855322-B2

Inventors

  • 内藤 剛
  • 安田 龍一郎

Assignees

  • キヤノン株式会社

Dates

Publication Date
20260508
Application Date
20210816

Claims (10)

  1. An imaging device comprising an image sensor for capturing an image, an imaging optical system for forming an image of light from an object on the imaging surface of the image sensor, and a photometering means for measuring the light of the shooting environment, A motion vector calculation means that calculates a motion vector based on a plurality of images continuously captured by the image sensor, A reliability calculation means for calculating the reliability of the motion vector, The system includes a control means for controlling development parameters for a plurality of images continuously captured by the image sensor, according to the photometric results from the photometric means and the reliability calculated by the reliability calculation means. The imaging apparatus is characterized in that, when the reliability of the motion vector calculated based on a first plurality of images continuously captured by the image sensor is lower than a predetermined threshold, the control means performs a first development process for generating an image used when the motion vector calculation means calculates the motion vector, and a second development process for generating an image used when the images are aligned and combined, on the output signals from the image sensor corresponding to each of the second plurality of images continuously captured by the image sensor, the gain in the first development process is set higher than the gain in the second development process .
  2. The imaging apparatus according to claim 1, further comprising image synthesis means for aligning and combining multiple images captured in succession.
  3. The motion vector calculation means calculates motion vectors based on the images generated by the first development process on the output signals from the image sensor corresponding to each of the second plurality of images, The imaging apparatus according to claim 2, characterized in that the image synthesis means aligns and synthesizes images generated by the second development process on the output signals from the image sensor corresponding to each of the second plurality of images, based on the calculated motion vector.
  4. The imaging apparatus according to any one of claims 1 to 3, characterized in that the reliability calculation means calculates the reliability according to the number of motion vectors calculated by the motion vector calculation means from a plurality of images continuously captured by the image sensor during preparation for shooting.
  5. The imaging apparatus according to any one of claims 1 to 4, characterized in that the reliability calculation means calculates the reliability according to the variance of the motion vector calculated by the motion vector calculation means from a plurality of images continuously captured by the image sensor during preparation for shooting.
  6. The imaging apparatus according to any one of claims 1 to 5, further comprising a shooting condition determination means for determining the shooting conditions when capturing a second plurality of images, in accordance with the reliability of the motion vector calculated based on the first plurality of images, the shooting conditions when capturing the first plurality of images, and the photometric results of the photometric means.
  7. The imaging apparatus according to claim 6, characterized in that the shooting condition determination means changes the exposure time, which is one of the shooting conditions when shooting the second plurality of images, according to the brightness of the shooting environment determined by the photometric result of the photometric means, when the reliability of the motion vector calculated based on the first plurality of images is lower than the predetermined threshold.
  8. The system further comprises a blur correction means for correcting blur in at least one of the image sensor and the imaging optical system, The imaging apparatus according to claim 6 or 7 , characterized in that the shooting condition determination means changes the shooting conditions when capturing the second plurality of images according to the image stabilization performance of the image stabilization means.
  9. A control method for an imaging device comprising an image sensor for capturing an image, an imaging optical system for forming an image of light from an object onto the imaging surface of the image sensor, and a photometric means for measuring the light of the shooting environment, A motion vector calculation step, which calculates a motion vector based on a plurality of images continuously captured by the image sensor, A reliability calculation step for calculating the reliability of the motion vector, The system includes a control step that controls development parameters for a plurality of images continuously captured by the image sensor, according to the photometric results from the photometric means and the reliability calculated in the reliability calculation step. The control step is characterized in that, if the reliability of the motion vector calculated based on a first plurality of images continuously captured by the image sensor is lower than a predetermined threshold, a first development process is performed on the output signal from the image sensor corresponding to each of the second plurality of images continuously captured by the image sensor to generate an image used when calculating the motion vector by the motion vector calculation means, and a second development process is performed to generate an image used when aligning and combining the images, and the gain in the first development process is set higher than the gain in the second development process .
  10. A program for causing a computer to function as a motion vector calculation means, a reliability calculation means, and a control means for an imaging apparatus described in any one of claims 1 to 8 .

Description

This invention relates to an imaging device, a control method thereof, and a program, and more particularly to an imaging device, a control method thereof, and a program that perform image blur correction during shooting by driving an image sensor and an imaging optical system. In recent years, with the advancement of imaging devices, many image sensors and imaging optics have been equipped with image stabilization mechanisms. These image stabilization mechanisms make it possible for users to reduce the impact of camera shake on captured images when shooting handheld with an imaging device. Furthermore, advancements in imaging technology have led to faster image sensor readout speeds, and techniques for correcting and reducing camera shake by aligning and combining multiple consecutive images are now known. However, this technique has a drawback: shortening the exposure time of each image to reduce camera shake in each of the combined images worsens the signal-to-noise ratio (SNR) of each image, increasing noise. To address such challenges, Non-Patent Document 1 discloses a technique for determining the exposure time for each image based on the balance between the magnitude of blur (camera shake, subject blur) and noise. Orly Liba and 13 others, “Handheld Mobile Photography in Very Low Light”, [online], October 24, 2019, [searched on July 5, 2021], Internet <URL: https://arxiv.org/list/cs.CV/1910?skip=500&show=25> This is a block diagram showing the central cross-sectional view and electrical configuration of the imaging device according to Embodiment 1 of the present invention.This figure illustrates the alignment image synthesis in Example 1 of the present invention.This is a diagram to explain the reliability of motion vectors.This is a flowchart of the basic control process according to Embodiment 1 of the present invention.Figure 4 is a flowchart of the motion vector reliability determination process in step S4003.Figure 4 is a flowchart of the motion vector confidence reassessment process in step S4006.Figure 4 is a flowchart of the process for determining the shooting conditions in step S4007.Figure 4 is a flowchart of the alignment and synthesis process in step S4009.This is a modified example of the flowchart for the alignment and synthesis process in step S4009 of Figure 4.This is a block diagram showing the electrical configuration for determining the exposure time and shooting conditions of each image captured in the alignment and synthesis process in Embodiment 2 of the present invention.This is a flowchart of the basic control process according to Embodiment 2 of the present invention.Figure 11 is a flowchart of the process for determining the shooting conditions in step S1102.Figure 11 is a flowchart of the alignment and synthesis process in step S1104. Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. (Example 1) Hereinafter, with reference to Figures 1 to 9, an imaging device comprising a camera body 1 and a lens barrel 2 attached thereto according to Embodiment 1 of the present invention will be described. Figure 1(a) is a central cross-sectional view of the imaging device according to this embodiment, and Figure 1(b) is a block diagram showing the electrical configuration. Components with the same reference numerals in Figures 1(a) and 1(b) indicate the same configuration. As shown in Figure 1, the camera body 1 has a lens barrel 2 attached to it. When this attachment is made, the camera body 1 and the lens barrel 2 can communicate via electrical contacts 11. The lens barrel 2 comprises a photographic optical system 3 consisting of multiple lenses, including a blur-correcting lens 3a located on the optical axis 4; a lens system control unit 12; a lens-side blur correction means 13; and a lens-side blur detection means 16. The camera body 1 comprises a camera system control unit 5, an image sensor 6, an image processing unit 7, a memory unit 8, a display unit 9, an operation detection unit 10, a camera-side image stabilization unit 14, a camera-side image stabilization unit 15, a shutter 17, a photometering unit 18, and a viewfinder 19. The camera system control unit 5 includes a motion vector calculation means 5a for calculating motion vectors, and an image synthesis means 5b for aligning and combining multiple acquired images. The display means 9 consists of a rear display device 9a provided on the back of the camera body 1, and an EVF (electronic viewfinder) 9b provided within the viewfinder 19. The operation detection unit 10 detects signals from operating means, including a shutter release button (not shown). The lens-side image stabilization mechanism 13 drives the image stabilization lens 3a, which performs image stabilization, in a plane perpendicular to the optical axis 4. The camera-side image stabilization means 14 drives the image sensor 6 in a plane perpendicular to the optical axis 4. The camera-side