Search

JP-7856417-B2 - Image processing apparatus and control method thereof

JP7856417B2JP 7856417 B2JP7856417 B2JP 7856417B2JP-7856417-B2

Inventors

  • 植草 友貴
  • 山下 豪
  • 大輪 寧司
  • 宇佐美 貴弘
  • 江幡 裕也
  • 形川 浩靖
  • 谷口 浩之
  • 相田 徹
  • 小貝 侑弘

Assignees

  • キヤノン株式会社

Dates

Publication Date
20260511
Application Date
20211210

Claims (8)

  1. A first tracking means that performs subject tracking using an image acquired by an imaging means, A second tracking means, which performs subject tracking using the image acquired by the imaging means, has a lower computational load than the first tracking means, The system includes a control means that switches between enabling both the first tracking means and the second tracking means, or disabling one of them, based on feature points detected from the image acquired by the imaging means. The control means is Within the region of the tracked subject, the number of feature points detected from the image is greater than the second threshold. If, outside the area of the tracked subject, the number of feature points detected from the image is less than the fourth threshold, An image processing apparatus characterized by disabling the first tracking means and enabling the second tracking means.
  2. The control means is Within the region of the tracked subject, the number of feature points detected from the image is greater than the second threshold. If, outside the area of the tracked subject, the number of feature points detected from the image is greater than the fourth threshold, The image processing apparatus according to claim 1, characterized in that both the first tracking means and the second tracking means are enabled.
  3. The control means is Within the region of the tracked subject, the number of feature points detected from the image is less than the second threshold. If, outside the area of the tracked subject, the number of feature points detected from the image is greater than the fourth threshold, The image processing apparatus according to claim 1 or 2, characterized in that the first tracking means is enabled and the second tracking means is disabled.
  4. The control means is Within the region of the tracked subject, the number of feature points detected from the image is less than the second threshold. If, outside the area of the tracked subject, the number of feature points detected from the image is less than the fourth threshold, The image processing apparatus according to any one of claims 1 to 3, characterized in that both the first tracking means and the second tracking means are enabled, while the operating rate is lower than in other cases.
  5. The image processing apparatus according to any one of claims 1 to 4, characterized in that the feature points are detected by performing horizontal and vertical differential filtering on the image acquired by the imaging means.
  6. The image processing apparatus according to any one of claims 1 to 4, characterized in that the first tracking means performs subject tracking using a multilayer neural network trained using deep learning.
  7. The image processing apparatus according to any one of claims 1 to 4, characterized in that the second tracking means performs subject tracking based on the similarity of color configuration or pattern matching.
  8. A first tracking step in which a first tracking means performs tracking of a subject using an image acquired by an imaging means, A second tracking step in which a second tracking means performs tracking of a subject using an image acquired by the imaging means, and the second tracking means has a lower computational load than the first tracking means, The control step includes switching between enabling both the first tracking means and the second tracking means, or disabling one of them, based on feature points detected from the image acquired by the imaging means. In the control step described above, Within the region of the tracked subject, the number of feature points detected from the image is greater than the second threshold. If, outside the area of the tracked subject, the number of feature points detected from the image is less than the fourth threshold, The first tracking means is disabled, and the second tracking means is enabled. An image processing method characterized by the following:

Description

This invention relates to an image processing apparatus and a control method for subject tracking. Some imaging devices, such as digital cameras, have a function to track feature regions (subject tracking function) by applying the detection of feature regions, such as faces, over time. Furthermore, devices that track subjects using pre-trained neural networks are also known (Patent Document 1). Japanese Patent Publication No. 2017-156886 Block diagram showing an example of the functional configuration of the imaging device according to the first embodiment.Operation flow diagram of the tracking control unit 113 in the imaging device according to the first embodiment.A diagram showing the live view display in the subject tracking process according to the first embodiment.Operational flowchart of the control unit 102 in the imaging device according to the second embodiment.Table showing the relationship between the shooting scene and the operating modes of the detection unit 110 and the tracking unit 105 according to the second embodiment.Table showing the operating modes of the detection unit 110 and the tracking unit 105 according to the second embodiment.Operation flow diagram of the control unit 102 in the third embodimentFlowchart of the feature point detection process performed by the feature point detection unit 201 of the third embodiment. Preferred embodiments of the present invention will be described below with reference to the attached drawings. (First embodiment) The present invention will be described in detail below with reference to the attached drawings, based on exemplary embodiments thereof. Note that the following embodiments do not limit the invention to the claims. Furthermore, while multiple features are described in the embodiments, not all of them are essential to the invention, and the multiple features may be combined arbitrarily. In addition, in the attached drawings, the same or similar configurations are given the same reference numeral, and redundant descriptions are omitted. In the following embodiments, the present invention will be described in relation to its implementation using an imaging device such as a digital camera. However, the present invention can also be implemented using any electronic device having an imaging function. Such electronic devices include computer equipment (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, and dashcams. These are examples, and the present invention can also be implemented using other electronic devices. Figure 1 is a block diagram showing an example of the functional configuration of an imaging device 100, which is an example of an image processing apparatus according to the first embodiment. The optical system 101 has multiple lenses, including a movable lens such as a focusing lens, and forms a detailed image of the shooting range on the image-forming plane of the image sensor 103. The control unit 102 has a CPU and, for example, loads programs stored in ROM 123 into RAM 122 and executes them. The control unit 102 realizes the functions of the imaging device 100 by controlling the operation of each functional block. ROM 123 is, for example, a rewritable non-volatile memory that stores programs, setting values, GUI data, etc., that the CPU of the control unit 102 can execute. RAM 122 is system memory used to load programs executed by the CPU of the control unit 102 and to save values necessary during program execution. Although omitted in Figure 1, the control unit 102 is connected to each functional block in a communication manner. The image sensor 103 may be, for example, a CMOS image sensor having a primary color Bayer array color filter. The image sensor 103 has multiple pixels arranged two-dimensionally, each having a photoelectric conversion region. The image sensor 103 converts the optical image formed by the optical system 101 into a group of electrical signals (analog image signals) using its multiple pixels. The analog image signals are converted into digital image signals (image data) by an A/D converter in the image sensor 103 and output. The A/D converter may be located outside the image sensor 103. The evaluation value generation unit 124 generates signals and evaluation values used for autofocus detection (AF) and calculates evaluation values used for automatic exposure control (AE) from image data obtained from the image sensor 103. The evaluation value generation unit 124 outputs the generated signals and evaluation values to the control unit 102. Based on the signals and evaluation values obtained from the evaluation value generation unit 124, the control unit 102 controls the focus lens position of the optical system 101 and determines shooting conditions (exposure time, aperture value, ISO sensitivity, etc.). The evaluation value generation unit 124 may also generate signals and evaluation values from display image data generated by the p