US-12626321-B2 - Image processing device, image processing method, and recording medium

US12626321B2US 12626321 B2US12626321 B2US 12626321B2US-12626321-B2

Abstract

An image processing device ( 3000 ) comprises an input unit ( 3020 ) and a presentation unit ( 3040 ). The input unit ( 3020 ) accepts an input of an operation for movement, on a captured image captured by a camera, of a first image which is superimposed on the captured image on the basis of a predetermined camera parameter indicating the position and attitude of the camera and which indicates a target object having a predetermined shape and a predetermined size set in a real space. The presentation unit ( 3040 ) presents the first image indicating the target object in a manner of view corresponding to a position on the captured image after the movement on the basis of the camera parameter.

Inventors

Hiroo Ikeda

Assignees

NEC CORPORATION

Dates

Publication Date: 20260512
Application Date: 20240725
Priority Date: 20140919

Claims (20)

1 . An image processing apparatus comprising: one or more memories storing instructions; and one or more processors configured to execute the instructions to: control a displayed screen to display a first image indicating a rectangular parallelepiped object superimposed on an image at a first position, the first image being determined based on a predetermined size, wherein the predetermined size corresponds to a size in real space; obtain an input indicating a selected position on the image for moving the rectangular parallelepiped object relative to a represented object shown on the image; and control the displayed screen to display, based on the input, a second image indicating the rectangular parallelepiped object superimposed on the image at the selected position, wherein an appearance of the rectangular parallelepiped object indicated by the first image is different from an appearance of the rectangular parallelepiped object indicated by the second image; and control the displayed screen to display a projective line of the rectangular parallelepiped object on a plane representing a reference plane, wherein a position of the projective line moves based on the input.
2 . The image processing apparatus according to claim 1 , wherein displaying the first image includes displaying the first image based on a camera parameter of a camera.
3 . The image processing apparatus according to claim 1 , wherein displaying the first image includes displaying the first image based on a position and an attitude of an imaging device.
4 . The image processing apparatus according to claim 1 , wherein the input includes sequentially selecting a plurality of coordinates on the displayed screen by a user.
5 . The image processing apparatus according to claim 1 , wherein the image comprises a static object which has a same appearance before and after the input.
6 . The image processing apparatus according to claim 1 , wherein the first image and the second image have different angles of rotation.
7 . The image processing apparatus according to claim 1 , wherein the first image represents the rectangular parallelepiped object depicted in the real space at a first depth and the second image represents the rectangular parallelepiped object depicted in the real space at a second depth that is farther away than the first depth, and wherein the first image is larger than the second image.
8 . The image processing apparatus according to claim 1 , wherein the first image is displayed at the first position is an initial position before the second image is displayed at the second position.
9 . The image processing apparatus according to claim 1 , wherein the one or more processors configured to execute the instructions to: control the displayed screen to move the rectangular parallelepiped object on the image.
10 . The image processing apparatus according to claim 1 , wherein the image is captured by a camera.
11 . An image processing method comprising: by one or more processors, controlling a displayed screen to display a first image indicating a rectangular parallelepiped object superimposed on an image at a first position, the first image being determined based on a predetermined size, wherein the predetermined size corresponds to a size in real space; obtaining an input indicating a selected position on the image for moving the rectangular parallelepiped object relative to a represented object shown on the image; controlling a displayed screen to display, based on the input, a second image indicating the rectangular parallelepiped object superimposed on the image at the selected position, wherein an appearance of the rectangular parallelepiped object indicated by the first image is different from an appearance of the rectangular parallelepiped object indicated by the second image; and controlling the displayed screen to display a projective line of the rectangular parallelepiped object on a plane representing a reference plane, wherein a position of the projective line moves based on the input.
12 . The image processing method according to claim 11 , wherein displaying the first image includes displaying the first image based on a camera parameter of a camera.
13 . The image processing method according to claim 11 , wherein the image comprises a static object which has a same appearance before and after the input.
14 . The image processing method according to claim 11 , wherein the first image represents the rectangular parallelepiped object depicted in the real space at a first depth and the second image represents the rectangular parallelepiped object depicted in the real space at a second depth that is farther away than the first depth, and wherein the first image is larger than the second image.
15 . A non-transitory program storage medium storing a computer program that causes a computer to execute: controlling a displayed screen to display a first image indicating a rectangular parallelepiped object superimposed on an image at a first position, the first image being determined based on a predetermined size, wherein the predetermined size corresponds to a size in real space; obtaining an input indicating a selected position on the image for moving the rectangular parallelepiped object relative to a represented object shown on the image; controlling the displayed screen to display, based on the input, second image indicating the rectangular parallelepiped object superimposed on the image at the selected position, wherein an appearance of the rectangular parallelepiped object indicated by the first image is different from an appearance of the rectangular parallelepiped object-indicated by the second image; and controlling the displayed screen to display a projective line of the rectangular parallelepiped object on a plane representing a reference plane, wherein a position of the projective line moves based on the input.
16 . The non-transitory program storage medium according to claim 15 , wherein displaying the first image includes displaying the first image based on a camera parameter of a camera.
17 . The non-transitory program storage medium according to claim 15 , wherein the image comprises a static object which, has a same appearance before and after the input.
18 . The non-transitory program storage medium according to claim 15 , wherein the first image represents the rectangular parallelepiped object depicted in the real space at a first depth and the second image represents the rectangular parallelepiped object depicted in the real space at a second depth that is farther away than the first depth, and wherein the first image is larger than the second image.
19 . The image processing apparatus according to claim 1 , wherein the projective line is a top view of the rectangular parallelepiped object.
20 . The image processing method according to claim 11 , wherein the projective line is a top view of the rectangular parallelepiped object.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/241,301, filed Sep. 1, 2023, which is a continuation of U.S. patent application Ser. No. 17/131,306, filed Dec. 22, 2020, which is a continuation of U.S. patent application Ser. No. 15/512,340, filed Mar. 17, 2017, now U.S. Pat. No. 10,911,645, which is a National Stage of International Application No. PCT/JP2015/071750 filed Jul. 31, 2015, which claims priority from Japanese Patent Application No. 2014-257137, filed on Dec. 19, 2014, and Japanese Patent Application No. 2014-191480, filed on Sep. 19, 2014. The entire disclosures of the above-referenced applications are incorporated herein by reference in their entirety. TECHNICAL FIELD The present invention relates to an image processing technique. BACKGROUND ART As one method for monitoring facilities and the like, there is a method that uses a video of a monitoring camera installed in the facilities and the like. An actual size and position of a person or object appearing in a video of the monitoring camera may be calculated using information (hereinafter, referred to as camera parameters) on a position and attitude (posture) of the camera and a size and position on an image of the person or object appearing in the video (image). Through such calculation, it is possible to grasp, when, for example, an important person (a criminal of a case or the like) is appearing in a video of a monitoring camera, a height and the like of the person using the video of the monitoring camera. Camera parameters used in the above-described purpose and the like are estimated, for example, by calibration. NPL 1 discloses a method in which a calibration pattern is image-captured by a camera and camera parameters (a rotation and translation of the camera) indicating a position and attitude of the camera are estimated from an association relation between three-dimensional coordinates of the calibration pattern in a real world and two-dimensional coordinates of the calibration pattern of the captured image. Further, there is a case in which already-estimated camera parameters are acquired and used. For example, camera parameters previously calculated by executing calibration for a camera having been a past target may be acquired, or camera parameters defined on the basis of information such as a position and attitude upon installation of the camera may be acquired. CITATION LIST Non Patent Literature NPL 1: Gang Xu and Saburo Tsuji, “Three-dimensional Vision”, Kyoritsu Shuppan, pp. 79-82, 1998 SUMMARY OF INVENTION Technical Problem It is difficult for camera parameters to always appropriately indicate a position and attitude or the like of a camera that is a target. For example, in a method for calculating camera parameters by calibration, due to a cause such as an input error of a corresponding point, lens distortion, and the like, camera parameters indicating a position and attitude different from an actual position and attitude of a camera may be calculated. Further, also when an already-estimated cameral parameter is acquired, it is difficult to understand whether the camera parameters are appropriate. It is possible that, for example, with an elapsed time, a position and attitude of a camera may change, and therefore camera parameters estimated in the past and a current position and attitude of the camera may differ from each other. When the camera parameters do not appropriately indicate a position and attitude or the like of a camera that is a target, a problem that an error in a calculation result occurs upon calculating, for example, a height of an important person appearing in a video of the above-described monitoring camera is produced. In view of the above-described problem, an object of the present invention has been achieved. The object of the present invention is to provide a technique enabling a use to easily confirm whether camera parameters are appropriate. Solution to Problem A first image processing device provided by the present invention includes: an input means configured to accept inputting of an operation for movement, on a captured image captured by a camera, to a first image that is superimposed on the captured image on the basis of predetermined camera parameters indicating a position and attitude of the camera and indicates a target object having a predetermined shape and a predetermined size set on a real space; and a presentation means configured to present the first image indicating the target object in a manner of view relating to a position on the captured image after the movement on the basis of the camera parameters. A second image processing device provided by the present invention includes: a display means configured to display a captured image captured by a camera; a parameter acquisition means configured to acquire a cameral parameter indicates a position and an attitude of the camera; an input means configured to accept designa