JP-7854636-B2 - Image processing device and program

Inventors

• 平光 昌寛
• 谷川 一也

Assignees

• 株式会社ミラボ

Dates

Publication Date

20260507

Application Date

20220125

Claims (5)

1. A means of receiving image data to be processed. A primary detection means for detecting a predetermined object from the received image data, A secondary detection means that performs a three-dimensional object detection process within the imaging range of the detected object. And, Image processing means that uses the information relating to the three-dimensional object detected by the secondary detection means to perform predetermined processing on the image data that is the target of processing. Includes, The secondary detection means refers to a reflectance database storing reflectance information for each detectable object to obtain the surface reflectance of the object detected by the primary detection means, and uses the obtained surface reflectance to perform a three-dimensional object detection process within the imaging range of the object .
2. An image processing apparatus according to claim 1, The primary detection means is an image processing device that performs a three-dimensional object detection process to detect a predetermined three-dimensional object from the received image data.
3. An image processing apparatus according to claim 1 or 2, The secondary detection means is an image processing device that performs a three-dimensional object detection process to detect a predetermined three-dimensional object within the imaging range of the object detected by the primary detection means.
4. An image processing apparatus according to any one of claims 1 to 3, The image processing means is an image processing device that performs a removal process as a predetermined process, which removes the image of the three-dimensional object detected by the secondary detection means from the image data subject to the processing.
5. A computer is a means of receiving image data to be processed. A primary detection means for detecting a predetermined object from the received image data, A secondary detection means that performs a three-dimensional object detection process within the imaging range of the detected object. And, Image processing means that uses the information relating to the three-dimensional object detected by the secondary detection means to perform predetermined processing on the image data that is the target of processing. To make it function as, A program that, when used as the secondary detection means, causes the computer to refer to a reflectance database storing reflectance information for each detectable object, to obtain the surface reflectance of the object detected by the primary detection means, and to use the obtained surface reflectance to perform a three-dimensional object detection process within the imaging range of the object .

Description

This invention relates to an image processing apparatus and a program. When photographing scenes containing objects with relatively high reflectivity, such as metal, the intended image may not be obtained due to reflections from these objects. For example, in scenes containing mirrored surfaces, the photographer's own image may be captured, resulting in an image that is not what the photographer intended. Furthermore, even in images captured for industrial purposes, reflections can sometimes prevent the desired outcome from being achieved. For example, when imaging dents in a vehicle, if surrounding images are reflected on the vehicle's surface, the dent may become less noticeable, or images that could be mistaken for the dent may be captured. Jie Yang, et al., Seeing Deeply and Bidirectionally: A Deep Learning Approach for Single Image Reflection Removal, Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 654-669 This is a block diagram showing an example configuration of an image processing apparatus according to an embodiment of the present invention.This is a functional block diagram showing an example of an image processing apparatus according to an embodiment of the present invention.This is an explanatory diagram illustrating an example of operation of an image processing apparatus according to an embodiment of the present invention.This is a flowchart illustrating an example of the operation of an image processing apparatus according to an embodiment of the present invention. Embodiments of the present invention will be described with reference to the drawings. The image processing apparatus 1 according to an embodiment of the present invention, as illustrated in Figure 1, includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and an input/output unit 15. Here, the control unit 11 is a program control device such as a CPU, and operates according to the program stored in the memory unit 12. In this embodiment, the control unit 11 receives image data to be processed and detects a predetermined object from the received image data. Furthermore, the control unit 11 performs a three-dimensional object detection process within the range in which the detected object is imaged, and uses the information related to the three-dimensional object detected in this detection process to perform a predetermined process on the image data that was the target of processing. The detailed operation of this control unit 11 will be described later. The storage unit 12 is a memory device, disk device, etc., and holds the program executed by the control unit 11. This storage unit 12 also functions as the work memory for the control unit 11. The operation unit 13 includes a mouse, keyboard, etc., and accepts user input, outputting the content of the accepted operation to the control unit 11. The display unit 14 is a display, etc., and displays and outputs information according to instructions input from the control unit 11. The input/output unit 15 is a USB interface, network interface, etc., and communicates with external devices, server equipment, etc. In one example of this embodiment, this input/output unit 15 is connected to the camera C and receives input of multiple image data (for example, a moving image including a series of continuously captured image data) captured by the camera C while changing the line of sight to the subject, and outputs the image data to the control unit 11 as the target of processing. Next, an example of the operation of the control unit 11 will be described. In this embodiment, the control unit 11 executes the program stored in the storage unit 12, thereby functionally realizing a configuration including a receiving unit 21, a primary detection unit 22, a secondary detection unit 23, and a processing execution unit 24, as illustrated in Figure 2. The receiving unit 21 receives multiple image data to be processed via the input/output unit 15. In this example, each of these image data is assumed to be monocular image data without depth information (distance information), captured by a single camera in a smartphone or the like. The primary detection unit 22 detects the area (image range) in which a predetermined object is imaged from each image data received by the receiving unit 21. Here, the primary detection unit 22 performs general object detection processing from the image data. The objects detected will include, for example, if the image data being processed is an image of a vehicle, the entire vehicle (hereinafter referred to as the overall object), as well as its various parts such as the body, tires, wheels, glass surfaces, and mirrors (hereinafter referred to as component objects). Similarly, if the image data being processed is a landscape image, the objects detected will include not only the overall object such as the imaged building, but also its component objects such as its walls and glass surfaces. As an example, when perfo