US-12619913-B2 - Data creation device, program creation device, object detection device, data creation method, and object detection method

Abstract

A data creation device that is a training data creation device for creating training data which is used for deep learning of an object detection program for detecting whether or not an object is included in an image, includes: an acquisition unit that acquires an anchor that is information on a frame specifying an area for each cell for detecting presence or absence of the object from the image; and a creation unit that associates area information of the object with image data to create the training data that includes a plurality of image data in which the area information is included. The creation unit determines a frame of the area information, based on a position of the anchor.

Inventors

• Kenta Nakao
• Kiichi Sugimoto
• Satoshi Iio

Assignees

• MITSUBISHI HEAVY INDUSTRIES, LTD.

Dates

Publication Date

20260505

Application Date

20201225

Priority Date

20200331

Claims (18)

1. 1 . A program creation device comprising: a data creation device; and a learning unit that performs deep learning on training data created by the data creation device to create a trained program for extracting an object from an image, wherein the data creation device is a training data creation device for creating the training data which is used for the deep learning of an object detection program for detecting whether or not the object is included in the image, and comprises: an acquisition unit that acquires an anchor that is information on a frame specifying an area for each cell for detecting presence or absence of the object from the image; and a creation unit that associates area information of the object with image data to create the training data that includes a plurality of image data in which the area information is included, and the creation unit determines a frame of the area information, based on a position of the anchor, calculates a degree of coincidence between an anchor and a target area in a case of being placed at each position of the image data, and moves the frame of the target area in a direction approaching an outer shape of the anchor having a highest degree of coincidence.
2. 2 . The program creation device according to claim 1 , wherein the creation unit sets the frame of the area information at a position overlapping with the anchor.
3. 3 . The program creation device according to claim 1 , wherein the creation unit performs at least one of moving the frame of the area information, changing a size of the frame with respect to the image, and changing an aspect ratio, based on a position of the anchor.
4. 4 . An object detection device comprising: a storage unit that stores the trained program created by the program creation device according to claim 1 ; and a calculation unit that executes the trained program stored in the storage unit, processes image data of an area that is included in an anchor set with respect to a cell, and executes object detection processing from the image data, wherein the calculation unit executes the trained program with respect to the image data, calculates a score in each cell, and determines that there is an object in a case where there is a cell whose score is equal to or higher than a first threshold value, and determines that there is an object at a boundary of an anchor when the score satisfies a predetermined condition, based on a score of a cell and a score of a cell related to the cell, in a case where there is no cell whose score is the first threshold value.
5. 5 . The object detection device according to claim 4 , wherein the calculation unit determines that there is an object at a boundary of the anchor in a case where both the score of the cell and the score of the cell related to the cell are lower than the first threshold value and equal to or higher than a second threshold value higher than 0.
6. 6 . The object detection device according to claim 4 , wherein the cell and the cell related to the cell are cells adjacent to each other.
7. 7 . The object detection device according to claim 6 , wherein the cells adjacent to each other include at least one cell of four cells adjacent to a reference cell in up-down and right-left directions.
8. 8 . The object detection device according to claim 4 , wherein the cell and the cell related to the cell include a cell of an area in which at least a part overlaps in image data having a different number of divisions.
9. 9 . An object detection device comprising: a storage unit that stores a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a calculation unit that executes the trained program stored in the storage unit, processes image data of an area that is included in an anchor set with respect to a cell, and executes object detection processing from the image data, wherein the calculation unit executes the trained program with respect to the image data, calculates a score in each cell, creates image data in which a position of the image data is moved by a distance shorter than the cell in which an anchor is installed, executes the trained program, repeats processing of calculating the score in each cell, and executes the object detection processing from the image data, based on score calculation results with respect to a plurality of image data whose positions have been moved.
10. 10 . The object detection device according to claim 9 , wherein the calculation unit sets the distance shorter than the cell in which the anchor is installed to be half the distance of the cell.
11. 11 . A data creation method for creating training data which is used for deep learning of an object detection program for detecting whether or not an object is included in an image, the method comprising: a step of acquiring an anchor which is information on a frame specifying an area for each cell for detecting presence or absence of the object from the image; and a step of associating area information of the object with image data to create training data that includes a plurality of image data in which the area information is included, wherein in the step of creating the training data, a frame of the area information is determined based on a position of the anchor.
12. 12 . An object detection method comprising: a step of storing a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a step of executing the stored trained program, processing image data of an area that is included in an anchor set with respect to a cell, and executing object detection processing from the image data, wherein in the step of executing object detection processing, the trained program is executed with respect to the image data, a score of each cell is calculated, and in a case where there is a cell whose score is equal to or higher than a first threshold value, it is determined that there is an object, and in a case where there is no cell whose score is the first threshold value, it is determined that there is an object at a boundary of the anchor when the score satisfies a predetermined condition, based on a score of a cell and a score of a cell related to the cell.
13. 13 . An object detection method comprising: a step of storing a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a step of executing the stored trained program, processing image data of an area that is included in an anchor set with respect to a cell, and executing object detection processing from the image data, wherein in the step of executing object detection processing, the trained program is executed with respect to the image data, a score of each cell is calculated, image data in which a position of the image data is moved by a distance shorter than the cell in which an anchor is installed is created, the trained program is executed, processing of calculating a score in each cell is repeated, and the object detection processing is executed from the image data, based on score calculation results with respect to a plurality of image data whose positions have been moved.
14. 14 . An object detection device comprising: a storage unit that stores a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a calculation unit that executes the trained program stored in the storage unit, processes image data of an area that is included in an anchor set with respect to a cell, and executes object detection processing from the image data, wherein the calculation unit executes the trained program with respect to the image data, calculates a score in each cell, and determines that there is an object in a case where there is a cell whose score is equal to or higher than a first threshold value, and determines that there is an object at a boundary of an anchor when the score satisfies a predetermined condition, based on a score of a cell and a score of a cell related to the cell, in a case where there is no cell whose score is the first threshold value.
15. 15 . The object detection device according to claim 14 , wherein the calculation unit determines that there is an object at a boundary of the anchor in a case where both the score of the cell and the score of the cell related to the cell are lower than the first threshold value and equal to or higher than a second threshold value higher than 0.
16. 16 . The object detection device according to claim 14 , wherein the cell and the cell related to the cell are cells adjacent to each other.
17. 17 . The object detection device according to claim 16 , wherein the cells adjacent to each other include at least one cell of four cells adjacent to a reference cell in up-down and right-left directions.
18. 18 . The object detection device according to claim 14 , wherein the cell and the cell related to the cell include a cell of an area in which at least a part overlaps in image data having a different number of divisions.

Description

TECHNICAL FIELD The present invention relates to a data creation device, a program creation device, an object detection device, a data creation method, and an object detection method. BACKGROUND ART As a system for detecting an object from an acquired image, there is a system for detecting an object by using a trained program in which deep learning is performed with a large number of images. In object detection using general deep learning, first, a feature quantity is extracted by performing convolution processing using a specific filter coefficient with respect to an input image. Next, in a feature quantity space with a different resolution obtained in the process of the convolution processing, a rectangular area (a bounding box) called an anchor is disposed, and a score indicating object-likeness is calculated from the feature quantity in the area for each anchor. An anchor whose score is equal to or higher than a threshold value is adjusted in size by regression processing by using the calculated score, and is output as a detection result. CITATION LIST Patent Literature [PTL 1] Japanese Unexamined Patent Application Publication No. 2017-146840[PTL 2] Japanese Unexamined Patent Application Publication No. 2018-165948 SUMMARY OF INVENTION Technical Problem In deep learning, the accuracy of detecting an object can be improved by setting a plurality of types of anchor shapes and performing detection of an object by using anchors having different shapes. However, if the number of anchors increases, the amount of processing of the arithmetic processing also increases. Further, if the choice of an anchor to be used increases, the amount of processing when deciding the conditions for the deep learning also increases. Due to the above, it is required to improve the accuracy of detecting an object while suppressing the amount of processing. At least one embodiment of the present disclosure has, in order to solve the above problem, an object to provide a data creation device, a program creation device, an object detection device, a data creation method, and an object detection method, in which it is possible to detect an object with high accuracy while suppressing the amount of processing. Solution to Problem According to the present disclosure, there is provided a data creation device that is a training data creation device for creating training data which is used for deep learning of an object detection program for detecting whether or not an object is included in an image, the device including: an acquisition unit that acquires an anchor that is information on a frame specifying an area for each cell for detecting presence or absence of the object from the image; and a creation unit that associates area information of the object with image data to create the training data that includes a plurality of image data in which the area information is included, in which the creation unit determines a frame of the area information, based on a position of the anchor. Further, according to the present disclosure, there is provided a program creation device including: the training data creation device described above; and a learning unit that performs deep learning on the training data created by the training data creation device to create a trained program for extracting an object from an image. Further, according to the present disclosure, there is provided an object detection device including: a storage unit that stores a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a calculation unit that executes the trained program stored in the storage unit, processes image data of an area that is included in an anchor set with respect to a cell, and executes object detection processing from the image data, in which the calculation unit executes the trained program with respect to the image data, calculates a score in each cell, and determines that there is an object in a case where there is a cell whose score is equal to or higher than a first threshold value, and determines that there is an object at a boundary of an anchor when the score satisfies a predetermined condition, based on a score of a cell and a score of a cell related to the cell, in a case where there is no cell whose score is the first threshold value. Further, according to the present disclosure, there is provided an object detection device including: a storage unit that stores a trained program created by associating area information of an object with image data and performing deep learning on training data that includes a plurality of image data in which the area information is included; and a calculation unit that executes the trained program stored in the storage unit, processes image data of an area that is included in an anchor set with respect to a cell, and executes object detection proc