CN-114127799-B - Image recognition evaluation program product, image recognition evaluation method, evaluation device, and evaluation system

CN114127799BCN 114127799 BCN114127799 BCN 114127799BCN-114127799-B

Abstract

The image recognition evaluation program is executed by an evaluation device that evaluates recognition accuracy of an image recognition device that performs image division, and causes the evaluation device to perform image processing on an input image input to the image recognition device, generate a plurality of processed input images, input the generated plurality of processed input images to the image recognition device, perform image division based on the image recognition device, thereby acquiring a plurality of output images classified by category, and calculate variance values of the output images based on the acquired plurality of output images.

Inventors

Sugawara Toshi
TAGUCHI KEISUKE

Assignees

京瓷株式会社

Dates

Publication Date: 20260508
Application Date: 20200610
Priority Date: 20190719

Claims (5)

1. An image recognition evaluation program product is executed by an evaluation device that evaluates recognition accuracy of an image recognition device that performs image division, wherein, Causing the evaluation device to execute: Image processing is performed on the input image input to the image recognition device to generate a plurality of processed input images, Inputting the generated plurality of processing input images to the image recognition device, performing image segmentation by the image recognition device, thereby obtaining a plurality of output images classified by category, Calculating a variance value of the output image based on the acquired plurality of output images, The variance value of the output image is a variance value of a class that establishes a correspondence with each pixel of the output image, A threshold value for determining whether or not the estimation based on the category classification of the image recognition apparatus is in a point estimation state is set in advance, The evaluation device is further caused to perform: determining whether or not in a point estimation state based on the calculated variance value of the output image and the threshold value, and Determining whether a variance value of each category of the output image is greater than the threshold value, and determining whether the estimation is in a point estimation state for each category, The point estimation state is a state in which the image recognition apparatus performs learning with low robustness in the learning process.
2. The image recognition evaluation program product according to claim 1, wherein, The image processing includes at least one of berlin noise processing, gaussian noise processing, gamma conversion processing, white balance processing, and blurring processing.
3. An image recognition evaluation method performed by an evaluation device that evaluates recognition accuracy of an image recognition device that performs image division, wherein the image recognition evaluation method is performed as follows: Image processing is performed on the input image input to the image recognition device to generate a plurality of processed input images, Inputting the generated plurality of processing input images to the image recognition device, performing image segmentation based on the image recognition device, acquiring a plurality of output images classified by category, Calculating a variance value of the output image based on the acquired plurality of output images, The variance value of the output image is a variance value of a class that establishes a correspondence with each pixel of the output image, A threshold value for determining whether or not the estimation based on the category classification of the image recognition apparatus is in a point estimation state is set in advance, The evaluation device is further caused to perform: determining whether or not in a point estimation state based on the calculated variance value of the output image and the threshold value, and Determining whether a variance value of each category of the output image is greater than the threshold value, and determining whether the estimation is in a point estimation state for each category, The point estimation state is a state in which the image recognition apparatus performs learning with low robustness in the learning process.
4. An evaluation device for evaluating the recognition accuracy of an image recognition device for image division, wherein, The device comprises: an input/output unit for inputting an input image to the image recognition device and acquiring an output image generated by the image recognition device, and A control unit that performs image processing on the input image input to the image recognition device, generates a plurality of processed input images, inputs the generated plurality of processed input images to the image recognition device, performs image division by the image recognition device, acquires a plurality of output images classified by categories, calculates a variance value of the output image based on the acquired plurality of output images, the variance value of the output image being a variance value of a category that corresponds to each pixel of the output image, The control unit sets a threshold value for determining whether or not an estimation based on the category classification of the image recognition device is in a point estimation state, The control unit further causes the evaluation device to execute: Determining whether or not in a point estimation state based on the calculated variance value of the output image and the threshold value, and Determining whether a variance value of each category of the output image is greater than the threshold value, and determining whether the estimation is in a point estimation state for each category, The point estimation state is a state in which the image recognition apparatus performs learning with low robustness in the learning process.
5. An evaluation system, wherein, The device comprises: the evaluation device according to claim 4, and The image recognition device performs image segmentation on the plurality of processed input images input from the evaluation device, and outputs the plurality of output images classified by category to the evaluation device.

Description

Image recognition evaluation program product, image recognition evaluation method, evaluation device, and evaluation system Technical Field The present invention relates to an image recognition evaluation program, an image recognition evaluation method, an evaluation device, and an evaluation system. Background As an image recognition technique, semantic Segmentation (semantic segmentation) using Fully Convolutional Network (FCN: full convolution network) is known (for example, refer to non-patent document 1). Semantic segmentation classifies a digital image input as an input image by pixel (estimate inference)). That is, the semantic division classifies the pixels of the digital image, and as a result of the estimation, the classified pixels are labeled with the types, thereby dividing the digital image into a plurality of types of image areas, and outputting the image areas as an output image. As a technique for evaluating image recognition accuracy, a method called Bayesian SegNet is known (for example, refer to non-patent document 2). In Bayesian SegNet, the internal state of the Network is randomly vibrated by a method called DropOut, and fluctuation of the estimation result is calculated. The reliability (recognition accuracy) is determined to be low when the calculated estimation result fluctuates significantly, and the reliability (recognition accuracy) is determined to be high when the calculated estimation result does not fluctuate. Prior art literature Non-patent literature Non-patent literature 1：Hengshuang Zhao, et al. "Pyramid scene parsing network" IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). 2017 Non-patent literature 2：Alex Kendall, et al. "Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding" arXiv:1511.02680v2 [cs.CV], 10 Oct 2016 Disclosure of Invention Problems to be solved by the invention In non-patent document 2, since the internal state of the Network is randomly vibrated, it is necessary to change the Network structure. Here, as the Network to be evaluated, there is a so-called Black Box Network in which the Network structure is black-boxed. In this case, non-patent document 2 assumes a Network structure to be changed, but on the other hand, black Box Network cannot be changed. Therefore, the method of non-patent document 2 cannot be applied to Black Box Network, and it is difficult to evaluate the recognition accuracy of the Network. The invention aims to provide an image recognition evaluation program, an image recognition evaluation method, an evaluation device and an evaluation system, which can evaluate the recognition accuracy of an image recognition device even if the image recognition device is blackened. Technical means for solving the problems An image recognition evaluation program according to one aspect is an image recognition evaluation program executed by an evaluation device that evaluates recognition accuracy of an image recognition device that performs image segmentation, wherein the evaluation device is configured to perform image processing on an input image input to the image recognition device, generate a plurality of processed input images, input the generated plurality of processed input images to the image recognition device, acquire a plurality of output images classified by categories by the image recognition device, and calculate a variance value of the output images based on the acquired plurality of output images. An image recognition evaluation method according to one aspect is performed by an evaluation device that evaluates recognition accuracy of an image recognition device that performs image segmentation, and the image recognition evaluation method is performed by performing image processing on an input image input to the image recognition device, generating a plurality of processed input images, inputting the generated plurality of processed input images to the image recognition device, performing image segmentation based on the image recognition device, acquiring a plurality of output images classified by category, and calculating a variance value of the output images based on the acquired plurality of output images. An evaluation device according to one aspect is an evaluation device for evaluating recognition accuracy of an image recognition device that performs image segmentation, and the evaluation device includes an input/output unit that inputs an input image to the image recognition device and acquires an output image generated by the image recognition device, and a control unit that performs image processing on the input image input to the image recognition device, generates a plurality of processed input images, inputs the generated plurality of processed input images to the image recognition device, performs image segmentation by the image recognition device, acquires a plurality of output images classified by category, and calculates a variance value of the ou