US-20260127845-A1 - INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING SYSTEM, AND OUTPUT METHOD

US20260127845A1US 20260127845 A1US20260127845 A1US 20260127845A1US-20260127845-A1

Abstract

An information processing device includes an acquisition unit that acquires an inference object image and a learned model, an inference unit that makes an inference by using the inference object image and the learned model, a generation unit that generates a heat map by using an inference result, an extraction unit that extracts a plurality of features based on the heat map, and an output unit. The generation unit generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image. The inference unit makes the inference by using the modification images and the learned model. The generation unit generates heat maps by using a plurality of inference results. The output unit outputs the modification images and the heat maps, or inference basis information.

Inventors

Takafumi Koike

Assignees

MITSUBISHI ELECTRIC CORPORATION

Dates

Publication Date: 20260507
Application Date: 20251230

Claims (14)

1 . An information processing device comprising: acquiring circuitry to acquire an inference object image and a learned model; inferring circuitry to make an inference by using the inference object image and the learned model; generating circuitry to generate a heat map that indicates a basis of an inference result by using the inference result; extracting circuitry to extract a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map; and outputting circuitry, wherein when the plurality of features has been extracted, the generating circuitry generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image, when the plurality of modification images has been generated, the inferring circuitry makes the inference by using the plurality of modification images and the learned model, when the inference has been made by using the plurality of modification images and the learned model, the generating circuitry generates a plurality of heat maps by using a plurality of inference results, and the outputting circuitry outputs the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps.
2 . The information processing device according to claim 1 , wherein the generating circuitry generates an image in a color opposite in hue to a color as the feature as the modification image by using the inference object image.
3 . The information processing device according to claim 1 , wherein when the feature is a color of a peripheral part in the region, the generating circuitry generates an image in a same color as the color of the peripheral part in the region as the modification image by using the inference object image.
4 . The information processing device according to claim 1 , wherein the generating circuitry generates an image with lightness opposite to lightness as the feature as the modification image by using the inference object image.
5 . The information processing device according to claim 1 , wherein when the feature is lightness of a peripheral part in the region, the generating circuitry generates an image with lightness the same as lightness of the peripheral part in the region as the modification image by using the inference object image.
6 . The information processing device according to claim 1 , wherein the generating circuitry generates the modification image, in which a frequency component as the feature has been removed, by using the inference object image.
7 . The information processing device according to claim 1 , further comprising identifying circuitry to identify a heat map having a greatest difference from the heat map corresponding to the inference object image out of the plurality of heat maps, wherein the generating circuitry generates an image as the inference basis information based on a difference between the inference object image and the modification image corresponding to the identified heat map.
8 . The information processing device according to claim 1 , further comprising identifying circuitry to identify a plurality of heat maps having a difference from the heat map corresponding to the inference object image as a plurality of identified heat maps out of the plurality of heat maps, wherein the generating circuitry generates a plurality of images as the inference basis information based on differences between the inference object image and a plurality of modification images corresponding to the plurality of identified heat maps.
9 . The information processing device according to claim 8 , wherein the generating circuitry generates a plurality of inference basis level images based on differences between the heat map corresponding to the inference object image and the plurality of identified heat maps, and the outputting circuitry outputs the plurality of inference basis level images.
10 . The information processing device according to claim 1 , further comprising identifying circuitry to identify a heat map having a greatest difference from the heat map corresponding to the inference object image out of the plurality of heat maps, wherein the generating circuitry generates a feature, that was modified when generating the modification image corresponding to the identified heat map, as the inference basis information being textual information.
11 . The information processing device according to claim 1 , further comprising identifying circuitry to identify a plurality of heat maps having a difference from the heat map corresponding to the inference object image as a plurality of identified heat maps out of the plurality of heat maps, wherein the generating circuitry generates a plurality of features, that were modified when generating a plurality of modification images corresponding to the plurality of identified heat maps, as the inference basis information being a plurality of pieces of textual information.
12 . An information processing system including a plurality of information processing devices, comprising: acquiring circuitry to acquire an inference object image and a learned model; inferring circuitry to make an inference by using the inference object image and the learned model; generating circuitry to generate a heat map that indicates a basis of an inference result by using the inference result; extracting circuitry to extract a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map; and outputting circuitry, wherein when the plurality of features has been extracted, the generating circuitry generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image, when the plurality of modification images has been generated, the inferring circuitry makes the inference by using the plurality of modification images and the learned model, when the inference has been made by using the plurality of modification images and the learned model, the generating circuitry generates a plurality of heat maps by using a plurality of inference results, and the outputting circuitry outputs the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps.
13 . An output method performed by an information processing device, the output method comprising: acquiring an inference object image and a learned model; making an inference by using the inference object image and the learned model; generating a heat map that indicates a basis of an inference result by using the inference result; extracting a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map; generating a plurality of modification images by making a modification in regard to each of the features by using the inference object image; making the inference by using the plurality of modification images and the learned model; generating a plurality of heat maps by using a plurality of inference results; and outputting the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps.
14 . An information processing device comprising: a processor to execute a program; and a memory to store the program which, when executed by the processor, performs processes of, acquiring an inference object image and a learned model, making an inference by using the inference object image and the learned model, generating a heat map that indicates a basis of an inference result by using the inference result, extracting a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map, generating a plurality of modification images by making a modification in regard to each of the features by using the inference object image, making the inference by using the plurality of modification images and the learned model, generating a plurality of heat maps by using a plurality of inference results, and outputting the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a continuation application of International Application No. PCT/JP2023/024901 having an international filing date of Jul. 5, 2023, which is hereby expressly incorporated by reference into the present application. BACKGROUND OF THE INVENTION 1. Field of the Invention The present disclosure relates to an information processing device, an information processing system, and an output method. 2. Description of the Related Art Learned models are used in recent years. The learned model is formed in complex structure. Accordingly, there is a problem in that an inference result of the learned model cannot be used without worry. In such a circumstance, Explainable Artificial Intelligence (XAI) is known as a technology for outputting an inference basis. A technology regarding the XAI has been proposed (see Patent Reference 1). An image output device in the Patent Reference 1 generates a heat map indicating the basis of the inference result by using the XAI. Patent Reference 1: Japanese Patent Application Publication No. 2022-96379 In the above-described technology, the heat map is generated. However, there are cases where a user cannot understand a detailed inference basis even by viewing the heat map. SUMMARY OF THE INVENTION An object of the present disclosure is to output data that enable the user to understand the inference basis. An information processing device according to an aspect of the present disclosure is provided. The information processing device includes an acquisition unit that acquires an inference object image and a learned model, an inference unit that makes an inference by using the inference object image and the learned model, a generation unit that generates a heat map that indicates a basis of an inference result by using the inference result, an extraction unit that extracts a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map, and an output unit. When the plurality of features has been extracted, the generation unit generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image. When the plurality of modification images has been generated, the inference unit makes the inference by using the plurality of modification images and the learned model. When the inference has been made by using the plurality of modification images and the learned model, the generation unit generates a plurality of heat maps by using a plurality of inference results. The output unit outputs the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps. According to the present disclosure, data that enable the user to understand the inference basis can be outputted. BRIEF DESCRIPTION OF THE DRAWINGS The present disclosure will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present disclosure, and wherein: FIG. 1 is a diagram showing hardware included in an information processing device in a first embodiment; FIG. 2 is a block diagram showing functions of the information processing device in the first embodiment; FIG. 3 is a diagram showing a concrete example of an inference object image and a heat map in the first embodiment; FIG. 4 is a diagram showing an example of a generation process in the first embodiment; FIG. 5 is a diagram showing a concrete example of output information in the first embodiment; FIG. 6 is a flowchart showing an example of a process executed by the information processing device in the first embodiment; FIG. 7 is a block diagram showing functions of an information processing device in a second embodiment; FIG. 8 is a diagram showing a concrete example of a generation process in the second embodiment; FIG. 9 is a flowchart showing an example of a process executed by the information processing device in the second embodiment; FIG. 10 is a diagram showing a concrete example of a generation process in a first modification of the second embodiment; FIG. 11 is a diagram showing a concrete example of a generation process in a second modification of the second embodiment; and FIG. 12 is a diagram showing an information processing system in a third embodiment. DETAILED DESCRIPTION OF THE INVENTION Embodiments will be described below with reference to the drawings. The following embodiments are just examples and a variety of modifications are possible within the scope of the present disclosure. First Embodiment FIG. 1 is a diagram showing hardware included in an information processing device in a first embodiment. The information processing