EP-3767537-B1 - INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING SYSTEM

EP3767537B1EP 3767537 B1EP3767537 B1EP 3767537B1EP-3767537-B1

Inventors

TSUKAMOTO, YUSUKE
ISHIKAWA, KAZUNOBU
TSUKIZAWA, SOTARO

Dates

Publication Date: 20260506
Application Date: 20200709

Claims (4)

An information processing method for improving an accuracy of an object detection model in autonomous driving to be executed on a computer, the method comprising: acquiring image data; acquiring a first object detection result by inputting the image data to a first trained model that performs object detection processing; executing a first determination based on the first object detection result and reference information about the image data, the first determination being a determination of an error or an omission included in the first object detection result with respect to the image data; acquiring a second model through first training using machine learning, the first training being conducted using training data that includes at least one of first image data and data similar to the first image data, the first image data being the image data associated with the first object detection result including an error or an omission,wherein the data similar to the first image data is identical or nearly identical in composition to said first image data and is captured by changing an image-capture setting or obtained through image processing performed on said first image data; acquiring a second object detection result by inputting the image data to the second model; executing a second determination based on the second object detection result and the reference information about the image data, the second determination being a determination of an error or an omission included in the second object detection result with respect to the image data; acquiring a third model through second training using machine learning, the second training being conducted using training data that includes at least one of second image data and data similar to the second image data, the second image data being the image data associated with the second object detection result including an error or an omission that is not included in the first object detection result,wherein the data similar to the second image data is identical or nearly identical in composition to said second image data and is captured by changing an image-capture setting or obtained through image processing performed on said second image data; acquiring a third object detection result by inputting the image data to the third model; executing a third determination based on the third object detection result and the reference information about the image data, the third determination being a determination of an error or an omission included in the third object detection result with respect to the image data; and when the error or the omission included in the third object detection result is recognized as being identical to the error or the omission included in the first object detection result, outputting information about a reached limit of improvement in the performance of the first trained model.
The information processing method according to claim 1, wherein the object detection result includes a first bounding box that is a bounding box of an object obtained as a result of the object detection processing, the reference information includes a second bounding box that is a bounding box of an object serving as a reference, and the error or the omission is determined based on the first bounding box and the second bounding box.
The information processing method according to any one of claims 1 to 2, wherein the outputting of the information provides a notification of a possibility that the training limit has been reached, and the notification is provided via an image or audio.
An information processing system for improving an accuracy of an object detection model in autonomous driving comprising: an object detection processor (10); an anomaly determiner (20); a model trainer (60); and a notification controller (50), wherein the object detection processor (10) acquires image data and acquires a first object detection result by inputting the image data to a first trained model that performs object detection processing, the anomaly determiner (20) executes a first determination based on the first object detection result and reference information about the image data, the first determination being a determination of an error or an omission included in the first object detection result with respect to the image data, the model trainer (60) acquires a second model through first training using machine learning, the first training being conducted using training data that includes at least one of first image data and data similar to the first image data, the first image data being the image data associated with the first object detection result including an error or an omission, wherein the data similar to the first image data is identical or nearly identical in composition to said first image data and that is captured by changing an image-capture setting or obtained through image processing performed on said first image data, the object detection processor (10) further acquires a second object detection result by inputting the image data to the second model, the anomaly determiner (20) further executes a second determination based on the second object detection result and the reference information about the image data, the second determination being a determination of an error or an omission included in the second object detection result with respect to the image data, the model trainer (60) further acquires a third model through second training using machine learning, the second training being conducted using training data that includes at least one of second image data and data similar to the second image data, the second image data being the image data associated with the second object detection result including an error or an omission that is not included in the first object detection result, wherein the data similar to the second image data is identical or nearly identical in composition to said second image data and is captured by changing an image-capture setting or obtained through image processing performed on said second image data, the object detection processor (10) further acquires a third object detection result by inputting the image data to the third model, the anomaly determiner (20) further executes a third determination based on the third object detection result and the reference information about the image data, the third determination being a determination of an error or an omission included in the third object detection result with respect to the image data, and when the error or the omission included in the third object detection result is recognized as being identical to the error or the omission included in the first object detection result, the notification controller (50) outputs information about a reached limit of improvement in the performance of the first trained model.

Description

Field The present disclosure relates to an information processing method to be executed on a computer, and an information processing system for executing the information processing method. Background Object detection techniques using deep learning (see NPL 1, for example), for which high-precision implementation examples have been reported, are expected to become commercially practical in various applications. As a countermeasure against anomalies in detection (including false positive detection and false negative detection) in the object detection techniques using machine learning such as deep learning, machine learning-based model training is conducted using training data that includes extra data to be detected with which an anomaly in detection has occurred (see PTL1). Citation List Patent Literature PTL 1: Japanese Unexamined Patent Application Publication No. 2002-342739 Non Patent Literature NPL 1: "M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network" by Qijie Zhao and other six members, November 2018, [online], arXiv, retrieved on February 26, 2020 from Internet <URL: https: //arxiv.org/abs/1811.04533>.NPL 2: "Object Detection with Discriminatively Trained Part-Based Models" by FELZENSZWALB P.F. et al., IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 32, no. 9, 1 September 2010 (2010-09-01), pages 1627-1645, ISSN: 0162-8828, DOI: 10.1109/ TPAMI.2009.167. Summary Technical Problem However, there are cases where an anomaly in detection cannot be eliminated even by training using modified training data. In such a case, it is difficult to determine whether the elimination of an anomaly in detection is possible or not, i.e., whether a training limit has been reached. The present disclosure provides an information processing method and the like that enable determining the limit of machine learning-based training. Solution to Problem An information processing method according to one aspect of the present disclosure is a method to be executed on a computer, as defined by claim 1. An information processing system according to one aspect of the present disclosure is defined by independent claim 4. Note that these comprehensive or specific aspects may be implemented as a device, an integrated circuit, or a computer-readable recording medium such as a CD-ROM, in addition to the above-described method or system, or may be implemented as any combination of a device, a system, an integrated circuit, a method, a computer program, and a recording medium. Advantageous Effects With the information processing method and the like according to the present disclosure, it is possible to determine the limit of machine learning-based training. Brief Description of Drawings [FIG. 1]FIG. 1 is a block diagram illustrating a functional configuration example of an information processing system that executes an information processing method according to an embodiment.[FIG. 2]FIG. 2 is a flowchart illustrating an example of the procedure of anomaly determination in the information processing method according to the embodiment.[FIG. 3]FIG. 3 is a flowchart illustrating an example of the procedure of the information processing method according to the embodiment. Description of Embodiment Findings Forming Basis of Present Disclosure The inventors of the present disclosure have found the following problems with conventional techniques. For example in the case of applying object detection techniques to applications such as autonomous driving that require high reliability, taking a countermeasure against anomalies in detection such as false positive detection and false negative detection is indispensable because such anomalies in detection may cause accidents that can threaten human lives. In object detection techniques using machine learning methods such as deep learning, as a countermeasure against anomalies in detection, in general machine learning-based model training is conducted using training data that includes data to be detected with which an anomaly in detection has occurred. However, even if an observed anomaly in detection has been eliminated from the model obtained as a result of this training, determination as to whether another anomaly in detection has occurred in this model, i.e., whether this training contributes to an overall improvement in the accuracy of object detection using this model or has reached its limit, depends greatly on experimental knowledge or intuition of a person in charge of the model training under present circumstances. For example, there is a possibility that, although a highly experienced person in charge might consider alternatives such as changing the network configuration in order to obtain a more highly accurate object detection model, in reality a little experienced person in charge may choose to conduct further training using modified training data and observe results obtained by a resultant model. In this way, the