US-12620048-B2 - Information processing system, information processing method, and non-transitory storage medium

Abstract

To improve accuracy of a result acquired by analyzing, by using a machine learning model, an image in which a road is captured, an information processing system 100 is an information processing system for collecting a machine learning image, and includes a generation unit 122 and a storage processing unit 123 . The generation unit 122 processes an image in which a road is captured, and generates capturing situation information indicating a situation related to capturing of the image. The storage processing unit 123 performs processing for storing an image, based on the capturing situation information.

Inventors

• Shin Tominaga
• Yusuke Imai
• Kazutoshi SAGI
• Yuzo Senda

Assignees

• NEC CORPORATION

Dates

Publication Date

20260505

Application Date

20230726

Priority Date

20220928

Claims (19)

1. 1 . An information processing system for collecting a machine learning image, the information processing system comprising: a memory configured to store instructions; and a processor configured to execute the instructions to execute: processing an image in which a road is captured, and generating capturing situation information indicating a situation related to capturing of the image; deciding whether to store the image, based on the capturing situation information and a storage condition; and storing the image in learning information storage for machine learning when it is decided that the image is to be stored, wherein, the capturing situation information includes a capturing condition and road information about the road captured in the image, a first analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the capturing condition, a second analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the road information, and processing the image and generating capturing situation information includes acquiring a capturing condition by using the first analysis model that has been trained to process a captured image of a road and acquire the capturing condition, and acquiring road information by using the second analysis model that has been trained to process a captured image of a road and acquire the road information.
2. 2 . The information processing system according to claim 1 , wherein the capturing situation information further includes at least one of a capturing timing, and apparatus information about a capturing apparatus that captures the image.
3. 3 . The information processing system according to claim 2 , wherein the capturing condition includes at least one of an irradiation state, weather, exposure information, and brightness information about the image, the apparatus information includes at least one of a capturing range, information for identifying the capturing apparatus, and a depression angle of the capturing apparatus in a capturing direction, and the road information includes at least one of a road state indicating a state of the road, an event that occurs on the road, and road unique information being predetermined information about the road.
4. 4 . The information processing system according to claim 1 , further comprising acquiring image information including the image from a capturing apparatus, wherein processing the image and generating capturing situation information further includes at least one of holding setting information set for processing the image, and acquiring external information from an external apparatus, and in generating the capturing situation information, generating the capturing situation information by using at least one of the capturing condition and the road information, the image information, and at least one of the setting information and the external information.
5. 5 . The information processing system according to claim 1 , wherein the storage condition includes a target capturing condition, which determines a capturing condition that needs to be satisfied by an image to be a storage target, and an image amount condition, which is related to a data amount of an image, and deciding whether to store the image includes deciding whether the capturing situation information generated for the image satisfies the target capturing condition, and deciding whether to store the image, based on a data amount of the machine learning image that satisfies the target capturing condition, and the image amount condition, when it is decided that the target capturing condition is satisfied.
6. 6 . An information processing method for collecting a machine learning image by one or more computers, the information processing method comprising, by the one or more computers: processing an image in which a road is captured, and generating capturing situation information about a situation related to capturing of the image; deciding whether to store the image, based on the capturing situation information and a storage condition, and performing processing for storing the image in learning information storage for machine learning when it is decided that the image is to be stored, wherein the capturing situation information includes a capturing condition and road information about the road captured in the image, a first analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the capturing condition, a second analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the road information, and processing the image and generating capturing situation information includes acquiring a capturing condition by using the first analysis model that has been trained to process a captured image of a road and acquire the capturing condition, and acquiring road information by using the second analysis model that has been trained to process a captured image of a road and acquire the road information.
7. 7 . The information processing method according to claim 6 , wherein the capturing situation information further includes at least one of a capturing timing, and apparatus information about a capturing apparatus that captures the image.
8. 8 . The information processing method according to claim 7 , wherein the capturing condition includes at least one of an irradiation state, weather, exposure information, and brightness information about the image, the apparatus information includes at least one of a capturing range, information for identifying the capturing apparatus, and a depression angle of the capturing apparatus in a capturing direction, and the road information includes at least one of a road state indicating a state of the road, an event that occurs on the road, and road unique information being predetermined information about the road.
9. 9 . The information processing method according to claim 6 , further comprising acquiring image information including the image from a capturing apparatus, wherein processing the image and generating capturing situation information further includes at least one of holding setting information set for processing the image, and acquiring external information from an external apparatus, and in generating the capturing situation information, generating the capturing situation information by using at least one of the capturing condition and the road information, the image information, and at least one of the setting information and the external information.
10. 10 . The information processing method according to claim 6 , wherein the storage condition includes a target capturing condition, which determines a capturing condition that needs to be satisfied by an image to be a storage target, and an image amount condition, which is related to a data amount of an image, and deciding whether to store the image includes deciding whether the capturing situation information generated for the image satisfies the target capturing condition, and deciding whether to store the image, based on a data amount of the machine learning image that satisfies the target capturing condition, and the image amount condition, when it is decided that the target capturing condition is satisfied.
11. 11 . A non-transitory storage medium storing a program for causing one or more computers to collect a machine learning image, the program for causing the one or more computers to execute: processing an image in which a road is captured, and generating capturing situation information about a situation related to capturing of the image; deciding whether to store the image, based on the capturing situation information and a storage condition; and storing the image in learning information storage for storing the machine learning image when it is decided that the image is to be stored, wherein the capturing situation information includes a capturing condition and road information about the road captured in the image, a first analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the capturing condition, a second analysis model is trained, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the road information, and processing the image and generating capturing situation information includes acquiring a capturing condition by using the first analysis model that has been trained to process a captured image of a road and acquire the capturing condition, and acquiring road information by using the second analysis model that has been trained to process a captured image of a road and acquire the road information.
12. 12 . The non-transitory storage medium storing the program according to claim 11 , wherein the capturing situation information further includes at least one of a capturing timing, and apparatus information about a capturing apparatus that captures the image.
13. 13 . The non-transitory storage medium storing the program according to claim 12 , wherein the capturing condition includes at least one of an irradiation state, weather, exposure information, and brightness information about the image, the apparatus information includes at least one of a capturing range, information for identifying the capturing apparatus, and a depression angle of the capturing apparatus in a capturing direction, and the road information includes at least one of a road state indicating a state of the road, an event that occurs on the road, and road unique information being predetermined information about the road.
14. 14 . The non-transitory storage medium storing the program according to claim 11 , further comprising acquiring image information including the image from a capturing apparatus, wherein processing the image and generating capturing situation information further includes at least one of holding setting information set for processing the image, and acquiring external information from an external apparatus, and in generating the capturing situation information, generating the capturing situation information by using at least one of the capturing condition and the road information, the image information, and at least one of the setting information and the external information.
15. 15 . The information processing system according to claim 1 , wherein the first analysis model and the second analysis model are formed by using neural networks independent of each other.
16. 16 . The information processing system according to claim 5 , wherein the image amount condition is defined by using at least one of a number of images, a time of a video formed of the image, and a data amount of the image.
17. 17 . The information processing system according to claim 16 , wherein the target capturing condition includes a condition related to at least one of a capturing timing, a capturing condition, apparatus information about a capturing apparatus that captures the image, and road information about a road captured in the image.
18. 18 . The information processing system according to claim 1 , wherein the processor is further configured to execute the instructions to execute: training the first analysis model, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the capturing condition, and training the second analysis model, using the image stored in the learning information storage as an input, to process a captured image of a road and acquire the road information.
19. 19 . The information processing system according to claim 18 , wherein the image is decided to be stored when the capturing situation information satisfies the storage condition, and the storing the image includes storing the image in the learning information storage when the capturing situation information satisfies the storage condition.

Description

RELATED ART The present invention relates to an information processing system, an information processing method, and a non-transitory storage medium. Various apparatuses that analyze an image in which a road is captured are proposed. For example, an information processing apparatus described in PTL 1 (International Publication No. WO2019/187291) acquires video data from a camera that continuously captures a traffic state of a predetermined road, and generates statistical information about traffic on the road from the video data. Further, for example, PTL 2 (International Publication No. WO2018/047393) discloses a video output system installed in a vehicle. The video output system includes an interface apparatus and a camera. The interface apparatus described in PTL 2 includes a condition storage unit, and a video output unit that outputs a video received from the camera. When a recording condition is input from a vehicle-mounted apparatus, the condition storage unit determines a recording condition ID associated to the recording condition from a recording condition table, and outputs the recording condition ID to the camera. The camera described in PTL 2 functions as a video generation unit that captures surroundings of a vehicle and generates a captured video, a recognition unit, a comparison unit, and a camera calculation unit that outputs the generated captured video to the interface apparatus. The recognition unit performs recognition processing on, as a recognition target, a target object such as a pedestrian, a vehicle, a two-wheeled vehicle, a sign, and a road division line, and an environment such as a daytime, a nighttime, and a backlight state. When the target object or the environment being the recognition target is recognized in a frame being a processing target, the recognition unit outputs identification information and additional information. The comparison unit decides whether a frame of a captured image in which the recognition unit recognizes a specific target object or a specific environment satisfies a recording condition. When the recognition unit outputs the identification information and the additional information, and the comparison unit decides that the recording condition is satisfied, the camera calculation unit adds the identification information and the additional information to a captured video, and outputs the added captured video to the interface apparatus. When the recognition unit does not output the identification information and the additional information, or when the comparison unit decides that the recording condition is not satisfied, the camera calculation unit does not add the identification information and the additional information, and outputs only a captured video to the interface apparatus. SUMMARY PTL 1 does not disclose a technique for improving accuracy of statistical information about traffic on a road. PTL 2 does not disclose a technique for improving accuracy of recognition of a recognition target in recognition processing. One example of an object of the present invention is, in view of the problem described above, to provide an information processing system, an information processing method, a non-transitory storage medium storing a program and the like that solve a challenge to improve accuracy of a result acquired by analyzing, by using a machine learning model, an image in which a road is captured. One aspect of the present invention provides an information processing system for collecting a machine learning image, the information processing system including: a generation means for processing an image in which a road is captured, and generating capturing situation information indicating a situation related to capturing of the image; anda storage processing means for performing processing for storing the image, based on the capturing situation information. One aspect of the present invention provides an information processing apparatus for collecting a machine learning image, the information processing apparatus including: a generation means for processing an image in which a road is captured, and generating capturing situation information about a situation related to capturing of the image; anda storage processing means for performing processing for storing the image, based on the capturing situation information. One aspect of the present invention provides an information processing method for collecting a machine learning image by one or more computers, the information processing method including, by the one or more computers:processing an image in which a road is captured, and generating capturing situation information about a situation related to capturing of the image; andperforming processing for storing the image, based on the capturing situation information. One aspect of the present invention provides a program for causing one or more computers to collect a machine learning image, the program for causing the one or more computers to execute: p