US-12619889-B2 - Learning-type real space information formation system

US12619889B2US 12619889 B2US12619889 B2US 12619889B2US-12619889-B2

Abstract

A learning-type real space information generation system uses sensor data that can be efficiently uploaded from an information terminal device to a server, and which is capable of generating highly accurate real space information. A plurality of information terminal devices and a server are connected via in a communicable state via a network. The control unit of each information terminal device controls a transmission unit, based on the importance of elements configuring real space information, the importance being determined by an importance determination unit of the server such that sensor data corresponding to an element configuring the real space information and having high importance is preferentially transmitted. The generation unit of the server uses a feature model to generate real space information, based on elements configuring real space information and having high importance extracted by an extraction unit from the most recent sensor data received by a reception unit.

Inventors

Ryoichi SHINKUMA

Assignees

KYOTO UNIVERSITY

Dates

Publication Date: 20260505
Application Date: 20191002
Priority Date: 20181011

Claims (9)

1 . A learning-type real space information generation system comprising: a plurality of information terminal devices; and a server computer connected to the plurality of information terminal devices via a network in a communicable state, the server computer being configured to collect sensor data acquired by each of the plurality of information terminal devices and generate real space information by learning, wherein the information terminal device includes: a data acquisition unit configured to acquire the sensor data of a real space: a data holding unit configured to hold a plurality of sensor data acquired by the data acquisition unit; a transmission unit configured to transmit the sensor data held by the data holding unit to the server computer via the network; and a control unit configured to control a transmission of the sensor data by the transmission unit, wherein the server computer includes: a reception unit configured to receive the sensor data transmitted by the information terminal device via the network; an extraction unit configured to extract elements configuring the real space information from the sensor data received by the reception unit; an importance determination unit configured to determine an importance of each element of the elements configuring the real space information, wherein the importance of each element of the elements is transmitted to the control unit of the information terminal device, wherein a first set of elements have a high importance for the importance of each element in the first set of elements based on feature amounts of the first set of elements varying over a period of time, and wherein a second set of elements have a low importance for the importance of each element in the second set of elements based on feature amounts of the second set of elements not varying over the period of time; a learning unit configured to train and generate a feature model of the real space information of a predetermined pattern based on the first set of elements and the second set of elements that configure the real space information extracted by the extraction unit from past sensor data received by the reception unit; and a generation unit configured to generate the real space information using the feature model based on the elements configuring the real space information extracted by the extraction unit from most recent sensor data received by the reception unit, wherein the control unit of the information terminal device controls the transmission unit to transmit the sensor data corresponding to the first set of elements having the high importance to configure the real space information based on the importance of each element determined by the importance determination unit of the server, for a plurality of sensor data stored in the data holding unit, wherein the generation unit of the server computer generates the real space information using the feature model based on the first set of elements configuring the real space information extracted by the extraction unit from the most recent sensor data received by the reception unit, and wherein the control unit controls the transmission unit to transmit all the sensor data corresponding to the elements configuring the real space information when a communication environment is a high-speed communication environment and transmits the sensor data corresponding to the first set of elements configuring the real space information when the communication environment is a low-speed communication environment.
2 . The learning-type real space information generation system as recited in claim 1 , wherein the control unit controls the transmission unit to transmit all the sensor data corresponding to the elements configuring the real space information when a stored energy of the information terminal device is equal to or greater than a predetermined value, and transmits the sensor data corresponding to the first set of elements configuring the real space information when the stored energy of the information terminal device is less than the predetermined value.
3 . The learning-type real space information generation system as recited in claim 1 , wherein the learning unit generates a feature model of a pattern in which the elements configuring the real space information are arranged multidimensionally.
4 . The learning-type real space information generation system as recited in claim 3 , wherein the learning unit generates a feature model of a pattern in which the elements configuring the real space information are arranged two dimensionally, the pattern being composed of any one of a combination of time information and spot information, time information and identification information, longitude information and latitude information, and spot information and word information.
5 . The learning-type real space information generation system as recited in claim 1 , wherein the importance determination unit determines the importance of each element based on a change in a feature amount of each element configuring the real space information in the feature model of the real space information of a predetermined pattern.
6 . A learning-type real space information generation system comprising: a plurality of information terminal devices; and a server computer connected to the plurality of information terminal devices via a network in a communicable state, the server computer being configured to collect sensor data acquired by each of the plurality of information terminal devices and generate real space information by learning, wherein the information terminal device includes: a data acquisition unit configured to acquire the sensor data of a real space: a data holding unit configured to hold a plurality of sensor data acquired by the data acquisition unit; a transmission unit configured to transmit the sensor data held by the data holding unit to the server computer via the network; and a control unit configured to control a transmission of the sensor data by the transmission unit, wherein the server computer includes: a reception unit configured to receive the sensor data transmitted by the information terminal device via the network; an extraction unit configured to extract elements configuring the real space information from the sensor data received by the reception unit; an importance determination unit configured to determine an importance of each element of the elements configuring the real space information, wherein the importance of each element of the elements is transmitted to the control unit of the information terminal device, wherein a first set of elements have a high importance for the importance of each element in the first set of elements based on feature amounts of the first set of elements varying over a period of time, and wherein a second set of elements have a low importance for the importance of each element in the second set of elements based on feature amounts of the second set of elements not varying over the period of time; a learning unit configured to train and generate a feature model of the real space information of a predetermined pattern based on the first set of elements and the second set of elements that configure the real space information extracted by the extraction unit from past sensor data received by the reception unit; and a generation unit configured to generate the real space information using the feature model based on the elements configuring the real space information extracted by the extraction unit from most recent sensor data received by the reception unit, wherein the control unit of the information terminal device controls the data acquisition unit to acquire the sensor data corresponding to the first set of elements having the high importance to configure the real space information based on the importance of each element determined by the importance determination unit of the server, wherein the generation unit of the server computer generates the real space information using the feature model based on the first set of elements configuring the real space information extracted by the extraction unit from the most recent sensor data received by the reception unit, and wherein the control unit controls the data acquisition unit to acquire all the sensor data corresponding to the elements configuring the real space information when a communication environment is a high-speed communication environment and acquires the sensor data corresponding to the first set of elements configuring the real space information when the communication environment is a low-speed communication environment.
7 . The learning-type real space information generation system as recited in claim 6 , wherein the control unit controls the data acquisition unit to acquire all the sensor data corresponding to the elements configuring the real space information when a stored energy of the information terminal device is equal to or greater than a predetermined value, and acquires the sensor data corresponding to the first set of elements configuring the real space information when the stored energy of the information terminal device is less than the predetermined value.
8 . A learning-type real space information generation system comprising: a plurality of information terminal devices; and a server computer connected to the plurality of information terminal devices via a network in a communicable state, the server computer being configured to collect sensor data acquired by each of the plurality of information terminal devices and generate real space information by learning, wherein the information terminal device includes: a data acquisition unit configured to acquire the sensor data of a real space: a data holding unit configured to hold a plurality of sensor data acquired by the data acquisition unit; a transmission unit configured to transmit the sensor data held by the data holding unit to the server computer via the network; and a control unit configured to control a transmission of the sensor data by the transmission unit, wherein the server computer includes: a reception unit configured to receive the sensor data transmitted by the information terminal device via the network; an extraction unit configured to extract elements configuring the real space information from the sensor data received by the reception unit; an importance determination unit configured to determine an importance of each element of the elements configuring the real space information, wherein the importance of each element of the elements is transmitted to the control unit of the information terminal device, wherein a first set of elements have a high importance for the importance of each element in the first set of elements based on feature amounts of the first set of elements varying over a period of time, and wherein a second set of elements have a low importance for the importance of each element in the second set of elements based on feature amounts of the second set of elements not varying over the period of time; a learning unit configured to train and generate a feature model of the real space information of a predetermined pattern based on the first set of elements and the second set of elements that configure the real space information extracted by the extraction unit from past sensor data received by the reception unit; and a generation unit configured to generate the real space information using the feature model based on the elements configuring the real space information extracted by the extraction unit from most recent sensor data received by the reception unit, wherein the control unit of the information terminal device controls a movement of the information terminal device to acquire the sensor data corresponding to the first set of elements having the high importance to configure the rear space information by the data acquisition unit based on the importance of each element determined by the importance determination unit of the server, and wherein the generation unit of the server computer generates the real space information using the feature model based on the first set of elements configuring the real space information extracted by the extraction unit from the most recent sensor data received by the reception unit and wherein the control unit controls the movement of the information terminal device so that the data acquisition unit acquires all the sensor data corresponding to the elements configuring the real space information when a communication environment is a high-speed communication environment and the data acquisition unit acquires the sensor data corresponding the first set of elements configuring the real space information when the communication environment is a low-speed communication environment.
9 . The learning-type real space information generation system as recited in claim 8 , wherein the control unit controls the movement of the information terminal device so that the data acquisition unit acquires all the sensor data corresponding to the elements configuring the real space information when a stored energy of the information terminal device is equal to or greater than a predetermined value and the data acquisition unit transmits the sensor data corresponding to the first set of elements configuring the real space information when the stored energy of the information terminal device is less than the predetermined value.

Description

TECHNICAL FIELD The present invention relates to a learning-type real space information generation system in which a plurality of information terminal devices and a server computer are connected in a communicable state via a network, sensor data acquired by the information terminal devices are collected, and real space information is generated by learning by the server computer. BACKGROUND ART In recent years, there has been an increasing need for a service in which a plurality of moving or movable mobile devices equipped with various sensors collects sensor data acquired by the various sensors via a communication network, real space information is generated by integrally processing the collected sensor data with a server computer, such as, e.g., a cloud server computer, and the generated real space information is provided as knowledge. The collection of sensor data acquired by a plurality of mobile devices equipped with sensors via a communication network as described above is called “Mobile Crowd Sensing”, which is expected as a solution for various problems in society and industries (for example, see Non-Patent Documents 1 and 2). Recently, as a sensor for acquiring large-capacity sensor data, a camera, a three-dimensional image sensor, and a three-dimensional radar are often used, and multimedia data configured by mixing an image, such as, e.g., a moving image and a static image, a three-dimensional image, and a sound has become included in the sensor data. For example, it is conceivable to collect images, such as, e.g., static images and moving images, captured as sensor data by mobile devices equipped with cameras, and real space information (for example, the number of automobiles at each time in a plurality of spots) is generated by extracting objects (for example, license plates of automobiles) from the images. PRIOR ART DOCUMENT Non-Patent Document Non-Patent Document 1: X. Wang, W. Wu, and D. Qi, “Mobility-Aware Participant Recruitment for Vehicle-based Mobile Crowdsensing”, IEEE Transactions on Vehicular Technology, 2017Non-Patent Document 2: Z. He, J. Cao, and X. Liu, “High quality participant recruitment in vehicle-based crowdsourcing using predictable mobility”, IEEE INFOCOM, pp. 2542-2550, 2015. SUMMARY OF THE INVENTION Problems to be Solved by the Invention When a mobile device uploads sensor data, such as, e.g., an image, to a server computer, an uplink communication for a mobile communication is used, but its communication speed is slow, and the communication speed is even slower when the mobile device is moving. For example, in a case where 100 units of mobile devices each send a 10 MB image every second, the transmission rate results in 8 Gbps. In the future, an omni-directional camera, a high-resolution camera, and a three-dimensional image sensor will be used, which will further increase the transmission rate. For this problem, the sensor data transmitted from mobile devices to a server in real time must be limited, and it is difficult to accurately generate real space information from such limited sensor data. The present invention has been made in view of the above-described problems, and aims to provide a learning-type real space information generation system capable of efficiently uploading sensor data from information terminal devices, such as, e.g., mobile devices, to a server, and thus capable of generating highly accurate real space information. Means for Solving the Problem To achieve the above-described object, the present invention is directed to a learning-type real space information generation system comprising: a plurality of information terminal devices; anda server computer connected to the plurality of information terminal devices via a network in a communicable state, the server computer being configured to collect sensor data acquired by each of the plurality of information terminal devices and generate real space information by learning,wherein the information terminal device includes:a data acquisition unit configured to acquire the sensor data of a real space:a data holding unit configured to hold a plurality of sensor data acquired by the data acquisition unit;a transmission unit configured to transmit the sensor data held by the data holding unit to the server computer via the network; anda control unit configured to control a transmission of the sensor data by the transmission unit,wherein the server computer includes:a reception unit configured to receive the sensor data transmitted by the information terminal device via the network;an extraction unit configured to extract elements configuring the real space information from the sensor data received by the reception unit;a learning unit configured to generate a feature model of the real space information of a predetermined pattern by learning based on the elements configuring the real space information extracted by the extraction unit from past sensor data received by the reception unit;an importance determina