Search

US-20260127816-A1 - INFORMATION PROCESSING METHOD, INFORMATION PROCESSING DEVICE, COMPUTER PROGRAM, AND INFORMATION PROCESSING SYSTEM

US20260127816A1US 20260127816 A1US20260127816 A1US 20260127816A1US-20260127816-A1

Abstract

To enable efficient and accurate execution of processing for an object (processing target) existing around a moving body. An information processing system according to an embodiment acquires an image captured by an imaging device provided in a drone and/or a position of the imaging device. Then, the information processing system acquires processing candidate information indicating that an object on which processing based on a corresponding three-dimensional model is executable exists around the drone and/or the three-dimensional model of the object existing around the drone on the basis of the image captured by the imaging device and/or the position of the imaging device.

Inventors

  • Shinichiro Abe
  • Takuto MOTOYAMA
  • Masahiko Toyoshi

Assignees

  • Sony Group Corporation

Dates

Publication Date
20260507
Application Date
20240207
Priority Date
20230331

Claims (20)

  1. 1 - 8 . (canceled)
  2. 9 . An image processing apparatus comprising: circuitry configured to receive an image captured by an autonomous moving body, the image including an object, receive a pose of the autonomous moving body, acquire candidate information indicating at least one candidate for the object or an area around the autonomous moving body, the candidate information being acquired according to at least one of the acquired image or the acquired pose of the autonomous moving body, and acquire at least one of a three-dimensional (3D) model of the object or a 3D model of the area around the autonomous moving body to be used for autonomous movement based on a selected candidate from among the acquired candidate information.
  3. 10 . The image processing apparatus according to claim 9 , wherein the circuitry is further configured to transmit the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body to the autonomous moving body.
  4. 11 . The image processing apparatus according to claim 10 , wherein the circuitry receives the image captured by the autonomous moving body and the pose information of the autonomous moving body from the autonomous moving body, wherein the circuitry is further configured to transmit the image and the pose information to a server, transmit information of the selected candidate to the server, and receive the at least one of the 3D model of the object or the 3D model of the area from the server, and wherein the circuitry transmits the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body to the autonomous moving body.
  5. 12 . The image processing apparatus according to claim 11 , wherein the circuitry is further configured to receive the candidate information from the server, display the at least one acquired candidate in a display, and acquire a user selection with respect to the at least one candidate displayed in the display.
  6. 13 . The image processing apparatus according to claim 11 , wherein the circuitry is further configured to receive a relative positional relationship between the autonomous moving body and the object from the server, the relative positional relationship being estimated based on the image and the pose, and transmit the relative positional relationship to the autonomous moving body.
  7. 14 . The image processing apparatus according to claim 9 , wherein the circuitry is further configured to transmit the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body to a client device.
  8. 15 . The image processing apparatus according to claim 14 , wherein the circuitry receives the image captured by the autonomous moving body and the pose information of the autonomous moving body from the client device, and wherein the circuitry is further configured to transmit the acquired candidate information to the client device, and receive information of the selected candidate from the client device.
  9. 16 . The image processing apparatus according to claim 15 , wherein the circuitry is further configured to estimate a relative positional relationship between the autonomous moving body and the object based on the image and the pose, and transmit the relative positional relationship to the client device.
  10. 17 . The image processing apparatus according to claim 15 , wherein the selected candidate is based on candidate information displayed on a display of the client device.
  11. 18 . The image processing apparatus according to claim 15 , wherein the circuitry is further configured to determine whether the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body is stored in a storage, and wherein the circuitry acquires the candidate information indicating the at least one candidate for the object or the area from the storage when the at least one of the 3D model of the object or the 3D model of the area is stored in the storage.
  12. 19 . The image processing apparatus according to claim 18 , wherein the circuitry is further configured to perform an object recognition on the received image, and determine whether the 3D model of the object corresponding to the recognized object is stored in the storage, and wherein the circuitry acquires the candidate information indicating the at least one candidate for the object from the storage when the 3D model of the object is stored in the storage.
  13. 20 . The image processing apparatus according to claim 18 , wherein the circuitry is further configured to perform a positioning process based on the received image and the received pose, the positioning process being for specifying an area around the autonomous moving body, and determine whether the 3D model corresponding to the area specified in the positioning process is stored in the storage, and wherein the circuitry acquires the candidate information indicating the at least one candidate for the area from the storage when the 3D model of the area is stored in the storage.
  14. 21 . The image processing apparatus according to claim 18 , wherein the circuitry is further configured to notify the client device that candidate information does not exist when the at least one of the 3D model of the object or the 3D model of the area is not stored in the storage.
  15. 22 . The image processing apparatus according to claim 9 , wherein the candidate information includes a position of the object corresponding to the 3D model in the image or a name of the 3D model of the object or the 3D model of the area.
  16. 23 . The image processing apparatus according to claim 9 , the candidate information includes a name or an address of the area.
  17. 24 . The image processing apparatus according to claim 9 , wherein the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body includes information to cause the autonomous moving body to generate a navigation route for the autonomous movement.
  18. 25 . An image processing method comprising: receiving an image captured by an autonomous moving body and a pose of the autonomous moving body, the image including an object; acquiring candidate information indicating at least one candidate for the object or an area around the autonomous moving body, the candidate information being acquired according to at least one of the acquired image or the acquired pose of the autonomous moving body; and acquiring at least one of a three-dimensional (3D) model of the object or a 3D model of the area around the autonomous moving body to be used for autonomous movement based on a selected candidate from among the acquired candidate information.
  19. 26 . The image processing method according to claim 25 , further comprising: transmitting the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body to the autonomous moving body.
  20. 27 . The image processing method according to claim 25 , further comprising: transmitting the at least one of the 3D model of the object or the 3D model of the area around the autonomous moving body to a client device.

Description

TECHNICAL FIELD The present disclosure relates to an information processing device, an information processing method, a computer program, and an information processing system. BACKGROUND ART Autonomous moving bodies such as drones are being used in various fields. Drones are already used in practice for applications such as aerial photography, survey, inspection, photogrammetry (three-dimensional model generation), disaster relief, and transportation logistics. For example, survey or inspection can be performed by using autonomous movement of the drone. In this case, for example, after a subject as a survey target is set, a movement route of the drone for imaging the subject is generated. Thereafter, the drone autonomously moves according to the movement route and images the subject. In a case of using such autonomous movement, a desired work can be performed more efficiently than in a case where a person controls the drone. On the other hand, currently, the above-described subject setting is manually performed. As a specific work, for example, after the subject on the image captured by the drone is surrounded by a bounding box, imaging processing is executed on the subject in the bounding box. Then, a rough three-dimensional model of the subject is generated on the basis of the captured image. In this way, the subject setting is completed. Then, the movement route is generated on the basis of the generated rough three-dimensional model. However, in the above-described subject setting method, the imaging processing for generating a rough three-dimensional model takes time. In addition, in the imaging processing, the drone usually also captures an image of a range other than the subject. Therefore, it takes time and effort to remove an unnecessary image, and it also takes time to generate a rough three-dimensional model. Therefore, in the current subject setting method, for example, there is room for improvement regarding improvement efficiency of work performed before autonomous movement. CITATION LIST Patent Document Patent Document 1: JP 2022-107269 A SUMMARY OF THE INVENTION Problems to Be Solved by the Invention The present disclosure has been made in view of the above circumstances, and it is desirable to provide an information processing method, an information processing device, a computer program, and an information processing system that enable efficient and accurate execution of processing for an object (processing target) existing around a moving body. Solutions to Problems An information processing method according to an embodiment of the present disclosure includes: an information acquisition step of acquiring an image captured by an imaging device provided in a moving body and/or a position of the imaging device; and a model information acquisition step of acquiring processing candidate information indicating that an object for which processing based on a corresponding three-dimensional model is executable exists around the moving body and/or the three-dimensional model of the object existing around the moving body on the basis of the image captured by the imaging device and/or the position of the imaging device. An information processing device according to an embodiment of the present disclosure includes: an information acquisition unit that acquires an image captured by an imaging device provided in a moving body and/or a position of the imaging device; and a model information acquisition unit that acquires processing candidate information indicating that an object for which processing based on a corresponding three-dimensional model is executable exists around the moving body and/or the three-dimensional model of the object existing around the moving body on the basis of the image captured by the imaging device and/or the position of the imaging device. A computer program according to an embodiment of the present disclosure causes a computer to execute: an information acquisition step of acquiring an image captured by an imaging device provided in a moving body and/or a position of the imaging device; and a model information acquisition step of acquiring processing candidate information indicating that an object for which processing based on a corresponding three-dimensional model is executable exists around the moving body and/or the three-dimensional model of the object existing around the moving body on the basis of the image captured by the imaging device and/or the position of the imaging device. An information processing system according to an embodiment of the present disclosure includes: a moving body; a first information processing device that communicates with the moving body; and a second information processing device that communicates with the first information processing device, in which at least one of the first information processing device or the second information processing device includes: an information acquisition unit that acquires an image captured by an imaging devic