Search

EP-4737065-A1 - CONTROL SYSTEM

EP4737065A1EP 4737065 A1EP4737065 A1EP 4737065A1EP-4737065-A1

Abstract

A control system includes a state recognition unit, an emotion determination unit, and an action determination unit, in which an electronic apparatus is detachably provided on a movement unit having a wheel for autonomous traveling, the electronic apparatus includes one or plural sensors that collect information for controlling rotation of the wheel, and device operations include setting first action content of leading a user by controlling the wheel.

Inventors

  • SON, MASAYOSHI

Assignees

  • SoftBank Group Corp.

Dates

Publication Date
20260506
Application Date
20240712

Claims (20)

  1. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus is detachably provided on a movement unit having a wheel for autonomous traveling, the electronic apparatus includes one or a plurality of sensors that collect information for controlling rotation of the wheel, and the device operations include setting first action content of leading the user by controlling the wheel.
  2. The control system according to claim 1, wherein the action determination unit spontaneously or periodically detects an action of the user on the basis of the sensors, and in a case in which it is determined to lead the user as the action of the electronic apparatus on the basis of the detected action of the user and specific information stored in advance, the action determination unit determines the first action content.
  3. The control system according to claim 2, wherein the first action content includes reproduction of at least one of vocal sound or an image for assisting in walking of the user.
  4. The control system according to claim 3, wherein the reproduction includes reproduction of at least one of vocal sound or an image for guiding the user along a route different from a set guidance route.
  5. The control system according to claim 4, wherein the reproduction includes reproduction of at least one of vocal sound or an image for guiding the user along a path for returning the user to the set guidance route in a case in which the user led along the guidance route deviates from the guidance route.
  6. The control system according to claim 3, wherein the action determination unit determines whether or not the action of the user has been corrected by detecting the action of the user while the movement unit was leading the user or after the movement unit reproduced at least one of the vocal sound or the image, and determines second action content different from the first action content in a case in which the action of the user has been corrected.
  7. The control system according to claim 6, wherein the second action content includes reproduction of at least one of vocal sound praising the user for the action, vocal sound expressing gratitude for the action of the user, an image praising the user for the action, or an image expressing gratitude for the action of the user.
  8. The control system according to claim 3, wherein the action determination unit determines whether or not the action of the user has been corrected by detecting the action of the user while the movement unit was leading the user or after the movement unit reproduced at least one of the vocal sound or the image, and determines third action content different from the first action content in a case in which the action of the user has not been corrected.
  9. The control system according to claim 8, wherein the third action content includes at least one of transmission of specific information to a person other than the user, reproduction of sound that attracts an interest of the user, or reproduction of an image that attracts the interest of the user.
  10. The control system according to claim 1, wherein the electronic apparatus is a communication terminal, and the action determination unit determines any of a plurality of types of actions of an agent for interacting with the user, including no action, as an action of the communication terminal.
  11. The control system according to claim 10, wherein the action determination model is a sentence generation model having an interaction function, and the action determination unit inputs text representing at least one of the user state, a state of the agent, the emotion of the user, or an emotion of the agent and text for inquiry about an action of the agent to the sentence generation model, and determines the action of the communication terminal on the basis of an output of the sentence generation model.
  12. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus detachably provided on a neck strap includes one or a plurality of sensors that collect information, and the device operations include setting first action content of leading the user by the electronic apparatus reproducing at least one of vocal sound or an image.
  13. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus detachably provided on a neck strap includes one or a plurality of sensors that collect information, and the device operations include setting first action content of leading the user by the electronic apparatus reproducing at least one of vocal sound or an image.
  14. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user, an emotion of another user existing around the user, or an emotion of the electronic apparatus; an action determination unit that, in a case in which an emotion value indicating an intensity of the emotion of the another user changes by a predetermined value or more, determines to start capturing a moving image of the another user with a camera as an action of the electronic apparatus; and an action control unit that controls the electronic apparatus to execute the action of the electronic apparatus determined by the action determination unit.
  15. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein surrounding environment information is acquired from one or a plurality of sensors that collect information, the sensors being included in the electronic apparatus detachably provided on a body of the user, and the action determination unit checks a health state of the user by the electronic apparatus outputting at least one of vocal sound or text according to the acquired environment information, and the electronic apparatus makes an appropriate proposal to the user.
  16. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and another user; an emotion determination unit that determines an emotion of the user and another user; and an action determination unit that determines any of a plurality of types of device operations as an operation of an electronic apparatus by using at least one of the user state or the emotion of the user and an action determination model at a predetermined timing, wherein the state recognition unit recognizes a surrounding state from one or a plurality of sensors of the electronic apparatus detachably provided on a body of the user, and the action determination unit determines that the electronic apparatus makes speech toward the another user in a case in which the another user exists near the user.
  17. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the emotion determination unit estimates an emotion of another person around the user from a video of the another person acquired from one or a plurality of sensors that collect information, the sensors being included in the electronic apparatus detachably provided on a body of the user, and the action determination unit makes an appropriate proposal to the user in a case in which the emotion of the another person is in trouble.
  18. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus detachably provided on a neck strap includes one or a plurality of sensors that collect information, the sensors including an imaging unit that captures an image of surroundings, and the device operations include issuing a warning to a person satisfying a predetermined condition among persons approaching the imaging unit.
  19. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus detachably provided on a neck strap includes one or a plurality of sensors that collect information, and the device operations include setting action content of supporting work performed by the user at a position lower than eyes of the user by the electronic apparatus reproducing at least one of vocal sound or an image generated by the action determination model on the basis of the information collected by the electronic apparatus.
  20. A control system comprising: a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus; an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus; and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, wherein the electronic apparatus is detachably provided on a movement unit having a wheel for autonomous traveling, the electronic apparatus includes one or a plurality of sensors that collect information regarding a surrounding situation of the user, the electronic apparatus controls rotation of the wheel on the basis of the information regarding the surrounding situation of the user, and the action determination unit determines, in a case in which it is detected that there is a dangerous obstacle around the user, first action content of leading the user to a safe place so that the user does not approach the dangerous obstacle as the action of the electronic apparatus.

Description

Technical Field The present invention relates to a control system. Background Art Patent Literature 1 discloses a technique for determining an appropriate action of a robot with respect to a state of a user. In the related art of Patent Literature 1, a reaction of a user in a case in which the robot executes a specific action is recognized, and in a case in which an action of the robot with respect to the recognized reaction of the user cannot be determined, the action of the robot is updated by receiving information regarding an action suitable for the recognized state of the user from a server. Patent Literature 2 discloses a communication terminal with a camera including a neck strap. Prior Art Document Patent Literature Patent Literature 1: Japanese Patent No. 6053847Patent Literature 2: Japanese Patent Application Laid-Open (JP-A) No. 2019-215444 SUMMARY OF INVENTION Technical Problem However, in the related art, since an electronic apparatus supports a user in a state of being stationary at a specific position, there is room for improvement in supporting the user. Although the related art can suppress the reflection of the neck strap on the camera, this does not support the user's action, and thus there is room for improvement in supporting the user. Although the related art can suppress the reflection of the neck strap on the camera, this does not prevent a suspicious person from approaching a child. Therefore, there is room for improvement in preventing a suspicious person from approaching a child. Although the related art can suppress the reflection of the neck strap on the camera, this does not support the user's action, and thus there is room for improvement in supporting the user. Solution to Problem According to a first aspect of the present disclosure, a control system is provided. The control system includes a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus, an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus, and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, in which the electronic apparatus is detachably provided on a movement unit having a wheel for autonomous traveling, the electronic apparatus includes one or a plurality of sensors that collect information for controlling rotation of the wheel, and the device operations include setting a first action content of leading the user by controlling the wheel. The electronic apparatus includes a communication terminal. Here, the communication terminal includes a device that performs a physical operation, a device that outputs a video or vocal sound without performing a physical operation, and an agent that operates on software. According to a second aspect of the disclosure, a control system is provided. The control system includes a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus, an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus, and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plurality of types of device operations including non-operation as an action of the electronic apparatus, in which the electronic apparatus detachably provided on a neck strap includes one or a plurality of sensors that collect information, and the device operations include setting first action content of leading the user by the electronic apparatus reproducing at least one of vocal sound or an image. Here, the electronic apparatus includes a communication terminal. The communication terminal includes a device that performs a physical operation, a device that outputs a video or vocal sound without performing a physical operation, and an agent that operates on software. According to a third aspect of the disclosure, a control system is provided. The control system includes a state recognition unit that recognizes a user state including an action of a user and a state of an electronic apparatus, an emotion determination unit that determines an emotion of the user or an emotion of the electronic apparatus, and an action determination unit that determines, by using at least one of the user state, the state of the electronic apparatus, the emotion of the user, or the emotion of the electronic apparatus, and an action determination model at a predetermined timing, any of a plural