KR-102964651-B1 - ROBOT AND METHOD FOR CONTROLLING THEREOF

KR102964651B1KR 102964651 B1KR102964651 B1KR 102964651B1KR-102964651-B1

Abstract

A robot is disclosed. The robot according to the present disclosure includes a camera, a depth sensor, a memory, and a processor. The processor performs an interaction with a first user who has the highest level of interest among a plurality of users present around the robot. While performing the interaction with the first user, the processor acquires gaze information of the plurality of users and distance information of the plurality of users. The processor determines the level of immersion of the first user in the interaction using the gaze information and distance information of the first user among the plurality of users, determines the level of interest of another user using the gaze information and distance information of the first user and another user among the plurality of users. If the level of immersion of the first user is less than a threshold value, the processor terminates the interaction with the first user and performs an interaction with another user based on the level of interest of the other user.

Inventors

한흥우
송무근
이상윤
이유정

Assignees

삼성전자주식회사

Dates

Publication Date: 20260513
Application Date: 20201211

Claims (17)

In robots, camera; Depth sensor; Memory; and Includes a processor; The above processor is, Using the camera and the depth sensor, perform interaction with the first user with the highest level of interest among a plurality of users present around the robot, and While performing interaction with the first user, the camera is used to acquire gaze information of the plurality of users, and the depth sensor is used to acquire distance information of the plurality of users. The level of immersion of the first user in the interaction is determined using the gaze information and distance information of the first user among the plurality of users, and the level of interest of the other user is determined using the gaze information and distance information of the other user among the plurality of users. Matching and storing information about the aforementioned other user and the interest level of the aforementioned other user by time period with time information, and If the immersion level of the first user is lower than a threshold, the interaction with the first user is terminated, and the same service provided to the first user based on the matched information is provided to the other user. Robot.
In Article 1, The above processor is, While performing interaction with the first user, a first map including gaze information of each of the plurality of users and a second map including distance information of each of the plurality of users are stored in the memory, and A fusion map is generated by fusing the first map and the second map, and the fusion map is stored in the memory. Performing interaction with the other user based on the fusion map Robot.
In Article 2, The above processor The above fusion maps are accumulated for each time period and stored in the above memory, and Deleting the first map and the second map from the memory over time Robot.
In Article 1, The above processor is, Using the above camera, an image including the plurality of users is obtained, and By analyzing the above image, gaze information of the plurality of users is obtained, and Distance information of the plurality of users is obtained using the depth sensor above, and Based on the distance information of the plurality of users, density information of the plurality of users is obtained, and Based on the gaze information, density information, and distance information of the plurality of users, the level of interest of each of the plurality of users is calculated, and Identifying the first user with the highest level of interest among the plurality of users based on the level of interest of each of the plurality of users. Robot.
In Paragraph 4, The above density information includes a density weight corresponding to each of the plurality of users, and The above processor is, Calculate a first density weight corresponding to the first user based on at least one of the distance between the first user and other users and the number of other users included within a preset range from the first user, and Calculating the interest level of the first user based on the first density weighting above Robot.
In Article 5, The above processor is, The greater the first density weight, the higher the interest level of the first user is calculated. Robot.
In Article 1, The above processor is, Time information corresponding to the interest level of each of the aforementioned plurality of users is obtained and stored in the memory, Based on the above time information, identifying another user as an interaction target who has an interest level greater than a threshold within a preset range from the time when the interaction with the first user ends. Robot.
In Article 1, Includes a microphone, The above processor is, Acquire a voice signal using the above microphone, and Identifying the user corresponding to the voice signal among the plurality of users above, and Calculating the level of interest of the identified user based on the above voice signal Robot.
In a method for controlling a robot, A step of acquiring the level of interest of each of a plurality of users present around the robot using a camera and a depth sensor included in the robot; A step of performing interaction with the first user with the highest level of interest among the plurality of users based on the interest level obtained above; A step of acquiring gaze information of the plurality of users using the camera while performing interaction with the first user, and acquiring distance information of the plurality of users using the depth sensor; A step of determining the level of immersion of the first user in the interaction using the gaze information and distance information of the first user among the plurality of users, and determining the level of interest of the other user using the gaze information and distance information of the other user among the plurality of users; A step of matching and storing information about the other user and the interest of the other user by time period with time information; and If the immersion level of the first user is less than a threshold, the method comprises the step of terminating the interaction with the first user and providing the same service to the other user as the service provided to the first user based on the matched information. Control method.
In Article 9, While performing interaction with the first user, the step of storing a first map containing gaze information of each of the plurality of users and a second map containing distance information of each of the plurality of users; and The method further includes the step of fusing the first map and the second map to generate a fusion map and storing the fusion map. The step of performing interaction with the other user mentioned above is, Performing interaction with the other user based on the fusion map Control method.
In Article 10, The above fusion maps for each time period are accumulated and stored, The first map and the second map are deleted from memory over time. Control method.
In Article 9, The step of acquiring the level of interest for each of the aforementioned plurality of users is, A step of acquiring an image including the plurality of users using the camera, A step of obtaining gaze information of the plurality of users by analyzing the above image, A step of obtaining distance information of the plurality of users using the depth sensor, A step of obtaining density information of the plurality of users based on the distance information of the plurality of users, and A step comprising calculating the interest level of each of the plurality of users based on the gaze information, density information, and distance information of the plurality of users. Control method.
In Article 12, The above density information includes a density weight corresponding to each of the plurality of users, and The step of calculating the level of interest of each of the aforementioned multiple users is, A step of calculating a first density weight corresponding to the first user based on at least one of the distance between the first user and another user and the number of other users included within a preset range from the first user, and A step comprising calculating the interest level of the first user based on the first density weighting. Control method.
In Article 13, The step of calculating the interest level of the first user above is, The greater the first density weight, the higher the interest level of the first user is calculated. Control method.
In Article 12, The step of acquiring the level of interest for each of the aforementioned plurality of users is, A step of acquiring and storing time information corresponding to the interest level of each of the plurality of users, Based on the above time information, the method includes the step of identifying another user as an interaction target who has an interest level greater than or equal to a threshold within a preset range from the time when the interaction with the first user ends. Control method.
In Article 9, The step of acquiring the level of interest for each of the aforementioned plurality of users is, A step of acquiring a voice signal using a microphone included in the robot, A step of identifying a user corresponding to the voice signal among the plurality of users, and A step comprising calculating the interest of the identified user based on the voice signal. Control method.
A computer-readable, non-transitory recording medium having a program for executing the method of any one of claims 9 through 16 on a computer.

Description

Robot and Method for Controlling Thereof The present disclosure relates to a robot and a control method thereof, and more specifically, to a robot and a control method thereof that collects and stores information about surrounding users even while performing an interaction, and performs an interaction based on information about the users. Thanks to advancements in electronic technology, robots that interact with users in real life are being actively utilized. For example, service robots provide directions to users at specific locations such as airports or museums, or provide users with information about those places. Meanwhile, for a robot to perform interaction with a user, a process of identifying the interaction target is required first. Conventional robots, when multiple users are present in the vicinity, calculate a score based on the location or distance of each user and select the user with the highest score as the interaction target. However, conventional robots did not consider information regarding user density when calculating scores for multiple users. For example, even if many users are densely clustered, if a specific user not included in that cluster is located close to the robot, that user is selected as the interaction target, resulting in a situation where more people could not receive the service simultaneously. Consequently, there was a problem of reduced service efficiency. Furthermore, conventional robots did not collect information about other users in the vicinity while interacting with an interaction target. Consequently, conventional robots returned to their initial position once the interaction with the target ended, causing inconvenience for other users located near the target who wished to receive services from the robot, as they had to move to the robot's initial position. Therefore, there is a need for robots and control methods that can further increase service utility and improve user convenience. FIG. 1 is a drawing for explaining the operation of a robot according to one embodiment of the present disclosure. FIG. 2 is a block diagram illustrating the configuration of a robot according to one embodiment of the present disclosure. FIG. 3a is a drawing illustrating a map according to one embodiment of the present disclosure. FIG. 3b is a drawing illustrating a map according to one embodiment of the present disclosure. FIG. 3c is a drawing for illustrating a fusion map according to one embodiment of the present disclosure. FIG. 4 is a table showing the level of interest of a user according to one embodiment of the present disclosure. FIG. 5 is a drawing for explaining a method for acquiring interest according to one embodiment of the present disclosure. FIG. 6 is a table showing the level of user immersion according to one embodiment of the present disclosure. FIG. 7 is a flowchart illustrating a control method of a robot according to one embodiment of the present disclosure. The terms used in this specification will be briefly explained, and the present disclosure will be described in detail. The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure. The embodiments of the present disclosure are subject to various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the scope of the disclosed spirit and technology. In describing the embodiments, if it is determined that a detailed description of related prior art may obscure the essence, such detailed description is omitted. Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. Terms are used solely for the purpose of distinguishing one component from another. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more