KR-102963353-B1 - The System, Method And Computer-Readable Medium that Automatically Generate Conversations of NPCs for Each Situation in A Metaverse Space Using A Language Generation Model

KR102963353B1KR 102963353 B1KR102963353 B1KR 102963353B1KR-102963353-B1

Abstract

The present invention relates to a system, method, and computer-readable medium for automatically generating conversation content of an NPC in a metaverse space by utilizing a language generation model, wherein the system implements the NPC in the metaverse space based on NPC information regarding the NPC, detects an interaction between a user connected to the metaverse and the NPC, determines the situation type and emotional state regarding the interaction, and generates conversation content for the NPC to respond to the user based on the user's text information, situation type information, emotional state information, and NPC information.

Inventors

황보택근
오기성
김제현
정원준
최형선
한일겸

Assignees

가천대학교 산학협력단

Dates

Publication Date: 20260511
Application Date: 20230724

Claims (15)

A service system comprising one or more processors and one or more memories, and performing a method for automatically generating dialogue content of situational NPCs within a metaverse space using a language generation model, An NPC module that stores NPC information including the NPC's own information and information about the NPC's past interactions with other NPCs or users; A metaverse implementation unit that implements and manages the metaverse by implementing a user's avatar in the metaverse according to user input and implementing an NPC in the metaverse according to information managed by the NPC module; An interaction detection unit that detects an interaction between a user located in the metaverse space and the NPC and generates interaction information regarding the interaction; A first classifier that derives situation type information related to the interaction based on the interaction information using a trained deep learning-based first inference model; A second classifier that uses a trained deep learning-based second inference model to derive emotional state information related to the interaction based on the interaction information; A language generation model interfacing unit that inputs text information entered by the user regarding the above NPC, the above situation type information, the above emotional state information, and the above NPC information into a language generation model; and A language generation model that generates generated dialogue content containing text and transmits it to a user by an NPC based on information received from the language generation model interfacing unit, and transmits the generated dialogue content to the corresponding language generation model interfacing unit; The above metaverse implementation unit is a service system that implements the above-mentioned generated conversation content within the metaverse and interacts with the user.
In claim 1, The above service system is, It further includes an NPC update unit that updates the NPC information by transmitting the text information and the generated dialogue content to the NPC module during the interaction between the user and the NPC. A service system that performs the process of updating the NPC information multiple times whenever the interaction between the user and the NPC continues.
In claim 1, The above NPC information is, Including one or more of the age, gender, occupation, and personality of the NPC, A service system that further includes the affinity between the said NPC and other NPCs or users located within the metaverse space.
In claim 1, The above interaction information is, It is input to the first and second classifiers in the form of an image, and Includes text information and motion information output to the NPC from a user located within a preset distance from the NPC, and The above motion information is a service system that includes a user's preset movement or preset facial expression.
In claim 4, The above situation type information includes multiple situation types, and Each of the multiple situation types is derived with different weights based on the user's text information and user's motion information included in the corresponding interaction information, and The above emotional state information includes multiple emotional states, and A service system in which each of multiple emotional states is derived with different weights based on the user's text information and user's motion information included in the corresponding interaction information.
In claim 5, The above language generation model is, Includes a Large Language Model, A service system that generates conversational content for an interaction based on weights corresponding to each of the plurality of situation types and weights corresponding to each of the plurality of emotional states.
A method for automatically generating dialogue content for situational NPCs within a metaverse space by utilizing a language generation model executed in a service system comprising one or more processors and one or more memories, wherein The above service system includes an NPC module, a metaverse implementation unit, an interaction detection unit, a first classifier, a second classifier, a language generation model interfacing unit, and a language generation model. An NPC information storage step that stores NPC information including self-information about the NPC and information about the NPC's past interactions with other NPCs or users, by means of an NPC module; A metaverse implementation step that implements and manages the metaverse by implementing a user's avatar in the metaverse according to user input through a metaverse implementation unit, and implementing an NPC in the metaverse according to information managed by the NPC module; An interaction detection step that detects an interaction between a user located in the metaverse space and the NPC by means of an interaction detection unit, and generates interaction information regarding the interaction; A situation type information derivation step for deriving situation type information related to the corresponding interaction based on the interaction information, using a deep learning-based first inference model learned by a first classifier; An emotional state information derivation step for deriving emotional state information related to the corresponding interaction based on the interaction information, by utilizing a deep learning-based second inference model trained by a second classifier; A language generation model interfacing step of inputting text information entered by a user regarding the NPC, the situation type information, the emotion state information, and the NPC information into a language generation model by means of a language generation model interfacing unit; and A generated dialogue content transmission step comprising: generating generated dialogue content that an NPC including text transmits to a user based on information received through the language generation model interfacing step by means of a language generation model, and transmitting the generated dialogue content to the corresponding language generation model interfacing unit; The above metaverse implementation unit implements the above-mentioned generated conversation content within the metaverse and automatically generates conversation content that interacts with a user.
In claim 7, The above service system further includes an NPC update section, and The method for automatically generating the above conversation content is, The method further includes an NPC update step in which the text information and the generated dialogue content during the interaction between the user and the NPC are transmitted to the NPC module to update the NPC information by means of the above NPC update unit; A method for automatically generating dialogue content, wherein the process of updating the NPC information is performed multiple times whenever the interaction between the user and the NPC continues.
In claim 7, The above NPC information is, Including one or more of the age, gender, occupation, and personality of the NPC, A method for automatically generating conversation content that includes additional intimacy with other NPCs or users located within the metaverse space for the NPC in question.
In claim 7, The above interaction information is, It is input to the first and second classifiers in the form of an image, and Includes text and motion information output to the NPC from a user located within a preset distance from the NPC, and A method for automatically generating dialogue content, wherein the motion information above includes a user's preset movements or preset facial expressions.
In claim 10, The above situation type information includes multiple situation types, and Each of the multiple situation types is derived with different weights based on the user's text information and user's motion information included in the corresponding interaction information, and The above emotional state information includes multiple emotional states, and A method for automatically generating conversation content in which each of multiple emotional states is derived with different weights based on the user's text information and user's motion information included in the corresponding interaction information.
In claim 11, The above language generation model is, Includes a Large Language Model, A method for automatically generating dialogue content for an interaction based on a weight corresponding to each of the plurality of situation types and a weight corresponding to each of the plurality of emotional states.
A computer-readable recording medium for implementing a method for automatically generating dialogue content of a situational NPC in a metaverse space by utilizing a language generation model executed in a service system comprising one or more processors and one or more memories, wherein the computer-readable recording medium comprises computer-executable instructions that cause the service system to perform the following steps. The above service system includes an NPC module, a metaverse implementation unit, an interaction detection unit, a first classifier, a second classifier, a language generation model interfacing unit, and a language generation model. The steps below are: An NPC information storage step that stores NPC information including self-information about the NPC and information about the NPC's past interactions with other NPCs or users, by means of an NPC module; A metaverse implementation step that implements and manages the metaverse by implementing a user's avatar in the metaverse according to user input through a metaverse implementation unit, and implementing an NPC in the metaverse according to information managed by the NPC module; An interaction detection step that detects an interaction between a user located in the metaverse space and the NPC by means of an interaction detection unit, and generates interaction information regarding the interaction; A situation type information derivation step for deriving situation type information related to the corresponding interaction based on the interaction information, using a deep learning-based first inference model learned by a first classifier; An emotional state information derivation step for deriving emotional state information related to the corresponding interaction based on the interaction information, by utilizing a deep learning-based second inference model trained by a second classifier; A language generation model interfacing step of inputting text information entered by a user regarding the NPC, the situation type information, the emotion state information, and the NPC information into a language generation model by means of a language generation model interfacing unit; and A generated dialogue content transmission step comprising: generating generated dialogue content that an NPC including text transmits to a user based on information received through the language generation model interfacing step by means of a language generation model, and transmitting the generated dialogue content to the corresponding language generation model interfacing unit; The above metaverse implementation unit implements the above-mentioned generated conversation content within the metaverse and interacts with the user, a computer-readable recording medium.
In claim 13, The above NPC information is, Including one or more of the age, gender, occupation, and personality of the NPC, A computer-readable recording medium that further includes the affinity of the said NPC with other NPCs or users located within the said metaverse space.
In claim 13, The above interaction information is, It is input to the first and second classifiers in the form of an image, and Includes text and motion information output to the NPC from a user located within a preset distance from the NPC, and The above motion information is a computer-readable recording medium including a user's preset movement or preset facial expression.

Description

The System, Method And Computer-Readable Medium that Automatically Generate Conversations of NPCs for Each Situation in A Metaverse Space Using A Language Generation Model The present invention relates to a system, method, and computer-readable medium for automatically generating conversation content of an NPC in a metaverse space by utilizing a language generation model, wherein the system implements the NPC in the metaverse space based on NPC information regarding the NPC, detects an interaction between a user connected to the metaverse and the NPC, determines the situation type and emotional state regarding the interaction, and generates conversation content for the NPC to respond to the user based on the user's text information, situation type information, emotional state information, and NPC information. The Metaverse is a virtual world that replicates real-world interactions within a virtual space, consisting of forms such as 3D experiential virtual reality, augmented reality, mixed reality, or extended reality. Recently, the applications of the Metaverse are not limited solely to the entertainment sector, such as virtual reality games and virtual concerts; they are also being applied to frontline businesses and industrial sites, utilizing it for three-dimensional and precise tasks in virtual spaces, such as design and process operations. As the scope of Metaverse applications expands in this way, demand for the Metaverse is increasing, and related technologies are developing rapidly. Meanwhile, since the Metaverse is a technology that replicates real-world space into a virtual one, the core of the Metaverse economy lies in ensuring that users connected to the Metaverse do not feel a sense of disconnect from reality within the Metaverse space; consequently, many developers are currently working on technologies to achieve this goal in the implementation of the Metaverse. Meanwhile, while users connected to the metaverse can converse naturally with one another just as in the real world, NPCs located within the metaverse to assist users with content only output text pre-stored in the system implementing the metaverse; consequently, users and NPCs are forced into unnatural interactions. As a result, metaverse users are unable to easily immerse themselves in the metaverse worldview, leading to rising dissatisfaction among users. Recently, artificial intelligence systems capable of processing vast amounts of natural language data and generating responses that are often indistinguishable from human-generated text, such as OpenAI’s GPT (Generative Pre-trained Transformer) series and Google’s BERT (Bidirectional Encoder Representations from Transformers) models, have been gaining attention. Language generation models, including LLMs (Large Language Models), are primarily built using deep learning technology and can be utilized in various fields, leading to increasingly active research in this area. However, conventional language generation technology generates conversational content by considering only the user's text input into the generation model; since this is based on somewhat fragmentary information, it suffers from low reliability. For example, even sentences composed of the same text can carry different meanings depending on the context, making it difficult to elicit natural interactions with responses generated solely from text. Therefore, there is a need to develop technology that enables NPCs to interact naturally with users by generating dialogue content that considers various additional elements besides text. FIG. 1 schematically illustrates a service system that performs a method for automatically generating conversation content of NPCs in a metaverse space by utilizing a language generation model according to an embodiment of the present invention. FIG. 2 schematically illustrates NPC information according to one embodiment of the present invention. FIG. 3 schematically illustrates the conditions for performing interaction between a user and an NPC according to one embodiment of the present invention. FIG. 4 schematically illustrates situation type information and emotional state information according to one embodiment of the present invention. FIG. 5 schematically illustrates the process of a language generation model according to an embodiment of the present invention generating dialogue content. FIG. 6 schematically illustrates the process of generating dialogue content of a language generation model according to one embodiment of the present invention. FIG. 7 schematically illustrates the execution steps for an NPC update unit according to an embodiment of the present invention. FIG. 8 schematically illustrates the interaction between a user and an NPC based on updated NPC information according to an embodiment of the present invention. FIG. 9 illustrates, in an exemplary manner, the internal configuration of a computing device according to one embodiment of the present invention. Hereinafte