US-20260129099-A1 - VIRTUAL AVATAR BEHAVIOR SAFETY MANAGEMENT SYSTEM BASED ON AI MONITORING AND PREDICTION AND METHOD THEREOF

US20260129099A1US 20260129099 A1US20260129099 A1US 20260129099A1US-20260129099-A1

Abstract

A virtual avatar behavior safety management system based on AI monitoring and prediction and a method thereof are disclosed. In the system, a speech conversation between a user-controlled virtual avatar and another virtual avatar are monitored in real time, and a voice recognition technology and a speech-to-text technology are used to convert the speech conversation into text messages, and a conversation prediction message is generated through a retrieval-augmented generation (RGA) technology, and input into a pre-trained safety protection language model to predict whether the virtual avatar exhibits an inappropriate behavior characteristic. When the virtual avatar exhibits the inappropriate behavior characteristic, user's control over the virtual avatar is immediately prohibited, thereby achieving the technical effect of enhancing interactive safety in the virtual world.

Inventors

Chuan-Cheng Chiu
Tao Yu
Yi-Ming ZHAN

Assignees

SQ Technology (Shanghai) Corporation
INVENTEC CORPORATION

Dates

Publication Date: 20260507
Application Date: 20250114
Priority Date: 20241101

Claims (10)

1 . A virtual avatar behavior safety management system based on AI monitoring and prediction, comprising: a non-transitory computer-readable storage medium, configured to store computer readable instructions, and a pre-trained safety protection language model; and a hardware processor, electrically connected to the non-transitory computer-readable storage medium, and configured to execute the computer readable instructions to operate: when a user operates a virtual avatar to perform conversation, extracting the conversation of the virtual avatar to generate a conversation speech, and converting the conversation speech into conversation messages through a voice recognition technology and a speech-to-text technology; using a retriever of retrieval-augmented generation (RGA) to retrieve knowledge messages related to the conversation messages from an external knowledge base, and using a generator of the retrieval-augmented generation to generate a conversation prediction message, which is accurate and highly relevant, based on the knowledge message and a natural language processing technology; and inputting the conversation prediction message into the safety protection language model to predict whether the virtual avatar exhibits an inappropriate behavior characteristic, and when the virtual avatar exhibits the inappropriate behavior characteristic, immediately prohibiting the user from controlling the virtual avatar.
2 . The virtual avatar behavior safety management system based on AI monitoring and prediction according to claim 1 , wherein the inappropriate behavior characteristic comprises the conversation related to inappropriate language, fraudulent intent, unusual behavior, or potential safety threat, wherein when the conversation prediction message matches the inappropriate behavior characteristic, warning points are accumulated, and a control time of prohibiting the user from controlling the virtual avatar is dynamically adjusted based on the warning points, and the warning points and the control time are negatively correlated to each other.
3 . The virtual avatar behavior safety management system based on AI monitoring and prediction according to claim 1 , wherein the safety protection language model comprises bidirectional encoder representations from transformers (BERT) technology, generative pre-trained transformer (GPT) technology, and is pre-trained with training data comprising text, images, sounds, or combinations thereof related to inappropriate language, fraudulent intent, unusual behavior, and potential safety threat.
4 . The virtual avatar behavior safety management system based on AI monitoring and prediction according to claim 1 , wherein the external knowledge base comprises news, pictures, sounds, images, and a combination thereof related to fraud, bullying, emotional blackmail, and emotional control, and the external knowledge base is embedded with vectors corresponding to the news, pictures, sounds, images, and a combination thereof for the retriever to perform a similarity search.
5 . The virtual avatar behavior safety management system based on AI monitoring and prediction according to claim 1 , wherein when the safety protection language model predicts whether the virtual avatar exhibits the inappropriate behavior characteristic, and dynamic adjustment of weights based on a conversation topic and an interpersonal relationship of the user is allowed to increase prediction accuracy.
6 . A virtual avatar behavior safety management method based on AI monitoring and prediction, wherein the virtual avatar behavior safety management method is executed by a hardware processor and comprises: loading a pre-trained safety protection language model, by the hardware processor; when a user operates a virtual avatar to perform conversation, extracting the conversation of the virtual avatar to generate a conversation speech, and converting the conversation speech into conversation messages through voice recognition technology and speech-to-text technology, by the hardware processor; using a retriever of retrieval-augmented generation (RGA) to retrieve knowledge messages related to the conversation message from an external knowledge base, and using a generator of the RGA to generate a conversation prediction message, which is accurate and highly relevant, based on the knowledge messages and a natural language processing technology, by the hardware processor; and inputting the conversation prediction messages to the safety protection language model to predict whether the virtual avatar exhibits an inappropriate behavior characteristic, and when the virtual avatar exhibits the inappropriate behavior characteristic, immediately prohibiting the user from controlling the virtual avatar, by the hardware processor.
7 . The virtual avatar behavior safety management method based on AI monitoring and prediction according to claim 6 , wherein the inappropriate behavior characteristic comprises the conversation related to inappropriate language, fraudulent intent, unusual behavior, or potential safety threat, wherein when the conversation prediction message matches the inappropriate behavior characteristic, warning points are accumulated, and a control time of prohibiting the user from controlling the virtual avatar is dynamically adjusted based on the warning points, and the warning points and the control time are negatively correlated to each other.
8 . The virtual avatar behavior safety management method based on AI monitoring and prediction according to claim 6 , wherein the safety protection language model comprises bidirectional encoder representations from transformers (BERT) technology, generative pre-trained transformer (GPT) technology, and is pre-trained with training data comprising text, images, sounds, or combinations thereof related to inappropriate language, fraudulent intent, unusual behavior, and potential safety threat.
9 . The virtual avatar behavior safety management method based on AI monitoring and prediction according to claim 6 , wherein the external knowledge base comprises news, pictures, sounds, images, and a combination thereof related to fraud, bullying, emotional blackmail, and emotional control, and the external knowledge base is embedded with vectors corresponding to the news, pictures, sounds, images, and a combination thereof for the retriever to perform a similarity search.
10 . The virtual avatar behavior safety management method based on AI monitoring and prediction according to claim 6 , wherein when the safety protection language model predicts whether the virtual avatar exhibits the inappropriate behavior characteristic, and dynamic adjustment of weights based on a conversation topic and an interpersonal relationship of the user is allowed to increase prediction accuracy.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a safety management system and a method thereof, more particularly to a virtual avatar behavior safety management system based on AI monitoring and prediction and a method thereof. 2. Description of the Related Art In recent years, with the widespread development of various immersive technologies, the application of virtual avatar has emerged rapidly. However, how to improve interaction safety in the virtual world has been one of the key challenges that companies are eager to solve. Conventional virtual avatars are usually controlled by users, for example, to move or interact in the virtual world. However, since the user controls the virtual avatar in the virtual world, there are no real-world constraints, it leads to frequent occurrences of malicious harassment, fraud, inappropriate language, and other issues. Therefore, the virtual world faces a problem of insufficient interaction safety. In response to this, some companies have proposed methods using keyword detection to determine whether certain pre-set keywords (such as offensive language) appear in a conversation content, and muting manner to mute the virtual avatar when such keywords are identified. However, this method can only work for the pre-set keywords and is ineffective when the keywords have not been pre-configured, it is obvious that the method has limited applicability and cannot detect cases where the users substitute words to bypass keyword detection, such as using English, numbers, or homophones for abusive behavior. Therefore, the existing technologies still fail to effectively solve the problem of insufficient interaction safety in the virtual world. According to above-mentioned contents, what is needed is to develop an improved solution to solve the conventional problem of insufficient interaction safety in the virtual world. SUMMARY OF THE INVENTION An objective of the present invention is to disclose a virtual avatar behavior safety management system based on AI monitoring and prediction and a method thereof, to solve the conventional problem. To achieve the objective, the present invention discloses a virtual avatar behavior safety management system based on AI monitoring and prediction, and virtual avatar behavior safety management system includes a non-transitory computer-readable storage medium and a hardware processor. The non-transitory computer-readable storage medium is configured to store computer readable instructions, and a pre-trained safety protection language model. The hardware processor is electrically connected to the non-transitory computer-readable storage medium, and configured to execute the computer readable instructions to operate: when a user operates a virtual avatar to perform conversation, extracting the conversation of the virtual avatar to generate a conversation speech, and converting the conversation speech into conversation messages through a voice recognition technology and a speech-to-text technology; using a retriever of retrieval-augmented generation (RGA) to retrieve knowledge messages related to the conversation messages from an external knowledge base, and using a generator of the retrieval-augmented generation to generate a conversation prediction message, which is accurate and highly relevant, based on the knowledge message and a natural language processing technology; inputting the conversation prediction message into the safety protection language model to predict whether the virtual avatar exhibits an inappropriate behavior characteristic, and when the virtual avatar exhibits the inappropriate behavior characteristic, immediately prohibiting the user from controlling the virtual avatar. To achieve the objective, the present invention discloses a virtual avatar behavior safety management method based on AI monitoring and prediction, wherein the virtual avatar behavior safety management method is executed by a hardware processor and includes the steps of: loading a pre-trained safety protection language model, by the hardware processor; when a user operates a virtual avatar to perform conversation, extracting the conversation of the virtual avatar to generate a conversation speech, and converting the conversation speech into conversation messages through voice recognition technology and speech-to-text technology, by the hardware processor; using a retriever of retrieval-augmented generation (RGA) to retrieve knowledge messages related to the conversation message from an external knowledge base, and using a generator of the RGA to generate a conversation prediction message, which is accurate and highly relevant, based on the knowledge messages and a natural language processing technology, by the hardware processor; inputting the conversation prediction messages to the safety protection language model to predict whether the virtual avatar exhibits an inappropriate behavior characteristic, and when the virtua