KR-102961437-B1 - METHOD FOR CONTROLLING IN-VEHICLE ARTIFICAIL INTELLIGENCE AGENT AND SYSTEM THEREFOR

KR102961437B1KR 102961437 B1KR102961437 B1KR 102961437B1KR-102961437-B1

Abstract

A method and system for controlling an artificial intelligence agent for a vehicle are provided. A method for controlling an artificial intelligence agent for a vehicle according to one embodiment of the present disclosure may include, in a method performed by an infotainment system for a vehicle, receiving vehicle situation information including vehicle-related information and external environment information from a vehicle subsystem and generating a first data packet regarding a situation related to the vehicle; receiving driver state information from at least one sensor included in the vehicle subsystem and generating a second data packet regarding driver biosignal analysis information; generating a first trigger condition to generate a prompt based on at least one of the first data packet and the second data packet; when the first trigger condition is detected, generating the prompt based on at least one of the first data packet and the second data packet and inputting the prompt into an LLM (Large-scale Language Model); and providing briefing information based on result data output from the LLM through at least one output interface.

Inventors

황슬기
이지호
안근백
안병진

Assignees

(주)카네비모빌리티

Dates

Publication Date: 20260508
Application Date: 20250804

Claims (15)

In a method performed by a vehicle infotainment system, A step of receiving vehicle situation information including vehicle-related information and external environment information from a vehicle subsystem, and generating a first data packet regarding a situation related to the vehicle; A step of receiving driver status information from at least one sensor included in the vehicle subsystem and generating a second data packet regarding driver biosignal analysis information; A step of generating a first trigger condition to generate a prompt based on at least one of the first data packet and the second data packet; When the first trigger condition is detected, the step of generating the prompt based on at least one of the first data packet and the second data packet, and inputting the prompt into an LLM (Large-scale Language Model); and The method includes the step of providing briefing information based on result data output from the LLM through at least one output interface, The step of inputting the above prompt into the LLM model is, A step of identifying the speaker of the speech data that served as the basis for generating the above prompt; and The step of generating a customized prompt corresponding to a customized mode corresponding to the identified speaker, wherein The step of identifying the above speaker is, A step of identifying a registered speaker by comparing the speaker vectors of a driver and a passenger previously registered in the vehicle infotainment system with the embedding of the speech data, and obtaining the registration profile information of the registered speaker; and If the embedding of the above speech data is not previously registered in the vehicle infotainment system, the method includes the step of analyzing the acoustic features of the above speech data to identify a new speaker and generating temporary profile information corresponding to the new speaker. The step of providing the above briefing information is, If the identified speaker is a speaker registered in the vehicle infotainment system, the method comprises the step of inputting a customized prompt corresponding to the identified speaker into the LLM and providing the output first result data according to the registered profile information; and If the identified speaker is a speaker not registered in the vehicle infotainment system, the method includes the step of inputting a customized prompt corresponding to the identified speaker into the LLM and providing the output second result data according to the temporary profile information. Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of generating the above-mentioned first trigger condition is, A method comprising the step of generating the first trigger condition based on user routine information stored in the vehicle infotainment system. Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The above first trigger condition is, Based on the request of a user riding in the above vehicle, Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of entering the above prompt into the LLM is, A step of obtaining user personal setting information set by a user of the vehicle from the above vehicle infotainment system; and A step comprising generating the prompt using user interest information included in the user personal setting information, the first data packet and the second data packet, Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of entering the above prompt into the LLM is, A step of obtaining user personal setting information from a user terminal associated with the above driver; and A step comprising generating the prompt using user interest information included in the user personal setting information, the first data packet and the second data packet, Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of generating the first data packet above is, A step of obtaining information regarding the current location and destination of the vehicle from the navigation system of the vehicle; and The method includes the step of obtaining weather information on the current location, the destination, and the driving path from the current location to the destination. The step of entering the above prompt into the LLM is, A method comprising the step of generating a first prompt requesting a briefing regarding the above weather information, Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of providing the above briefing information is, A step of providing a summary of the above result data as voice data; Regarding the above voice data, the step of receiving an interaction requesting detailed confirmation from the driver; and A method comprising the step of providing detailed information about the result data through a display of the vehicle infotainment system in response to the above interaction. Method for controlling an artificial intelligence agent for vehicles.
In Article 1, The step of generating the second data packet above is, A step of recognizing the face of the driver and obtaining information on the driver's emotional state; and The method includes the step of quantifying the above emotional state information and calculating an emotional state score, The step of generating the above-mentioned first trigger condition is, A step comprising generating an option to switch the mode of the artificial intelligence agent providing the briefing information when the above emotional state score is above a threshold value, Method for controlling an artificial intelligence agent for vehicles.
In Article 8, The step of entering the above prompt into the LLM is, The method includes a step of automatically switching the mode of the above artificial intelligence agent from the current mode to a suggested mode, The step of providing the above briefing information is, A step comprising providing the briefing information according to the above-mentioned switching proposal mode, Method for controlling an artificial intelligence agent for vehicles.
In Article 8, The step of entering the above prompt into the LLM is, With respect to the above option, the driver receives a request to switch modes of the artificial intelligence agent; and In response to receiving the above switching request, the method includes the step of switching the mode of the artificial intelligence agent from the current mode to the selected mode. The step of providing the above briefing information is, A step comprising providing the briefing information according to the above selection mode, Method for controlling an artificial intelligence agent for vehicles.
In Article 8, The step of generating the first data packet above is, The method includes the step of obtaining real-time traffic information of the section where the vehicle is traveling using the vehicle situation information. The step of calculating the above emotional state score is, The step of classifying the above real-time traffic information according to predefined classification criteria and generating emotional state inference data; and A step comprising correcting the emotional state score using the emotional state inference data above, Method for controlling an artificial intelligence agent for vehicles.
delete
delete
In Article 1, The step of entering the above prompt into the LLM is, A step of obtaining driver personal schedule information from a user terminal associated with the above driver; A step of obtaining information regarding the current location of the vehicle, the destination, and the driving path from the current location to the destination from the first data packet; A step of obtaining the estimated travel time required to arrive at the destination from the current location; A step of calculating the difference time between the estimated arrival time, which is calculated by adding the estimated driving time to the current time at which the vehicle is located at the current location, and the schedule start time of the priority schedule included in the driver's personal schedule information; A step of generating a first prompt for searching for a point of interest on the driving path that the driver may pass through during the difference time; and A step comprising inputting the above first prompt into the above LLM, Method for controlling an artificial intelligence agent for vehicles.
Communication interface; Memory where a computer program is loaded; and The computer program described above includes one or more processors on which it is executed, The above computer program is, The operation of receiving vehicle situation information including vehicle-related information and external environment information from a vehicle subsystem, and generating a first data packet regarding a situation related to the vehicle; The operation of receiving driver status information from at least one sensor included in the vehicle subsystem and generating a second data packet regarding driver biosignal analysis information; An operation to generate a first trigger condition to generate a prompt based on at least one of the first data packet and the second data packet; When the first trigger condition is detected, the operation of generating the prompt based on at least one of the first data packet and the second data packet, and inputting the prompt into an LLM (Large-scale Language Model); and Includes instructions that perform an operation of providing briefing information based on result data output from the LLM through at least one output interface, The action of inputting the above prompt into the LLM model is, An action to identify the speaker of the speech data that served as the basis for generating the above prompt; and The operation includes generating a custom prompt corresponding to a custom mode corresponding to the identified speaker, wherein The action of identifying the above speaker is, The operation of identifying a registered speaker by comparing the speaker vectors of a driver and passenger previously registered in a vehicle infotainment system with the embeddings of the speech data, and obtaining the registration profile information of the registered speaker; and If the embedding of the above speech data is not previously registered in the vehicle infotainment system, the method includes the operation of analyzing the acoustic features of the above speech data to identify a new speaker and generating temporary profile information corresponding to the new speaker. The operation of providing the above briefing information is, If the identified speaker is a speaker registered in the vehicle infotainment system, the operation of inputting a customized prompt corresponding to the identified speaker into the LLM and providing the output first result data according to the registered profile information; and If the identified speaker is a speaker not registered in the vehicle infotainment system, the method includes the operation of inputting a customized prompt corresponding to the identified speaker into the LLM and providing the output second result data according to the temporary profile information. Vehicle AI agent control system.

Description

Method for controlling an artificial intelligence agent in a vehicle and system therewith The present disclosure relates to a method and system for controlling an artificial intelligence agent for a vehicle. More specifically, it relates to a method and system for controlling an artificial intelligence agent for a vehicle that actively provides information using information on situations related to the vehicle and biosignal analysis of the driver. Existing vehicle systems have limitations in actively providing information by comprehensively recognizing real-time driving conditions, such as driver fatigue or emotional state, as well as external environmental data, and personalized interaction with the driver is difficult due to the fixed conversational style of AI agents. Accordingly, there is a need to provide a safer and more personalized in-vehicle user experience by dynamically adjusting the conversation mode of the AI agent based on the driver's condition or passenger characteristics. FIG. 1 is a system configuration diagram for explaining the configuration and operation of a vehicle artificial intelligence agent providing system according to some embodiments of the present disclosure. FIG. 2 is a flowchart for explaining the operation of a method for controlling an artificial intelligence agent for a vehicle according to some embodiments of the present disclosure. FIG. 3 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 4 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 5 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 6 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 7 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 8 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 9 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 10 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 11 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 12 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 13 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 14 is a detailed flowchart for explaining the detailed operation of a vehicle artificial intelligence agent control method according to some embodiments of the present disclosure, described with reference to FIG. 2. FIG. 15 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 16 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 17 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 18 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 19 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 20 is a drawing relating to an example of a user interface screen in which briefing information is provided according to some embodiments of the present disclosure. FIG. 21 is a drawing relating to an example o