Search

CN-122009226-A - Vehicle control method and device based on in-vehicle personnel emotion data and computer readable storage medium

CN122009226ACN 122009226 ACN122009226 ACN 122009226ACN-122009226-A

Abstract

The embodiment of the invention relates to the technical field of intelligent vehicle control and discloses a vehicle control method, a device and a computer readable storage medium based on emotion data of personnel in a vehicle, wherein the method comprises the steps of obtaining multi-modal sensing data of the personnel in the vehicle, wherein the multi-modal sensing data comprises face data, voice data, non-contact physiological signals and contact physiological signals; the method comprises the steps of carrying out fusion analysis according to the multi-mode sensing data to obtain emotion state data of personnel in a vehicle, generating a vehicle control instruction according to the emotion state data, and sending the vehicle control instruction to a vehicle control execution unit so that the vehicle control execution unit executes the vehicle control instruction to complete vehicle control based on the emotion data of the personnel in the vehicle. By applying the technical scheme of the invention, the crossing from passive alarming to active control can be realized, the active safety of driving is improved, and the emotion care and comfortable experience of the user in the vehicle are considered.

Inventors

  • GUO CONGYAO

Assignees

  • 阿维塔科技(重庆)股份有限公司

Dates

Publication Date
20260512
Application Date
20260330

Claims (10)

  1. 1. A vehicle control method based on in-vehicle personnel emotion data, the method comprising: Acquiring multi-modal sensing data of personnel in a vehicle, wherein the multi-modal sensing data comprises facial data, voice data, non-contact physiological signals and contact physiological signals; performing fusion analysis according to the multi-mode sensing data to obtain emotion state data of the personnel in the vehicle; Generating a vehicle control instruction according to the emotion state data; and sending the vehicle control instruction to a vehicle control execution unit so that the vehicle control execution unit executes the vehicle control instruction to complete vehicle control based on the emotion data of the personnel in the vehicle.
  2. 2. The method of claim 1, wherein generating vehicle control instructions from the emotional state data comprises: Acquiring a preset emotion state control strategy mapping table, wherein the preset emotion state control strategy mapping table comprises a corresponding relation between emotion state data and a vehicle control instruction; and searching the preset emotion state control strategy mapping table according to the emotion state data to obtain a vehicle control instruction.
  3. 3. The method of claim 1, wherein the performing fusion analysis according to the multimodal perception data to obtain the emotional state data of the person in the vehicle comprises: Inputting the multi-modal sensing data into an emotion classification recognition model to obtain an emotion state label and a confidence level corresponding to the emotion state label, wherein the emotion classification recognition model is input as multi-modal sensing data, and output as the emotion state label and the confidence level corresponding to the emotion state label; and obtaining the emotion state data according to the emotion state label and the confidence level.
  4. 4. The method of claim 3, wherein inputting the multimodal sensory data into an emotion classification recognition model to obtain an emotion state label and a confidence level corresponding to the emotion state label comprises: Desensitizing the multi-mode sensing data line to obtain a desensitizing result; Inputting the desensitization result into an initial emotion classification recognition model for prediction to obtain a prediction result; Optimizing and iterating the initial emotion classification recognition model according to the prediction result to obtain an optimized emotion classification recognition model; and obtaining an emotion state label and a confidence coefficient corresponding to the emotion state label according to the emotion classification and identification model.
  5. 5. The method of claim 1, wherein the emotional state data comprises a confidence level; Before the vehicle control instruction is generated according to the emotion state data, the method further comprises the following steps: And executing the step of generating a vehicle control instruction according to the emotion state data when the confidence coefficient is greater than or equal to a preset confidence coefficient threshold value.
  6. 6. The method of claim 1, wherein the acquiring multi-modal awareness data of the person in the vehicle, wherein the multi-modal awareness data includes facial data, voice data, non-contact physiological signals, and contact physiological signals, comprises: acquiring a face video stream of a person in the vehicle acquired by a camera; acquiring voice signals of the personnel in the vehicle, which are acquired by a microphone array; Extracting features of the face video stream and the voice signal to obtain face data and voice data; Acquiring heart rate and respiratory frequency acquired by a millimeter wave radar to obtain a non-contact physiological signal; And acquiring heart rate and skin electric response data of the personnel in the vehicle acquired by the embedded contact sensor to obtain a contact physiological signal.
  7. 7. The method of claim 6, wherein the feature extracting the face video stream and the voice signal to obtain face data and voice data comprises: Extracting facial action unit characteristics of the facial video stream to obtain facial data; and extracting the intonation and the speech speed of the voice signal to obtain voice data.
  8. 8. A vehicle control apparatus based on in-vehicle human emotion data, the apparatus comprising: The system comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for acquiring multi-mode sensing data of personnel in a vehicle, and the multi-mode sensing data comprises face data, voice data, non-contact physiological signals and contact physiological signals; the fusion analysis module is used for carrying out fusion analysis according to the multi-mode sensing data to obtain the emotion state data of the personnel in the vehicle; the instruction generation module is used for generating a vehicle control instruction according to the emotion state data; and the vehicle control module is used for sending the vehicle control instruction to a vehicle control execution unit so that the vehicle control execution unit executes the vehicle control instruction to complete vehicle control based on the emotion data of the personnel in the vehicle.
  9. 9. The vehicle control device based on the emotion data of the personnel in the vehicle is characterized by comprising a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus; The memory is configured to store at least one executable instruction that causes the processor to perform the operations of the vehicle control method based on in-vehicle human emotion data as recited in any one of claims 1 to 7.
  10. 10. A computer-readable storage medium, wherein at least one executable instruction is stored in the storage medium, which when executed on a vehicle control apparatus/device based on in-vehicle human emotion data, causes the vehicle control apparatus/device based on in-vehicle human emotion data to perform the operations of the vehicle control method based on in-vehicle human emotion data according to any one of claims 1 to 7.

Description

Vehicle control method and device based on in-vehicle personnel emotion data and computer readable storage medium Technical Field The embodiment of the invention relates to the technical field of intelligent vehicle control, in particular to a vehicle control method and device based on in-vehicle personnel emotion data and a computer readable storage medium. Background Along with the improvement of the intelligent level of the whole vehicle, the vehicle not only needs to sense the external road environment, but also needs to grasp the physiological state and psychological state of the driver and the passengers in the driving process in real time so as to cope with the situation that fatigue, tension, mood fluctuation or sudden physiological abnormality and the like have influence on the driving safety. The traditional technical scheme generally adopts driver monitoring or passenger monitoring to detect the face state, sight line behavior or physiological index of the driver, and sends a prompt or warning to the driver in a sound or image mode when the preset condition is met, but the state recognition result is often isolated from the actual driving control of the vehicle, and the driver only stays at the prompt level. Disclosure of Invention In view of the above problems, the embodiments of the present invention provide a vehicle control method and apparatus based on emotion data of personnel in a vehicle, and a computer readable storage medium, which are used to solve the problem in the prior art that it is difficult to accurately identify abnormal emotion or physiological state of personnel in the vehicle, which may affect driving safety, and generate a vehicle safety decision. According to an aspect of the embodiment of the invention, there is provided a vehicle control method based on in-vehicle personnel emotion data, the method comprising: Acquiring multi-modal sensing data of personnel in the vehicle, wherein the multi-modal sensing data comprises facial data, voice data, non-contact physiological signals and contact physiological signals; fusion analysis is carried out according to the multi-mode sensing data, so that emotion state data of personnel in the vehicle are obtained; Generating a vehicle control instruction according to the emotion state data; And sending the vehicle control instruction to the vehicle control execution unit so that the vehicle control execution unit executes the vehicle control instruction to complete vehicle control based on the emotion data of the personnel in the vehicle. According to another aspect of the embodiment of the present invention, there is provided a vehicle control device based on in-vehicle human emotion data, including: the system comprises a data acquisition module, a data processing module and a data processing module, wherein the data acquisition module is used for acquiring multi-mode sensing data of personnel in a vehicle, and the multi-mode sensing data comprises face data, voice data, non-contact physiological signals and contact physiological signals; the fusion analysis module is used for carrying out fusion analysis according to the multi-mode sensing data to obtain the emotion state data of the personnel in the vehicle; The instruction generation module is used for generating a vehicle control instruction according to the emotion state data; and the vehicle control module is used for sending the vehicle control instruction to the vehicle control execution unit so as to enable the vehicle control execution unit to execute the vehicle control instruction to complete vehicle control based on the emotion data of the personnel in the vehicle. According to another aspect of the embodiment of the invention, there is provided a vehicle control device based on in-vehicle human emotion data, comprising a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus; The memory is configured to store at least one executable instruction that causes the processor to perform operations of the vehicle control method based on in-vehicle human emotion data as described above. According to still another aspect of an embodiment of the present invention, there is provided a computer-readable storage medium having stored therein at least one executable instruction for causing an in-vehicle human emotion data-based vehicle control apparatus/device to perform the operations of the above in-vehicle human emotion data-based vehicle control method. According to the embodiment of the invention, the facial data, the voice data, the non-contact physiological signals and the contact physiological signals of the personnel in the vehicle are acquired, different types of data are comprehensively analyzed to acquire the emotion state data of the personnel in the vehicle, and then the vehicle control command is generated according to