CN-122008860-A - Vehicle interaction method based on visual tracking and related equipment

CN122008860ACN 122008860 ACN122008860 ACN 122008860ACN-122008860-A

Abstract

The application discloses a vehicle interaction method based on visual tracking and related equipment, and relates to the technical field of vehicle human-computer interaction, wherein the method comprises the steps of starting a visual tracking mode, and further obtaining a face image obtained by shooting the face of a driver by an infrared camera; the method comprises the steps of obtaining a gazing direction of eyeballs of a driver according to a facial image, judging effective gazing when the gazing time length of the driver in the gazing direction exceeds a preset time length threshold, mapping the gazing direction to a coordinate area of a vehicle-mounted central control screen when the gazing of the driver is effective gazing, further determining a functional module of the coordinate area, obtaining interactive operation of the driver on a multifunctional knob on the steering wheel, executing the function of the functional module according to an input signal of the interactive operation, and dynamically displaying an execution result on the vehicle-mounted central control screen. According to the application, the fixation area of the driver is identified through the infrared camera, and the rapid and safe operation of the vehicle central control system is realized by combining the multifunctional knob on the steering wheel.

Inventors

LV ZHIYONG
Chen Ruishu
LIU GANG
ZHANG YING
ZHANG YING
Nie Zixiong
LUO LINGYUE

Assignees

武汉理工大学

Dates

Publication Date: 20260512
Application Date: 20260112

Claims (10)

1. A vehicle interaction method based on visual tracking, the method comprising the steps of: starting a visual tracking mode, and further acquiring a face image obtained by shooting the face of the driver by an infrared camera; acquiring the gazing direction of the eyeballs of a driver according to the facial image; when the fixation time length of the driver in the fixation direction exceeds a preset time length threshold value, judging that the fixation is effective; When the driver's gaze is regarded as the effective gaze, mapping the gaze direction to a coordinate area of a vehicle-mounted central control screen, thereby determining a functional module of the coordinate area; Acquiring interactive operation of a driver on a multifunctional knob on a steering wheel; and executing the functions of the function modules according to the input signals of the interactive operation, and dynamically displaying the execution results on the vehicle-mounted central control screen.
2. The vehicle interaction method based on visual tracking according to claim 1, wherein the acquiring of the face image obtained by capturing the face of the driver with the infrared camera comprises the steps of: The face image of the driver is captured by the infrared camera mounted above the front windshield of the vehicle.
3. A vehicle interaction method based on visual tracking according to claim 1, wherein said obtaining the gaze direction of the driver's eyeballs from said facial image comprises the steps of: Positioning the pupil center of the driver and determining the head posture of the driver according to the facial image by utilizing a visual recognition algorithm; And fusing the pupil center and the head gesture, and further calculating and obtaining the gazing direction vector of the eyeball of the driver.
4. The vehicle interaction method based on visual tracking according to claim 1, wherein the step of acquiring the interaction operation performed by the driver on the multifunctional knob on the steering wheel comprises the steps of: acquiring parameter adjustment operation or option switching operation executed by a driver on a rotatable outer ring driving plate in the multifunctional knob; or a function confirmation operation or a mode exit operation executed by a driver on a depressible central key in the multifunction knob is acquired.
5. A vehicle interaction method based on visual tracking according to claim 1, characterized in that the method further comprises the steps of: When the temperature or pressure in the dangerous chemical transportation bin exceeds a safety threshold, the vehicle-mounted central control screen displays high-brightness warning information so that a driver can execute temperature or pressure adjusting operation through the multifunctional knob, wherein a vehicle driven by the driver is loaded with the dangerous chemical transportation bin.
6. A vehicle interaction method based on visual tracking according to any of claims 1 to 5, further comprising the steps of: and when the operation of the multifunctional knob or the vehicle-mounted central control screen by the driver is not detected within the set time, the visual tracking mode is exited and the standby monitoring state is returned.
7. The visual tracking-based vehicle interaction method of claim 6, further comprising the steps of: After exiting the visual tracking mode, a record of the last selected functional module is kept for the driver to resume operation of the functional module at the next activation.
8. A vehicle interaction device based on visual tracking, the device comprising: A face image acquisition unit for starting a visual tracking mode and further acquiring a face image obtained by shooting the face of the driver by an infrared camera; a gaze direction determining unit configured to acquire a gaze direction of an eyeball of a driver from the face image; the effective gazing detection unit is used for judging effective gazing when detecting that the gazing duration of the driver in the gazing direction exceeds a preset duration threshold; The function module determining unit is used for mapping the gazing direction to a coordinate area of the vehicle-mounted central control screen when the driver's gaze is regarded as the effective gaze, so as to determine a function module of the coordinate area; An interactive operation acquisition unit for acquiring interactive operation performed by a driver on a multifunctional knob on a steering wheel; And the interaction unit is used for executing the functions of the function modules according to the input signals of the interaction operation and dynamically displaying the execution results on the vehicle-mounted central control screen.
9. An electronic device comprising a memory storing a computer program and a processor implementing the method of any of claims 1 to 7 when the computer program is executed by the processor.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method according to any one of claims 1 to 7.

Description

Vehicle interaction method based on visual tracking and related equipment Technical Field The application relates to the technical field of vehicle man-machine interaction, in particular to a vehicle interaction method based on visual tracking and related equipment. Background Along with the improvement of the vehicle-mounted intelligent level, a driver can control the vehicle through a central control touch screen or a voice recognition system, such as adjusting the temperature of an air conditioner, switching a navigation interface, checking the state of a warehouse and the like. However, the conventional touch and voice interaction mode still has obvious defects in the dangerous chemical transportation vehicle scene. (1) The disadvantage of the touch interaction mode is that most vehicles currently adopt the touch of a central control screen as a main interaction mode. The driver needs to move his or her line of sight from the road to the screen and click, slide, etc. The method has the following problems that the driver needs to pay attention to in the operation process, the driving distraction risk is increased, the dangerous chemical transport vehicle usually runs in a long-distance and high-vibration environment, the touch operation stability is poor, and when the driver wears gloves or hands to be stained with greasy dirt, the touch sensitivity is obviously reduced, so that the operation precision is affected. (2) In recent years, a voice recognition system is becoming popular in intelligent cabins, and a driver can control a vehicle through voice instructions. However, the mode has various limitations in practical use, such as long interaction time, long interaction chain, large cognitive burden of a driver, clear expression of the command and waiting for system response, influence on driving concentration, poor suitability of dialects and voice recognition, sensitivity of the voice system to dialects, accents and speed difference, low recognition rate, obvious influence of environmental noise, easy interference on voice recognition due to frequent engine booming or noise of refrigerating equipment of dangerous chemicals transportation vehicles, influence on quietness of the environment in the vehicle due to speech output required by voice operation, and possibility of disturbing escort or rest of other vehicle-carrying personnel. Disclosure of Invention The embodiment of the application mainly aims to provide a vehicle interaction method based on visual tracking and related equipment so as to realize rapid and safe human-vehicle interaction. To achieve the above object, an aspect of an embodiment of the present application provides a vehicle interaction method based on visual tracking, the method including the following steps: starting a visual tracking mode, and further acquiring a face image obtained by shooting the face of the driver by an infrared camera; acquiring the gazing direction of the eyeballs of a driver according to the facial image; when the fixation time length of the driver in the fixation direction exceeds a preset time length threshold value, judging that the fixation is effective; When the driver's gaze is regarded as the effective gaze, mapping the gaze direction to a coordinate area of a vehicle-mounted central control screen, thereby determining a functional module of the coordinate area; Acquiring interactive operation of a driver on a multifunctional knob on a steering wheel; and executing the functions of the function modules according to the input signals of the interactive operation, and dynamically displaying the execution results on the vehicle-mounted central control screen. In some embodiments, the acquiring a face image obtained by capturing a face of a driver with an infrared camera includes the steps of: The face image of the driver is captured by the infrared camera mounted above the front windshield of the vehicle. In some embodiments, the capturing the gaze direction of the driver's eye from the facial image includes the steps of: Positioning the pupil center of the driver and determining the head posture of the driver according to the facial image by utilizing a visual recognition algorithm; And fusing the pupil center and the head gesture, and further calculating and obtaining the gazing direction vector of the eyeball of the driver. In some embodiments, the step of obtaining the interaction operation performed by the driver on the multifunctional knob on the steering wheel comprises the following steps: acquiring parameter adjustment operation or option switching operation executed by a driver on a rotatable outer ring driving plate in the multifunctional knob; or a function confirmation operation or a mode exit operation executed by a driver on a depressible central key in the multifunction knob is acquired. In some embodiments, the method further comprises the steps of: When the temperature or pressure in the dangerous chemical transportation bin exceeds