CN-121993010-A - Vehicle window control method and vehicle window control device

CN121993010ACN 121993010 ACN121993010 ACN 121993010ACN-121993010-A

Abstract

The invention relates to a multi-mode vehicle window control method and device. According to the method, voice recognition and computer vision technologies are combined, a vehicle-mounted microphone is used for detecting voice wake-up words of a user, a vehicle-mounted camera is started, the face and the head of the user are detected, the sight, the facial posture and the head posture of the user are analyzed, and therefore the attention area of the user is estimated and registered with a vehicle body coordinate system. And according to the calculated specific coordinates of the user attention on the vehicle window, the intelligent control of the vehicle window is realized. Even if the face or eyes of the user are not detected, the intention of the user can be judged through the face or head gesture, so that accurate window operation can be performed.

Inventors

ZHOU JUN
FAN YICHENG
YANG SHENGBAO
YAO YAO

Assignees

现代自动车株式会社
起亚株式会社

Dates

Publication Date: 20260508
Application Date: 20241107

Claims (8)

1. A window control method for controlling a plurality of windows of a vehicle, comprising: A voice awakening step, namely detecting voice awakening words sent by a user by using a vehicle-mounted microphone, and starting a vehicle-mounted camera when a specific awakening word is detected; a face and head detection step of performing face detection and head detection on the user by using the vehicle-mounted camera to acquire image data of the face and head of the user; An attention area estimation step of analyzing a face state of the user using the image data of the face and the head obtained in the face and head detection step, estimating a line of sight of the user when the face of the user is detected and the eyes of the user are detected, taking a projection area of the vehicle toward which the line of sight is directed as an attention area, estimating a face posture of the user when the face of the user is detected but the eyes of the user are not detected, taking a projection area of the vehicle toward which the face of the user is directed as an attention area, and performing a head detection of the user and estimating a head posture of the user when the face of the user is not detected, taking the projection area of the vehicle toward which the head of the user is directed as an attention area; An attention area processing step of registering the attention area calculated by the attention area estimating step with a vehicle body coordinate system to determine specific coordinates of the user's attention on the window, and And a window control step of performing switching control of the window based on the wake-up word according to the coordinates of the window obtained in the attention area processing step.
2. The window control method according to claim 1, characterized in that: The attention area estimation step further includes a face key point detection and eyebrow depth estimation step of performing key point detection on the detected face of the user, acquiring coordinate information of facial feature points, and performing eyebrow depth estimation using the coordinate information of the facial feature points to determine depth information of the face of the user in a three-dimensional space.
3. The window control method according to claim 1 or 2, characterized in that: In the window control step, it is further determined whether or not a window control operation is required to be performed based on a duration threshold, and the window control operation is performed only when a gazing time of the user at a specific position exceeds a preset duration threshold.
4. The window control method according to claim 1 or 2, characterized in that: And a camera dynamic calibration step is further included between the voice awakening step and the face and head detection step, and the vehicle-mounted camera is dynamically calibrated to acquire external parameter information of the vehicle-mounted camera relative to a vehicle body coordinate system so as to ensure the accuracy of the camera at different positions.
5. A window control device that controls a plurality of windows of a vehicle, comprising: The voice recognition module is used for detecting voice wake-up words sent by a user by using the vehicle-mounted microphone, and when specific wake-up words are detected, the vehicle-mounted camera is started; A face and head detection module that performs face detection and head detection on the user using the in-vehicle camera to acquire image data of the face and head of the user; An attention area estimation module that analyzes a face state of the user using the image data of the face and the head obtained by the face and head detection module, estimates a line of sight of the user when the face of the user is detected and the eyes of the user are detected by the face of the user, takes a projection area of the vehicle toward which the line of sight is directed as an attention area, estimates a face posture of the user when the face of the user is detected but the eyes of the user are not detected by the face of the user, takes a projection area of the vehicle toward which the face of the user is directed as an attention area, and detects and estimates a head posture of the user when the face of the user is not detected by the face of the user, and takes a projection area of the vehicle toward which the head of the user is directed as an attention area; an attention area processing module registering the attention area calculated by the attention area estimating module with a vehicle body coordinate system to determine specific coordinates of the user's attention on the window, and And the vehicle window control module is used for controlling the opening and closing of the vehicle window based on the wake-up word according to the coordinates of the vehicle window obtained by the attention area processing module.
6. The window control device according to claim 5, wherein: The attention area estimation module further comprises a human face key point detection and eyebrow depth estimation module, wherein the human face key point detection and eyebrow depth estimation module is used for carrying out key point detection on the detected face of the user, obtaining coordinate information of facial feature points, and carrying out eyebrow depth estimation by using the coordinate information of the facial feature points to determine depth information of the face of the user in a three-dimensional space.
7. The window control device according to claim 5 or 6, characterized in that: the window control module also determines whether a window control operation is required to be performed according to a duration threshold, and the window control operation is performed only when the gazing time of the user at a specific position exceeds a preset duration threshold.
8. The window control device according to claim 5 or 6, characterized in that: the camera dynamic calibration module is used for carrying out dynamic calibration on the vehicle-mounted camera and obtaining external parameter information of the vehicle-mounted camera relative to a vehicle body coordinate system so as to ensure the accuracy of the camera at different positions.

Description

Vehicle window control method and vehicle window control device Technical Field The invention relates to the technical field of automobile control, in particular to a multi-mode vehicle window control method combining computer vision and voice recognition. Background In modern automotive design, comfort, convenience and intelligent control of the vehicle are key factors in design and manufacture. With the advancement of technology, window control systems are also gradually evolving towards automation and intelligence. However, conventional window control relies primarily on manual operation, i.e. lifting and lowering of the window is achieved by means of physical buttons or switches. This approach, while simple and intuitive, has certain limitations in practical use. First, conventional manual control requires the driver or passenger to directly operate the buttons, which may be inconvenient in some cases. For example, when the driver needs to concentrate on driving, operating the window button may be distracted, affecting driving safety. Furthermore, in emergency situations, a quick response to open or close the window may be limited. Second, the prior art generally lacks intelligent awareness of the in-vehicle environment and occupant intent. The open/close state of the window often cannot be dynamically adjusted according to the change of the environment inside and outside the vehicle and the demand of passengers. For example, the system may not automatically adjust the window to improve air circulation when the air quality in the vehicle is poor, or may not automatically close the window to reduce interference when the external noise is excessive. In response to the above problems, some research and technological advances in recent years have begun to attempt to introduce computer vision and artificial intelligence techniques to promote the level of intelligence in window control. For example, by means of cameras and image processing techniques, the system can recognize facial expressions, gaze directions, and even gesture actions of the person in the vehicle. By combining the voice recognition technology, the car window control system can receive and analyze voice instructions of passengers, so that more visual and convenient operation is realized. For example, patent CN113867527a proposes a window automatic control method combining line-of-sight detection and gesture/voice control. According to the method, an image to be detected of the environment in the vehicle is obtained through the camera, and a sight gaze area of a person in the vehicle is identified from the image. When the sight line gazing area coincides with the position of the vehicle window and corresponding control information is received, the system automatically executes opening and closing actions of the vehicle window. The control information may be a gesture instruction or a voice instruction. The technology improves the automation degree of the car window operation, reduces the requirement of manual operation, improves the car window control convenience to a certain extent, but the vision control of the car window control device is extremely dependent on the sight direction of eyes, and when eyes are shielded, blurred and lost or faces are lost, the estimation of the sight direction is lost, so that the loss of control information is caused, and the user experience is greatly influenced. And accuracy, security, convenience are limited, have hardly satisfied convenient and fast's intelligent cabin demand. Disclosure of Invention In view of the technical defects, the invention aims to provide a window control method with multi-mode fusion of vision and voice, which can accurately control the window even if human eyes are lost and human faces are lost. The invention provides a vehicle window control method for controlling a plurality of vehicle windows of a vehicle, which is characterized by comprising a voice awakening step, a vehicle-mounted camera and a vehicle-mounted control step, wherein the voice awakening step utilizes a vehicle-mounted microphone to detect voice awakening words sent by a user, and when a specific awakening word is detected, the vehicle-mounted camera is started; a face and head detection step of performing face detection and head detection on the user by using the vehicle-mounted camera to acquire image data of the face and head of the user; an attention area estimating step of analyzing a face state of the user using the image data of the face and the head obtained in the face and head detecting step, estimating a line of sight of the user when the face of the user is detected and the eyes of the user are detected by the face of the user, taking the projected area of the vehicle toward which the line of sight is directed as an attention area, estimating a face posture of the user when the face of the user is detected but the eyes of the user are not detected by the face of the user, taking the projected area