Search

CN-117037113-B - Pedestrian crossing early warning method and device under condition of limited visual field, vehicle and medium

CN117037113BCN 117037113 BCN117037113 BCN 117037113BCN-117037113-B

Abstract

The application relates to a pedestrian crossing early warning method, device, vehicle and medium under the condition of limited visual field, wherein the method comprises the steps of collecting at least one pedestrian image of a target vehicle, inputting the at least one pedestrian image into a pre-built pedestrian detection model, outputting detection duration of each pedestrian image, collecting running speed of the target vehicle, calculating window time of the target vehicle based on the running speed, comparing the detection duration with the window time, retraining the pedestrian detection model if the detection duration is greater than or equal to the window time, and controlling the target vehicle to perform pedestrian crossing early warning on a driver if the detection duration is less than the window time. The application tracks and detects pedestrians in a view-limited scene by utilizing the neural network model, so that emergency braking measures can be taken under the necessary condition, the performance and stability of pedestrian detection are effectively improved, the efficiency of vehicle response is improved, and the occurrence probability of traffic accidents is reduced.

Inventors

  • Long Guoshuai

Assignees

  • 芜湖雄狮汽车科技有限公司
  • 奇瑞汽车股份有限公司

Dates

Publication Date
20260505
Application Date
20230804

Claims (6)

  1. 1. The pedestrian crossing early warning method under the condition of limited visual field is applied to an online analysis stage and is characterized by comprising the following steps of: collecting at least one pedestrian image of a target vehicle, inputting the at least one pedestrian image into a pre-constructed pedestrian detection model, and outputting the detection duration of each pedestrian image; collecting the running speed of the target vehicle, and calculating the window time of the target vehicle based on the running speed; Wherein the calculating the window time of the target vehicle includes: determining a projection axial direction according to the running direction of the target vehicle; Calculating the absolute value of the difference between the projection positions of the driver and the vehicle-mounted camera in the projection axial direction, and calculating the window time according to the running speed of the target vehicle and the absolute value; Comparing the detection duration with the window time, retraining the pedestrian detection model if the detection duration is greater than or equal to the window time, and controlling the target vehicle to perform pedestrian crossing early warning on a driver if the detection duration is less than the window time; and if the detection duration is less than the window time, controlling the target vehicle to perform pedestrian crossing early warning on a driver, including: Calculating the distance between the current vehicle and the pedestrian; when the distance is greater than a preset first safety threshold, controlling the target vehicle to enter a primary alarm mode, and controlling the target vehicle to remind the driver in an acoustic and/or optical mode; When the distance is smaller than or equal to the preset first safety threshold value and larger than a preset second safety threshold value, controlling the target vehicle to enter a secondary alarm mode, and controlling the running speed and the running direction of the target vehicle; and when the distance is smaller than or equal to the preset second safety threshold value, controlling the target vehicle to enter a three-level alarm mode, and controlling the target vehicle to execute emergency brake operation.
  2. 2. A pedestrian crossing early warning method under the condition of limited visual field, which is applied to an offline training stage, and is characterized in that the pedestrian crossing early warning method under the condition of limited visual field, which is applied to an online analysis stage, as claimed in claim 1 is adopted, wherein the method comprises the following steps: Acquiring structural information of a current vehicle, and determining an optimal installation position of a vehicle-mounted camera according to the structural information; the method for acquiring the structural information of the current vehicle and determining the optimal installation position of the vehicle-mounted camera according to the structural information comprises the following steps: constructing a limited-view pedestrian crossing scene model based on the structural information; Obtaining an optimal observation point of the vehicle-mounted camera by using the limited-view pedestrian crossing scene model, and determining an optimal mounting position of the vehicle-mounted camera according to the optimal observation point; the determining the optimal installation position of the vehicle-mounted camera according to the optimal observation point comprises the following steps: Determining an optimal horizontal observation point according to the cockpit structure of the current vehicle, and determining the horizontal installation position of the vehicle-mounted camera; determining a vertical installation position of the vehicle-mounted camera based on shielding information of the current vehicle on the vehicle-mounted camera in an actual running process; determining the optimal mounting position according to the horizontal mounting position and the vertical mounting position; Collecting at least one pedestrian image in front of the current vehicle by a vehicle-mounted camera mounted at the optimal mounting position, generating a training set according to a preset pedestrian database and the at least one pedestrian image, and And training a preset pedestrian detection network by using the training set training model to construct a pedestrian detection model.
  3. 3. A pedestrian crossing warning device under limited field of view for implementing the pedestrian crossing warning method under limited field of view applied to an online analysis stage as claimed in claim 1, the device comprising: The second acquisition module is used for acquiring at least one pedestrian image of the target vehicle, inputting the at least one pedestrian image into a pre-constructed pedestrian detection model and outputting the detection duration of each pedestrian image; The third acquisition module is used for acquiring the running speed of the target vehicle and calculating the window time of the target vehicle based on the running speed; the early warning module is used for comparing the detection duration with the window time, retraining the pedestrian detection model if the detection duration is greater than or equal to the window time, and controlling the target vehicle to perform pedestrian crossing early warning on a driver if the detection duration is less than the window time; The second acquisition module comprises a fifth determination unit, a second calculation unit and a window time calculation unit, wherein the fifth determination unit is used for determining a projection axial direction according to the running direction of the target vehicle; the early warning module comprises: a first calculation unit for calculating a distance between a current vehicle and a pedestrian; The first control unit is used for controlling the target vehicle to enter a primary alarm mode when the distance is larger than a preset first safety threshold value, and controlling the target vehicle to remind the driver in an acoustic and/or optical mode; The second control unit is used for controlling the target vehicle to enter a secondary alarm mode and controlling the running speed and the running direction of the target vehicle when the distance is smaller than or equal to the preset first safety threshold value and larger than the preset second safety threshold value; and the third control unit is used for controlling the target vehicle to enter a three-level alarm mode and controlling the target vehicle to execute emergency brake operation when the distance is smaller than or equal to the preset second safety threshold value.
  4. 4. A visual field limited pedestrian crossing warning device for implementing the visual field limited pedestrian crossing warning method applied to the offline training phase as claimed in claim 2, the device comprising: The acquisition module is used for acquiring the structural information of the current vehicle and determining the optimal installation position of the vehicle-mounted camera according to the structural information; a first acquisition module for acquiring at least one pedestrian image in front of the current vehicle through an on-vehicle camera installed at the optimal installation position, generating a training set according to a preset pedestrian database and the at least one pedestrian image, and The training module is used for training a preset pedestrian detection network by utilizing the training set training model to construct a pedestrian detection model; The acquisition module further comprises a second determination unit, a third determination unit, a fourth determination unit and a third determination unit, wherein the second determination unit is used for determining an optimal horizontal observation point according to the cockpit structure of the current vehicle and determining the horizontal installation position of the vehicle-mounted camera; the acquisition module comprises a modeling unit and a first determining unit, wherein the modeling unit is used for constructing a limited-view pedestrian crossing scene model based on the structural information, and the first determining unit is used for obtaining an optimal observation point of the vehicle-mounted camera by utilizing the limited-view pedestrian crossing scene model and determining an optimal installation position of the vehicle-mounted camera according to the optimal observation point.
  5. 5. A vehicle comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the pedestrian crossing warning method in the limited field of view condition of claim 1 or 2.
  6. 6. A computer readable storage medium having stored thereon a computer program, the program being executable by a processor for implementing a pedestrian crossing warning method in a limited field of view situation according to claim 1 or 2.

Description

Pedestrian crossing early warning method and device under condition of limited visual field, vehicle and medium Technical Field The application relates to the technical field of pedestrian detection, in particular to a pedestrian crossing early warning method and device under the condition of limited visual field, a vehicle and a medium. Background Pedestrian detection has been the main research direction in vehicle safety systems due to the risk asymmetry of pedestrians and vehicles in traffic accidents. When the field of vision of pedestrians and drivers of traveling vehicles is blocked by other objects in the environment, serious traffic accidents are easily caused. At present, the research on pedestrian detection and active pedestrian avoidance in the wide view is mature, but the research on the same category in the limited view scene is lack, and from the aspect of vehicle driving safety, a driver early warning system capable of coping with the crossing of the limited view pedestrian is needed. At present, the technical direction of crossing a scene by a pedestrian with limited vision can be generally divided into the direction of detecting the pedestrian based on millimeter waves and the direction of establishing a V2X car networking system for early warning. The V2X vehicle networking detection method is dependent on a large number of sensor deployment and communication networking establishment, the sensor can acquire dead angle-free information of the whole limited-view pedestrian crossing scene, and the dead angle-free information is sent to a vehicle driving system through the communication networking to provide information support for vehicle automatic driving or auxiliary driving further decision behaviors. However, in the millimeter wave detection method, the detection capability of the millimeter wave radar on non-mobile objects, especially human bodies, is limited by the detection principle, the situation of insufficient detection or excessive detection exists, an auxiliary driving system implements unstable driving auxiliary behaviors based on unstable information, such as unexpected sudden braking, which disturb driving judgment and operation of a driver and cannot guarantee driving safety, a large number of sensors are required to be installed in the popularization and implementation process of the V2X vehicle networking detection method, the cost is high, the urban and rural road coverage rate of equipment is low, and secondly, from the safety consideration of networking, once the sensors or local communication network are deliberately damaged, the maintenance of the system also needs manpower investment, and the V2X system is difficult to realize in a large scale in a short time. In addition, the related technology can also use an on-board camera to establish a machine vision system and cooperate with a neural network to detect pedestrians, however, the method is limited in the traditional detection field by the main-stream RNN and CNN neural network system, and usually requires stronger calculation power support and energy supply, and the detection speed is difficult to meet the requirement of the pedestrians with limited vision crossing the complex scenes. In summary, when the prior art detects a pedestrian crossing scene with a limited field of view, detection data is unstable, detection cost or calculation cost is high, detection efficiency is low, danger early warning is difficult to timely and effectively performed on a driver, safety of pedestrians and the driver is greatly affected, and the problem needs to be solved. Disclosure of Invention The application provides a pedestrian crossing early warning method, device, vehicle and medium under the condition of limited visual field, which are used for solving the problems that detection data are unstable, detection cost or calculation cost is high, detection efficiency is low, and danger early warning is difficult to be effectively carried out on a driver in time when the limited visual field pedestrian crossing scene is detected in the prior art. An embodiment of the first aspect of the application provides a pedestrian crossing early warning method under the condition of limited visual field, which is applied to an offline training stage and comprises the following steps of obtaining structural information of a current vehicle, determining an optimal installation position of a vehicle-mounted camera according to the structural information, collecting at least one pedestrian image in front of the current vehicle through the vehicle-mounted camera installed at the optimal installation position, generating a training set according to a preset pedestrian database and the at least one pedestrian image, and training a preset pedestrian detection network by utilizing the training set training model to construct a pedestrian detection model. Optionally, in one embodiment of the present application, the obtaining structural information of