CN-121989929-A - Intelligent auxiliary safety protection system and method for vehicle facing complex industrial scene

CN121989929ACN 121989929 ACN121989929 ACN 121989929ACN-121989929-A

Abstract

The invention relates to the technical field of image processing and vehicle safety protection, and discloses an intelligent auxiliary safety protection system and method for vehicles facing complex industrial scenes, the method comprises the steps of collecting video stream data, identifying obstacles, generating space distribution information, dynamically constructing a three-dimensional warning space, analyzing intruding obstacles, tracking motion tracks, generating a hierarchical braking control signal and sending the hierarchical braking control signal to a braking executing mechanism. The system comprises a video acquisition unit, an obstacle recognition unit, a space mapping unit, a warning space construction unit, a space analysis unit, a target tracking unit, a decision unit and an instruction output unit. According to the invention, the obstacle is accurately identified through image identification and space mapping, the self-adaptive three-dimensional warning space is dynamically constructed according to the running parameters of the vehicle, and the classification braking signals are generated by combining the types of the obstacle and the movement track, so that the accuracy and the reliability of intelligent auxiliary safety protection of the vehicle in a complex industrial scene are realized.

Inventors

WEI QING
Cui Ruikai
SUN HAO
CAO KANG

Assignees

太原市廉顺交通服务有限公司

Dates

Publication Date: 20260508
Application Date: 20260327

Claims (10)

1. The intelligent auxiliary safety protection method for the vehicle facing the complex industrial scene is characterized by comprising the following steps of: s1, collecting video stream data in the traveling direction of a vehicle; s2, identifying obstacles in an image frame in the video stream data, and determining position information and category information of the obstacles in an image coordinate system, wherein the position information and the category information appear in the image frame; S3, mapping the position information into a physical space coordinate system taking the vehicle as a center to generate space distribution information comprising the physical position and the category of the obstacle; S4, dynamically constructing a three-dimensional guard space matched with the predicted running path of the vehicle according to the current running parameters of the vehicle; S5, carrying out space relation analysis on the space distribution information and the three-dimensional guard space, and judging whether an invasive barrier which invades the three-dimensional guard space exists or not; S6, when an invasive obstacle exists, carrying out association tracking on the same invasive obstacle in the continuous multi-frame image frames to generate a motion track of the invasive obstacle in a physical space; S7, generating a graded braking control signal comprising braking time and braking intensity according to the motion trail and the type of the intrusion barrier; And S8, transmitting the graded braking control signal to a vehicle braking executing mechanism to trigger the vehicle to execute corresponding deceleration or braking operation.
2. The method according to claim 1, wherein the step S2 of identifying the obstacle for the image frame specifically includes: Inputting the image frames into a pre-trained deep convolutional neural network, performing step-by-step convolution and pooling operation on the image frames through a multi-layer convolution kernel group of the deep convolutional neural network, and extracting a layered visual feature map comprising edges, textures and component layers; the method comprises the steps of respectively accessing a layered visual feature map into a first parallel branch and a second parallel branch of the network, wherein the first parallel branch generates a candidate target frame through a regional proposal network, finely adjusts the position of the candidate target frame through a boundary frame regressive, and outputs a boundary frame marking the outline of an obstacle, and the second parallel branch carries out feature classification on an image region covered by the boundary frame through a full-connection layer and outputs an obstacle category label corresponding to the boundary frame, wherein the obstacle category label at least comprises personnel, equipment and sundries.
3. The method according to claim 1, wherein the step S4 of dynamically constructing a three-dimensional alert space specifically comprises: Periodically acquiring an instantaneous speed pulse signal of a vehicle, an angle signal of a steering wheel angle sensor and pitch angle and roll angle signals of a vehicle body posture sensor from a vehicle CAN bus; Inputting an instantaneous speed pulse signal, an angle signal, a pitch angle and a roll angle signal into a vehicle kinematic model, and iteratively generating an expected travelling track curve consisting of discrete path points according to the inherent wheelbase, minimum turning radius and full-load braking distance parameters of the vehicle; and generating a three-dimensional enveloping body extending along the datum line according to the outline size of the vehicle by taking the expected travelling track curve as the datum line, wherein the width of the horizontal section of the three-dimensional enveloping body expands outwards when the curvature of the curve increases, and narrows when the curvature of the curve decreases, and the height of the vertical section of the three-dimensional enveloping body covers the highest point at the top of the vehicle to form a three-dimensional warning space.
4. The method according to claim 1, wherein the step S6 of performing association tracking on the same intrusion obstacle specifically comprises: Extracting image blocks falling into an invasive obstacle boundary frame from continuous multi-frame image frames, carrying out scale-invariant feature transformation on the image blocks, and generating feature description vectors composed of key points on the surface of the invasive obstacle and gradient direction histograms thereof; Performing similarity matching by comparing Euclidean distances of feature description vectors of the intrusion barriers between two adjacent frames of image frames in time, judging the intrusion barriers with the similarity exceeding a preset threshold as the same physical target, and distributing a unique tracking identifier for the physical target to form a position sequence with a time stamp; And (3) performing differential processing on position coordinates of adjacent moments in the position sequence to generate an instantaneous pixel displacement vector of the physical object on an image plane, and converting the instantaneous pixel displacement vector into an instantaneous motion direction and a motion rate in a physical space by combining the mapping relation of the step (S3).
5. The method according to claim 1, wherein generating the step S7 includes: Presetting a first braking threshold interval and a second braking threshold interval, wherein the first braking threshold interval corresponds to a first safety distance range between the vehicle and an intruding obstacle, and the second braking threshold interval corresponds to a second safety distance range which is larger than the first safety distance range; When the class label of the intruding obstacle is personnel, the movement direction points to the expected travelling track curve, and the real-time distance between the vehicle and the intruding obstacle falls into a first braking threshold value interval, generating a high-level pulse width modulation signal as an emergency braking signal, and driving a hydraulic valve in a braking executing mechanism to act at the maximum opening degree by the signal so as to enable the braking deceleration to reach the physical limit value of a vehicle braking system; when the class label of the intruding obstacle is equipment, the movement rate is zero or the movement direction deviates from the expected travelling track curve, and the real-time distance between the vehicle and the intruding obstacle falls into a second braking threshold value interval, a low-level pulse width modulation signal is generated as an early warning braking signal, and the signal drives a hydraulic valve in a braking executing mechanism to act with partial opening, so that the braking deceleration is kept below a preset comfortable deceleration threshold value, and simultaneously an audible and visual alarm connected with a CAN bus of the vehicle is triggered to emit intermittent alarm sound and yellow flashing light.
6. A vehicle intelligent auxiliary safety protection system for complex industrial scenarios, for implementing the method of any one of claims 1-5, comprising: The video acquisition unit is configured to acquire video stream data in the traveling direction of the vehicle; An obstacle recognition unit configured to recognize an obstacle for an image frame in the video stream data, and determine position information and category information of the obstacle in the image coordinate system, which appears in the image frame; A spatial mapping unit configured to map the position information into a physical spatial coordinate system centered on the vehicle, and generate spatial distribution information including physical positions and categories of the obstacle; The warning space construction unit is configured to dynamically construct a three-dimensional warning space matched with the predicted running path of the vehicle according to the current running parameters of the vehicle; A space analysis unit configured to perform a space relationship analysis on the space distribution information and the three-dimensional guard space, and determine whether an intrusion barrier that intrudes into the three-dimensional guard space exists; A target tracking unit configured to perform association tracking on the same invasive obstacle in successive multi-frame image frames when the invasive obstacle exists, and generate a motion track of the invasive obstacle in a physical space; a decision unit configured to generate a hierarchical brake control signal including a brake timing and a brake intensity according to the motion trajectory and the type of the intrusion obstacle; and the command output unit is configured to send the graded brake control signal to the vehicle brake actuating mechanism to trigger the vehicle to execute corresponding deceleration or braking operation. .
7. The system of claim 6, wherein the obstacle recognition unit is further configured to: Inputting the image frames into a pre-trained deep convolutional neural network, performing step-by-step convolution and pooling operation on the image frames through a multi-layer convolution kernel group of the deep convolutional neural network, and extracting a layered visual feature map comprising edges, textures and component layers; the method comprises the steps of respectively accessing a layered visual feature map into a first parallel branch and a second parallel branch of the network, wherein the first parallel branch generates a candidate target frame through a regional proposal network, finely adjusts the position of the candidate target frame through a boundary frame regressive, and outputs a boundary frame marking the outline of an obstacle, and the second parallel branch carries out feature classification on an image region covered by the boundary frame through a full-connection layer and outputs an obstacle category label corresponding to the boundary frame, wherein the obstacle category label at least comprises personnel, equipment and sundries.
8. The system of claim 6, wherein the alert space construction unit is further configured to: Periodically acquiring an instantaneous speed pulse signal of a vehicle, an angle signal of a steering wheel angle sensor and pitch angle and roll angle signals of a vehicle body posture sensor from a vehicle CAN bus; Inputting an instantaneous speed pulse signal, an angle signal, a pitch angle and a roll angle signal into a vehicle kinematic model, and iteratively generating an expected travelling track curve consisting of discrete path points according to the inherent wheelbase, minimum turning radius and full-load braking distance parameters of the vehicle; and generating a three-dimensional enveloping body extending along the datum line according to the outline size of the vehicle by taking the expected travelling track curve as the datum line, wherein the width of the horizontal section of the three-dimensional enveloping body expands outwards when the curvature of the curve increases, and narrows when the curvature of the curve decreases, and the height of the vertical section of the three-dimensional enveloping body covers the highest point at the top of the vehicle to form a three-dimensional warning space.
9. The system of claim 6, wherein the target tracking unit is further configured to: Extracting image blocks falling into an invasive obstacle boundary frame from continuous multi-frame image frames, carrying out scale-invariant feature transformation on the image blocks, and generating feature description vectors composed of key points on the surface of the invasive obstacle and gradient direction histograms thereof; Performing similarity matching by comparing Euclidean distances of feature description vectors of the intrusion barriers between two adjacent frames of image frames in time, judging the intrusion barriers with the similarity exceeding a preset threshold as the same physical target, and distributing a unique tracking identifier for the physical target to form a position sequence with a time stamp; And carrying out differential processing on position coordinates of adjacent moments in the position sequence to generate an instantaneous pixel displacement vector of the physical object on an image plane, and converting the instantaneous pixel displacement vector into an instantaneous motion direction and a motion rate in a physical space by combining the mapping relation of the space mapping unit.
10. The system of claim 6, wherein the decision unit is further configured to: Presetting a first braking threshold interval and a second braking threshold interval, wherein the first braking threshold interval corresponds to a first safety distance range between the vehicle and an intruding obstacle, and the second braking threshold interval corresponds to a second safety distance range which is larger than the first safety distance range; When the class label of the intruding obstacle is personnel, the movement direction points to the expected travelling track curve, and the real-time distance between the vehicle and the intruding obstacle falls into a first braking threshold value interval, generating a high-level pulse width modulation signal as an emergency braking signal, and driving a hydraulic valve in a braking executing mechanism to act at the maximum opening degree by the signal so as to enable the braking deceleration to reach the physical limit value of a vehicle braking system; when the class label of the intruding obstacle is equipment, the movement rate is zero or the movement direction deviates from the expected travelling track curve, and the real-time distance between the vehicle and the intruding obstacle falls into a second braking threshold value interval, a low-level pulse width modulation signal is generated as an early warning braking signal, and the signal drives a hydraulic valve in a braking executing mechanism to act with partial opening, so that the braking deceleration is kept below a preset comfortable deceleration threshold value, and simultaneously an audible and visual alarm connected with a CAN bus of the vehicle is triggered to emit intermittent alarm sound and yellow flashing light.

Description

Intelligent auxiliary safety protection system and method for vehicle facing complex industrial scene Technical Field The invention relates to the technical field of image processing and vehicle safety protection, in particular to an intelligent auxiliary safety protection system and method for a vehicle facing a complex industrial scene. Background Along with the large-scale development of industrial production, a large number of large-scale loaders, bracket trucks, mining dump trucks and other engineering vehicles are operated in complex industrial scenes such as underground coal mines, surface mines and the like, the vehicles are huge in size and wide in blind area range, mobile personnel, auxiliary equipment and various sundries exist in the operation environment at the same time, and collision accidents are easy to occur, so that casualties and equipment damage are caused. In the prior art, a vehicle auxiliary safety protection system mainly relies on an ultrasonic radar or a millimeter wave radar to detect obstacles, however, in the environments with high dust, high humidity and poor illumination under a coal mine, ultrasonic signals are easy to be subjected to dust attenuation and roadway wall reflection interference, millimeter wave radar has weak recognition capability on static obstacles and cannot distinguish the types of the obstacles, so that a large number of false positives and false negatives are caused, meanwhile, the existing system usually adopts a circular or sector warning area with a fixed distance, cannot be dynamically adjusted according to the real-time running state of the vehicle, has poor adaptability in complex road sections such as curves, slopes and the like, is difficult to meet the precision requirement on the safety protection of the vehicle under complex industrial scenes, Disclosure of Invention The invention aims to provide an intelligent auxiliary safety protection system and method for a vehicle facing a complex industrial scene, so as to solve the problems in the background technology. In order to achieve the purpose, the invention provides the following technical scheme that the intelligent auxiliary safety protection method for the vehicle facing the complex industrial scene comprises the following steps: s1, collecting video stream data in the traveling direction of a vehicle; s2, identifying obstacles in an image frame in the video stream data, and determining position information and category information of the obstacles in an image coordinate system, wherein the position information and the category information appear in the image frame; S3, mapping the position information into a physical space coordinate system taking the vehicle as a center to generate space distribution information comprising the physical position and the category of the obstacle; S4, dynamically constructing a three-dimensional guard space matched with the predicted running path of the vehicle according to the current running parameters of the vehicle; S5, carrying out space relation analysis on the space distribution information and the three-dimensional guard space, and judging whether an invasive barrier which invades the three-dimensional guard space exists or not; S6, when an invasive obstacle exists, carrying out association tracking on the same invasive obstacle in the continuous multi-frame image frames to generate a motion track of the invasive obstacle in a physical space; S7, generating a graded braking control signal comprising braking time and braking intensity according to the motion trail and the type of the intrusion barrier; And S8, transmitting the graded braking control signal to a vehicle braking executing mechanism to trigger the vehicle to execute corresponding deceleration or braking operation. As a preferred technical solution of the present invention, the step S2 of identifying the obstacle for the image frame specifically includes: Inputting the image frames into a pre-trained deep convolutional neural network, performing step-by-step convolution and pooling operation on the image frames through a multi-layer convolution kernel group of the deep convolutional neural network, and extracting a layered visual feature map comprising edges, textures and component layers; the method comprises the steps of respectively accessing a layered visual feature map into a first parallel branch and a second parallel branch of the network, wherein the first parallel branch generates a candidate target frame through a regional proposal network, finely adjusts the position of the candidate target frame through a boundary frame regressive, and outputs a boundary frame marking the outline of an obstacle, and the second parallel branch carries out feature classification on an image region covered by the boundary frame through a full-connection layer and outputs an obstacle category label corresponding to the boundary frame, wherein the obstacle category label at least comprises personne