CN-121979202-A - Remote control driving safety control method, device, medium and vehicle

CN121979202ACN 121979202 ACN121979202 ACN 121979202ACN-121979202-A

Abstract

The application relates to the technical field of remote control driving safety control, and particularly discloses a remote control driving safety control method, a device, a medium and a vehicle. The method comprises the steps of firstly merging three sensor data acquired by an image, a first point cloud device and a second point cloud device, carrying out space-time synchronization and a coordinate system on the three sensor data, then identifying whether a medium which interferes with the detection capability of the first point cloud acquisition device exists or not based on the image, dividing an unreliable area from the first point cloud data according to the medium, preferentially using second point cloud data with better revenues to detect obstacles in the area, preferentially using first point cloud data with higher precision to detect the other areas, and finally combining two detection results to determine the position and the form of the obstacles so as to generate a vehicle motion control instruction, thereby improving the perception reliability and the driving safety in complex scene of fire such as dense smoke.

Inventors

XIE JIAYOU
ZHU GUANGYOU
ZHANG HUAQI
XIONG SHUNJIN
XU JIALONG

Assignees

湖南中联重科应急装备有限公司

Dates

Publication Date: 20260505
Application Date: 20251226

Claims (11)

1. A remote control driving safety control method, characterized by comprising: acquiring environment image data, first point cloud data and second point cloud data respectively acquired by image acquisition equipment, first point cloud acquisition equipment and second point cloud acquisition equipment; Carrying out data space-time synchronization and coordinate system unified processing on each acquisition device so as to correlate data from different sources under the same space-time reference; based on the environmental image data, identifying whether a medium causing interference to the detection capability of the first point cloud acquisition device exists; Distinguishing an untrusted point cloud area corresponding to the medium from the first point cloud data according to the spatial distribution information of the medium in the environment image data; in the space range corresponding to the unreliable point cloud area, performing obstacle detection based on the second point cloud data preferentially to obtain a first detection result; in the space range except the unreliable point cloud area, performing obstacle detection based on the first point cloud data preferentially to obtain a second detection result; and synthesizing the first detection result and the second detection result to generate a corresponding vehicle motion control instruction.
2. The method of claim 1, wherein performing data space-time synchronization and coordinate system integration on each acquisition device to correlate data from different sources to the same space-time reference comprises: Synchronizing clock references of the image acquisition equipment, the first point cloud acquisition equipment and the second point cloud acquisition equipment based on an external clock source; Selecting a plurality of feature point calculation feature vectors in space based on the first point cloud data and the second point cloud data, and determining a space transformation matrix from a coordinate system where the second point cloud data is positioned to a coordinate system of the first point cloud data; downsampling the first point cloud data through direct filtering and voxel filtering, and determining the processed first point cloud data; And according to the space transformation matrix, the second point cloud data, the processed first point cloud data and a coordinate system where the projection is located on the environment image data.
3. The security control method according to claim 2, wherein selecting a plurality of feature point calculation feature vectors in space based on the first point cloud data and the second point cloud data, determining a space transformation matrix from a coordinate system where the second point cloud data is located to a coordinate system of the first point cloud data, includes: selecting at least three non-collinear space feature points from the first point cloud data, constructing a local coordinate system and calculating three orthogonal feature vectors corresponding to the local coordinate system; identifying homonymous feature points corresponding to the feature point space positions in the first point cloud data in the second point cloud data, constructing a local coordinate system based on the homonymous feature points, and calculating three corresponding orthogonal feature vectors; and according to the corresponding relation between the two sets of feature vectors, solving a rotation matrix and a translation vector which align the two sets of coordinate systems, thereby obtaining a space transformation matrix for transforming the second point cloud data coordinate system to the first point cloud data coordinate system.
4. The security control method of claim 1, wherein the identifying whether a medium that interferes with the detection capability of the first point cloud acquisition device exists based on the environmental image data comprises: Detecting the environmental image data based on a pre-trained smoke recognition neural network; And under the condition that the smoke confidence degree output by the neural network exceeds a preset threshold, judging that a medium which causes interference to the detection capability of the first point cloud acquisition equipment exists.
5. The security control method according to claim 1, wherein the distinguishing the untrusted point cloud region corresponding to the medium from the first point cloud data according to the spatial distribution information of the medium in the environment image data includes: Determining a smoke point cloud boundary based on echo intensities in the first point cloud data if it is determined that a medium causing interference to the detection capability of the first point cloud acquisition device exists; calculating the space position and the range of a smoke area based on the smoke point cloud boundary, and determining the minimum circumscribed bounding box of the smoke area; and taking the minimum circumscribed bounding box of the smoke area as the unreliable point cloud area.
6. The security control method according to claim 1, wherein in the spatial range corresponding to the untrusted point cloud area, obstacle detection is preferentially performed based on the second point cloud data, and obtaining a first detection result includes: acquiring the second point cloud data of the current moment and the previous historical moment; Based on vehicle pose information, uniformly transforming the second point cloud data of the historical moment to a coordinate system of the current moment to form multi-frame fusion point clouds; Extracting obstacle point cloud data of obstacles behind the smoke; And clustering the multi-frame fusion point cloud based on a multi-frame joint clustering method, calculating the position, the azimuth and the boundary information of the obstacle, and projecting the obstacle point cloud data under a vehicle coordinate system by combining the position and the course angle change of the vehicle at the current moment and the previous historical moment to generate the first detection result.
7. The security control method according to claim 1, wherein the performing obstacle detection based on the first point cloud data preferentially in the spatial range other than the untrusted point cloud area, and obtaining a second detection result includes: Determining coordinates of a center point and a boundary point of the obstacle based on the first point cloud data; and determining the second detection result according to the coordinates of the central point and the boundary point.
8. The safety control method according to claim 1, wherein the generating a corresponding vehicle motion control instruction by integrating the first detection result and the second detection result includes: And determining the height of the obstacle based on the first detection result and the second detection result, judging the distance between the vehicle and the obstacle under the condition that the height of the obstacle is higher than a preset height threshold value, and generating a vehicle motion control instruction by combining the vehicle braking state and the vehicle speed state.
9. A remote control driving safety control device, characterized by comprising: a memory configured to store instructions; A processor configured to invoke the instructions from the memory and when executing the instructions is capable of implementing the remote control driving safety control method according to any one of claims 1 to 8.
10. A machine-readable storage medium having stored thereon instructions for causing a machine to perform the remote driving safety control method according to any one of claims 1 to 8.
11. A remotely piloted vehicle, comprising: a remote control driving safety control device for executing a remote control driving safety control method; And a controller for controlling the vehicle speed.

Description

Remote control driving safety control method, device, medium and vehicle Technical Field The application relates to the technical field of remote control driving safety control, in particular to a remote control driving safety control method, a device, a medium and a vehicle. Background In the fire scene rescue task, adverse factors such as extremely complex environment, dense smoke, high temperature, low visibility and the like seriously threaten the life safety of firefighters. In order to reduce the risk of casualties and improve the rescue efficiency, the development of remote control driving technology in a fire scene becomes an important direction at present. By remotely controlling or semi-autonomously driving the fire-fighting vehicle, the fire-extinguishing and investigation tasks can be executed in dangerous areas which cannot be accessed by personnel, so that a safe operation mode of 'man-machine separation and remote control' is realized. However, existing remote control driving systems generally rely on a single sensor, which is difficult to adapt to complex scenes or conditions. If a visible light camera or a laser radar is used for environment sensing, the performance is severely limited under severe conditions such as dense smoke, the visible light camera can not image due to shielding of smoke, the laser radar point Yun Yi is interfered by scattering of smoke, and the traditional millimeter wave Lei Dadian cloud is sparse and has insufficient precision, so that the position and the form of an obstacle can not be accurately identified. Although a part of systems are provided with a plurality of sensors, due to lack of a data fusion mechanism, the data of each sensor are not aligned in time and space, and trust weights are not dynamically adjusted according to scenes, so that the system has low reliability of obstacle detection and poor system adaptability. Therefore, a remote control driving safety control technology which can integrate advantages of a multi-source sensor and adapt to complex fire scene environments is needed to achieve all-weather and full-scene obstacle detection capability of the fire engine, and safety of remote control driving in the fire scene is improved. Disclosure of Invention The embodiment of the application aims to provide a remote control driving safety control method, a remote control driving safety control device, a remote control driving safety control medium and a vehicle, which are used for solving the problems that in the prior art, under a scene with limited visual field, the detection capability of an obstacle is insufficient, the scene adaptability is poor, and the anti-collision detection mechanism of the vehicle is difficult to exert an effect. In order to achieve the above object, a first aspect of the present application provides a remote control driving safety control method, including: acquiring environment image data, first point cloud data and second point cloud data respectively acquired by image acquisition equipment, first point cloud acquisition equipment and second point cloud acquisition equipment; Carrying out data space-time synchronization and coordinate system unified processing on each acquisition device so as to correlate data from different sources under the same space-time reference; Based on the environmental image data, identifying whether a medium causing interference to the detection capability of the first point cloud acquisition device exists; distinguishing an untrusted point cloud area corresponding to the medium from the first point cloud data according to the spatial distribution information of the medium in the environment image data; in a space range corresponding to the unreliable point cloud area, performing obstacle detection based on the second point cloud data preferentially to obtain a first detection result; In the space range except for the unreliable point cloud area, performing obstacle detection based on the first point cloud data preferentially to obtain a second detection result; and generating a corresponding vehicle motion control instruction by integrating the first detection result and the second detection result. In the embodiment of the application, data space-time synchronization and coordinate system unified processing are carried out on each acquisition device so as to correlate data from different sources to the same space-time reference, wherein the method comprises the steps of synchronizing clock references of an image acquisition device, a first point cloud acquisition device and a second point cloud acquisition device based on an external clock source, selecting a plurality of space internal characteristic point calculation characteristic vectors based on the first point cloud data and the second point cloud data, determining a space transformation matrix from a coordinate system where the second point cloud data is located to the coordinate system of the first point cloud data, carrying out