Search

CN-122024191-A - Image key point-based method and device for detecting drivable area and vehicle

CN122024191ACN 122024191 ACN122024191 ACN 122024191ACN-122024191-A

Abstract

The invention discloses a method and a device for detecting a drivable area based on image key points and a vehicle. And extracting the characteristics of the RGB image by using a preset model, and outputting a characteristic diagram. And calculating parameter information of the feature map, wherein the parameter information comprises a confidence thermodynamic diagram, a class map and an offset. And screening out key points from each column of pixel points of the feature map according to the parameter information. And sequentially connecting the plurality of key points screened from each column of pixel points to generate a 2D travelable area boundary. And projecting the 2D travelable region boundary into a 3D space coordinate system by utilizing a preset transformation rule, and generating the space coordinate of the 3D travelable region boundary. Compared with the prior art, the method for detecting the drivable region based on the image key points has the remarkable advantages of accuracy, robustness, calculation efficiency and the like.

Inventors

  • GUO XINYANG
  • YANG YINGBIN
  • WANG HONGHUI

Assignees

  • 城市之光(深圳)无人驾驶有限公司

Dates

Publication Date
20260512
Application Date
20251226

Claims (10)

  1. 1. The method for detecting the drivable region based on the image key points is characterized by comprising the following steps of: acquiring RGB images acquired by a vehicle-mounted camera of a vehicle; extracting features of the RGB image by using a preset model, and outputting a feature map; calculating parameter information of the feature map, wherein the parameter information comprises a confidence thermodynamic diagram, a class map and an offset; According to the parameter information, key points are screened out from each column of pixel points of the feature map; Sequentially connecting a plurality of key points screened from each column of pixel points to generate a 2D drivable area boundary; Projecting the 2D travelable region boundary into a 3D space coordinate system by using a preset transformation rule to generate space coordinates of the 3D travelable region boundary; A travelable region of the vehicle is determined based on the spatial coordinates of the 3D travelable region boundary.
  2. 2. The method for detecting a travelable region based on image key points as claimed in claim 1, wherein: and extracting the characteristics of the RGB image by using a preset model, namely extracting the characteristics of the RGB image by using a deep neural network, and outputting a downsampled characteristic map, wherein the downsampling multiplying power is 8 or 16 times.
  3. 3. The method for detecting a travelable region based on image key points as claimed in claim 1, wherein: The activation function of the confidence coefficient thermodynamic diagram is a Sigmoid function, and the output value is 0 and 1, which is used for indicating the confidence coefficient of whether the corresponding area of the RGB image corresponding to each pixel point in the feature diagram is a vehicle drivable area.
  4. 4. The method for detecting a travelable region based on image key points as claimed in claim 3, wherein the key points are screened out from each column of pixel points of the feature map according to the parameter information, specifically comprising: Dividing the feature map into a plurality of longitudinal columns, wherein each column comprises a plurality of pixel points; according to the confidence thermodynamic diagram, the pixel point with the highest confidence is selected from each column of pixel points to be used as a key point; generating sub-pixel offset of the key point based on the offset; and adding the sub-pixel offset to the ordinate of the key point.
  5. 5. The method for detecting a travelable region based on image keypoints as claimed in claim 3, wherein the method for generating a 2D travelable region boundary comprises the steps of: Connecting a plurality of key points screened from each column of pixel points in sequence in the transverse direction to generate a 2D travelable area boundary; And giving corresponding category information to each key point according to the category map, and forming category information of the boundary of the 2D drivable area.
  6. 6. The method for detecting a travelable region based on image key points as claimed in claim 1, wherein: The activation function of the class diagram is a Softmax function, and probability distribution representing the class to which each pixel belongs is output.
  7. 7. The method for detecting a travelable region based on image key points as claimed in claim 1, wherein: the preset transformation rule adopts an inverse perspective transformation rule.
  8. 8. A drivable region detection apparatus based on image key points, characterized in that the drivable region detection apparatus comprises: the acquisition module is used for acquiring RGB images acquired by the vehicle-mounted camera of the vehicle; The extraction module is used for carrying out feature extraction on the RGB image by utilizing a preset model and outputting a feature map; The computing module is used for computing parameter information of the feature map, wherein the parameter information comprises a confidence thermodynamic diagram, a class map and an offset; The screening module is used for screening key points from each column of pixel points of the feature map according to the parameter information; the generation module is used for sequentially connecting a plurality of key points screened from each column of pixel points to generate a 2D travelable area boundary; the transformation module is used for projecting the 2D drivable area boundary into a 3D space coordinate system by utilizing a preset transformation rule, and generating space coordinates of the 3D drivable area boundary; and the output module is used for determining the drivable area of the vehicle based on the space coordinates of the boundary of the 3D drivable area.
  9. 9. An electronic device, comprising: At least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform a travelable region detection method based on image keypoints as claimed in any one of claims 1-7.
  10. 10. A vehicle characterized in that it comprises the travelable region detection apparatus as claimed in claim 8 or the vehicle comprises the electronic device as claimed in claim 9.

Description

Image key point-based method and device for detecting drivable area and vehicle Technical Field The invention relates to the technical field of automatic driving, in particular to a method and a device for detecting a drivable area based on image key points and a vehicle. Background With the development of automatic driving technology, unmanned vehicles have increasingly high requirements on environmental perception technology. The detection of the drivable area is a vital part of an automatic driving system, directly affecting the safety, driving accuracy and navigation ability of the vehicle. The existing detection method of the travelable region mainly comprises a detection method based on semantic segmentation, laser radar and image fusion and traditional characteristics. Although the existing detection methods of the drivable area improve the environment sensing capability of the intelligent driving system to a certain extent, the methods generally have the problems of insufficient precision, poor robustness and large consumption of computing resources, and are to be improved. Disclosure of Invention In order to solve the technical problems of insufficient precision, poor robustness and high consumption of calculation resources in the prior art, the invention aims to provide a method and a device for detecting a drivable area based on image key points and a vehicle, which have remarkable advantages in the aspects of precision, robustness, calculation efficiency and the like, can provide a more reliable, efficient and low-cost solution for an automatic driving system, and promote the wide application of intelligent driving technology. The aim of the invention is achieved by the following technical scheme: In a first aspect, the present invention provides a method for detecting a drivable region based on an image key point, the method comprising: acquiring RGB images acquired by a vehicle-mounted camera of a vehicle; extracting features of the RGB image by using a preset model, and outputting a feature map; calculating parameter information of the feature map, wherein the parameter information comprises a confidence thermodynamic diagram, a class map and an offset; According to the parameter information, key points are screened out from each column of pixel points of the feature map; Sequentially connecting a plurality of key points screened from each column of pixel points to generate a 2D drivable area boundary; Projecting the 2D travelable region boundary into a 3D space coordinate system by using a preset transformation rule to generate space coordinates of the 3D travelable region boundary; A travelable region of the vehicle is determined based on the spatial coordinates of the 3D travelable region boundary. In a possible implementation manner, the feature extraction is performed on the RGB image by using a preset model, specifically, the feature extraction is performed on the RGB image by using a deep neural network, a downsampled feature map is output, and the downsampling magnification is 8 or 16 times. In one possible implementation manner, the activation function of the confidence thermodynamic diagram is a Sigmoid function, and the output value is [0,1] for indicating the confidence that whether the corresponding area of the RGB image corresponding to each pixel point in the feature diagram is a vehicle drivable area. In one possible implementation manner, according to the parameter information, the key points are screened from each column of pixel points of the feature map, and specifically include: Dividing the feature map into a plurality of longitudinal columns, wherein each column comprises a plurality of pixel points; according to the confidence thermodynamic diagram, the pixel point with the highest confidence is selected from each column of pixel points to be used as a key point; generating sub-pixel offset of the key point based on the offset; and adding the sub-pixel offset to the ordinate of the key point. In one possible implementation manner, the method for generating the 2D travelable region boundary sequentially connects a plurality of key points screened from each column of pixel points specifically includes: Connecting a plurality of key points screened from each column of pixel points in sequence in the transverse direction to generate a 2D travelable area boundary; And giving corresponding category information to each key point according to the category map, and forming category information of the boundary of the 2D drivable area. In a possible implementation manner, the activation function of the class diagram is a Softmax function, and a probability distribution representing a class to which each pixel point belongs is output. In one possible implementation, the preset transformation rule employs an inverse perspective transformation rule. In a second aspect, the present invention provides a drivable region detection apparatus based on image key points, the drivable region