US-20260125080-A1 - METHODS AND ELECTRONIC DEVICES FOR CONTROLLING OPERATION OF A SELF-DRIVING CAR

US20260125080A1US 20260125080 A1US20260125080 A1US 20260125080A1US-20260125080-A1

Abstract

Methods and electronic devices for controlling operation of a self-driving car (SDC) are disclosed. The method includes receiving sensor data, generating a map of the environment using the sensor data, and generating a grid structure with a plurality of cells corresponding to respective portions of the map. A given cell is associated with a probability value indicative of a probability that an object is present in the respective portion of the map. The method includes, in response to the probability value being above a detection threshold: generating a bounding shape covering the given cell. The method includes, in response to the probability value being between the detection threshold and a second threshold: determining that an undetected object is potentially present. The method includes, in response to the determining that the undetected object is potentially present: triggering the SDC to perform a remedial action.

Inventors

Aleksey SOLOVYEV

Assignees

Y.E. Hub Armenia LLC

Dates

Publication Date: 20260507
Application Date: 20251008
Priority Date: 20241101

Claims (20)

1 . A method of controlling operation of a self-driving car (SDC), the method including: receiving sensor data about an environment of the SDC; generating, using a Neural Network (NN), a map of the environment using the sensor data; generating, using the NN, a grid structure with a plurality of cells corresponding to respective portions of the map, a given cell from the plurality of cells being associated with a probability value indicative of a probability that an object is present in the respective portion of the map; in response to the probability value being above a detection threshold: generating a bounding shape covering the given cell, the bounding shape being indicative of that a detected object is present in the respective portion of the map; in response to the probability value being between the detection threshold and a second threshold, the second threshold being inferior to the detection threshold: determining that an undetected object is potentially present in the respective portion of the map; and in response to the determining that the undetected object is potentially present in the respective portion of the map: triggering the SDC to perform a remedial action.
2 . The method of claim 1 , wherein the sensor data comprises first sensor data from a first sensor, and second sensor data from a second sensor.
3 . The method of claim 2 , wherein the first sensor data is a point cloud and the first sensor is a LIDAR sensor.
4 . The method of claim 2 , wherein the method further comprises generating fused sensor data by combining the first sensor data and the second sensor data, and wherein the generating the map of the environment comprises generating the map of the environment using the fused sensor data.
5 . The method of claim 1 , wherein the map of the environment is a Bird Eye View (BEV) map of the environment.
6 . The method of claim 1 , wherein the bounding shape is a bounding box.
7 . The method of claim 1 , wherein the remedial action is a reduction of speed of the SDC.
8 . The method of claim 1 , wherein the triggering the SDC to perform the remedial action is executed independently from one or more path planning operations.
9 . The method of claim 1 , wherein the generating the bounding shape comprises executing a Non-Maximum Suppression (NMS) algorithm onto a plurality of candidate bounding shapes.
10 . A method of controlling operation of a self-driving car (SDC), the method including: receiving sensor data about an environment of the SDC; generating, using a Neural Network, a map of the environment using the sensor data; generating, using the NN, a grid structure with a plurality of cells corresponding to respective portions of the map, the plurality of cells being associated with respective probability values indicative of a probability that an object is present in the respective portions of the map; executing a two-stage object detection process onto the grid structure, including: during a first stage: generating a bounding shape covering a first cell from the plurality of cells based on a first probability value of the first cell, the bounding shape being indicative of that a detected object is present in a first portion of the map corresponding to the first cell, the first cell being a bounded cell; during a second stage: determining that an undetected object is potentially present in a second portion of the map corresponding to a non-bounded cell based on a second probability value of the non-bounded cell; triggering control of the SDC based on the presence of the detected object in the first portion and the potential presence of the undetected object in the second portion.
11 . An electronic device for controlling operation of a self-driving car (SDC), the electronic device being configured to: receive sensor data about an environment of the SDC; generate, using a Neural Network (NN), a map of the environment using the sensor data; generate, using the NN, a grid structure with a plurality of cells corresponding to respective portions of the map, a given cell from the plurality of cells being associated with a probability value indicative of a probability that an object is present in the respective portion of the map; in response to the probability value being above a detection threshold: generate a bounding shape covering the given cell, the bounding shape being indicative of that a detected object is present in the respective portion of the map; in response to the probability value being between the detection threshold and a second threshold, the second threshold being inferior to the detection threshold: determine that an undetected object is potentially present in the respective portion of the map; and in response to determining that the undetected object is potentially present in the respective portion of the map: trigger the SDC to perform a remedial action.
12 . The electronic device of claim 11 , wherein the sensor data comprises first sensor data from a first sensor, and second sensor data from a second sensor.
13 . The electronic device of claim 12 , wherein the first sensor data is a point cloud and the first sensor is a LIDAR sensor.
14 . The electronic device of claim 12 , wherein the electronic device is further configured to generate fused sensor data by combining the first sensor data and the second sensor data, and wherein to generating the map of the environment comprises the electronic device configured to generate the map of the environment using the fused sensor data.
15 . The electronic device of claim 11 , wherein the map of the environment is a Bird Eye View (BEV) map of the environment.
16 . The electronic device of claim 11 , wherein the bounding shape is a bounding box.
17 . The electronic device of claim 11 , wherein the remedial action is a reduction of speed of the SDC.
18 . The electronic device of claim 11 , wherein to trigger the SDC to perform the remedial action comprises the electronic device to perform the remedial action independently from one or more path planning operations.
19 . The electronic device of claim 11 , wherein to generating the bounding shape comprises the electronic device configured to execute a Non-Maximum Suppression (NMS) algorithm onto a plurality of candidate bounding shapes.
20 . The electronic device of claim 11 , wherein the electronic device is a local electronic device of the SDC.

Description

CROSS-REFERENCE The present application claims priority to Russian Patent Application No. 2024132913, entitled “Methods and Electronic Devices for Controlling Operation of a Self-Driving Car”, filed Nov. 1, 2024, the entirety of which is incorporated herein by reference. TECHNICAL FIELD The present technology relates generally to autonomous driving, and more particularly, to methods and electronic devices for controlling operation of a Self-Driving Car (SDC). BACKGROUND Autonomous driving is a technology that enables a vehicle to drive itself without human (or with little) human intervention by using various sensors, computer systems, and algorithms. For example, some sensors used for autonomous driving include inter alia cameras, lidars, radars, and GPS. Cameras are optical devices that capture images of the surrounding environment. They can provide visual information such as color, texture, shape, and motion of the objects in the scene. Cameras can also recognize road signs, traffic lights, and lane markings. A lidar is a sensor that emits laser beams and measures the time it takes for them to bounce back from the objects in the environment. Lidars can create a 3D point cloud that represents the shape, size, and location of the objects in the scene. Lidars can also measure the distance and velocity of the objects. A radar is a sensor that emits radio waves and measures the time it takes for them to bounce back from the objects in the environment. GPS is a system that uses satellites to determine the geographic location and altitude of the vehicle. GPS can provide coarse information about the position and orientation of the vehicle. In order to enable autonomous driving, a computer system needs to perform at least three functions: perception, planning, and control. These functions can be implemented via separate computer modules that communicate and cooperate with each other to achieve the desired behavior of the vehicle. Each module can use different sensors, models, and algorithms to perform its respective tasks depending on inter alia the level of autonomy and the requirements of a given scenario. Designing a system to safely drive a vehicle autonomously is difficult. An autonomous vehicle should be capable of performing as a functional equivalent of an attentive driver who draws upon a perception and action system that has an incredible ability to identify and react to moving and static obstacles in a complex environment, to avoid colliding with other objects or structures along the path of the vehicle. Thus, the ability to detect instances of animate (e.g., objects cars, pedestrians, etc.) and other parts of an environment is necessary for autonomous driving perception systems. Conventional perception methods rely on cameras or lidar sensors to detect objects in an environment, and a variety of approaches have been developed using Deep Neural Networks (DNNs) to perform object detection. Some DNNs perform “Bird's Eye View” (BEV) object detection. A BEV map is a result of transforming a multi-dimensional representation of the surroundings into a 2D image that shows the scene from a top-down perspective. This can help to reduce the complexity of the data and make it easier to apply computer vision techniques for object detection and localization. US Patent Publication 2022/0289237 discloses a map-free generic obstacle detection for collision avoidance systems. SUMMARY Developers of the present technology have realized at least some drawbacks with known solutions for object detection in an environment of a Self-Driving Car (SDC). Generally speaking, an object detection module of a SDC is configured to inter alia locate and classify objects in the environment of the SDC. It can be said that an object is “detected” when the object detection module generates a bounding shape for a portion of a map of the environment. The object detection module may also assign a label/class to the bounding shape indicative of a class of object located in the corresponding portion of the map. Initially, the object detection module gathers data from a variety of sensors, such as cameras, lidars, and radars, for example, and which is indicative of a vehicle's surroundings. This data may undergo pre-processing to correct distortions and/or remove noise, ensuring that the information is accurate and synchronized across different sensor types. Data from different sensors can be combined or “fused” in a combined representation of the surroundings. This combined representation may include a plurality of features such as edges, shapes, colors, and patterns, for example, and which can be used for distinguishing objects in the environment. This combined representation including a plurality of features is analyzed by a Neural Network (NN). The NN is configured to generate a grid structure to discretize the combined representation and uses features to assign probabilities to respective cells of the grid structure, indicative of a likelih