Search

US-20260126524-A1 - OBJECT DETECTION USING RADAR SENSORS

US20260126524A1US 20260126524 A1US20260126524 A1US 20260126524A1US-20260126524-A1

Abstract

Provided are methods for generation of representations of radar data. Some methods described include: receiving ADC raw data of a radar sensor of a vehicle; performing range FFT, Doppler FFT, and azimuth FFT on the ADC raw data; generating a 1D range heat map tensor representing the range FFT, a 2D RD heat map tensor representing a combination of the range FFT and the Doppler FFT, a 2D RA heat map tensor representing a combination of the range FFT and the azimuth FFT, or a 3D RAD matrix tensor representing a combination of the range FFT, the Doppler FFT, and the azimuth FFT; and inputting at least one of the 1D range heat map tensor, the 2D RD heat map tensor, the 2D RA heat map tensor, or the 3D RAD matrix tensor, to a machine learning model for detecting objects on a road network around the vehicle.

Inventors

  • Ting Wang
  • Yun Lin
  • Ken Power

Assignees

  • MOTIONAL AD LLC

Dates

Publication Date
20260507
Application Date
20251229

Claims (17)

  1. 1 . A method, comprising: receiving, by at least one processor, analog-to-digital converter (ADC) raw data of a radar sensor of a vehicle; performing, by the at least one processor, at least one of range fast Fourier transform (FFT), Doppler FFT, and azimuth FFT on the ADC raw data; generating, by the at least one processor, (a) a one-dimensional (1D) range heat map tensor representing the range FFT, (b) a two-dimensional (2D) range-Doppler (RD) heat map tensor representing a combination of the range FFT and the Doppler FFT, (c) a 2D range-azimuth (RA) heat map tensor representing a combination of the range FFT and the azimuth FFT, or (d) a three-dimensional (3D) range-azimuth-Doppler (RAD) matrix tensor representing a combination of the range FFT, the Doppler FFT, and the azimuth FFT; and inputting, by the at least one processor, at least one of the 1D range heat map tensor, the 2D RD heat map tensor, the 2D RA heat map tensor, or the 3D RAD matrix tensor, to a machine learning model for detecting objects on a road network around the vehicle.
  2. 2 . The method of claim 1 , wherein inputting the 1D range heat map tensor, the 2D RD heat map tensor, the 2D RA heat map tensor, or the 3D RAD matrix tensor, to the machine learning model further comprises: receiving camera images from a camera of the vehicle; fusing the camera images with the 1D range heat map tensor, the 2D RD heat map tensor, the 2D RA heat map tensor, or the 3D RAD matrix tensor; and inputting fused data to the machine learning model.
  3. 3 . The method of claim 1 , wherein the 3D RAD matrix tensor includes the azimuth FFT representing a quantity of ADC channels, the Doppler FFT representing a quantity of chirps in each ADC channel, and the range FFT representing a quantity of samples per chirp.
  4. 4 . The method of claim 3 , wherein a size of the 3D RAD matrix tensor is configured based on one or more of sensing range, distance resolution, velocity resolution, angular resolution, a bandwidth of a chirp, a period of the chirp, the quantity of samples per chirp or a sampling rate, or the quantity of ADC channels.
  5. 5 . The method of claim 1 , wherein the 2D RD heat map tensor includes the Doppler FFT representing a quantity of chirps in each ADC channel and the range FFT representing a quantity of samples per chirp.
  6. 6 . The method of claim 5 , wherein a size of the 2D RD heat map tensor is configured based on one or more of a sensing range, a distance resolution, a velocity resolution, a bandwidth of a chirp, a period of the chirp, the quantity of samples per chirp, or a sampling rate.
  7. 7 . The method of claim 1 , wherein the 2D RA heat map tensor includes the azimuth FFT representing a quantity of ADC channels and the range FFT representing a quantity of samples per chirp.
  8. 8 . The method of claim 7 , wherein a size of the 2D RA heat map tensor is configured based on one or more of a sensing range, a distance resolution, an angular resolution, a bandwidth of the chirp, or the quantity of ADC channels.
  9. 9 . The method of claim 1 , wherein the machine learning model comprises a detection head and a segmentation head.
  10. 10 . A system, comprising: at least one processor; and a memory storing instructions thereon that, when executed by the at least one processor, cause the at least one processor to perform operations, comprising: receiving analog-to-digital converter (ADC) raw data of a radar sensor of a vehicle; performing range fast Fourier transform (FFT), Doppler FFT, and azimuth FFT on the ADC raw data; generating a three-dimensional (3D) range-azimuth-Doppler (RAD) matrix tensor representing a combination of the range FFT, the Doppler FFT, and the azimuth FFT; and inputting the 3D RAD matrix tensor to a machine learning model for detecting objects on a road network around the vehicle.
  11. 11 . The system of claim 10 , wherein inputting the 3D RAD matrix tensor to the machine learning model further comprises: receiving camera images from a camera of the vehicle; fusing the camera images with the 3D RAD matrix tensor; and inputting fused data to the machine learning model.
  12. 12 . The system of claim 11 , wherein the 3D RAD matrix tensor includes the azimuth FFT representing a quantity of ADC channels, the Doppler FFT representing a quantity of chirps in each ADC channel, and the range FFT representing a quantity of samples per chirp.
  13. 13 . The system of claim 12 , wherein a size of the 3D RAD matrix tensor is configured based on one or more of a sensing range, a distance resolution, a velocity resolution, an angular resolution, a bandwidth of a chirp, a period of the chirp, the quantity of samples per chirp or a sampling rate, or the quantity of ADC channels.
  14. 14 . A non-transitory, computer-readable storage medium having instructions stored thereon, that when executed by at least one processor, cause the at least one processor to perform operations, comprising: receiving analog-to-digital converter (ADC) raw data of a radar sensor of a vehicle; performing range fast Fourier transform (FFT), Doppler FFT, and azimuth FFT on the ADC raw data; generating a three-dimensional (3D) range-azimuth-Doppler (RAD) matrix tensor representing a combination of the range FFT, the Doppler FFT, and the azimuth FFT; and inputting the 3D RAD matrix tensor to a machine learning model for detecting objects on a road network around the vehicle.
  15. 15 . The computer-readable storage medium of claim 14 , wherein inputting the 3D RAD matrix tensor to the machine learning model further comprises: receiving camera images from a camera of the vehicle; fusing the camera images with the 3D RAD matrix tensor; and inputting fused data to the machine learning model.
  16. 16 . The computer-readable storage medium of claim 14 , wherein the 3D RAD matrix tensor includes the azimuth FFT representing a quantity of ADC channels, the Doppler FFT representing a quantity of chirps in each ADC channel, and the range FFT representing a quantity of samples per chirp.
  17. 17 . The computer-readable storage medium of claim 16 , wherein a size of the 3D RAD matrix tensor is configured based on one or more of a sensing range, a distance resolution, a velocity resolution, an angular resolution, a bandwidth of a chirp, a period of the chirp, the quantity of samples per chirp or a sampling rate, or the quantity of ADC channels.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a division of, and claims priority to, U.S. patent application Ser. No. 18/105,183, filed Feb. 2, 2023, which claims priority to U.S. Provisional Patent Application No. 63/416,459, filed Oct. 14, 2022, the entire contents of each of which are incorporated herein by reference. BACKGROUND Radar sensors transmit electromagnetic wave signals that are reflected by objects in the environment. Radar sensors capture the reflected signals, and the reflected signals are processed to determine various properties of the environment. BRIEF DESCRIPTION OF THE FIGURES FIG. 1 is an example environment in which a vehicle including one or more components of an autonomous system can be implemented; FIG. 2 is a diagram of one or more systems of a vehicle including an autonomous system; FIG. 3 is a diagram of components of one or more devices and/or one or more systems of FIGS. 1 and 2; FIG. 4 is a diagram of certain components of an autonomous system; FIG. 5 is a diagram of an implementation of a process that detects objects using a sensor suite; FIGS. 6 (a)-(c) are different example pipelines of a radar sensor; FIGS. 7 (a)-(d) are different example representations of radar data; FIG. 8 is a diagram illustrating generation of an example Range-Azimuth-Doppler (RAD) matrix tensor representing radar data; FIGS. 9 (a)-(c) are different example pipelines of sensor data fusion; and FIG. 10 is an example flow chart of a process for generation of representations of radar data. DETAILED DESCRIPTION In the following description numerous specific details are set forth in order to provide a thorough understanding of the present disclosure for the purposes of explanation. It will be apparent, however, that the embodiments described by the present disclosure can be practiced without these specific details. In some instances, well-known structures and devices are illustrated in block diagram form in order to avoid unnecessarily obscuring aspects of the present disclosure. Specific arrangements or orderings of schematic elements, such as those representing systems, devices, modules, instruction blocks, data elements, and/or the like, are illustrated in the drawings for ease of description. However, it will be understood by those skilled in the art that the specific ordering or arrangement of the schematic elements in the drawings is not meant to imply that a particular order or sequence of processing, or separation of processes, is required unless explicitly described as such. Further, the inclusion of a schematic element in a drawing is not meant to imply that such element is required in all embodiments or that the features represented by such element may not be included in or combined with other elements in some embodiments unless explicitly described as such. Further, where connecting elements such as solid or dashed lines or arrows are used in the drawings to illustrate a connection, relationship, or association between or among two or more other schematic elements, the absence of any such connecting elements is not meant to imply that no connection, relationship, or association can exist. In other words, some connections, relationships, or associations between elements are not illustrated in the drawings so as not to obscure the disclosure. In addition, for ease of illustration, a single connecting element can be used to represent multiple connections, relationships, or associations between elements. For example, where a connecting element represents communication of signals, data, or instructions (e.g., “software instructions”), it should be understood by those skilled in the art that such element can represent one or multiple signal paths (e.g., a bus), as may be needed, to affect the communication. Although the terms first, second, third, and/or the like are used to describe various elements, these elements should not be limited by these terms. The terms first, second, third, and/or the like are used only to distinguish one element from another. For example, a first contact could be termed a second contact and, similarly, a second contact could be termed a first contact without departing from the scope of the described embodiments. The first contact and the second contact are both contacts, but they are not the same contact. The terminology used in the description of the various described embodiments herein is included for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well and can be used interchangeably with “one or more” or “at least one,” unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will