CN-122029763-A - Camera triggering based on GPTP time synchronization

Abstract

Methods for triggering a camera in an Autonomous Vehicle (AV). The methods include receiving universal precision time protocol (gPTP) information with a programmable circuit, gPTP information for synchronizing the programmable circuit with a gPTP master clock (GM), generating a Pulse Per Second (PPS) signal having a period of one second based on gPTP time information with the programmable circuit, generating a synchronization signal having a period less than the period of the PPS signal with the programmable circuit, generating a trigger signal based on the PPS signal with the programmable circuit, wherein the period of the trigger signal is the same as the period of the synchronization signal, and transmitting a trigger signal to one or more cameras of an AV to trigger the one or more cameras to capture one or more images with the programmable circuit. Systems and computer program products are also provided.

Inventors

• S. Menon
• P. Fuzu Enmaye

Assignees

• 动态AD有限责任公司

Dates

Publication Date

20260512

Application Date

20240820

Priority Date

20230821

Claims (18)

1. 1.A method, comprising: receiving common precision time protocol information, gPTP information, with a programmable circuit, the gPTP information being used to synchronize the programmable circuit with a gPTP master clock, gPTP GM; Generating a pulse-per-second signal, PPS, signal having a period of one second based on gPTP time information using the programmable circuit; Generating, with the programmable circuit, a synchronization signal having a period less than a period of the PPS signal, wherein respective PPS signal pulses correspond to a set of synchronization signal pulses; Generating a trigger signal based on the PPS signal using the programmable circuit, wherein the trigger signal has a period identical to the period of the synchronization signal, and With the programmable circuit, the trigger signal for triggering the one or more cameras to capture one or more images is sent to one or more cameras of an autonomous vehicle, i.e. an AV.
2. 2. The method of claim 1, wherein the programmable circuit is a field programmable gate array, FPGA.
3. 3. The method of claim 1, wherein a rising edge of a respective trigger signal pulse is synchronized with a start of each scan of the one or more LiDAR sensors of the AV.
4. 4. The method of claim 3, wherein the PPS signal has a period that is X times a period of the synchronization signal, wherein X is a number of revolutions per second of the one or more LiDAR sensors, wherein the respective PPS signal pulses correspond to X synchronization signal pulses.
5. 5. The method of claim 4, wherein the PPS signal has a period at least 20 times that of the synchronization signal, and the respective PPS signal pulses correspond to at least 20 synchronization signal pulses.
6. 6. The method of claim 1, wherein a rising edge of the respective PPS signal pulse is synchronized with a rising edge of a first pulse in the set of synchronization signal pulses.
7. 7. The method of claim 1, wherein the synchronization signal is generated using a state machine comprising an idle state, a deferred state, an on-standby state, and a triggered state.
8. 8. The method of claim 7, wherein the idle state is switched to the triggered state upon arrival of the PPS signal pulse.
9. 9. The method of claim 8, wherein the triggered state is switched to the waiting state after a first predetermined period of time.
10. 10. The method of claim 9, wherein the first predetermined period of time is at least 10 nanoseconds.
11. 11. The method of claim 9, wherein the waiting state is switched to the triggered state if (i) the PPS signal pulse arrives and a time since generation of a latest synchronization signal is greater than or equal to a second predetermined period of time, or (ii) a time since generation of the latest synchronization signal is greater than or equal to a third predetermined period of time, wherein the second predetermined period of time is less than the third predetermined period of time.
12. 12. The method of claim 11, wherein the second predetermined period of time is at least 25 milliseconds and the third predetermined period of time is at least 50 milliseconds.
13. 13. The method of claim 9, wherein the waiting state is switched to the deferred state in the event that the PPS signal pulse arrives and a time since generation of a latest synchronization signal is less than a second predetermined period of time.
14. 14. The method of claim 13, wherein the deferred state is switched to the waiting state after the first predetermined period of time.
15. 15. The method of claim 13, wherein the first predetermined period of time is at least 10 nanoseconds and the second predetermined period of time is at least 25 milliseconds.
16. 16. The method of claim 1, further comprising applying a time offset to the synchronization signal to generate the trigger signal, wherein the time offset is a fixed period of time.
17. 17. A system, comprising: at least one programmable circuit configured to perform operations comprising: receiving common precision time protocol information, gPTP information, with a programmable circuit, the gPTP information being used to synchronize the programmable circuit with a gPTP master clock, gPTP GM; Generating a pulse-per-second signal, PPS, signal having a period of one second based on gPTP time information using the programmable circuit; Generating, with the programmable circuit, a synchronization signal having a period less than a period of the PPS signal, wherein respective PPS signal pulses correspond to a set of synchronization signal pulses; Generating a trigger signal based on the PPS signal using the programmable circuit, wherein the trigger signal has a period identical to the period of the synchronization signal, and With the programmable circuit, the trigger signal for triggering the one or more cameras to capture one or more images is sent to one or more cameras of an autonomous vehicle, i.e. an AV.
18. 18. A non-transitory computer-readable storage medium having instructions stored thereon, which when executed by programmable circuitry, cause the programmable circuitry to perform operations comprising: receiving common precision time protocol information, gPTP information, with a programmable circuit, the gPTP information being used to synchronize the programmable circuit with a gPTP master clock, gPTP GM; Generating a pulse-per-second signal, PPS, signal having a period of one second based on gPTP time information using the programmable circuit; Generating, with the programmable circuit, a synchronization signal having a period less than a period of the PPS signal, wherein respective PPS signal pulses correspond to a set of synchronization signal pulses; Generating a trigger signal based on the PPS signal using the programmable circuit, wherein the trigger signal has a period identical to the period of the synchronization signal, and With the programmable circuit, the trigger signal for triggering the one or more cameras to capture one or more images is sent to one or more cameras of an autonomous vehicle, i.e. an AV.

Description

Camera triggering based on GPTP time synchronization Cross Reference to Related Applications The application claims the benefit of U.S. provisional application No. 63/533,872 filed on 8/21 of 2023, the disclosure of which is incorporated herein by reference in its entirety. Background In an Autonomous Vehicle (AV), cameras and light detection and ranging (LiDAR) sensors capture data associated with the environment. The captured data is used to perceive the environment and enable the vehicle to make driving decisions. Drawings FIG. 1 is an example environment in which a vehicle including one or more components of an autonomous system may be implemented; FIG. 2 is a diagram of one or more systems of a vehicle including an autonomous system; FIG. 3 is a diagram of components of one or more devices and/or one or more systems of FIGS. 1 and 2; FIG. 4 is a diagram of certain components of an autonomous system; FIG. 5 is a diagram of an implementation of an architecture for AV computation; FIG. 6 is a diagram of an example system for synchronizing a camera with a LiDAR sensor; Fig. 7A illustrates an example time sequence in which PPS signal pulses arrive earlier than the 20 th pulse in the previous set of synchronization signal pulses; fig. 7B illustrates an example time sequence in which PPS signal pulses arrive later than the 20 th pulse in the previous set of synchronization signal pulses; FIG. 8 illustrates an example state machine of a synchronization signal generator; FIG. 9 illustrates an example flowchart of a process for synchronizing a camera with a LiDAR sensor; Fig. 10 illustrates an example time sequence of gpp, PPS signal, synchronization signal, trigger signal, and LiDAR trigger signal. Detailed Description In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that the embodiments described in this disclosure may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring aspects of the present disclosure. In the drawings, for ease of description, specific arrangements or sequences of illustrative elements (such as those representing systems, devices, modules, blocks of instructions, and/or data elements, etc.) are illustrated. However, those of skill in the art will understand that a specific order or arrangement of elements illustrated in the drawings is not intended to require a specific order or sequence of processes, or separation of processes, unless explicitly described. Furthermore, the inclusion of a schematic element in a figure is not intended to mean that such element is required in all embodiments nor that the feature represented by such element is not included in or combined with other elements in some implementations unless explicitly described. Furthermore, in the drawings, connecting elements (such as solid or dashed lines or arrows, etc.) are used to illustrate a connection, relationship or association between or among two or more other schematic elements, the absence of any such connecting element is not intended to mean that no connection, relationship or association exists. In other words, some connections, relationships, or associations between elements are not illustrated in the drawings so as not to obscure the present disclosure. In addition, for ease of illustration, a single connection element may be used to represent multiple connections, relationships, or associations between elements. For example, if a connection element represents a communication of signals, data, or instructions (e.g., "software instructions"), those skilled in the art will understand that such element may represent one or more signal paths (e.g., buses) that may be required to effect the communication. Although the terms "first," "second," and/or "third," etc. may be used to describe various elements, these elements should not be limited by these terms. The terms "first," second, "and/or third" are used merely to distinguish one element from another element. For example, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact, without departing from the scope of the described embodiments. Both the first contact and the second contact are contacts, but they are not the same contacts. The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the specification of the various embodiments described and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, and may be used interchangeably with "one or more than one" or "at least one," unless the context clearly indicates otherwise. It will also be