US-12620240-B2 - Traffic light relevancy

US12620240B2US 12620240 B2US12620240 B2US 12620240B2US-12620240-B2

Abstract

Systems and methods are provided for vehicle navigation. In one implementation, a system for navigating a vehicle may include at least one processor configured to receive a first image frame; detect in the first image frame a representation of a traffic light and determine a color state associated with lamps included on the traffic light. The at least one processor may receive an additional image frame includes a representation of the at least one traffic light; and determine, based on a comparison of the first image frame and the additional image frame, whether the at least one traffic light includes a blinking lamp. If the at least one traffic light includes a blinking lamp, the processor may cause the vehicle to implement a navigational action relative the traffic light in accordance with the determination and also based on a detected color state for the blinking lamp.

Inventors

David Huberman
Jonathan BARLEV

Assignees

MOBILEYE VISION TECHNOLOGIES LTD.

Dates

Publication Date: 20260505
Application Date: 20220810

Claims (20)

1 . A method for generating a crowd-sourced map for use in vehicle navigation, the method comprising: receiving drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; grouping the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; linking each of two or more drivable paths for the road segment with at least one of the two or more traffic light groups; storing in the crowd-sourced map representations of the links between each of the two or more drivable paths with at least one of the two or more traffic light groups; and making the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
2 . The method of claim 1 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to generate the two or more drivable paths for the road segment based on aggregated motion characteristics of the plurality of vehicles as they traversed the road segment.
3 . The system of claim 2 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to store the generated two or more drivable paths in the crowd-sourced map.
4 . A non-transitory computer readable medium containing instructions that when executed by at least one processor, cause the at least one processor to perform operations comprising: receiving drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; grouping the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; linking each of two or more drivable paths for the road segment with at least one of the two or more traffic light groups; storing in a crowd-sourced map representations of the links between each of the two or more drivable paths with at least one of the two or more traffic light groups; and making the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
5 . The non-transitory computer readable medium of claim 4 , wherein traffic lights observed by any of the plurality of vehicles as having different color states from one another are grouped into different traffic light groups.
6 . The non-transitory computer readable medium of claim 4 , wherein traffic lights observed by the plurality of vehicles as having common color states are grouped into a common traffic light group.
7 . A system for generating a crowd-sourced map for use in vehicle navigation, the system comprising: at least one processor comprising circuitry and a memory, wherein the memory includes instructions that when executed by the circuitry cause the at least one processor to: receive drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; group the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; store in the crowd-sourced map representations of at least one of the two or more traffic light groups; and make the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
8 . The system of claim 7 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to link each of two or more drivable paths for the road segment with at least one of the two or more traffic light groups.
9 . The system of claim 8 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to store in the crowd-sourced map representations of the links between each of the two or more drivable paths with at least one of the two or more traffic light group.
10 . The system of claim 8 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to generate the two or more drivable paths for the road segment based on aggregated motion characteristics of the plurality of vehicles as they traversed the road segment.
11 . The system of claim 10 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to store the generated two or more drivable paths in the crowd-sourced map.
12 . A method for generating a crowd-sourced map for use in vehicle navigation, the method comprising: receiving drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; grouping the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; storing in the crowd-sourced map representations of at least one of the two or more traffic light groups; and making the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
13 . The method of claim 12 , further including linking each of two or more drivable paths for the road segment with at least one of the two or more traffic light groups.
14 . The method of claim 13 , further including storing in the crowd-sourced map representations of the links between each of the two or more drivable paths with at least one of the two or more traffic light group.
15 . A non-transitory computer readable medium containing instructions that when executed by at least one processor, cause the at least one processor to perform operations comprising: receiving drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; grouping the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; storing in a crowd-sourced map representations of at least one of the two or more traffic light groups; and making the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
16 . The non-transitory computer readable medium of claim 15 , further including instructions that when executed by the at least one processor, cause the at least one processor to generate the two or more drivable paths for the road segment based on aggregated motion characteristics of the plurality of vehicles as they traversed the road segment.
17 . The non-transitory computer readable medium of claim 16 , further including instructions that when executed by the at least one processor, cause the at least one processor to store the generated two or more drivable paths in the crowd-sourced map.
18 . A system for generating a crowd-sourced map for use in vehicle navigation, the system comprising: at least one processor comprising circuitry and a memory, wherein the memory includes instructions that when executed by the circuitry cause the at least one processor to: receive drive information collected from a plurality of vehicles that traversed a road segment, wherein the road segment is associated with a junction and wherein the junction includes a plurality of traffic lights; group the plurality of traffic lights into two or more traffic light groups based on analysis of the drive information collected from the plurality of vehicles, wherein grouping the plurality of traffic lights includes determining, based on a synchronization between a first signal pattern associated with a first traffic light and a second signal pattern associated with a second traffic light, an association between the first traffic light and the second traffic light, the first signal pattern and the second signal pattern being determined based on at least one indicator of color state for each of the plurality of traffic lights, the at least one indicator of color state having been determined based on analysis of images captured by the plurality of vehicles; link each of two or more drivable paths for the road segment with at least one of the two or more traffic light groups; store in the crowd-sourced map representations of the links between each of the two or more drivable paths with at least one of the two or more traffic light groups; and make the crowd-sourced map available to at least one host vehicle for navigation relative to the plurality of traffic lights.
19 . The system of claim 18 , wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to: generate drivable junction paths between each entrance and an associated exit of the junction and store the generated drivable junction paths in the crowd-sourced map, each drivable junction path being associated with at least one of an entrance point or an exit point stored in the crowd-sourced map.
20 . The system of claim 18 , wherein the navigation relative to the plurality of traffic lights includes at least one of a traffic light warning issued to a vehicle operator or autonomous braking relative to a detected color state of at least one traffic light determined to be in a traffic light group indicated in the crowd-sourced map as relevant to a current drivable path of the at least one host vehicle.

Description

CROSS REFERENCES TO RELATED APPLICATIONS This application is a continuation of PCT International Application No. PCT/US2021/032520, filed May 14, 2021, which claims the benefit of priority of U.S. Provisional Application No. 63/024,591, filed on May 14, 2020; U.S. Provisional Application No. 63/038,235, filed on Jun. 12, 2020; U.S. Provisional Application No. 63/135,440, filed on Jan. 8, 2021. The foregoing applications are incorporated herein by reference in their entirety BACKGROUND Technical Field The present disclosure relates generally to autonomous vehicle navigation. Background Information As technology continues to advance, the goal of a fully autonomous vehicle that is capable of navigating on roadways is on the horizon. Autonomous vehicles may need to take into account a variety of factors and make appropriate decisions based on those factors to safely and accurately reach an intended destination. For example, an autonomous vehicle may need to process and interpret visual information (e.g., information captured from a camera) and may also use information obtained from other sources (e.g., from a GPS device, a speed sensor, an accelerometer, a suspension sensor, etc.). At the same time, in order to navigate to a destination, an autonomous vehicle may also need to identify its location within a particular roadway (e.g., a specific lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, and travel from one road to another road at appropriate intersections or interchanges. Harnessing and interpreting vast volumes of information collected by an autonomous vehicle as the vehicle travels to its destination poses a multitude of design challenges. The sheer quantity of data (e.g., captured image data, map data, GPS data, sensor data, etc.) that an autonomous vehicle may need to analyze, access, and/or store poses challenges that can in fact limit or even adversely affect autonomous navigation. Furthermore, if an autonomous vehicle relies on traditional mapping technology to navigate, the sheer volume of data needed to store and update the map poses daunting challenges. SUMMARY Embodiments consistent with the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed systems may include one, two, or more cameras that monitor the environment of a vehicle. The disclosed systems may provide a navigational response based on, for example, an analysis of images captured by one or more of the cameras. In an embodiment, a system for navigating a vehicle may include at least one processor comprising circuitry and a memory. The memory may include instructions that when executed by the circuitry cause the at least one processor to receive a first image frame acquired by an image capture device, wherein the first image frame is representative of an environment of the vehicle, the environment including at least one traffic light; detect in the first image frame a representation of the at least one traffic light and determine a color state associated with one or more lamps included on the at least one traffic light; receive at least one additional image frame acquired by the image capture device, wherein the at least one additional image frame includes a representation of the at least one traffic light; determine, based on a comparison of at least a portion of the first image frame and at least a portion of the at least one additional image frame, whether the at least one traffic light includes a blinking lamp; and if the at least one traffic light is determined to include a blinking lamp, cause the vehicle to implement a navigational action relative the at least one traffic light in accordance with the determination that the at least one traffic light includes a blinking lamp and also based on a detected color state for the blinking lamp. In an embodiment, a method for navigating a vehicle may include receiving a first image frame acquired by an image capture device, wherein the first image frame is representative of an environment of the vehicle, the environment including at least one traffic light; detecting in the first image frame a representation of the at least one traffic light and determine a color state associated with one or more lamps included on the at least one traffic light; receiving at least one additional image frame acquired by the image capture device, wherein the at least one additional image frame includes a representation of the at least one traffic light; determining, based on a comparison of at least a portion of the first image frame and at least a portion of the at least one additional image frame, whether the at least one traffic light includes a blinking lamp; and if the at least one traffic light is determined to include a blinking