US-12623633-B2 - Vehicle camera lens obstruction

Abstract

A method for processing vehicle camera lens obstructions including obtaining an image from a vehicle camera, generating a first camera lens dirt map according to pixel texture richness and color spectrum levels extracted from the image, obtaining a sequential feed of multiple frames from the vehicle camera, generating a second camera lens dirt map according to at least one salient region extracted from the multiple frames, combining the first camera lens dirt map and the second camera lens dirt map to generate a combined camera lens dirt map, and initiating at least one of automated lens cleaning or a camera lens obstruction signal, according to the combined camera lens dirt map.

Inventors

• JIABIN SHEN
• Jay Blackwell
• Yun Qian Miao

Assignees

• GM Global Technology Operations LLC

Dates

Publication Date

20260512

Application Date

20230912

Claims (20)

1. 1 . A method for processing vehicle camera lens obstructions, the method comprising: obtaining an image from a vehicle camera; generating a first camera lens dirt map according to pixel texture richness and color spectrum levels extracted from the image; obtaining a sequential feed of multiple frames from the vehicle camera; generating a second camera lens dirt map according to at least one salient region extracted from the multiple frames; combining the first camera lens dirt map and the second camera lens dirt map to generate a combined camera lens dirt map; and initiating at least one of automated lens cleaning or a camera lens obstruction signal, according to the combined camera lens dirt map.
2. 2 . The method of claim 1 , wherein initiating automated lens cleaning includes at least one of dispensing a cleaning solution to a lens of the vehicle camera or moving a wiper across a surface of the lens.
3. 3 . The method of claim 1 , wherein initiating the camera lens obstruction signal includes alerting at least one vehicle movement control system of an obstruction of a lens of the vehicle camera, to inhibit generation of at least one of automated vehicle braking or audible alerts according to input received from the vehicle camera.
4. 4 . The method of claim 1 , further comprising: extracting texture sub-areas of the image; discretizing an image blurriness level according to the pixel texture richness of the texture sub-areas; and extracting color spectrum sub-areas of the image, wherein the first camera lens dirt map is generated by combining the image blurriness level and the color spectrum sub-areas.
5. 5 . The method of claim 1 , further comprising: extracting and tracking the at least one salient region in the multiple frames; and detecting at least one blockage area of the vehicle camera according to the at least one salient region, wherein the second camera lens dirt map is generated according to the at least one blockage area.
6. 6 . The method of claim 5 , further comprising: obtaining at least one vehicle motion signal; and generating a timeline of pixel statistics in the multiple frames, according to the at least one vehicle motion signal, wherein the at least one salient region is tracked according to the timeline of pixel statistics in the multiple frames, and wherein at least one blockage area is detected according to the timeline of pixel statistics in the multiple frames.
7. 7 . The method of claim 5 , wherein tracking the at least one salient region includes identifying a location in the multiple frames where detected objects repeatedly disappear during vehicle motion.
8. 8 . The method of claim 6 , wherein the at least one vehicle motion signal includes at least one of a steering wheel rotation signal and a wheel speed sensor signal.
9. 9 . The method of claim 1 , wherein combining the first camera lens dirt map and the second camera lens dirt map includes applying a weighted average to the first camera lens dirt map and the second camera lens dirt map to generate the combined camera lens dirt map.
10. 10 . The method of claim 1 , wherein: obtaining the image includes obtaining the image from a raw camera feed of the vehicle camera; and generating the first camera lens dirt map includes generating an information map based on the image from the raw camera feed, and determining a valid perception region based on the information map.
11. 11 . The method of claim 1 , wherein the vehicle camera comprises a rear vehicle backup camera.
12. 12 . A vehicle camera lens obstruction detection system comprising: a vehicle camera mounted on a vehicle, the vehicle camera including a lens, and the vehicle camera configured to capture images; and a vehicle control module in communication with the vehicle camera, the vehicle control module configured to: receive a sequential feed of multiple frames from the vehicle camera; obtain at least one vehicle motion signal; generate a timeline of pixel statistics in the multiple frames, according to the at least one vehicle motion signal; extract and track at least one salient region in the multiple frames according to the timeline of pixel statistics in the multiple frames; detect at least one blockage area of the vehicle camera according to the at least one salient region and the timeline of pixel statistics in the multiple frames; generate a camera lens dirt map according to the at least one blockage area; and initiate at least one of automated lens cleaning or a camera lens obstruction signal, according to the camera lens dirt map.
13. 13 . The vehicle camera lens obstruction detection system of claim 12 , wherein the vehicle control module is configured to track the at least one salient region by identifying a location in the multiple frames where detected objects repeatedly disappear during motion of the vehicle.
14. 14 . The vehicle camera lens obstruction detection system of claim 12 , wherein the at least one vehicle motion signal includes at least one of a steering wheel rotation signal and a wheel speed sensor signal.
15. 15 . The camera vehicle lens obstruction detection system of claim 12 , wherein initiating automated lens cleaning includes at least one of dispensing a cleaning solution to a lens of the vehicle camera or moving a wiper across a surface of the lens.
16. 16 . The camera vehicle lens obstruction detection system of claim 12 , wherein initiating the camera lens obstruction signal includes alerting at least one vehicle movement control system of an obstruction of a lens of the vehicle camera, to inhibit generation of at least one of automated vehicle braking or audible alerts according to input received from the vehicle camera.
17. 17 . The camera vehicle lens obstruction detection system of claim 12 , wherein the vehicle camera comprises a rear vehicle backup camera.
18. 18 . A vehicle camera lens obstruction detection system comprising: a vehicle camera mounted on a vehicle, the vehicle camera including a lens, and the vehicle camera configured to capture images; and a vehicle control module in communication with the vehicle camera, the vehicle control module configured to: receive an image from the vehicle camera; extract texture sub-areas of the image; discretize an image blurriness level according to pixel texture richness of the texture sub-areas; extract color spectrum sub-areas of the image; generate a camera lens dirt map according to the pixel texture richness of the texture sub-areas and the color spectrum sub-areas of the image; and initiate at least one of automated lens cleaning or a camera lens obstruction signal, according to the camera lens dirt map.
19. 19 . The vehicle camera lens obstruction detection system of claim 18 , wherein: initiating automated lens cleaning includes at least one of dispensing a cleaning solution to a lens of the vehicle camera or moving a wiper across a surface of the lens; and initiating the camera lens obstruction signal includes alerting at least one vehicle movement control system of an obstruction of a lens of the vehicle camera, to inhibit generation of at least one of automated vehicle braking or audible alerts according to input received from the vehicle camera.
20. 20 . The camera vehicle lens obstruction detection system of claim 18 , wherein the vehicle camera comprises a rear vehicle backup camera.

Description

INTRODUCTION The information provided in this section is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure. The present disclosure generally relates to vehicle camera lens obstruction, including detection of an obstruction of the vehicle camera lens to control automated lens cleaning and other vehicle control features. Some vehicles include cameras, such as rear vehicle cameras that provide a camera feed for drivers while backing up. Other vehicle systems may use camera images to sense surroundings and to control different vehicle functions, such as audible alerts when an object is detected in close proximity behind the vehicle. Sometimes dirt or other debris may collect on a lens of the vehicle camera, negatively impacting systems that rely on accurate images captured from the vehicle camera. SUMMARY A method for processing vehicle camera lens obstructions including obtaining an image from a vehicle camera, generating a first camera lens dirt map according to pixel texture richness and color spectrum levels extracted from the image, obtaining a sequential feed of multiple frames from the vehicle camera, generating a second camera lens dirt map according to at least one salient region extracted from the multiple frames, combining the first camera lens dirt map and the second camera lens dirt map to generate a combined camera lens dirt map, and initiating at least one of automated lens cleaning or a camera lens obstruction signal, according to the combined camera lens dirt map. In other features, initiating automated lens cleaning includes at least one of dispensing a cleaning solution to a lens of the vehicle camera or moving a wiper across a surface of the lens. In other features, initiating the camera lens obstruction signal includes alerting at least one vehicle movement control system of an obstruction of a lens of the vehicle camera, to inhibit generation of at least one of automated vehicle braking or audible alerts according to input received from the vehicle camera. In other features, the method includes extracting texture sub-areas of the image, discretizing an image blurriness level according to the pixel texture richness of the texture sub-areas, and extracting color spectrum sub-areas of the image, wherein the first camera lens dirt map is generated by combining the image blurriness level and the color spectrum sub-areas. In other features, the method includes extracting and tracking the at least one salient region in the multiple frames, and detecting at least one blockage area of the vehicle camera according to the at least one salient region, wherein the second camera lens dirt map is generated according to the at least one blockage area. In other features, the method includes obtaining at least one vehicle motion signal, and generating a timeline of pixel statistics in the multiple frames, according to the at least one vehicle motion signal, wherein the at least one salient region is tracked according to the timeline of pixel statistics in the multiple frames, and wherein at least one blockage area is detected according to the timeline of pixel statistics in the multiple frames. In other features, tracking the at least one salient region includes identifying a location in the multiple frames where detected objects repeatedly disappear during vehicle motion. In other features, the at least one vehicle motion signal includes at least one of a steering wheel rotation signal and a wheel speed sensor signal. In other features, combining the first camera lens dirt map and the second camera lens dirt map includes applying a weighted average to the first camera lens dirt map and the second camera lens dirt map to generate the combined camera lens dirt map. In other features, obtaining the image includes obtaining the image from a raw camera feed of the vehicle camera, and generating the first camera lens dirt map includes generating an information map based on the image from the raw camera feed, and determining a valid perception region based on the information map. In other features, the vehicle camera comprises a rear vehicle backup camera. A vehicle camera lens obstruction detection system includes a vehicle camera mounted on a vehicle, the vehicle camera including a lens, and the vehicle camera configured to capture images, and a vehicle control module in communication with the vehicle camera, the vehicle control module configured to receive a sequential feed of multiple frames from the vehicle camera, extract and track at least one salient region in the multiple frames, detect at least one blockage area of the vehicle camera according to the at least one salient region, generate a camera lens dirt m