Search

EP-4736468-A1 - OPTICAL SENSOR, ENVIRONMENTAL DETECTION SYSTEM AND VEHICLE

EP4736468A1EP 4736468 A1EP4736468 A1EP 4736468A1EP-4736468-A1

Abstract

The invention relates to an optical sensor (5), an environmental detection system and a vehicle, wherein the optical sensor (5) has a pixel matrix (10), a read-out unit (20) and an evaluation unit (30). The spectral sensitivity of pixels (12) of the pixel matrix (10) is determined according to a predefined order (40) which determines the spectral sensitivities of five successive pixels (12) at a time in a row (14) of the pixel matrix (10), wherein, on the basis of the predefined order (40), three pixels (12) of the five successive pixels (12) in a row (14) are configured to only detect light in the visible spectral range; one pixel (12) of the five successive pixels (12) is configured to only detect light outside of the visible spectral range; the predefined order (40) is repeatedly assigned per row (14) of the pixel matrix (10); between two consecutive rows (14), the predefined order (40) is asssigned shifted by two pixels (12) at a time; the read-out unit (20) is configured to read out generated charge quantities from the pixel matrix (10) and, on the basis thereof, to generate a brightness pattern that corresponds with the pixel matrix (10); and the evaluation unit (30) is configured to evaluate the brightness pattern pixel-by-pixel on the basis of a predefined evaluation pattern (60), by evaluating for each pixel (12) four further pixels (12') at a time which adjoin the pixel (12), to assign the respective brightness information from the pixels (12, 12') to be jointly evaluated to their respective spectral sensitivities and to provide this as group information for each pixel (12) to be evaluated.

Inventors

  • VOCK, DOMINIK MAXIMILIAN MARTIN
  • Kurz, Heiko Gustav
  • DEWALD, WILMA

Assignees

  • Volkswagen Aktiengesellschaft

Dates

Publication Date
20260506
Application Date
20240626

Claims (10)

  1. 1. Optical sensor (5) comprising: • a pixel matrix (10), • a reading unit (20), and • an evaluation unit (30), wherein • the pixel matrix (10) is formed from a plurality of light-sensitive cells, • a spectral sensitivity of pixels (12) of the pixel matrix (10) is determined according to a predefined sequence (40) of spectral sensitivities, which spectral sensitivities are determined for every five consecutive pixels (12) in a row (14) of the pixel matrix (10), • based on the predefined sequence (40) of spectral sensitivities o at least three pixels (12) of the five consecutive pixels (12) in a row (14) are designed to detect essentially only light in the visible spectral range, wherein the at least three pixels (12) each have spectral sensitivities that differ from one another, o at least one pixel (12) of the five consecutive pixels (12) is designed to detect essentially only light outside the visible spectral range, • the predefined sequence (40) of spectral sensitivities is assigned repeatedly depending on a number of pixels (12) per row (14), so that all pixels (12) of a row (14) are assigned one of the predefined spectral sensitivities according to the predefined sequence (40) of spectral sensitivities, • the predefined sequence (40) of spectral sensitivities is assigned between two consecutive rows (12) of the pixel matrix (10) in a predefined direction (50) offset by two pixels (12) each, • the read-out unit (20) is designed to read out photoelectrically generated charge quantities in the pixels (12) from the pixel matrix (10) and to generate a brightness image corresponding to the pixel matrix (10) on the basis of these, • the evaluation unit (30) is set up, o evaluating the brightness image pixel by pixel on the basis of a predefined evaluation pattern (60) by evaluating, for each pixel (12) to be evaluated, four further pixels (12') adjacent to the pixel (12) to be evaluated, which are arranged symmetrically with respect to the pixel (12) to be evaluated, so that the total of five pixels (12, 12') to be evaluated together have the spectral sensitivities of the predefined sequence (40) of spectral sensitivities, and o assigning the respective brightness information of the pixels (12, 12') to be evaluated together to their respective spectral sensitivities and providing this as group information for each pixel (12) to be evaluated.
  2. 2. Optical sensor (5) according to claim 1, wherein the evaluation unit (30) is set up to evaluate, by means of the evaluation pattern (60), in addition to the pixel (12) currently to be evaluated, those four pixels (12') of the pixel matrix (10) for generating the group information, • each of which has an edge adjacent to one of the edges of the pixel (12) to be evaluated, and/or • which each border diagonally on the pixel to be evaluated (12).
  3. 3. Optical sensor (5) according to one of the preceding claims, wherein the at least three pixels (12) which are configured on the basis of the predefined sequence (40) of spectral sensitivities to detect essentially only light in the visible spectral range with respectively differing spectral sensitivities are configured to detect red, green and blue light.
  4. 4. Optical sensor (5) according to one of the preceding claims, wherein the at least one pixel (12) of the five consecutive pixels (12) which is intended to detect light outside the visible range is arranged to detect light • in the near infrared range and/or in the short-wave infrared range, and/or • in a wavelength range of 700 nm to 1000 nm, and/or • in a wavelength range of 1000 nm to 2000 nm, and/or • in a wavelength range of 1150 nm to 1250 nm, and/or • in a wavelength range from 1350 nm to 1450 nm.
  5. 5. Optical sensor (5) according to one of the preceding claims, wherein a pixel (12) of the predefined sequence (40) of pixels (12) is arranged to receive light in the entire visible spectral range.
  6. 6. Optical sensor (5) according to one of the preceding claims, wherein the respective spectral sensitivities of the pixels (12) of the pixel matrix (10) are based on • a plurality of spectral filters are implemented which are arranged upstream of the respective pixels (12) in order to filter light incident on the optical sensor (5) before it strikes the pixel matrix (10), and/or • differently formed light-sensitive cells are realized.
  7. 7. Environment detection system (70) comprising an optical sensor (5) according to one of the preceding claims, wherein the environment detection system (70) is configured to verify the plausibility of and/or supplement environment information in the visible spectral range detected by the optical sensor (5) by means of environment information in the near infrared range and/or in the short-wave infrared range detected by the optical sensor (5).
  8. 8. Environment detection system (70) according to claim 7, wherein the environment detection system (70) is set up on the basis of measurement signals of the optical sensor (5), • Ice and/or water and/or fog, and/or • Persons and/or animals and/or plants, and/or • identify different materials.
  9. 9. Vehicle (80) comprising an optical sensor (5) according to one of claims 1 to 6 and/or an environment recognition system (70) according to claim 7 or 8.
  10. 10. Vehicle (80) according to claim 9, wherein the vehicle (80) is configured to use environmental information acquired by means of the optical sensor (5) and/or the environmental detection system (70) for a partially autonomous and/or fully autonomous ferry operation and/or for a driver assistance system of the vehicle (80).

Description

Description Optical sensor, environment recognition system and vehicle The invention relates to an optical sensor, an environment recognition system with such an optical sensor and a vehicle with such an optical sensor and/or with such an environment recognition system. Optical sensors are known from the state of the art, which are used, for example, in cameras that enable the detection of three different colors, usually red, green and blue (hereinafter also abbreviated as RGB). For this purpose, individual light-sensitive cells (i.e. pixels) of the optical sensors are usually preceded by color filters corresponding to the respective colors to be detected, which are distributed over a receiving surface of the optical sensors according to a predefined pattern (e.g. a Bayer pattern, or similar). Sensor surfaces with a silicon-based photosensitive layer are often used as the basis for such optical sensors, by means of which a spectral range of approximately 400 nm to 1000 nm can be detected. However, novel approaches for photosensitive layers enable the detection of a wider spectral range from approximately 400 nm to 1700 nm or in some cases even up to over 2000 nm, on the basis of which multispectral images with more than three or more than four different detectable spectral ranges are possible. Multispectral images for capturing more than four different spectral ranges have so far been achieved using a so-called color wheel, which is placed in front of the optical sensor. This color wheel is divided into a large number of sectors, each with a different spectral band filter, and is positioned in front of the optical sensor in such a way that an image generated by the optical sensor is completely captured by a single spectral band filter. This means that several images are recorded sequentially, each representing an exposure for a predefined spectral range. US 20070024931 A1 discloses an image sensor having a two-dimensional arrangement of first and second groups of pixels, wherein the pixels of the first group have a narrower spectral sensitivity than pixels of the second group, and wherein the first group has individual pixels corresponding to a set of at least two colors. The placement of the pixels of the first and second groups forms a pattern having a minimum repeat unit of at least 12 pixels. US 6803955 B1 discloses an image capture device having a pixel array formed as a two-dimensional periodic array of 6-pixel arrays as a unit array, wherein of the six pixels of each unit array, the first to third pixels have different color properties and the fourth to sixth pixels each have different sensitivities and identical color properties with respect to the first to third pixels. US 2007296841 A1 discloses, inter alia, a method for manufacturing a color image capture unit having a pixel group in which a plurality of pixels each have a photoelectric conversion element and a color filter, the method comprising, inter alia, a step for generating a random arrangement pattern in which color filters of at least one color component are randomly arranged. It is an object of the present invention to provide an optical sensor on the basis of which a simultaneous multispectral recording of visible light and light outside the visible spectral range is made possible. It is also an object of the present invention to provide an environment recognition system and a vehicle based on the optical sensor. The solution to the problem identified above is achieved by the features of the independent claim. The subclaims contain preferred developments of the invention. According to a first aspect of the present invention, an optical sensor is proposed which has a (two-dimensional) pixel matrix, a readout unit and an evaluation unit. The optical sensor is designed, for example, as a CCD sensor and/or as a CMOS sensor. A number of pixels per row and/or column and/or a size of the pixels are in principle not limited to specific values. Based on the optical sensor, it is possible, for example, to design a camera for a vehicle and/or a camera for optical quality monitoring of products and/or a component other than this. The pixel matrix is formed from a plurality of light-sensitive cells (also called pixels), wherein a spectral sensitivity of pixels of the pixel matrix is determined according to a predefined sequence of spectral sensitivities, which determines spectral sensitivities for five consecutive pixels in a row of the pixel matrix. Based on the predefined sequence of spectral sensitivities, at least three pixels of the five consecutive pixels in a row are set up to essentially only detect light in the visible spectral range, wherein the at least three pixels each have spectral sensitivities that differ from one another. It should be noted that the term "row", the pixel matrix in which the predefined sequence of spectral sensitivities is applied, does not necessarily have to correspond to a row definition resulting from a readou