Search

CN-122029827-A - Image processing method and image processing system

CN122029827ACN 122029827 ACN122029827 ACN 122029827ACN-122029827-A

Abstract

The image processing method includes detecting a plurality of optical signals corresponding to a plurality of pixels via an optical element having a plurality of light transmittances corresponding to the plurality of pixels for each of the plurality of light components (S101), obtaining a1 st compressed image signal obtained by adding the plurality of optical signals to adjacent pixel groups of two or more adjacent pixels among the plurality of pixels (S102), and generating a1 st restored image signal group having a plurality of restored image signals corresponding to the plurality of light components from the 1 st compressed image signal based on 1 st transmittance information related to the plurality of light transmittances (S103).

Inventors

  • Yago Motoki
  • SATO SATORU

Assignees

  • 松下知识产权经营株式会社

Dates

Publication Date
20260512
Application Date
20241120
Priority Date
20231129

Claims (16)

  1. 1. An image processing method, comprising: Detecting a plurality of optical signals corresponding to a plurality of pixels via an optical element having a plurality of light transmittances corresponding to the plurality of pixels for each of a plurality of light components; Obtaining a1 st compressed image signal obtained by adding the plurality of optical signals to adjacent pixel groups of two or more adjacent pixels among the plurality of pixels; A1 st restored image signal group having a plurality of restored image signals corresponding to the plurality of light components is generated from the 1 st compressed image signal according to the 1 st transmittance information related to the plurality of light transmittance.
  2. 2. The image processing method according to claim 1, The plurality of light components are a plurality of wavelength bands.
  3. 3. The image processing method according to claim 1 or 2, The 1 st transmittance information is obtained by integrating the plurality of light transmittances of the 2 nd transmittance information, which has the plurality of light transmittances corresponding to the plurality of pixels for each of the plurality of light components, as information, for adjacent pixel groups of the two or more adjacent pixels.
  4. 4. The image processing method according to claim 1 or 2, The optical signal corresponding to the 1 st pixel of the two or more adjacent pixels is the optical signal transmitted through the 1 st region of the optical element, The optical signal corresponding to the 2 nd pixel of the two or more adjacent pixels is the optical signal transmitted through the 2 nd region of the optical element, The light transmission spectrum for the 1 st region of the plurality of light components is different from the light transmission spectrum for the 2 nd region of the plurality of light components.
  5. 5. The image processing method according to claim 1 or 2, The optical element has a plurality of light transmission spectra corresponding to the plurality of pixels, The plurality of light transmission spectra are each light transmission spectra for the plurality of light components, The plurality of light transmission spectra have spatial randomness with respect to the plurality of pixels.
  6. 6. The image processing method according to claim 1 or 2, Further comprising obtaining a2 nd compressed image signal having the plurality of optical signals, The 1 st compressed image signal is obtained by adding the plurality of optical signals of the 2 nd compressed image signal to each of adjacent pixel groups of the two or more adjacent pixels.
  7. 7. The image processing method according to claim 1 or 2, And switching between a1 st mode and a 2 nd mode, the 1 st mode being a mode of generating the 1 st restored image signal group from the 1 st compressed image signal, and the 2 nd mode being a mode of generating the 2 nd restored image signal group from the 2 nd compressed image signal having the plurality of optical signals.
  8. 8. The image processing method according to claim 7, In the switching of the 1 st mode and the 2 nd mode, the 1 st mode and the 2 nd mode are switched based on an operation by a user.
  9. 9. The image processing method according to claim 7, The number of pixels of each restored image signal of the 1 st restored image signal group is smaller than the number of pixels of each restored image signal of the 2 nd restored image signal group.
  10. 10. The image processing method according to claim 1 or 2, The number of pixels of each restored image signal of the 1 st restored image signal group is smaller than the number of the plurality of pixels.
  11. 11. The image processing method according to claim 1 or 2, The number of pixels of each restored image signal of the 1 st restored image signal group is the same as the number of pixels of the 1 st compressed image signal.
  12. 12. The image processing method according to claim 1 or 2, The 1 st transmittance information has, as information, a plurality of light transmittances smaller than the plurality of light transmittances corresponding to the plurality of pixels, for each of the plurality of light components.
  13. 13. The image processing method according to claim 1, The plurality of light components are a plurality of polarization directions.
  14. 14. The image processing method according to claim 1 or 2, The 1 st compressed image signal is obtained by adding the plurality of optical signals by adjacent pixel groups of the two or more adjacent pixels and performing an averaging process.
  15. 15. An image processing method, comprising: Detecting a plurality of optical signals corresponding to a plurality of pixels via an optical element having a plurality of light transmission spectra corresponding to the plurality of pixels; Obtaining a1 st compressed image signal obtained by adding the plurality of optical signals to adjacent pixel groups of two or more adjacent pixels among the plurality of pixels; A1 st restored image signal group having a plurality of restored image signals corresponding to a plurality of bands is generated from the 1 st compressed image signal based on information related to the plurality of light transmission spectrums.
  16. 16. An image processing system is provided with: An image sensor that detects a plurality of optical signals corresponding to a plurality of pixels via an optical element having a plurality of light transmittances corresponding to the plurality of pixels for each of a plurality of light components, and A processing circuit that obtains a 1 st compressed image signal obtained by adding the plurality of optical signals to adjacent pixel groups of two or more adjacent pixels among the plurality of pixels, The processing circuit further generates a1 st restored image signal group having a plurality of restored image signals corresponding to the plurality of light components from the 1 st compressed image signal based on the 1 st transmittance information related to the plurality of light transmittance.

Description

Image processing method and image processing system Technical Field The present disclosure relates to an image processing method and the like. Background Patent document 1 describes a compressed sensing (sensing) method for performing merging (binning). Patent document 2 describes a method of calculating image data of a plurality of wavelength ranges by using compressed sensing. Non-patent document 1 describes compressed sensing across a spatial domain, a spectral domain, and a polarization domain. Non-patent document 2 describes a multispectral polarization filter array (FILTER ARRAY). Non-patent document 3 describes a multispectral polarization camera (camera). Non-patent documents 4 and 5 describe techniques related to hyperspectral images. Prior art literature Patent literature Patent document 1 Japanese patent application laid-open No. 2015-056903 Patent document 2 Japanese patent laid-open publication 2016-156801 Non-patent literature Non-patent document 1: chen Feng et al, volume （Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector）,Journal of the Optical Society of America A,32, 11 th edition, pages 2178-2188, 2015, compressed spectral polarization imaging based on pixelized polarizers and color pattern detectors Non-patent document 2, xiao Tian Yima et al, snap-shot multispectral polarization imaging based on Photonic Crystal Filter array, volume （Snapshot multispectral polarization imaging using a photonic crystal filter array）,Optics Express,26, 12 th edition, pages 15948-15971, 2018 Non-patent document 3, xiao Tian Yima et al, alignment-free filter array Snapshot multispectral polarization imaging （Alignment-free filter array: Snapshot multispectral polarization imaging based on a Voronoi-like random photonic crystal filter）,Optics Express,28 volume based on Voronoi-like random Photonic Crystal Filter, 26 th edition, pages 38867-38882, 2020 Non-patent document 4: the technology of combining the superlens and the artificial intelligence with a common digital camera and acquiring hyperspectral images and videos, namely integrating optics and the artificial intelligence, is realized for the first time worldwide, the general camera is a real-world, general-front, front-to-back, rear-to-back, front-to-back, front-back, front-on, front, a device, a device on day 11 and 16 of 2023, internet < URL https:// group. Ntt/jp/newsrelease/2022/10/24/221024a. Html > Non-patent document 5, ahasan Ahamed et al, use of a reconstruction spectroscopy technique for CMOS image sensors with random photon capturing nanostructures, （Reconstruction-based spectroscopy using CMOS image sensors with random photon-trapping nanostructure per sensor）,Proc. SPIE 11971,High-Speed Biomedical Imaging and Spectroscopy VII,1197106,2022, 3/2/year Disclosure of Invention However, there are cases where the sensitivity of the sensor is low. On the other hand, image processing for effectively improving the sensitivity of the sensor sometimes causes parallax to the image. Thus, an image processing method and the like are provided that can effectively improve the sensitivity of the sensor while suppressing parallax occurring in the image. An image processing method according to an aspect of the present disclosure includes detecting a plurality of optical signals corresponding to a plurality of pixels via an optical element having a plurality of light transmittances corresponding to the plurality of pixels, obtaining a1 st compressed image signal obtained by adding (summing) the plurality of optical signals to adjacent pixel groups (groups) of two or more adjacent pixels among the plurality of pixels, and generating a1 st restored image signal group from the 1 st compressed image signal based on 1 st transmittance information related to the plurality of light transmittances. The general and specific aspects may be implemented by a system, an apparatus, a method, an integrated circuit, a computer program, a non-transitory recording medium such as a computer-readable CD-ROM, or any combination of the system, the apparatus, the method, the integrated circuit, the computer program, and the recording medium. According to the image processing method and the like according to one aspect of the present disclosure, the effective sensitivity of the sensor can be improved while suppressing parallax generated in an image. Drawings Fig. 1 is a conceptual diagram showing the merging process in reference example 1. Fig. 2 is a conceptual diagram showing the merging process in reference example 2. Fig. 3 is a conceptual diagram showing the merging process in reference example 3. Fig. 4 is a conceptual diagram showing the merging process in the 4 th reference example. Fig. 5 is a conceptual diagram showing the merging process in the reference example 5. Fig. 6 is a conceptual diagram showing the merging process in the 6 th reference example. Fig. 7 is a block diagram showing an example of the configuration (confi