Search

CA-3035946-C - SYSTEM AND METHOD FOR DYNAMIC PIXEL MANAGEMENT OF A CROSS PIXEL INTERCONNECTED CMOS IMAGE SENSOR

CA3035946CCA 3035946 CCA3035946 CCA 3035946CCA-3035946-C

Abstract

A camera using a CMOS image sensor based on a shared pixel array technology avails both high definition (HD) and ultra-high definition (UHD) resolution mode formats. Dynamic pixel management allows for both sequential and binned timing formats of pixel signals using switched capacitor noise reduction techniques. When UHD resolution mode is selected, noise can be reduced using both digital double sampling (DDS) or differential digital double sampling (dDDS), and when HD resolution mode is selected noise can be reduced using DDS. Additionally, both rolling shutter and global shutter modes can be selected when HD resolution mode is selected.

Inventors

  • Petrus Gijsbertus CENTEN
  • Jeroen Rotte
  • Juul Josephus Johannes VAN DEN HEIJKANT
  • Rudolf VAN REE

Assignees

  • GVBB HOLDINGS, S.A.R.L.

Dates

Publication Date
20260505
Application Date
20170908
Priority Date
20161128

Claims (3)

  1. CLAIMS What is claimed is: 1. An image sensing system for providing dynamic pixel management to switch operational modes between high definition (HD) and ultra-high definition (UHD), the image sensing system compnsmg: a complementary metal oxide semiconductor (CMOS) image sensor including a shared pixel array having a plurality of rows of shared pixel units with each pixel unit comprising at least two photodiodes, a shared floating diffusion and at least two transfer gates configured to control readout of the at least two photodiodes, respectively, wherein the shared pixel array comprises a cross pixel interconnection with the at least two transfer gates of each shared pixel unit including a first transfer gate coupled to a corresponding transfer gate of a shared pixel unit in a row directly above the respective row of the shared pixel unit and a second transfer gate coupled to corresponding transfer gate of a shared pixel unit in a row directly below the respective row of shared pixel unit; a plurality of vertical and horizontal charge circuity coupled to the CMOS image sensor and configured to activate the plurality of shared pixel units during image capture based on a set operational mode of the CMOS image sensor; a dynamic pixel manager configured to switch the operational mode of the CMOS image sensor between a UHD mode and a HD mode in response to a user selection of an image resolution for the image captured by the image sensor; a UHD mode controller configured to control the plurality of vertical and horizontal charge circuity to sequentially transfer charge between the at least two photodiodes and the shared floating diffusion of each shared pixel unit when the dynamic pixel manager sets the operational mode of the CMOS image sensor to the UHD mode to individually sample output values of each of the at least two photodiode during the image capture by the image sensor, wherein the UHD mode controller controls the plurality of vertical and horizontal charge circuity to concurrently sample output values of respective photodiodes in adjacent rows of shared pixel units above or below each AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 other by applying a first common transfer gate control signal to the first transfer gate and the corresponding transfer gate of the shared pixel unit in the row directly above the respective row of the shared pixel unit and a second common transfer gate control signal to the second transfer gate and the corresponding transfer gate of the shared pixel unit in the row directly below the respective row of the shared pixel unit; an HD mode controller configured to control the plurality of vertical and horizontal charge circuity to bin charge concurrently between the at least two photodiodes and the shared floating diffusion of each shared pixel unit when the dynamic pixel manager sets the operational mode of the CMOS image sensor to the HD mode to collectively sample output values of each shared pixel unit that combines output values of the at least two photodiodes during the image capture by the image sensor; a column readout circuit having a plurality of storage capacitors selectively coupled to the shared pixel array that are each configured to store sampled output values of the at least two photodiode of each shared pixel unit during the image capture in the UHD mode and to store sampled output values of each shared pixel unit during the image capture in the HD mode; and an image generating unit configured to generate image data based on the stored sampled output values in the plurality of storage capacitors, the generated image configured to be displayed on a display device.
  2. 2. The image sensing system according to claim 1, wherein the plurality of rows of shared pixel units of the shared pixel array comprises a first row of shared pixel units comprising at least one first shared pixel unit and a second row of shared pixel units comprising at least one second shared pixel unit. 3. The image sensing system according to claim 2, wherein the column readout circuit is further configured to read the stored sampled output values from the shared floating diffusion of the at least one first shared pixel unit and the shared floating diffusion of the at least one second shared pixel unit. 4. The image sensing system according to claim 2, wherein the dynamic pixel manager is further configured to control the plurality of vertical and horizontal charge circuity to concurrently AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 transfer charge between at least one photodiode and the shared floating diffusion of the at least one first shared pixel unit and between at least one photodiode and the shared floating diffusion of the at least one second shared pixel unit when the dynamic pixel manager sets the operational mode of the CMOS image sensor to the HD mode. 5. The image sensing system according to claim 2, wherein the image generating unit comprises a pixel output calculator configured to digitally double sample the at least two photodiodes of each shared pixel unit to reduce noise during the image capture in the UHD mode. 6. The image sensing system according to claim 5, wherein the pixel output calculator digitally double samples the at least two photodiodes of each shared pixel unit by subtracting a dark sample output from a bright sample out of each of the at least two photodiodes, wherein the dark and the bright sample outputs are read sequentially by the column readout circuit from the shared floating diffusion of the first row. 7. The image sensing system according to claim 2, wherein the UHD mode controller is further configured to control the plurality of vertical and horizontal charge circuity to control the CMOS image sensor to sample pixel outputs of the shared pixel array according to a rolling shutter exposure sequence when the dynamic pixel manager sets the operational mode of the CMOS image sensor to the UHD mode. 8. The image sensing system according to claim 7, wherein the shared pixel array further comprises a third row of shared pixel units comprising at least one third shared pixel unit, wherein the rolling shutter exposure sequence transfers charge between at least one photodiode and a shared floating diffusion of the at least one third shared pixel unit after charge is transferred between the at least one photodiode and the shared floating diffusion of the at least one first shared pixel unit. 9. A camera for providing dynamic pixel management to switch operational modes between high definition (HD) and ultra-high definition (UHD), the camera comprising: an image sensor including a shared pixel array having a plurality of shared pixels that each comprises at least two photodiodes and a shared floating diffusion, wherein the shared pixel array comprises a cross pixel interconnection with each shared pixel including a first transfer gate AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 coupled to a corresponding transfer gate of a shared pixel unit in a row directly above the respective shared pixel unit and a second transfer gate coupled to corresponding transfer gate of a shared pixel unit in a row directly below the respective shared pixel unit, with each transfer gate configured to control readout of a respective photodiode coupled thereto of the respective shared pixel unit; a dynamic pixel manager configured to switch an operational mode of the image sensor between a UHD mode and a HD mode based on a selected image resolution for an image capture by the image sensor; a UHD mode controller configured to control the image sensor to sequentially transfer charge between the at least two photodiodes and the shared floating diffusion of each shared pixel when the dynamic pixel manager sets the operational mode of the image sensor to the UHD mode to individually sample photodiode output values of each of the at least two photodiode during the image capture by the image sensor; an HD mode controller configured to control the image sensor to bin charge concurrently between the at least two photodiodes and the shared floating diffusion of each shared pixel when the dynamic pixel manager sets the operational mode of the image sensor to the HD mode to collectively sample pixel output values of each shared pixel that combines output values of the at least two photodiodes during the image capture by the image sensor; and an image generating unit configured to generate image data based on at least one of the individually sampled photodiode output values during the UHD mode and the collectively sampled pixel output values during the HD mode. 10. The camera according to claim 9, further comprising: a column readout circuit having a plurality of storage capacitors selectively coupled to the shared pixel array that are each configured to store sampled output values of the at least two photodiode of each shared pixel unit during the image capture in the UHD mode and to store sampled output values of each shared pixel unit during the image capture in the HD mode, wherein the UHD mode controller is configured to control a plurality of vertical and horizontal charge circuity to concurrently sample output values of respective photodiodes in adjacent rows of shared pixel units above or below each other by applying a first common transfer AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 gate control signal to the first transfer gate and the corresponding transfer gate of the shared pixel unit in the row directly above the respective row of the shared pixel unit and a second common transfer gate control signal to the second transfer gate and the corresponding transfer gate of the shared pixel unit in the row directly below the respective row of the shared pixel unit. 11. The camera according to claim 9, wherein the plurality of rows of shared pixel units of the shared pixel array comprises a first row of shared pixel units comprising at least one first shared pixel unit and a second row of shared pixel units comprising at least one second shared pixel unit. 12. The camera according to claim 11, further comprising a column readout circuit configured to read the stored sampled output values from the shared floating diffusion of the at least one first shared pixel unit and the shared floating diffusion of the at least one second shared pixel unit. 13. The camera according to claim 11, wherein the dynamic pixel manager is further configured to control the image sensor to concurrently transfer charge between at least one photodiode and the shared floating diffusion of the at least one first shared pixel unit and between at least one photodiode and the shared floating diffusion of the at least one second shared pixel unit when the dynamic pixel manager sets the operational mode of the image sensor to the HD mode. 14. The camera according to claim 12, wherein the image generating unit comprises a pixel output calculator configured to digitally double sample the at least two photodiodes of each shared pixel unit to reduce noise during the image capture in the UHD mode. 15. The camera according to claim 14, wherein the pixel output calculator digitally double samples the at least two photodiodes of each shared pixel unit by subtracting a dark sample output from a bright sample out of each of the at least two photodiodes, wherein the dark and the bright sample outputs are read sequentially by the column readout circuit from the shared floating diffusion of the first row. 16. The camera according to claim 9, wherein the UHD mode controller is further configured to control the image sensor to control the image sensor to sample pixel outputs of the shared pixel AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 array according to a rolling shutter exposure sequence when the dynamic pixel manager sets the operational mode of the image sensor to the UHD mode. 17. The camera according to claim 16, wherein the shared pixel array further comprises a third row of shared pixel units comprising at least one third shared pixel unit, wherein the rolling shutter exposure sequence transfers charge between at least one photodiode and a shared floating diffusion of the at least one third shared pixel unit after charge is transferred between the at least one photodiode and the shared floating diffusion of the at least one first shared pixel unit. 18. A camera for providing dynamic pixel management to switch between operational modes to change image resolution for image capture, the camera comprising: a camera mode controller configured to switch the camera between a first operational mode and a second operational mode based on a selected image resolution for the camera; and an image sensor configured to individually sample sub-pixels of each pixel in an image sensor when the camera mode controller sets the camera to the first operational mode for an image capture and to collectively sample the sub-pixels of each pixel in the image sensor when the camera mode controller sets the camera to the second operational mode for the image capture, wherein the image sensor comprises a cross pixel interconnection configuration in which at least one pixel is coupled to a first pixel in a row above the at least one pixel and further coupled to a second pixel in a row below the at least one pixel; and a first operational mode controller configured to control the image sensor in the first operational mode to concurrently sample output values of respective sub-pixels in the at least one pixel and the first pixel in the row above the at least one pixel or the second pixel in the row below the at least one pixel. 19. The camera according to claim 18, wherein the first operational mode is an ultra-high definition (UHD) mode and the second operational mode is a high definition (HD) mode. 20. The camera according to claim 19, further comprising a UHD mode controller configured to control the image sensor to sequentially transfer charge between at least two sub-pixels and a shared floating diffusion of each pixel in the image sensor when the camera mode controller sets AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 the camera to the UHD mode to individually sample sub-pixel output values of each of the at least two sub-pixels during the image capture. 21. The camera according to claim 20, further comprising a HD mode controller configured to control the image sensor to bin charge concurrently between the at least two sub-pixels and the shared floating diffusion of each pixel when the camera mode controller sets the camera to the HD mode to collectively sample pixel output values of each shared pixel that combines output values of the at least two sub-pixels during the image capture. 22. The camera according to claim 21, further comprising an image generating unit configured to generate image data based on at least one of the individually sampled sub-pixel output values during the UHD mode and the collectively sampled pixel output values during the HD mode. 23. The camera according to claim 22, further comprising a column readout circuit having a plurality of storage capacitors selectively coupled to the image sensor that are each configured to store sampled output values of the at least two sub-pixels of each pixel during the image capture in the UHD mode and to store sampled output values of each pixel during the image capture in the HD mode. 24. The camera according to claim 22, wherein the image sensor comprises a shared pixel array having a first row of pixels comprising at least one first pixel and a second row of pixels comprising at least one second pixel. 25. The camera according to claim 24, wherein the HD mode controller is further configured to control the image sensor to concurrently transfer charge between at least one sub-pixel and the shared floating diffusion of the at least one first pixel and between at least one sub-pixel and the shared floating diffusion of the at least one second pixel when the camera mode controller sets the operational mode of the camera to the HD mode. 26. The camera according to claim 23, wherein the image generating unit comprises a pixel output calculator configured to digitally double sample the at least two sub-pixels of each pixel to reduce noise during the image capture in the UHD mode. AFDOCS/15379477.3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01 27. The camera according to claim 26, wherein the pixel output calculator digitally double samples the at least two sub-pixels of each pixel by subtracting a dark sample output from a bright sample output of each of the at least two sub-pixels, wherein the dark and the bright sample outputs are read sequentially by the column readout circuit from the shared floating diffusion of the first row. 28. The camera according to claim 24, wherein the UHD mode controller is further configured to control the image sensor to control the image sensor to sample pixel outputs of the shared pixel array according to a rolling shutter exposure sequence when the camera mode controller sets the operational mode of the camera to the UHD mode. 29. The camera according to claim 28, wherein the shared pixel array further comprises a third row of pixels comprising at least one third pixel. 30. The camera according to claim 29, wherein the rolling shutter exposure sequence transfers charge between at least one sub-pixel and a shared floating diffusion of the at least one third pixel after charge is transferred between the at least one sub-pixel and the shared floating diffusion of the at least one first pixel. AFDOCS/15379477.
  3. 3 WSLEGAL\061775\00194\32199959vl Date Re1rue/Date Received 2024-02-01

Description

SYSTEM AND METHODS FOR DYNAMIC PIXEL MANAGEMENT OF A CROSS PIXEL INTERCONNECTED CMOS IMAGE SENSOR CROSS-REFERENCE TO RELATED APPLICATIONS [0001) This application claims priority to U.S. Application Serial No. 15/697,349, filed September 6, 2017, which is a continuation-in-part of U.S. Application Serial No. 15/362,023, filed November 28, 2016, which claims priority to U.S. Provisional Application Serial No. 62/385,027, filed on September 8, 2016. This application also claims priority to U.S. Provisional Application Serial No. 62/385,204, filed on September 8, 2016. TECHNICAL FIELD [0002) The disclosure herein generally relates to digital cameras, and more particularly to cameras using complementary metal oxide semiconductor (CMOS) image sensors. BACKGROUND [0003) Complementary metal oxide semiconductor ("CMOS") image sensors are widely used in digital cameras to produce digital images by converting optical signals into electrical signals. In operation, CMOS image sensors may convert an optical signal into an electrical signal using a multitude of pixels that each include a photodiode and a read-out circuit. The photodi ode generates electric charges using absorbed light, converts the generated electric charges into an analog AFDOCS/15379477-3 WSLEGAL\061775\00194\32200010vl Date Re9ue/Date Received 2022-09-06 current, and delivers the analog current to the read-out circuit. The read-out circuit may convert the analog signal into a digital signal and outputs the digital signal. [0004) Certain CMOS image sensor pixel circuits are formed using four transistors and are known and referred to as 4T image sensor pixels or "4T pixels." FIG. 1 illustrates an exemplary design ofa 4T pixel 110 connected to a bit-line 120. As shown, the 4T CMOS image sensor pixel 110 includes a photodiode ("PD") that provides the photon to electron conversion, while a floating diffusion ("FD") point provides the electron to voltage conversion. The voltage per electron conversion of the FD is known as conversion gain ("CG") and is an important parameter for CMOS image sensors. Conversion gain boosts the pixel signal relative to the analog noise, thereby reducing the noise floor, and thereby enabling performance at lower light levels. [0005] For such CMOS image sensors, during the analog-to-digital conversion process, a comparator receives an analog voltage and compares the analog voltage with a ramp voltage. In one implementation of a CMOS image sensor, the comparator compares the analog voltage with the ramp voltage, and uses a counter to count until the ramp voltage is greater than an analog voltage. Once the counter stops counting, a count value is digital data corresponding to an analog voltage, that is, the count value is the digital data into which the analog voltage has been converted. [0006] Referring to FIG. 1, the pixel is reset when the reset transistor ("RST") and transfer gate ("TG") are turned on simultaneously, setting both the floating diffusion FD and the photodiode PD to the VDD voltage level. Next, the transfer gate TG is turned off (disconnecting the photodiode PD and floating diffusion FD) and the photodiode PD is left to integrate light. [0007] After integration, the signal measurement occurs. First, the reset transistor RST is turned on and off to reset the floating diffusion FD. Immediately after this, the reset level is sampled from the floating diffusion FD and stored on the column circuit, i.e., bit-line 120. Next, the transfer gate TG is turned on and off which allows charge on the photodiode PD to transfer to the floating diffusion (FD). Once the charge transfer is complete, this charge (the photodiode signal level plus the floating diffusion reset level) is measured and stored on bit-line 120 as well. [0008] These two stored voltages are then differenced (Dsig-Drst) to determine the photodiode signal level. The 4T pixel design 110 significantly improves the performance of other CMOS image sensors, reducing both read noise and image lag. In addition, the design reduces pixel source follower offsets and the like. SUMMARY [0009] In one exemplary aspect, an image sensing system is disclosed for providing dynamic pixel management to switch operational modes between high definition (HD) and ultra-high definition. In this aspect, the image sensor includes a complementary metal oxide semiconductor (CMOS) image sensor including a shared pixel array having a plurality of shared pixel units that each comprises at least two photodiodes and a shared floating diffusion; a plurality of vertical and horizontal charge circuity coupled to the CMOS image sensor and configured to activate the plurality of shared pixel units during image capture based on a set operational mode of the image sensor; and a dynamic pixel manager configured to switch the operational mode of the CMOS image sensor between a UHD mode and a HD mode in response to a user selection of an image resolution for the image captured by the image sensor. Moreove