US-12626116-B2 - Integrated optical neuromorphic computing apparatus

Abstract

A hybrid neuromorphic computing device is provided, in which artificial neurons include light-emitting devices that provide weighted sums of inputs as light output. The output is detected by a photodetector and converted to an electrical output. Each neuron may receive output from one or more other neurons as initial input. Interconnects between neurons may be optical, electrical, or a combination thereof. The neurons also may provide imaging sensor and/or display capabilities.

Inventors

• Michael Hack
• PAUL PRUCNAL
• Thomas Ferreira de Lima
• Michael S. Weaver
• Simon Bilodeau

Assignees

• UNIVERSAL DISPLAY CORPORATION
• THE TRUSTEES OF PRINCETON UNIVERSITY

Dates

Publication Date

20260512

Application Date

20210628

Claims (18)

1. 1 . A device comprising: an imaging device comprising a plurality of first optical sensors disposed in a two-dimensional array; and a neuromorphic computing device comprising a plurality of artificial neurons, the neuromorphic computing device configured to process image data captured by the imaging device, each of the plurality of artificial neurons comprising one or more integrated emissive optical components; wherein integration of the inputs to each artificial neuron of the plurality of artificial neurons is performed in the each neuron; wherein the optical sensors in the imaging device and the integrated emissive optical components in the neuromorphic computing device are formed from organic optical sensors, organic photovoltaic cells, OLEDs, or a combination thereof; and wherein the neuromorphic computing device provides image output data having a resolution different than the image data captured by the imaging device.
2. 2 . A device comprising: an imaging device comprising a plurality of first optical sensors disposed in a two-dimensional array; and a neuromorphic computing device comprising a plurality of artificial neurons, the neuromorphic computing device configured to process image data captured by the imaging device, each of the plurality of artificial neurons comprising one or more integrated emissive optical components; wherein integration of the inputs to each artificial neuron of the plurality of artificial neurons is performed in the each neuron; and wherein the optical sensors in the imaging device and the integrated emissive optical components in the neuromorphic computing device are formed from organic optical sensors, organic photovoltaic cells, OLEDs, or a combination thereof.
3. 3 . The device of claim 2 , further comprising a display configured to display image data captured by the imaging sensor, output of the neuromorphic computing device, or a combination thereof.
4. 4 . The device of claim 3 , wherein the display comprises a plurality of pixels, each pixel being driven by a set of instructions that indicate when the pixel should update based on current outputs of neighboring pixels, past outputs of neighboring pixels, or a combination thereof.
5. 5 . The device of claim 2 , further comprising an infrared sensor, wherein the device is configured to determine a body temperature of the person based upon data from the infrared sensor.
6. 6 . The device of claim 2 , wherein at least some of the plurality of artificial neurons operates as pixels or sub-pixels in an active-matrix display.
7. 7 . The device of claim 2 , wherein each of the plurality of artificial neurons comprises at least one of the plurality of first optical sensors.
8. 8 . The device of claim 2 , wherein each of the plurality of artificial neurons comprises: at least one second optical sensor configured to integrate outputs of the one or more integrated emissive optical components; wherein each output of each of the one or more integrated emissive optical components is determined by one or more inputs to the each integrated emissive optical component.
9. 9 . The device of claim 8 , wherein each output of each of the one or more integrated emissive optical components is determined by one or more weighted inputs to the each integrated emissive optical component.
10. 10 . The device of claim 8 , wherein the at least one second optical sensor configured to sum outputs is one of the first optical sensors disposed in the two-dimensional array.
11. 11 . The device of claim 2 , wherein the processing of the image data captured by the imaging device by the neuromorphic computing device comprises providing a visual output based on the imaging data.
12. 12 . The device of claim 11 , wherein at least some of the plurality of artificial neurons operates as pixels or sub-pixels in an active-matrix display.
13. 13 . The device of claim 11 , wherein the visual output is provided as an overlay of an image output provided by the imaging device.
14. 14 . The device of claim 11 , wherein a latency between acquisition of image data by the imaging device and output by the neuromorphic computing device is less than 20 ms.
15. 15 . The device of claim 11 , wherein a latency between acquisition of image data by the imaging device and display of the visual output is less than 20 ms.
16. 16 . The device of claim 2 , wherein, for a portion of image data captured by the imaging device having a data size D, the neuromorphic computing device is capable of outputting imaging data having a data size not more than R that is sufficient to display the image data, wherein R is not more than 0.1D.
17. 17 . An electronic device comprising: a device comprising: an imaging device comprising a plurality of optical sensors disposed in a two-dimensional array; and a neuromorphic computing device comprising a plurality of artificial neurons, the neuromorphic computing device configured to process image data captured by the imaging device, each of the plurality of artificial neurons comprising one or more integrated emissive optical components; wherein summation or integration of inputs to each artificial neuron of the plurality of artificial neurons is performed in the each neuron; and wherein the optical sensors in the imaging device and the integrated emissive optical components in the neuromorphic computing device are formed from organic optical sensors, organic photovoltaic cells, OLEDs, or a combination thereof.
18. 18 . The electronic device of claim 17 , wherein the device is at least one type selected from the group consisting of: a flat panel display, a curved display, a computer monitor, a medical monitor, a television, a billboard, a light for interior or exterior illumination, signaling, or a combination thereof, a heads-up display, a fully or partially transparent display, a flexible display, a rollable display, a foldable display, a stretchable display, a laser printer, a telephone, a cell phone, tablet, a phablet, a personal digital assistant (PDA), a wearable device, a laptop computer, a digital camera, a camcorder, a viewfinder, a micro-display that is less than 2 inches diagonal, a 3-D display, a virtual reality or augmented reality display, a vehicle, a video walls comprising multiple displays tiled together, a theater or stadium screen, a drone, eyeglasses, an autonomous vehicle, a smart camera, a health-monitoring device, a smart sensor, and a sign.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a non-provisional of, and claims the priority benefit of U.S. Provisional Patent Application Ser. No. 63/045,660, filed Jun. 29, 2020, the entire contents of which are incorporated herein by reference. GOVERNMENT SUPPORT This invention was made with government support under Contract No. HR0011-19-9-0049 awarded by the Defense Advanced Research Projects Agency (DARPA). The government has certain rights in the invention. FIELD The present invention relates to devices including neuromorphic computing devices made from artificial neurons, which may also provide imaging and/or display functions within a combined device such as through use of organic light emitting diodes, and devices including the same. BACKGROUND Opto-electronic devices that make use of organic materials are becoming increasingly desirable for a number of reasons. Many of the materials used to make such devices are relatively inexpensive, so organic opto-electronic devices have the potential for cost advantages over inorganic devices. In addition, the inherent properties of organic materials, such as their flexibility, may make them well suited for particular applications such as fabrication on a flexible substrate. Examples of organic opto-electronic devices include organic light emitting diodes/devices (OLEDs), organic phototransistors, organic photovoltaic cells, and organic photodetectors. For OLEDs, the organic materials may have performance advantages over conventional materials. For example, the wavelength at which an organic emissive layer emits light may generally be readily tuned with appropriate dopants. OLEDs make use of thin organic films that emit light when voltage is applied across the device. OLEDs are becoming an increasingly interesting technology for use in applications such as flat panel displays, illumination, and backlighting. Several OLED materials and configurations are described in U.S. Pat. Nos. 5,844,363, 6,303,238, and 5,707,745, which are incorporated herein by reference in their entirety. One application for phosphorescent emissive molecules is a full color display. Industry standards for such a display call for pixels adapted to emit particular colors, referred to as “saturated” colors. In particular, these standards call for saturated red, green, and blue pixels. Alternatively the OLED can be designed to emit white light. In conventional liquid crystal displays emission from a white backlight is filtered using absorption filters to produce red, green and blue emission. The same technique can also be used with OLEDs. The white OLED can be either a single EML device or a stack structure. Color may be measured using CIE coordinates, which are well known to the art. As used herein, the term “organic” includes polymeric materials as well as small molecule organic materials that may be used to fabricate organic opto-electronic devices. “Small molecule” refers to any organic material that is not a polymer, and “small molecules” may actually be quite large. Small molecules may include repeat units in some circumstances. For example, using a long chain alkyl group as a substituent does not remove a molecule from the “small molecule” class. Small molecules may also be incorporated into polymers, for example as a pendent group on a polymer backbone or as a part of the backbone Small molecules may also serve as the core moiety of a dendrimer, which consists of a series of chemical shells built on the core moiety. The core moiety of a dendrimer may be a fluorescent or phosphorescent small molecule emitter. A dendrimer may be a “small molecule,” and it is believed that all dendrimers currently used in the field of OLEDs are small molecules. As used herein, “top” means furthest away from the substrate, while “bottom” means closest to the substrate. Where a first layer is described as “disposed over” a second layer, the first layer is disposed further away from substrate. There may be other layers between the first and second layer, unless it is specified that the first layer is “in contact with” the second layer. For example, a cathode may be described as “disposed over” an anode, even though there are various organic layers in between. As used herein, “solution processible” means capable of being dissolved, dispersed, or transported in and/or deposited from a liquid medium, either in solution or suspension form. A ligand may be referred to as “photoactive” when it is believed that the ligand directly contributes to the photoactive properties of an emissive material. A ligand may be referred to as “ancillary” when it is believed that the ligand does not contribute to the photoactive properties of an emissive material, although an ancillary ligand may alter the properties of a photoactive ligand. As used herein, and as would be generally understood by one skilled in the art, a first “Highest Occupied Molecular Orbital” (HOMO) or “Lowest Unoccupied Molecular Orbital” (L