CN-122017877-A - Infrared multimode composite imaging system based on APD detector

CN122017877ACN 122017877 ACN122017877 ACN 122017877ACN-122017877-A

Abstract

The invention discloses an infrared multi-mode composite imaging system based on an APD detector, which realizes three modes of rapid conversion through a switchable optical component, wherein the detector receives a thermal radiation signal emitted by the target in a passive imaging mode, irradiates the target by a laser in an active gating mode and realizes set target depth of field imaging by controlling a time window of the detector, and obtains distance information of the target by emitting laser and receiving a reflected echo thereof in an active 3D imaging mode. Meanwhile, the invention designs a double-layer fusion frame based on self-adaptive weight, space distance information in an active 3D imaging mode is encoded to a chromaticity channel in a YCbCr color space, heat radiation information and object contour details in a passive infrared mode are encoded to a brightness channel, and an image fusing two-dimensional heat information and three-dimensional space information is output.

Inventors

SUI XIUBAO
LI YUXUAN
JIA XULIANG
GAO HANG
SUN XU
YANG ZIQIANG
CHEN ZHIPENG
CHEN QIAN
GU GUOHUA

Assignees

南京理工大学

Dates

Publication Date: 20260512
Application Date: 20260304

Claims (10)

1. An infrared multimode composite imaging system based on an APD detector, comprising: the APD detector array is used for receiving the optical signals and converting the optical signals into electric signals to further obtain digital signals; a laser emitting unit for emitting pulsed laser light to a target; And the FPGA control and processing unit is respectively connected with the APD detector array and the laser emission unit; the FPGA control and processing unit is configured to switch the system to one of three modes of operation: 1) A passive imaging mode, wherein the APD detector array is controlled to receive and process thermal radiation signals of a target to generate a thermal radiation intensity image; 2) An active gate control imaging mode, which comprises the steps of controlling the laser emitting unit to emit laser and adjusting the exposure delay time of the APD detector array And integration time Establishing a foreground distance from an imaging depth of field And depth range The linear control relation of the set depth of field is adopted to enable the detector window to be precisely matched with the laser echo in the set depth of field in time and space, so that the gating imaging of the slice with the set depth of field is realized; 3) And in the active 3D imaging mode, the laser emission unit is controlled to emit laser, and based on a time of flight (TOF), the laser flight time is converted into linear slope voltage through a high linearity integration circuit to be quantized, so that high-precision target distance measurement and three-dimensional image generation are realized, and a distance image is obtained.
2. The system of claim 1, wherein when the system is switched to a passive imaging mode, the APD detector array is controlled to receive thermal radiation signals from the target and process the thermal radiation signals to generate thermal radiation intensity images, wherein the thermal radiation intensity images are as follows: The method comprises the steps that thermal radiation emitted by a detection target is received by a lens of an APD detector array, signals are focused on the surface of the APD detector, the detector converts the received optical signals into electric signals based on photoelectric conversion characteristics of mercury cadmium telluride materials, the electric signals are subjected to gain processing through an amplifying circuit to obtain analog electric signals, the analog electric signals are converted into digital signals, the converted digital signals enter a signal processing module to be processed, thermal radiation intensity images are obtained, and the digital signals are finally displayed on a display screen in real time.
3. The infrared multimode composite imaging system based on APD detector as set forth in claim 1, wherein when switching the system to the active gated imaging mode, the laser emitting unit is controlled to emit laser light and by adjusting the detector exposure delay time And integration time Establishing a foreground distance from an imaging depth of field And depth range The linear control relation of the set depth of field slice is that the detector window is matched with the laser echo in the set depth of field in time and space accurately, so that the gating imaging of the set depth of field slice is realized, and the method specifically comprises the following steps: The FPGA control and processing unit sends a trigger signal to the APD detector array, and the trigger signal is used for indicating the APD detector array to start exposure integration after fixed internal delay so as to receive echo signals of pulse laser; controlling exposure delay time of laser before APD detector array starts exposure Emitting pulse laser to target, controlling the integration time of detector To determine a depth of field range of the detection target; by precise adjustment And The two time sequence parameters realize the selection and control of the imaging depth of field.
4. An infrared multimode composite imaging system based on APD detectors as set forth in claim 3, wherein the exposure delay time is adjusted Foreground distance by selecting depth of field of target Adjusting integration time To determine depth range of target depth of field The formula is satisfied: , , Wherein, the Is the light propagation speed.
5. The infrared multimode composite imaging system based on the APD detector according to claim 1, wherein when the system is switched to an active 3D imaging mode, the laser emitting unit is controlled to emit laser, and based on a time of flight method TOF, the laser flight time is converted into a linear ramp voltage through a high linearity integrating circuit to quantize, so as to realize high-precision target distance measurement and three-dimensional image generation, and a distance image is obtained, which is specifically as follows: 3-1), controlling the laser to emit pulse laser light to the target; 3-2) synchronously starting a high linearity integration circuit which charges or discharges the capacitor by means of a constant current source, thereby generating a defined slew rate which varies linearly with time Is set to the ramp voltage of (2) Converting the time quantity into a voltage quantity for high-precision measurement; 3-3), sampling the current voltage when the APD detector array receives the reflected echo of the pulse laser, and recording the current voltage as the sampling voltage To ensure the measurement accuracy of the flight time, the linear ramp voltage generated by the integral voltage The effective working interval of (2) needs to have good linearity, so the voltage is ramped Limiting the sampling range of the system to be within a high linearity range of 1V-3V so as to minimize nonlinear errors from a hardware level; 3-4) according to the sampling voltage And ramp voltage Calculating the time of flight of the laser pulse And from this the distance of the target is obtained Time of flight The formula is as follows: , Wherein, the The initial value is sampled for the ramp voltage, Representing the slew rate; the TOF measurement target distance formula is: , Wherein, the Is the speed of light.
6. An infrared multimode composite imaging system based on APD detectors according to claim 5, wherein the high linearity integrating circuit is in particular as follows: The high linearity integrating circuit comprises an N-channel enhanced MOSFET and a capacitor Resistance of resistor A first capacitor A reference voltage source, an operational amplifier and a second capacitor Switching diode and first resistor ; Constant voltage across the first resistor The switch diode is connected in series with the input end of the reference voltage source, the output end of the reference voltage source and the resistor Capacitance Series-connected ground, capacitor The non-inverting input ends of the N-channel enhancement MOSFET and the operational amplifier are connected in parallel, and a first capacitor is respectively connected between the output and input pins of the reference voltage source and the grounding end of the reference voltage source And a second capacitor The two capacitors are connected with the output end of the operational amplifier in parallel and are commonly connected to the ground; the operational amplifier is configured as a voltage follower with its non-inverting input connected to the capacitor The output end provides low impedance voltage to ensure capacitance The voltage at two ends is the same as the output voltage of the operational amplifier, and the capacitor Is connected to ground at one end thereof, another end and resistor Connected in series with the output end of the reference voltage source, constant current passing through the resistor Flow direction capacitor So that the capacitance The voltage is gradually increased, namely the voltage is ramped, and the charging current passes through the resistor The magnitude of the charging current is transferred by a resistor And the reference voltage of the operational amplifier, and the output voltage of the operational amplifier follows the capacitor The voltage on the reference voltage source is driven to the ground end of the reference voltage source, so that the voltage is ensured to be kept constant; the operational amplifier controls the output voltage to make the capacitor The relation between the voltage at two ends and the reference voltage is stable, the calculation formula of the slope voltage is as follows, and the values of the resistor and the capacitor are determined according to the integration time: Because the current is constant, the capacitance Voltage of (2) over time Linear rise: , In the above-mentioned method, the step of, Representing capacitance The voltage across the two terminals of the capacitor, Representing resistance Is used for the resistance value of the (a), Representing capacitance Is a capacitance value of (2); Slope slew rate calculation principle: , Wherein, the Representing a reference voltage; Time constant The method comprises the following steps: , Current flowing through the circuit The method comprises the following steps: , Rate of roll The method comprises the following steps: 。
7. The infrared multi-mode composite imaging system based on the APD detector is characterized in that the FPGA control and processing unit comprises an image processing module and an information fusion module, wherein the image processing module is used for carrying out non-uniformity correction, blind pixel detection and replacement, contrast enhancement and pseudo-color processing on digital signals acquired by an APD detector array so as to improve image quality and obtain an infrared image, the information fusion module is used for calling the infrared image and a distance image to carry out layered fusion by designing a double-layer fusion frame based on self-adaptive weight to obtain a fusion image, and finally synthesizing the fusion image in a YCbCr color space to output a multi-information fusion image capable of simultaneously displaying target space dimension information and heat information, so that the depth fusion of two-dimensional heat information and three-dimensional space information is realized.
8. The infrared multimode composite imaging system based on APD detectors of claim 7, wherein the adaptive weight based bilayer fusion framework is as follows: 1) The feature-oriented self-adaptive weight generation module extracts features with physical significance, namely significance of the structure and significance of details, from the range profile and the infrared image respectively, and calculates a guiding fusion space weight image according to the features; 2) The multi-scale decomposition module is used for carrying out multi-scale decomposition and layered fusion on the images in the two modes to obtain respective corresponding low-frequency components and high-frequency components, and implementing self-adaptive fusion rules on a low-frequency layer and a high-frequency layer by utilizing a weight map, wherein the low-frequency layer comprises more than 80% of basic structures of the images, and the high-frequency layer has more than 80% of detail textures of the images, so that a structure weight map and a detail weight map are correspondingly obtained; 3) The hierarchical fusion module is used for generating a fused low-frequency image by combining the structure weight graph and adopting a weighted average fusion strategy in low-frequency component fusion; in the high-frequency component fusion, combining the detail weight map, adopting a feature selection strategy to reserve significant thermal boundary and detail information in the infrared image, and generating a fused high-frequency image; 4) The color space synthesis module is used for synthesizing a channel based on a YCbCr color space, taking a fusion image as a brightness component (Y), converting a pseudo-color three-dimensional distance image into the YCbCr space, then extracting a chromaticity component as color information (Cb, cr), and generating a new YCbCr image by directly replacing the channel, wherein the brightness of the new YCbCr image is completely determined by the fusion image, the clear visibility of a thermal target and a contour is ensured, the chromaticity of the new YCbCr image is completely determined by the distance information, a red-to-purple continuous color code is formed, and a multi-information fusion image is obtained, and is converted back into an RGB space for output so as to be displayed on an upper computer.
9. The APD detector-based infrared multimode composite imaging system of claim 7, wherein the FPGA control and processing unit is configured to perform the following steps to achieve pixel level fusion, in particular: The method comprises the steps of 1, before fusion processing, registering a thermal radiation intensity image generated by a passive imaging mode with a distance image generated by an active 3D imaging mode by an image processing module, enabling the two imaging modes to share the same optical view field through switching an optical assembly to ensure that space references are consistent, sequentially triggering the two imaging modes on time sequence control, completing continuous acquisition in a very short time to form a space alignment and time correlation image pair for fusion, and sequentially carrying out non-uniformity correction, blind pixel detection and replacement, contrast enhancement and pseudo-color processing on the passive thermal radiation intensity image to obtain an infrared image; Step 2, the information fusion module performs feature analysis and weight generation, extracts features with physical significance, namely significance of a structure and significance of details from the distance image and the infrared image respectively, and calculates a spatial weight graph for guiding fusion according to the features, performs multi-scale decomposition and layered fusion on the images in two modes, decomposes the images to different frequency bands through guiding filtering, and implements self-adaptive fusion rules on a low-frequency layer and a high-frequency layer by utilizing the weight graph, wherein the low-frequency layer comprises more than 80% of basic structures of the images, and the high-frequency layer comprises more than 80% of detail textures of the images to obtain a fused image; and 3, performing color mapping on the fusion image, combining the structure and texture information in the fusion image and the pseudo-color coded distance information in a YCbCr color space to realize image reconstruction and generate a multi-information fusion image.
10. The infrared multimode composite imaging system of claim 9, wherein in step 1, the non-uniformity correction comprises single-point correction and two-point correction, the blind pixel detection and replacement comprises detection based on a neighborhood difference method and compensation based on gradient judgment, and the contrast enhancement adopts a contrast limited adaptive histogram equalization algorithm.

Description

Infrared multimode composite imaging system based on APD detector Technical Field The invention belongs to the technical field of infrared photoelectric detection and imaging, and particularly relates to a multimode and multidimensional information fusion imaging system based on an Avalanche Photodiode (APD) detector, which integrates and cooperates with three technologies of passive infrared imaging, active gating imaging and active 3D imaging to realize synchronous or selective acquisition of target radiation intensity, set depth of field detail and space distance information. Background Infrared imaging technology is a key means for target perception at night and in severe environments. The passive infrared imaging system realizes imaging by detecting the infrared radiation of the target, and has the unique advantage of strong concealment. However, such systems have inherent limitations in that firstly, they can only provide two-dimensional intensity information of the target, but cannot acquire critical distance and three-dimensional spatial information, and secondly, in the case of an extremely small or no temperature difference between the environment and the target, the imaging contrast is low, and details are difficult to distinguish. In a practical complex application scenario, it is far from sufficient to rely on two-dimensional thermal radiation intensity images alone. For example, in the presence of smoke, rain and snow or partial obstruction, there is a strong need for the ability to penetrate interference, clearly observe the target behind a set depth of field, while in many applications, accurate three-dimensional ranging and spatial modeling of the target is critical to achieving autonomous decision making and accurate operation. To compensate for the shortfall of passive imaging, active imaging techniques (such as laser gated imaging and lidar) provide an effective complement. The gating imaging can realize selective imaging of targets in a set distance section by controlling laser irradiation and a detector receiving window, and the laser radar can directly acquire high-precision three-dimensional point cloud data. However, the existing solutions often represent independent systems with a single function, or only have some of the above capabilities. The problems of system function cutting, incomplete information dimension, difficult collaborative fusion of data and the like are caused, so that a user cannot acquire comprehensive perception of a target under the same space-time reference. Therefore, there is an urgent need in the art for a multifunctional integrated imaging system, which can implement the intrinsic fusion and cooperative work of active and passive imaging modes on the same hardware platform, so as to synchronously acquire the radiation characteristics of the target, set depth of field details and accurate distance information, and fundamentally solve the problems that the information dimension of the existing imaging scheme is single and difficult to adapt to the complex application requirements. Disclosure of Invention The invention aims to provide an infrared multi-mode composite imaging system based on an APD detector, which aims to solve the problems of single function and insufficient information dimension of the existing infrared imaging system. The system integrates the functions of passive infrared imaging, active gate control imaging and active 3D imaging, and is more critical in realizing depth fusion and collaboration of information among different modes and improving the target detection and recognition capability under complex scenes. The scheme of the invention is that an infrared multi-mode composite imaging system based on an APD detector is characterized by comprising the following components: the APD detector array is used for receiving the optical signals and converting the optical signals into electric signals to further obtain digital signals; a laser emitting unit for emitting pulsed laser light to a target; And the FPGA control and processing unit is respectively connected with the APD detector array and the laser emission unit; the FPGA control and processing unit is configured to switch the system to one of three modes of operation: 1) And in a passive imaging mode, controlling the APD detector array to receive a thermal radiation signal of the target and processing the thermal radiation signal to generate a thermal radiation intensity image. 2) An active gate control imaging mode, which comprises the steps of controlling the laser emitting unit to emit laser and adjusting the exposure delay time of the APD detector arrayAnd integration timeEstablishing a foreground distance from an imaging depth of fieldAnd depth rangeThe linear control relation of the set depth of field slice is realized by enabling the detector window to be precisely matched with the laser echo in the set depth of field in time and space, so that the gating imaging of the set dep