Search

CN-121259113-B - Pixel response inconsistency correction method and system

CN121259113BCN 121259113 BCN121259113 BCN 121259113BCN-121259113-B

Abstract

The invention discloses a pixel response inconsistency correction method and a pixel response inconsistency correction system. The method relates to the technical field of pixel response correction, and comprises the following steps of pixel dynamic offset calculation, flat field coefficient acquisition, pixel-level comprehensive correction and dead pixel self-repair secondary optimization. The invention calculates the pixel dynamic offset by acquiring dark frame images and real-time temperature data and using a temperature-time dual-domain prediction model, fuses the calibrated basic value of the whole spectrum section of a miniature integrating sphere in a laboratory and the updated value of deep space and cloud top in-orbit observation, dead pixel detection and other processing to obtain a flat field coefficient, combines GPS, attitude angle and real-time temperature, uses the offset and the flat field coefficient to perform pixel level correction, outputs a gain compensation matrix by a deep reinforcement learning model comprising a convolutional neural network and a long-term and short-term memory network, secondarily optimizes images by a dead pixel self-repair module, improves the correction precision of pixel response inconsistency, and solves the problem of large correction error of pixel response inconsistency in the prior art.

Inventors

  • QIAO LEI
  • CHEN JIANPENG
  • SUN LI

Assignees

  • 星际光遥(北京)科技有限公司

Dates

Publication Date
20260505
Application Date
20250909

Claims (10)

  1. 1. A method for correcting response inconsistency of pixels, comprising the steps of: acquiring a dark frame image, combining camera temperature data acquired by a temperature sensor in real time, and calculating a pixel dynamic offset based on a temperature-time dual-domain prediction model; The method comprises the steps of obtaining a flat field coefficient, wherein the flat field coefficient is generated by a laboratory calibration basic value and an on-orbit uniform scene observation updating value through a fusion algorithm, wherein the laboratory calibration basic value is obtained through the calibration of a laboratory miniature integrating sphere and covers the whole spectrum of a camera; performing pixel-level correction on an original image of a remote sensing camera based on pixel dynamic offset and a flat field coefficient, and synchronously integrating GPS position data, attitude angle data and real-time temperature data of satellites to assist in optimizing correction precision in the correction process, wherein the attitude angle data is used for compensating pixel response deviation caused by camera sideslip, and the GPS position data is used for matching imaging parameters under different orbit heights; And after correction, performing secondary optimization on the image through a dead pixel self-repairing module, wherein the dead pixel self-repairing module is realized based on a pixel gain compensation matrix output by a deep reinforcement learning model, the input of the deep reinforcement learning model comprises real-time temperature, satellite attitude angle and pixel history response curve, a network structure adopts a convolutional neural network and a long-period memory network time sequence unit to combine, and the output pixel gain compensation matrix is used for compensating response deviation of individual pixels caused by long-term on-orbit aging and radiation damage.
  2. 2. The method for correcting the response inconsistency of pixels according to claim 1, wherein the specific process of acquiring the dark frame image is: in the satellite orbit shadow period, the mechanical shutter is closed to trigger exposure instantly, so that the non-interruption acquisition of dark frame images is realized, and the occupation of the effective time length of the camera to earth observation is avoided; The temperature data of the camera focal plane assembly are collected in real time through a temperature sensor integrated in a camera detector reading circuit, so that the accuracy of the temperature data of an input temperature-time two-domain prediction model is ensured; and calculating the dynamic dark level offset by adopting a temperature-time dual-domain prediction model based on the acquired temperature data and the dark frame image.
  3. 3. The method for correcting the response inconsistency of pixels according to claim 1, wherein the aging attenuation coefficient of the detector in the temperature-time dual-domain prediction model is updated periodically by dark frame data collected at a month level; acquiring dark frame data of a preset number of groups in a satellite orbit shadow period, fitting a response attenuation trend of the detector based on the plurality of groups of data, and calculating to obtain an aging attenuation coefficient of the detector in the current month, so as to realize dynamic adaptation of the model to the aging state of the detector; The temperature second-order fitting parameters in the temperature-time double-domain prediction model are determined by fitting together data in a laboratory stage and in-orbit stage, temperature-dark current data are acquired by simulating different temperature environments through a miniature integrating sphere in the laboratory stage, temperature sensor data and dark frame data are synchronized in the in-orbit stage, and the two groups of data are jointly fitted to obtain the temperature second-order fitting parameters so as to ensure the nonlinear response suitability of the model to temperature change and adapt to scenes with high temperature sensitivity in an infrared band.
  4. 4. The method of claim 1, wherein the acquiring a dark frame image further comprises: Filtering cosmic ray pollution through an inter-frame standard deviation detection algorithm, and constructing a pixel stacking area for multi-frame data of the same pixel in a dark frame image; Calculating standard deviation and median standard deviation of the multi-frame data construction pixel stacking area, judging whether the standard deviation is greater than three times of median standard deviation, if so, replacing polluted pixels by adopting a median value in a 3X 3 neighborhood range of the pixels, otherwise, keeping unchanged; after calculating the pixel dynamic offset based on the temperature-time double-domain prediction model, combining the clamping correction function of a camera focal plane circuit, clamping pixel output signals on a dark level reference corresponding to the dynamic offset, and enabling all effective pixels output by the linear array to be smaller than a preset quantization reference value after clamping.
  5. 5. The method for correcting the response inconsistency of pixels of claim 1, wherein said obtaining of said laboratory calibration basis values comprises: performing full-spectrum coverage type calibration on a calibration scene through a miniature integrating sphere in a laboratory, wherein the radiance calibration error of the miniature integrating sphere is smaller than or equal to a calibration error reference value, the radiance stability is smaller than or equal to a radiance reference value, the surface uniformity peak-valley value is smaller than or equal to a peak-valley value reference value and the angle uniformity is smaller than or equal to an angle uniformity reference value; The step of obtaining the on-orbit uniform scene observation updated value comprises the following steps: during the on-orbit operation of a camera, selecting a deep space and a cloud top as uniform observation scenes, performing dead point detection and inconsistent difference detection through time-space domain joint analysis, and generating scene observation updated values by adopting neighborhood interpolation processing; The dead pixel detection is realized through inter-frame standard deviation detection, and if the standard deviation of the pixel stack is more than three times of the median standard deviation, the dead pixel is judged to be dead pixel and median replacement is adopted; the inconsistent difference detection is based on time-space domain characteristics of camera imaging data, compares response differences of pixels at different moments and different positions, and identifies pixel areas with inconsistent responses; and the neighborhood interpolation processing adopts a mode of combining wavelet transformation with median filtering to repair the detected dead pixels and the response inconsistent areas.
  6. 6. The method for correcting the response inconsistency of a pixel of claim 1, wherein the step of obtaining the flat field coefficients comprises: Generating a flat field coefficient by a fusion algorithm, wherein the fusion weight decays along with the ageing index of the detector, the ageing index of the detector is obtained by monitoring month-level dark frame data, the fusion weight value is dynamically adjusted according to the ageing degree of the detector, the fusion weight value is in a fusion weight reference interval in an initial state, and gradually decreases along with the increase of the ageing degree of the detector, and the lowest fusion weight value is not lower than the fusion weight reference lower limit so as to ensure that the flat field coefficient can embody the basic precision of laboratory calibration.
  7. 7. The method for correcting the response inconsistency of pixels according to claim 1, wherein the specific process of performing the pixel level correction is: Acquiring triaxial stable attitude data of a satellite, extracting a camera sideslip angle, calculating a pixel response deviation compensation coefficient according to the sideslip angle, integrating the sideslip angle into a pixel level correction formula, and compensating pixel response non-uniformity caused by sideslip to reduce pixel response deviation in a sideslip state; Acquiring satellite GPS position data, calculating an imaging parameter matching coefficient according to the orbit height, and adjusting the local weight of the leveling field coefficient through the imaging parameter matching coefficient to match the imaging resolution requirements under different orbit heights based on a ground pixel resolution formula if the orbit height deviates from an orbit height reference value; And carrying out pixel level correction on the original image by adopting a correction formula, wherein the correction process realizes pipeline processing through irradiation resistance, and sequentially completing dark current subtraction, flat field correction, attitude deviation compensation and track parameter matching.
  8. 8. The method for correcting the response inconsistency of pixels according to claim 1, wherein the performing the second optimization on the image by the dead pixel self-repairing module comprises the following specific steps: acquiring real-time temperature data of a camera during on-orbit operation the satellite attitude angle data and the pixel history response curve are consistent with the updating frequency of the integration time; the method comprises the steps that a structure combining a convolutional neural network and a long-short-term memory network time sequence unit is adopted, the convolutional neural network is used for extracting spatial features in real-time temperature and attitude angle data, the long-short-term memory network time sequence unit is used for excavating time sequence change rules of a pixel historical response curve, and output features of the two are fused through a full-connection layer to generate a pixel gain compensation matrix; Taking the corrected pixel response non-uniformity residual error minimization as an objective function, adopting a laboratory simulation on-orbit aging and radiation damage pixel response data set to pretrain, and performing fine adjustment during on-orbit operation by utilizing newly acquired pixel response data so as to reduce the prediction error of individual pixel response deviation; Performing pixel-by-pixel matching on the corrected image and a pixel gain compensation matrix output by a depth reinforcement learning model, calculating the deviation rate of an actual response value and a theoretical response value of a pixel, judging whether the deviation rate is larger than a deviation rate reference value, if so, judging that the pixel is a response abnormal dead pixel caused by long-term on-orbit aging and radiation damage, and if not, not judging; And repairing the determined abnormal dead pixel response by adopting a gain compensation value of a corresponding pixel in a pixel gain compensation matrix so as to reduce gradient difference between the repaired pixel value and surrounding normal pixel values, and carrying out local MTF detection on the image after secondary optimization.
  9. 9. The method for correcting the response inconsistency of pixels according to claim 8, wherein said performing a secondary optimization of the image by the dead pixel self-repair module further comprises: realizing on-orbit uploading of model parameters through a satellite CAN bus, wherein a focal plane circuit is in a low-power-consumption working mode in the uploading process, and uploading data adopts frame verification; adopting a circulating storage mechanism, automatically covering earliest historical data when the stored data quantity reaches the upper limit, and periodically screening the historical data to remove invalid data caused by cosmic ray interference and temperature shock; After each track of imaging is finished, extracting a uniform region in the corrected image, calculating standard deviation of pixel response of the region, judging whether the standard deviation of pixel response is larger than a standard deviation reference value, if yes, triggering a model parameter fine tuning flow, wherein the fine tuning process only updates hidden layer parameters of a long-short-period memory network time sequence unit, and the fine tuning time is smaller than the fine tuning time reference value, so that the pixel gain compensation matrix output by the model is ensured to continuously adapt to on-track environment changes.
  10. 10. A pixel response inconsistency correction system, the pixel response inconsistency correction system applying the pixel response inconsistency correction method of any of claims 1-9, comprising a pixel dynamic offset calculation module, a flat field coefficient acquisition module, a pixel level comprehensive correction module, and a dead pixel self-repair secondary optimization module: The pixel dynamic offset calculating module is used for acquiring a dark frame image, combining camera temperature data acquired by a temperature sensor in real time and calculating pixel dynamic offset based on a temperature-time double-domain prediction model; The flat field coefficient acquisition module is used for acquiring a flat field coefficient, wherein the flat field coefficient is generated by a laboratory calibration basic value and an on-orbit uniform scene observation updated value through a fusion algorithm, the laboratory calibration basic value is acquired through the calibration of a laboratory miniature integrating sphere and covers the whole spectrum of a camera; The pixel-level comprehensive correction module is used for performing pixel-level correction on an original image of the remote sensing camera based on pixel dynamic offset and flat field coefficient, and in the correction process, satellite GPS position data, attitude angle data and real-time temperature data are synchronously integrated to assist in optimizing correction precision, the attitude angle data are used for compensating pixel response deviation caused by camera sideslip, and the GPS position data are used for matching imaging parameters under different track heights; The dead pixel self-repairing secondary optimization module is used for performing secondary optimization on the image through the dead pixel self-repairing module after correction, the dead pixel self-repairing module is realized based on a pixel gain compensation matrix output by the deep reinforcement learning model, the input of the deep reinforcement learning model comprises real-time temperature, satellite attitude angles and pixel historical response curves, a network structure adopts a convolutional neural network and a long-period memory network time sequence unit to be combined, and the output pixel gain compensation matrix is used for compensating response deviation of individual pixels caused by long-period on-orbit aging and radiation damage.

Description

Pixel response inconsistency correction method and system Technical Field The invention relates to the technical field of pixel response correction, in particular to a pixel response inconsistency correction method and system. Background In the field of remote sensing camera imaging, response characteristic parameters of each pixel are obtained through laboratory calibration, a pixel response inconsistency model is established, and a linear or nonlinear fitting method is generally adopted to describe the relationship between pixel response and radiation brightness. Secondly, based on a multi-frame image pixel-by-pixel non-uniformity correction technology, a 3D residual error network and other deep learning methods are utilized to dynamically compensate pixel response differences, and the method can effectively process non-uniformity noise in a remote sensing image and improve image quality. Then, geometric correction is carried out by combining ground control points, and a coordinate transformation relation between an original image and a corrected image is established by selecting ground characteristic points with known geographic coordinates, so that geometric distortion caused by factors such as sensor posture, earth curvature and the like is eliminated. And then resampling the image by adopting a control point correction method, ensuring that the corrected image has accuracy on geographic coordinates, and unifying a coordinate system and projection parameters. Finally, the digital quantized value of the pixel is converted into a radiation brightness value through radiation calibration, so that the influence of the response inconsistency of the pixel on quantitative remote sensing analysis is further eliminated. The whole process combines laboratory calibration, a deep learning algorithm and a geometric correction technology, and can remarkably improve the radiation consistency and geometric accuracy of remote sensing images. A pixel response non-uniformity correction method, a system, electronic equipment and a medium for the patent bulletin with the bulletin number of CN115567651B comprise the steps of obtaining micro-light observation image data output by a satellite-borne micro-light imager, preprocessing the micro-light observation image data, sequentially performing relative deviation calculation and histogram processing on the preprocessed micro-light observation image data to determine observed image bright line data, and sequentially performing bilinear interpolation processing and mapping correction processing on the observed image bright line data to obtain corrected micro-light observation image data. A correction device and method for non-linear response of pixel output in an image sensor, for example, an invention patent publication with publication number CN103118235B, comprises a photosensitive pixel array, a dark pixel, a row selection switch of the photosensitive pixel, a dark pixel switch, a photosensitive pixel bias current source, a dark pixel bias current source, a linearization circuit, an analog column readout circuit, a row selection and exposure control decoder. Compared with the traditional image sensor, the invention adds a nonlinear correction circuit consisting of a dark pixel and a linearization circuit, and the correction circuit occupies a smaller chip area. The above technology has at least the following technical problems: The traditional pixel response inconsistency correction is highly dependent on a basic flat field coefficient calibrated by integrating spheres and other devices in a laboratory stage, and the coefficient can only reflect the pixel response characteristic of a camera in a ground standard environment. However, after the camera enters in-orbit working, the camera is exposed to complex environments such as space radiation, temperature circulation change, detector aging and the like for a long time, so that the response characteristic of the pixels continuously drifts, in order to make up the limitation of laboratory calibration, partial technologies adopt an in-orbit scene statistical method (such as updating a flat field by using scenes such as uniform ground objects and cloud tops) to correct, but the methods rely on statistical average operation on global or local areas of images. In the operation process, high-frequency details (such as ground object edges, fine textures and the like) in the smooth image can be inevitably avoided, so that a modulation transfer function of the camera is lowered; For an infrared band camera, the pixel response is more obviously affected by temperature, and especially in a scene of large day and night temperature difference of a Tibet plateau and abrupt change of the temperature of an on-orbit camera, the nonlinear effect of temperature change on the pixel response cannot be dynamically tracked by the traditional two-point correction method (by determining correction coefficients through dark frames and bright frames).