CN-122016041-A - Unmanned aerial vehicle-mounted multi-sensor automatic radiometric calibration system and method
Abstract
The system comprises an unmanned aerial vehicle platform, a load integration module, a data acquisition module, a data processing module and a data storage and output module, wherein the system and the method can realize automatic identification of PIF ground features, relative normalization of multi-sensor data and real-time conversion of absolute reflectivity, and provide high-precision and high-reliability data support for quantitative analysis of unmanned aerial vehicle multi-source remote sensing data in a field complex environment.
Inventors
- CAO LEI
- ZHANG YING
- ZHOU YI
- LIU YINGYING
- Wu Bozhang
- SHI JIANBO
- LI HAO
- GAO CHAO
Assignees
- 国网湖北省电力有限公司神农架供电公司
- 国网湖北省电力有限公司电力科学研究院
Dates
- Publication Date
- 20260512
- Application Date
- 20251215
Claims (11)
- 1. The unmanned aerial vehicle-mounted multi-sensor automatic radiation calibration system is characterized by comprising an unmanned aerial vehicle platform, a load integration module, a data acquisition module, a data processing module and a data storage and output module: the unmanned plane platform adopts a multi-rotor or fixed wing aircraft type, has high-precision flight control capability, and can set a route, a height and a speed according to tasks; the load integration module comprises a multispectral camera, a hyperspectral camera and an upward-mounted high-precision spectral irradiance sensor, and is used for synchronously acquiring ground object images and downlink solar irradiance; The data acquisition module consists of a Field Programmable Gate Array (FPGA) synchronous control unit and a gigabit Ethernet/wireless transmission unit, ensures that the time stamp of the image and irradiance data is consistent and is transmitted to the data processing module in real time; The data processing module is used for running a PIF automatic identification and cross-sensor radiation normalization and absolute reflectivity inversion algorithm based on an embedded processor or an industrial personal computer to realize real-time data processing; The data storage and output module comprises a solid state disk, a USB 3.0, an HDMI and a network interface, and is used for storing original and calibration data and supporting local export and real-time display.
- 2. Based on the system, the invention provides an unmanned aerial vehicle-mounted multi-sensor automatic radiation calibration method, which is characterized by comprising the following steps of: step S1, system initialization and parameter configuration Before the unmanned aerial vehicle takes off, the whole calibration system is initialized and configured, and the method specifically comprises the following steps: Before the unmanned aerial vehicle takes off, initializing a calibration system, which comprises the following configurations: sensor parameters including input of internal, external and spectral response functions of multispectral and hyperspectral camera, sensitivity coefficient of irradiance sensor And dark current ; Flight parameters, namely, setting the flight altitude according to the operation area and the resolution requirements Heading overlap 80%, side overlap 70%, speed of flight Image acquisition frequency ; Algorithm parameters of setting spectrum stability threshold value of PIF identification Threshold of agreement with space, number of generations of robust regression And convergence threshold An atmospheric correction parameter; Step S2, synchronous data acquisition After the unmanned aerial vehicle takes off according to a preset route, the generation period of the synchronous control unit is as follows The synchronous trigger signals of the sensor are respectively sent to a main imaging sensor group and an irradiance measuring unit, and the main imaging sensor group and the irradiance measuring unit are controlled to synchronously start data acquisition: The multispectral camera and the hyperspectral camera in the main imaging sensor group collect ground object images simultaneously, and multispectral image data are obtained And hyperspectral image data Wherein For the pixel coordinates, Is a characteristic band wavelength of a multispectral camera, For the continuous band wavelengths of the hyperspectral camera, For the number of bands of the multi-spectral camera, For the number of bands of the hyperspectral camera, The irradiance measuring unit synchronously measures the total radiant flux density of the downlink sun to obtain irradiance data Wherein To acquire the time stamps, it is ensured that each frame of image data corresponds to unique irradiance data, The data acquisition module adds metadata information such as a time stamp, a sensor number and the like to the acquired image data and irradiance data, and then transmits the metadata information to the data processing module in real time through the data transmission unit; step S3, PIF ground object automatic identification and algorithm design The PIF automatic identification module in the data processing module processes the multispectral and hyperspectral images acquired synchronously, and automatically extracts PIF ground objects with stable spectrums; Step S4 Cross-sensor radiation normalization The core of cross-sensor radiation normalization is to establish a radiation response conversion relation between multispectral and hyperspectral sensors, normalize output data of different sensors to a unified radiation reference, eliminate radiation deviation caused by response difference of the sensors, and the steps are realized by three substeps of band matching, outlier rejection and robust regression modeling by taking DN values of PIF ground object areas as references; Step S5, absolute reflectivity conversion-real-time conversion algorithm based on coupling online irradiance The radiation data normalized by the cross-sensor is still a relative value, a 'real-time absolute reflectivity conversion algorithm coupled with on-line irradiance' is needed, irradiance data acquired by an unmanned aerial vehicle onboard sensor in real time is used as a dynamic reference, an atmosphere correction model is simplified, environmental interference of illumination intensity fluctuation and atmosphere transmission attenuation is eliminated, and a normalized DN value is converted into a standardized earth surface absolute reflectivity.
- 3. The method for automatic radiation calibration of the unmanned aerial vehicle multi-sensor according to claim 2, wherein in step S3, the following is specifically adopted: Step S3.1 image preprocessing S3.1.1 radiation pretreatment, eliminating system noise (1) Dark current correction is carried out by adopting a strategy of dark field calibration and band-by-band subtraction, and a mathematical model is as follows: , Wherein: pixel location in original image In the wave band Is a numerical value of (a); the dark current reference value corresponding to the wave band is obtained by collecting 50 frames of dark field images through a shielding lens and taking the average value; (2) Defective pixel repair Repairing by adopting a 3×3 neighborhood median filtering method, wherein if the deviation between DN value and neighborhood mean value of each pixel exceeds 3 times of standard deviation, judging that the pixel is a dead pixel; replacing the pixel value with a median value of the neighborhood center position; S3.1.2 geometric pretreatment, which is to eliminate spatial distortion Orthographic correction is carried out by adopting a rational polynomial coefficient RPC model, the pixel coordinates of the image are converted into geographic coordinates of the ground, the consistency expression of the ground object position is realized, Corrected pixel Corresponding to the ground coordinates The calculation formula of (2) is as follows: Wherein: the coordinates of the pixels of the image before correction, Corresponding ground projection coordinates are adopted, RPC model coefficients are obtained by a camera calibration experiment.
- 4. The method for automatic radiation calibration of multiple sensors on an unmanned aerial vehicle according to claim 3, wherein in step S3, the following is specific: Step S3.2 adaptive multiscale image segmentation-Generation of homogeneous region candidate set The regional level recognition strategy based on the spectrum similar homogeneous region is provided, a spectrum-space cooperative self-adaptive mean shift segmentation algorithm is designed, the image is segmented into homogeneous regions with consistent spectrum characteristics in advance, the subsequent recognition is carried out by taking the regions as units, the robustness and the recognition precision of the system under the complex environment are improved, S3.2.1 data input Input data spectral image As the height of the image is to be taken, For the width of the image to be the same, Is the number of wave bands; Preset parameters, empirical coefficients From the number of bands And (3) determining: when the time is taken to be 0.8, Taking 1.2, spectral complexity coefficients Spatial kernel bandwidth coefficient Region constraint threshold, number of pixels Degree of space compactness Homogeneity of spectrum ; S3.2.2 algorithm flow Step 1, feature vector construction For each pixel in the spectral image Constructing a spectrum-space joint feature vector: pixel A kind of electronic device A dimensional spectral vector; pixel Is a spatial coordinate vector of (a); step 2, local spectrum complexity calculation For each pixel Taking it out Neighborhood region Calculating local spectral complexity: wherein the method comprises the steps of Is the first in the neighborhood The variance of the wave band reflects the heterogeneity of the spectrum around the pixel; step 3, adaptive calculation of core bandwidth Splitting nuclear bandwidth into spectral nuclear bandwidth And space kernel bandwidth And (3) respectively calculating: The method comprises the steps of adapting the ground feature spectrum characteristics of a spectrum image as the local spectrum complexity increases and decreases; positively correlating with the neighborhood area to ensure the spatial neighborhood continuity of the pixels; Step 4, spectral-spatial binuclear weighted mean shift iteration For each pixel To For an initial point, a mean shift iteration is performed: ① Determining local area screening meets And is also provided with Is a pixel of (2) Constitutes a local area ; ② Calculating a dual-core weighted average: Wherein the weight coefficient Spectral nuclei Space kernel ; ③ Iterative updating, namely updating the initial point to be Repeating the steps 1-2 until ; ④ Cluster merging, namely dividing pixels converged to the same mean point into the same region ; Step 5, area constraint screening For the divided areas The regions satisfying the following conditions are screened as final homogeneous region candidate sets: ② Pixel number constraint: ② Space compactness constraint: ( for the number of pixels that the region contains, Number of area edge pixels); ③ Spectral homogeneity constraints: (Std is the standard deviation, max, of spectral vectors in the region Global maximum for the spectral vector of the spectral image); s3.2.3 algorithm output Output result of homogeneous region candidate set of the spectral image K is the total number of regions, each region satisfying the spectral-spatial homogeneity constraint.
- 5. The method for automatic radiation calibration of multiple sensors on an unmanned aerial vehicle according to claim 4, wherein in step S3, the following is specifically mentioned: Step S3.3, spectral stability analysis-double spectral index verification For each homogeneous region Extracting spectral feature vectors of the PIF ground object in multispectral and hyperspectral images, evaluating spectral stability by using a two-dimensional index of geometric similarity and statistical difference, ensuring that the spectral features of the identified PIF ground object are kept stable under different sensors in different time periods, S3.3.1 spectral angle matching SAM calculation For areas At adjacent acquisition moments Is the average spectral vector of (a) And The SAM calculation formula is as follows: Wherein: the spectrum angle is set in the range of the spectrum, Area of interest At the moment of time Is used for the average spectral reflectance or DN value, As a function of the wavelength(s), A common band set of multispectral and hyperspectral, a common band range, Setting spectral angle threshold in combination with experimental empirical data , When (when) When the spectrum characteristics of the region at two moments are considered to have geometric similarity; s3.3.2 spectral information divergence SID calculation Is provided with 、 Respectively are areas Normalized spectral vector at two moments, SID is defined as follows: wherein the method comprises the steps of For the Kullback-Leibler divergence (KL divergence), the calculation formula is as follows: Wherein: an extremely small positive number, preventing zero removal errors, The probability distribution is required to be normalized in advance, Setting SID empirical threshold , When SID When the spectral distribution of the region is considered to have statistical stability, And (3) the region meeting the SAM and SID threshold conditions is marked as a spectrum stability candidate region, and a subsequent space consistency verification link is entered.
- 6. The method for automatic radiation calibration of multiple sensors on an unmanned aerial vehicle according to claim 5, wherein in step S3, the following is specific: Step S3.4, space consistency verification-elimination of false stable regions S3.4.1 spatial heterogeneity analysis Computing regions Is of the spatial heterogeneity index of (2) Reflecting the degree of dispersion of pixel values within the region, the formula is as follows: Wherein: Area of interest Standard deviation of DN values for all pixels within, Area of interest The mean value of the DN values of all pixels within, Is a dimensionless index and is used for measuring the spectral uniformity in the region, Specifying spatial heterogeneity empirical thresholds When (when) When the method is used, the pixel characteristics in the description area are uniform, and the space consistency is good; s3.4.2 neighborhood continuity analysis Statistical region Number of regions belonging to "spectrally stable candidate region" in the 8 neighborhood of (2) The formula is as follows: Wherein: 8 number of spectral stability candidate regions in neighborhood and value range , The ratio of stable area in the neighborhood is normalized to the (0, 1) interval, and the neighborhood consistency empirical threshold is set When (when) When the method is used, the region is spatially communicated with the peripheral stable region, so as to exclude the isolated false stable region, And (3) comprehensively judging the area with the conditions meeting the following two conditions, finally determining the area as a PIF ground object area, and extracting DN values of all pixels of the area as a reference sample set for subsequent cross-sensor radiation normalization: 。
- 7. The method for automatic radiation calibration of the unmanned aerial vehicle multi-sensor according to claim 2, wherein in step S4, the following is specifically adopted: Step S4.1 band matching The band matching is carried out by adopting a spectrum response function convolution integral method, and the fusion formula is as follows: Wherein: multispectral wave band after fusion Is used for the digital quantization value DN of (c), Hyperspectral image at wavelength The DN value at the point(s), Multispectral camera first The spectral response function SRM of the band, Multispectral wave band Is defined by a wavelength range boundary of (a); Step S4.2, outlier rejection S4.2.1 initial residual calculation With multispectral DN values in the PIF region And the post-match hyperspectral DN value As an initial residual: Wherein: First of all The residual of the individual pixels is used, Multispectral camera in the wave band Is the first of (2) The value of the pixel is determined by the pixel value, The corresponding DN value of hyperspectral data after wave band matching; s4.2.2 weight calculation The adaptive weight of each pixel is calculated based on the median absolute deviation MAD of the residual, the weight function is as follows: Wherein: The median absolute deviation MAD of the residual, 1.4826 Conversion coefficient of MAD and standard deviation under normal distribution (MAD. Apprxeq. Sigma.) First of all The weight of each pixel is used for inhibiting the influence of abnormal values; s4.2.3 iterative optimization Updating model parameters by weights, calculating new residual errors based on the new parameters, and repeating the weight-parameter updating process until the iteration times reach a preset value Or the weight change is less than the convergence threshold ; S4.2.4 outlier deletion Rejection weights Wherein For the weight empirical threshold, the remaining pixels make up a purified PIF sample set for subsequent radiation normalization modeling.
- 8. The method for automatic radiation calibration of the unmanned aerial vehicle multi-sensor according to claim 7, wherein in step S4, the following is specifically adopted: Step S4.3 Multi-spectrum-hyperspectral adaptive weighted robust regression radiation normalization algorithm S4.3.1 algorithm inputs Input data: ① Multispectral image in wave band DN value matrix ; ② DN value matrix of hyperspectral image after band matching ; ③ Purified PIF sample set ( Is the number of samples); Wherein, the Multispectral camera in the wave band DN value of (2); The corresponding DN value of hyperspectral data after wave band matching; preset parameters, weight iteration convergence threshold Maximum number of iterations ; S4.3.2 algorithm flow Step 1, initializing PIF sample weight, and carrying out PIF sample concentration on each sample Initializing weights ; Step 2, self-adaptive weighted robust regression modeling, aiming at multispectral wave bands Iterative solution of linear transformation model Parameters of (2) : ① And (3) solving parameters of the t-th iteration, namely constructing an objective function by taking the minimum weighted residual square sum as a target: For a pair of And solving the bias guide and making the bias guide be 0 to obtain an analytic solution: , ② Weight update computing residual error of the t-th iteration Updating weights based on robust estimation of residuals: wherein the method comprises the steps of , ③ Convergence judgment, if Or (b) Stopping iteration and taking , Otherwise, repeating the step 1-2, Step 3, implementing cross-band radiation normalization, for all pixels in hyperspectral image Obtained by solving , The hyperspectral DN values are converted into DN values of the multispectral reference: step 4, model accuracy verification Calculating normalized residuals for PIF sample sets The following criteria were verified: Residual error mean value (Unbiased); Residual error standard deviation (The precision reaches the standard); S4.3.3 algorithm output Outputting a result: ① Multispectral wave bands Corresponding gain coefficient Coefficient of offset ; ② Normalized hyperspectral DN value matrix; Therefore, the radiation consistency of the multi-source remote sensing data is realized, and the subsequent quantitative analysis is supported.
- 9. The method for automatic radiation calibration of the unmanned aerial vehicle multi-sensor according to claim 2, wherein in step S5, the following is specifically mentioned: step S5.1 Algorithm Pre-preparation S5.1.1 input data formation Radiation basis data-DN value dataset normalized across sensor Wherein For the pixel coordinates, For the spectral band, the data has eliminated the multi-sensor response variance; Dynamic irradiance data-downlink solar total radiant flux density synchronously collected by airborne irradiance sensor , For collecting the time stamp, ensuring one-to-one correspondence with DN data of each frame; Auxiliary parameter data, namely unmanned aerial vehicle GPS/IMU data, sensor laboratory calibration parameters and real-time meteorological parameters; s5.1.2 condition hypothesis Based on the remote sensing radiation transmission theory, the following condition assumption is made: assume 1 that the sensor output DN value is linearly related to the input radiance, i.e This assumption is the physical basis for radiance conversion; 2, when the unmanned aerial vehicle flies at low altitude, aerosol is vertically distributed uniformly and has low concentration, and the scattering influence of the aerosol can be approximated by a simplified model; suppose 3 that downstream solar radiation can be considered to be evenly distributed at the moment of observation, this assumption is guaranteed by sensor mounting location optimization and data verification.
- 10. The method for automatic radiation calibration of multiple sensors on an unmanned aerial vehicle according to claim 9, wherein in step S5, the following is specifically mentioned: Step S5.2 dynamic correction of irradiance data To obtain high-precision downlink solar irradiation flux density, multistage dynamic correction must be performed on the raw data, specifically implemented as follows: s5.2.1 dark current dynamic correction The sensor still has weak output under the condition of no illumination, namely dark current, needs to be deducted in real time to eliminate baseline drift, Calculating instantaneous dark current by adopting a sliding window mean value strategy, wherein a mathematical model is as follows: Wherein: time of day Is a corrected dark current value of (2); sliding window length, taking 5 frames, and considering time response speed and stability; First of all Dark field data of the frame; S5.2.2 temperature drift compensation The sensitivity of the sensor drifts along with the change of the working temperature, the temperature coefficient is needed to compensate, Compensation model, linear temperature response model, temperature correction coefficient Describing by adopting a linear model: Wherein: Standard temperature The sensitivity coefficient of the sample is set to be lower, Wave band Temperature coefficient (unit: ) Obtained by calibration of a high-low temperature environment test (-10 ℃ to 50 ℃), Time of day Is measured (collected by a built-in temperature sensor); s5.2.3 random noise filtering Gaussian random noise is removed by adopting 5-point moving average filtering, and irradiance data after filtering The calculation is as follows: , s5.2.4 final irradiance determination The correction links are combined to obtain the moment Wave band True downlink solar irradiance of (2) The calculation formula is as follows: Wherein: the filtered original irradiance data; Dark current after sliding window correction; Temperature compensation coefficient.
- 11. The method for automatic radiation calibration of the unmanned aerial vehicle multi-sensor according to claim 10, wherein in step S5, the following is specifically mentioned: Step S5.3, calculation of atmospheric roof radiance Based on the normalized DN value, calculating the radiation brightness of the ground object on the atmosphere roof by combining with the camera laboratory calibration parameter Establishing a preliminary relation of sensor response characteristics, wherein the calculation formula is as follows: Wherein: pixel At the wavelength of The top radiation brightness of the atmosphere at the position, The digital quantized value DN normalized across the sensor, i.e. the data after radiation normalization, Camera at wavelength A gain factor at which, reflecting the sensor sensitivity, Offset coefficient, reflecting system zero point deviation; Substep S5.4 atmospheric correction and absolute reflectance calculation Based on simplification The model performs atmospheric correction, and according to the characteristics of low-altitude flight of the unmanned aerial vehicle, the complex scattering influence of aerosol is ignored, only atmospheric molecular scattering and water vapor absorption are considered, and the absolute reflectivity of the ground surface is calculated The formula of (2) is as follows: Wherein: The absolute reflectivity of the earth's surface, The radiation brightness of the atmosphere top is shown as follows, The radiation of a large gas path is adopted, The distance correction factor of the day and the ground, The solar irradiance after correction is used for the solar irradiance, The solar zenith angle is that, The air permeability is that the air permeability is high, S5.4.1 large gas path radiation Through dark field experiments or selection of deep water and shadow dark areas in images Estimated using the following formula: , s5.4.2 day-to-ground distance correction factor Because the revolution orbit of the earth is elliptical, the influence of the change of the distance between the sun and the earth on the solar irradiation is corrected, and the calculation formula is as follows: Wherein: DOY, annual date, value range Or 366 of the two or more, The relative proportion of the distance between the sun and the earth relative to the average distance, S5.4.3 solar zenith angle The real-time geographic position and the acquisition time of the unmanned aerial vehicle are calculated by firstly calculating the solar altitude angle : The zenith angle of the sun is: Wherein: the latitude of the observation point is shown as follows, The solar declination angle is calculated by the following formula: , Omega, time angle, and the calculation formula is as follows: , The time for the collection is given to the time, S5.4.4 atmospheric transmittance Divided into atmospheric molecular transmittance And water vapor transmission rate : , Wherein: The atmospheric optical thickness is obtained by actual measurement of a local weather station or inversion of hyperspectral data, Molecular scattering terms, calculated by Rayleigh scattering model: , rayleigh scattering coefficient, and wavelength In relation to each other, The moisture content (unit: g/cm 2) of the atmosphere column can be obtained by inversion of the moisture absorption spectrum or by inquiry from the U.S. standard atmosphere spectrum database (US Standard Atmosphere), The water vapor absorption term is obtained by an empirical model or a table look-up.
Description
Unmanned aerial vehicle-mounted multi-sensor automatic radiometric calibration system and method Technical Field The invention relates to the technical field of unmanned aerial vehicle remote sensing technology and radiation calibration, and is suitable for real-time radiation calibration processing of unmanned aerial vehicle carried hyperspectral, multispectral and other multi-type imaging sensors in a field dynamic environment. Background The radiation calibration is a key link for establishing a corresponding relation between a digital quantized value (DN) output by a sensor and the actual radiation characteristics of a ground object, and directly determines the comparability and quantitative analysis reliability of multi-source and multi-time phase data. The existing calibration method depends on ground calibration fields or calibration, and is not suitable for temporary field operation. According to the relative calibration method based on the pseudo-invariant feature (PIF), radiation normalization is achieved by identifying spectrum stable ground objects, but the prior art relies on manual PIF screening, the degree of automation is low, only relative radiation unification can be achieved, absolute reflectivity cannot be obtained, and the high-precision analysis requirement cannot be met. The on-line correction technology of the irradiance sensor carried by the unmanned aerial vehicle can only absolutely scale a single sensor, and the problem of data consistency of multiple sensors is not solved, and radiation deviation still exists during fusion of the multiple source data. Therefore, there is a need for a multi-sensor fully automatic radiometric calibration scheme that does not require a ground reference. Aiming at the problems, the invention combines PIF relative calibration and online irradiance absolute calibration technology, realizes automatic normalization and absolute reflectivity conversion of multi-sensor data through hardware integration and algorithm innovation, gets rid of dependence on ground calibration, and improves calibration performance in complex environments. Disclosure of Invention The invention aims to overcome the defects that the existing unmanned aerial vehicle radiometric calibration method depends on a ground calibration target, has low automation degree, cannot adapt to a dynamic illumination environment and has poor data consistency of multiple sensors, and provides an unmanned aerial vehicle-mounted multi-sensor automatic radiometric calibration system and method. The system and the method can realize automatic identification of PIF ground features, relative normalization of multi-sensor data and real-time conversion of absolute reflectivity, and provide high-precision and high-reliability data support for quantitative analysis of unmanned aerial vehicle multi-source remote sensing data in a field complex environment. The invention provides an unmanned aerial vehicle-mounted multi-sensor automatic radiation calibration system for achieving the purposes, which comprises an unmanned aerial vehicle platform, a load integration module, a data acquisition module, a data processing module and a data storage and output module: the unmanned plane platform adopts a multi-rotor or fixed wing aircraft type, has high-precision flight control capability, and can set a route, a height and a speed according to tasks; the load integration module comprises a multispectral camera, a hyperspectral camera and an upward-mounted high-precision spectral irradiance sensor, and is used for synchronously acquiring ground object images and downlink solar irradiance; The data acquisition module consists of a Field Programmable Gate Array (FPGA) synchronous control unit and a gigabit Ethernet/wireless transmission unit, ensures that the time stamp of the image and irradiance data is consistent and is transmitted to the data processing module in real time; The data processing module is used for running PIF automatic identification, cross-sensor radiation normalization and absolute reflectivity inversion algorithms based on an embedded processor or an industrial personal computer to realize real-time data processing; The data storage and output module comprises a solid state disk, a USB 3.0, an HDMI and a network interface, and is used for storing original and calibration data and supporting local export and real-time display. Based on the system, the invention also provides an unmanned aerial vehicle multi-sensor automatic radiation calibration method, which comprises the following steps: step S1, system initialization and parameter configuration Before the unmanned aerial vehicle takes off, the whole calibration system is initialized and configured, and the method specifically comprises the following steps: Before the unmanned aerial vehicle takes off, initializing a calibration system, which comprises the following configurations: sensor parameters including input of internal, external and spectral