CN-122023182-A - Boundary enhancement and time sequence stabilization method for medium wave infrared gas plume

Abstract

The invention discloses a boundary enhancement and time sequence stabilization method for a medium wave infrared gas plume, which comprises the following steps of 1, obtaining data and carrying out normalization processing on the data to obtain a standardized medium wave infrared gas plume time sequence frame set, 2, carrying out pre-training processing and division on the standardized MWIR time sequence frame set obtained in the step 1 to obtain an MWIR video time sequence segment, 3, inputting the MWIR video time sequence segment into an improved space-time enhancement network, training the improved space-time enhancement network to obtain the most optimal model, 4, inputting the MWIR video time sequence segment into the optimal model obtained in the step 3, outputting an enhancement sequence with the same resolution as the input, highlighting a plume soft boundary, inhibiting background interference and reducing cross-frame flicker, and 5, evaluating an output result of the step 4. The invention reduces the risk of false alarm and missing report, and improves the reliability and efficiency of leakage monitoring and security inspection.

Inventors

• MA ZONGFANG
• Bai Hanxue
• ZHANG GUOFEI
• SONG LIN

Assignees

• 西安建筑科技大学

Dates

Publication Date

20260512

Application Date

20251202

Claims (6)

1. 1. The boundary enhancement and time sequence stabilization method for the medium wave infrared gas plume is characterized by comprising the following steps of; Step 1, acquiring data and carrying out normalization processing on the data to obtain a standardized medium wave infrared gas plume time sequence frame set; Step 2, performing pre-training treatment and division on the standardized MWIR time sequence frame set obtained in the step 1, and constructing an MWIR video time sequence segment for modeling MWIR gas plume flow change characteristics; inputting the MWIR video time sequence segment into an improved space-time enhancement network, training the improved space-time enhancement network, and obtaining an optimal model for enhancing the visibility of MWIR gas plumes; Inputting the MWIR video time sequence segment into the optimal model obtained in the step 3, outputting and inputting an enhancement sequence with the same resolution, highlighting soft boundaries of the feather MWIR gas flow, inhibiting background interference and reducing cross-frame flicker; and 5, evaluating the output result of the step 4, and verifying that the visibility of the MWIR gas plume boundary is improved and the time-sequence stability is improved.
2. 2. The method for boundary enhancement and time sequence stabilization of medium wave infrared gas plume according to claim 1, wherein the step 1 is specifically: The MWIR frame sequence organized according to video is imported, effective frames are screened, and the length T=7 and the sampling interval are adopted =1 Is loaded as a timing slice and fixed window normalization to [0,1] is performed on each frame, resampled to uniform resolution (H, W) if necessary, resulting in a normalized MWIR timing frame set, where the original 16-bit gray scale frame is followed The normalization of (2) is: Wherein, clip () function is used to limit the values within [0,1], and W min and W max correspond to the minimum and maximum values of the frame image gray scale, respectively.
3. 3. The method for boundary enhancement and time sequence stabilization of medium wave infrared gas plume according to claim 2, wherein the step 2 is specifically based on the standardized MWIR time sequence frame set obtained in the step 1, and the step is a step size Forming training samples from each video sequence sampling segment to form And is proportionally divided into a training set, a verification set and a test set.
4. 4. The boundary enhancement and time sequence stabilization method for medium wave infrared gas plumes according to claim 3, wherein in the step 3, an improved space-time enhancement network is input to a time sequence segment, and the improved space-time enhancement network is composed of frame-by-frame feature coding and time displacement fusion, time bottleneck convolution, depth separable hole space pyramid pooling bottleneck, decoding and edge attention, and residual output; And in the training stage, the composite region of interest (ROI) gating and no labeling loss are adopted for optimization, and the learning rate is adjusted through a cosine annealing algorithm to obtain an optimal model.
5. 5. The method for boundary enhancement and time sequence stabilization of medium wave infrared gas plume according to claim 4, wherein in step 3, the specific operations of network and training comprise the following steps: step 3.1, fusion TSM of frame-by-frame coding and time displacement, adopting two-dimensional backbone to extract four-level space scale feature images of each frame, respectively recorded as Wherein F 4 is the deepest high semantic feature of the encoder, the channel number is C, and the channels are proportionally divided into forward shifts Backward movement of And rest at rest The TSM output for time index t is: forward movement feature Shift-in from last frame, shift-back feature Moving in from next frame, stationary features The method is kept unchanged, and zero filling or adjacent frame copying mode is adopted at the sequence boundary to compensate; step 3.2 for the highest layer features obtained in step 3.1 Applying a depth separable three-dimensional convolution only acting on the time dimension, and a convolution kernel shape (k, 1) for modeling slow diffusion and weak contrast change of medium wave infrared gas plumes in the time dimension, wherein the time convolution kernel of a channel c is set as The output of the channel at time index t and spatial position u= (i, j) is expressed as: Wherein the method comprises the steps of The feature sequence output in the step 3.1 after the TSM is fused through time displacement is marked as X; step 3.3 parallel computing 1×1 branches for the output Y of step 3.2 And void fraction aggregation Is divided into a plurality of depth separable cavity convolution branches; and (3) after splicing, carrying out projection fusion: Wherein the method comprises the steps of Representing a combination of normalization operators and activation functions; step 3.4, decoding and edge attention: the decoder upsamples step by step and matches the multi-scale encoded features extracted in step 3.1 Jump-connection fusion, setting a certain decoding characteristic as D, and its channel average Sobel core , Edge amplitude Wherein the method comprises the steps of To avoid unstable small constants caused by zero gradient, attention weights are obtained by 1X 1 convolution and Sigmoid Wherein W a is a point-by-point convolution kernel, As a Sigmoid function, final edge attention output: Wherein the method comprises the steps of Lambda is a coefficient for regulating and controlling the edge enhancement intensity, which is the Hadamard product; And 3.5, residual output and amplitude constraint. With 1 x 1 convolution kernels Edge attention feature to step 3.4 Performing point-by-point convolution, and predicting residual errors of a time index t: residual error adoption Amplitude compression is performed according to gain coefficient Modulating and finally outputting the pass interval projection operator Limited to the normalization range. The output of the enhanced image at time frame t, pixel position u= (i, j) is: Step 3.6, constructing a composite ROI and gating, namely three frames of differential motion heat, and defining the motion heat as follows when an input sequence is x t : for characterizing local luminance variations of the plume over time; Edge magnitudes were calculated with Sobel kernels K x and K y : Wherein the method comprises the steps of A small constant to prevent zero gradients from causing instability; Normalization operator Wherein, the For normalizing the input variables of the operator, the motion heat is taken in this step Or edge amplitude ; Combining the motion heat with the edge amplitude to obtain a fusion soft weight: Wherein the method comprises the steps of Is an edge suppression coefficient for avoiding false recognition of background texture as a plume region; Setting a split threshold Hard mask Morphological opening and closing by structural elements The representation is: Wherein the method comprises the steps of Is expanded by, Is corrosion; the composite soft mask is defined as: Wherein the method comprises the steps of S controls the balance between the hard mask and the probability map for the interval clipping function; Is provided with Let X be input for original enhancement result without ROI gating, and output for training period be Wherein the method comprises the steps of Controlling the differential constraint intensity of the plume region and the background region, ensuring that stronger enhanced supervision is applied to the MWIR plume region and silence is maintained to the background; Step 3.7, constructing a composite training target consisting of a plurality of unsupervised losses, implementing differential weight constraint on the ROI region generated in the step 3.6, highlighting plume details, adopting structural similarity SSIM and Charbonnier losses to combine to form an enhancement term and a structural consistency loss function Reconstructing a loss function with the ROI The definition is as follows: Wherein the method comprises the steps of Representing the edges to remove the border bright bands; charbonnier loss ; The gradient keeps the edge direction and intensity of the input/output consistent, avoids the de-texturing and overshoot ringing, Representing the Sobel spatial gradient, the gradient coincidence loss function is defined as: the time domain consistency suppresses the cross-frame flicker, applies strong constraint to the background area, ensures that the static background is not disturbed, applies constraint to the continuity of the enhancement sequence in the time dimension, and defines the time domain consistency loss function as: And applying an intensified silence constraint in the background area to stabilize the time dimension of the background, wherein the time domain background silence loss function is defined as: the inter-frame increment consistency enables the variation amplitude of the output along with the time to be fitted with the real variation of the input, and the increment consistency loss function is defined as: residual sequence smoothing limits the oscillation of the enhanced residual in time, further reduces Flicker, and residual smoothing loss function is defined as: Wherein the method comprises the steps of ; The edge hinge ensures that the output edge is not weaker than the input edge, enhances the visibility of plume boundary, is provided with As an edge-amplification factor, For Sobel gradients of input and output, reLU represents an element-by-element linear rectification function, then edge hinge loss is defined as: Wherein the method comprises the steps of A composite soft mask constructed for step 3.6 for applying a concentration of constraints to the plume region; imposing silence constraints outside the ROI, spatial background silence loss is defined as: This item ensures that the static background remains unchanged; defining the dc suppression loss is used to suppress overall offset and maintain the radiation consistency of the MWIR sequence: scaling parameters for BN layer channels at decoding end Applying an L1 sparse constraint for subsequent structured pruning, the sparse regularization penalty being defined as: Wherein the method comprises the steps of Is a sparsification coefficient; Combining all the non-labeling loss items, the final training targets are as follows: Each of which is Weighting coefficient forces for different loss terms; And 3.8, performing iterative training by adopting AdamW optimizers and cosine annealing learning rate scheduling, supporting automatic mixing precision AMP and exponential sliding average EMA, and periodically outputting visualizations and recording verification indexes in the training process until an optimal model is obtained.
6. 6. The method for boundary enhancement and time sequence stabilization of medium wave infrared gas plume according to claim 5, wherein the step 5 is specifically: the input image is noted as x, and is calculated under the condition of no label by adopting the following indexes: Tenengard sharpness, namely, gradient components G x and G y are obtained by adopting transverse and longitudinal Sobel convolution kernels K x and K y respectively and a space average operator is adopted The sharpness of Tenengrad was calculated, . The standard deviation of the Flicker cross time domain pixel is set Representing gray values at pixel u of time frame t in time dimension standard deviation Pixel spatial averaging Defining a flicker intensity across frames; Post-registration difference of warp-L1 optical flow For the enhanced image of the time frame t, For the optical flow field from frame t to t+1, the optical flow registration result is defined as The timing registration error is defined as WarpL1; EPR@90% edge retention to input gradient magnitude 90-Bit threshold of (2) For reference, EPR90 is defined as the proportion of pixels for which the gradient magnitude is not below the threshold; CNR and BF1 in the comparative noise ratio evaluation, the ROI mask function is set as Region of ROI With background area The average value is respectively 、 Standard deviations are respectively 、 Then: and obtaining the predicted edge by Canny To Is approximating the true value boundary by the morphological gradient of (a) Tolerance of Pixel, dilation operator The method is used for allowing edge position deviation, recording accuracy is P, recall is R, and boundary F1 index is defined as BF1; 。

Description

Boundary enhancement and time sequence stabilization method for medium wave infrared gas plume Technical Field The invention relates to the technical field of computer vision, in particular to a boundary enhancement and time sequence stabilization method for medium wave infrared gas plumes. Background The medium wave infrared imaging is widely applied to the scenes of natural gas/volatile organic compound leakage monitoring, petrochemical device safety inspection, factory unorganized emission control, environment-friendly law enforcement evidence collection and the like. Compared with visible light or long-wave infrared, the MWIR wave band (typical 3-5 μm) gas plume has the characteristics of low contrast, soft boundary, strong time variation and the like, wherein the plume has small difference of radiation with the background, the boundary is mostly in gradual transition, and is influenced by turbulence, refractive gradient and background thermal disturbance, cross-frame deformation and brightness fluctuation (flicker) are obvious, meanwhile, the industrial infrared camera often has the defects of non-uniform response (NUC) residues, stripe noise, hot spots and the like, so that the plume boundary is 'stealth' in videos, and the stable detection and manual inspection are difficult. The existing monitoring means still takes manual observation or auxiliary interpretation based on simple image enhancement as a main part, and engineering requirements of boundary visibility and time sequence stability are difficult to be considered. The traditional single-frame method (histogram equalization/CLAHE, non-sharpening mask, edge enhancement and the like) surrounding the visibility is easy to introduce ringing and overshoot while improving the overall contrast, further amplifies sensor noise and streak artifacts, and conventional denoising/stripping and multi-frame time domain smoothing can weaken the very weak plume gradient, so that the boundary is smoothed and the details are lost. In the aspect of 'steady state flicker suppression', the scheme based on optical flow registration or three-dimensional convolution/space-time filtering has strong dependence on texture and heavy calculation load, and flow field estimation is unstable in pictures with sparse plume texture and remarkable non-rigid deformation, so that smear, ghost or time domain jitter is easy to generate. More importantly, the MWIR plume boundary lacks artificial annotation which can be obtained in a large scale, and an ideal clear target which is not enhanced is difficult to be used as a supervision signal in a real scene, so that the mobility of a supervision learning method is poor and the cost is high, and the differences between simulation/synthesis data and a thermal background, turbulence statistics and a sensor noise model of an actual working condition exist, so that the interdomain fall further weakens generalization capability. In the aspect of industrial field application, the real-time performance and deployment constraint must be satisfied, namely, the device surface monitoring and mobile inspection scene requires the edge end or the industrial personal computer to stably run, and the algorithm needs to keep continuous video frame processing capability under the limited calculation power and storage bandwidth, and meanwhile, has self-adaptive robustness to scene change (day and night/season/load). On the other hand, the end-to-end framework directly relying on the detection/segmentation of the general target needs a large amount of frame line/boundary true values, and is difficult to cover the multi-scale and multi-state distribution of the plume soft boundary. Based on the above, there is an urgent need for a method that can implement "boundary enhancement+timing stabilization" while maintaining lightweight computation without labeling, by improving plume edge visibility under soft boundary and low contrast conditions, and by strictly suppressing modification in a background static region, reducing cross-frame flicker, and having good deployability and engineering maintainability. Disclosure of Invention In order to overcome the defects of the prior art, the invention aims to provide the boundary enhancement and time sequence stabilization method for the medium-wave infrared gas plume, which can obviously enhance the boundary visibility of the MWIR gas plume and inhibit frame-crossing flicker without manual marking, and can control the quantity and the calculated quantity of model parameters to meet the requirements of real-time performance and deployment in industrial fields, thereby reducing false alarm and missing report risks and improving the reliability and efficiency of leakage monitoring and security inspection. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: A boundary enhancement and time sequence stabilization method facing medium wave infrared gas plumes comprises the followi