CN-122024111-A - Unmanned aerial vehicle remote sensing blade accurate detection method and system based on rotation perception Gaussian encoding

CN122024111ACN 122024111 ACN122024111 ACN 122024111ACN-122024111-A

Abstract

The invention belongs to the field of remote sensing and image processing, and discloses an unmanned aerial vehicle remote sensing blade accurate detection method and system based on rotation perception Gaussian encoding, wherein the method comprises the steps of acquiring a multi-view RGB image dataset of a forest to be detected by utilizing an unmanned aerial vehicle remote sensing technology, and further generating corresponding instance labels, a multi-scale feature map and Gaussian parameters; introducing rotatable elliptic Gaussian position codes into a multi-scale feature map after Gaussian parameter modulation and updating, fusing the multi-scale feature map with sinusoidal position codes, generating a central significant thermodynamic diagram based on high-resolution features by using a significant guiding mechanism, initializing partial queries by using central candidate points, and fusing Gaussian alignment features on the high-resolution features to enhance the separability of boundary and adhesion regions, so as to obtain a blade instance mask and a corresponding detection result. The invention can obtain the blade example mask and the detection result with higher precision and stronger robustness under the conditions of complicated forest stand illumination disturbance, background disturbance and strong shielding.

Inventors

XU SHENG
Lv Jialong
WU DONGYANG
XIA SHAOBO
LI CHENGHUA
SHEN YU

Assignees

南京林业大学

Dates

Publication Date: 20260512
Application Date: 20260305

Claims (10)

1. An unmanned aerial vehicle remote sensing blade accurate detection method based on rotation perception Gaussian encoding is characterized by comprising the following steps: acquiring a multi-view RGB image dataset of a forest to be detected by using an unmanned aerial vehicle remote sensing technology; Generating instance labels corresponding to the images to be detected and multi-scale feature images of different resolution levels based on the RGB image data set, and obtaining initial learnable Gaussian parameters based on the instance labels; Based on the multi-scale feature map, modulating the Gaussian parameters by utilizing image conditional branches, and predicting Gaussian offset vectors corresponding to each RGB image And adaptively updating the Gaussian center position, the principal axis scale and the rotation angle according to the Gaussian center position and the principal axis scale; introducing rotatable elliptic Gaussian position codes and fusing with sinusoidal position codes to generate fusion coding features for the multi-scale feature map after the Gaussian parameter modulation updating is completed, and carrying out inverse flattening rearrangement on the multi-scale feature map according to the original space size of each scale based on the constraint of the fusion feature codes on cross-scale feature interaction, and recovering the multi-scale feature map to a two-dimensional feature map form with corresponding resolution to obtain injection direction-scale prior and complete image features after multi-scale interaction; simultaneously, an instance Gaussian embedded vector is constructed, mapped into a position updating vector, and then combined with a central clue to be fused through a learnable gate so as to realize the self-adaptive updating of the query position item; And carrying out Gaussian alignment feature fusion on the high-resolution features to enhance the separability of the boundary and the adhesion region, and obtaining a blade instance mask and a corresponding detection result.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises, Based on the RGB image dataset, the method for generating the multi-scale feature map with different resolution levels comprises the following steps: Based on the RGB image data set, a backbone network back is extracted by utilizing features to perform forward computation, shallow-to-deep level features are extracted in a layer-by-layer downsampling process to generate multi-scale feature maps of different resolution levels, wherein shallow-level features represent leaf edge textures, leaf vein details and local boundary information, deep-level features represent whole forms of blades, context relations and semantic information under shielding conditions, and high-resolution features are used for describing small-scale blades and fine boundaries in the generated multi-scale feature map, and low-resolution features are used for capturing whole structures of large-scale blades and crowns and semantic distinguishing capability under complex backgrounds.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises, Based on the instance labels, the method for obtaining the initial learnable Gaussian parameters comprises the following steps: Performing instance geometry statistics on each blade instance mask, and calculating instance centroid coordinates of each blade instance mask to form a sample point set for initializing Gaussian centers; Estimating covariance matrixes in each cluster according to pixel coordinate distribution of corresponding blade masks, and carrying out feature decomposition on the covariance matrixes, wherein a main feature vector is used for determining initial orientation/rotation angle of a Gaussian ellipse, and square root of a feature value is used for determining initial standard deviation of two principal axis directions, so that rotatable anisotropic elliptical Gaussian initialization parameters are obtained; and introducing the obtained Gaussian center, scale and rotation angle into a network to train as a learnable parameter, and carrying out end-to-end joint optimization updating together with the model weight to obtain a final learnable Gaussian parameter set.
4. The method of claim 1, wherein the step of determining the position of the substrate comprises, Based on the multi-scale feature map, the method for modulating the Gaussian parameters by using image conditional branches comprises the following steps: Gaussian parameter modulation by a gaussian offset vector Realization of the Gaussian offset vector The method comprises the following steps: , Wherein, the For translational compensation of the gaussian center, For adaptive scaling compensation of gaussian principal axis dimensions, The method is used for carrying out self-adaptive correction on the Gaussian rotation angle, and the corresponding Gaussian parameter updating relation satisfies the following conditions: , , , In the formula, , , And the modulated Gaussian parameters are used for updating Gaussian centers, scales and rotation angles.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises, The rotatable elliptic Gaussian position code is constructed by two-dimensional anisotropic Gaussian basis functions, and pixel coordinates The gaussian response at that point satisfies: , Wherein, the pixel coordinates in the image are , Is the first The center of the individual gaussian groups, In the form of a covariance matrix, Adopt rotatable oval decomposition with coding direction and scale information, satisfy: , Wherein, the As the rotation angle of the rotation, Is of a dimension of two principal axes, Is a two-dimensional rotation matrix.
6. The method of claim 1, wherein the step of determining the position of the substrate comprises, The method for generating a central salient thermodynamic diagram based on high-resolution features using a salient guidance mechanism comprises: the center significant thermodynamic diagram is generated by taking the center of a blade instance as a supervision, wherein the center of the blade instance is obtained by performing Euclidean distance transformation on an instance mask and taking a global maximum point, the Euclidean distance transformation is used for representing the distance from a pixel to an instance boundary, and the center significant thermodynamic diagram is predicted by high-resolution features.
7. The method of claim 6, wherein the step of providing the first layer comprises, The method for initializing the partial query by using the center candidate points comprises the following steps: performing local peak detection on the central significant thermodynamic diagram to obtain a central candidate point, and mapping the coordinates of the central candidate point into a central position vector And initializing the query according to the following gating fusion type: , Wherein, the In order to be able to learn the gating scalar, And partial inquiry takes a central candidate point as an initial anchor point in the mode of peak sampling and gating initialization.
8. The method of claim 7, wherein the step of determining the position of the probe is performed, The method for adaptively updating the query location items comprises the following steps: based on the upper layer Instance area corresponding to each query Aggregating Gaussian responses in the example area on each Gaussian response graph to obtain an example Gaussian embedding vector: , , and obtaining a position update vector from the instance Gaussian embedded vector through a projection network: , Wherein the method comprises the steps of Is a projection network; The central position vector and the position update vector are subjected to characteristic splicing and are mapped by a multi-layer perceptron, and are fused through a learning gate so as to carry out self-adaptive update on the queried position item: , , , Wherein the method comprises the steps of Is a learnable gating scalar.
9. The method of claim 1, wherein the step of determining the position of the substrate comprises, The method for obtaining the blade instance mask and the corresponding detection result comprises the following steps: And introducing a Gaussian alignment feature fusion mechanism on a high-resolution feature in the multi-scale image features, aligning and fusing the Gaussian channel feature and the high-resolution feature, and further, utilizing semantic details and direction-scale geometric prior at the pixel level prediction stage, improving the blade boundary detail recovery capability and the separability of adjacent adhesion examples, and finally obtaining a blade example mask and a corresponding detection result.
10. An unmanned aerial vehicle remote sensing blade precision detection system based on rotation perception Gaussian encoding, which uses the method of any one of claims 1-9, and is characterized by comprising an image acquisition device, an image initial processing module, a Gaussian modulation module, a fusion encoding module, an adaptive updating module and a detection output module; The image acquisition equipment is an unmanned aerial vehicle carrying a remote sensing technology, and obtains a multi-view RGB image dataset of a forest to be detected based on the unmanned aerial vehicle remote sensing technology; The image initial processing module is used for generating instance labels corresponding to the images to be detected and multi-scale feature images with different resolution levels based on the RGB image data set, and obtaining initial learnable Gaussian parameters based on the instance labels; The Gaussian modulation module is used for modulating the Gaussian parameters by utilizing image conditional branches based on the multi-scale feature map and predicting Gaussian offset vectors corresponding to each RGB image And adaptively updating the Gaussian center position, the principal axis scale and the rotation angle according to the Gaussian center position and the principal axis scale; the fusion coding module is used for introducing rotatable elliptic Gaussian position codes and fusing the rotatable elliptic Gaussian position codes with sinusoidal position codes to generate fusion coding features for the multi-scale feature images after the Gaussian parameter modulation updating is completed, performing inverse flattening rearrangement on the multi-scale feature images according to the original space size of each scale based on the constraint of the fusion feature codes on the trans-scale feature interaction, recovering the multi-scale feature images to a two-dimensional feature image form with corresponding resolution, and obtaining injection direction-scale priori and completing the image features after the multi-scale interaction; the self-adaptive updating module is used for generating a central salient thermodynamic diagram based on high-resolution characteristics by utilizing a salient guiding mechanism, initializing partial inquiry by using central candidate points, constructing an instance Gaussian embedded vector, mapping the instance Gaussian embedded vector into a position updating vector, and fusing by combining a central clue through learning gating to realize the self-adaptive updating of inquiry position items; the detection output module is used for carrying out Gaussian alignment feature fusion on the high-resolution features to enhance the separability of the boundary and the adhesion region, and a blade instance mask and a corresponding detection result are obtained.

Description

Unmanned aerial vehicle remote sensing blade accurate detection method and system based on rotation perception Gaussian encoding Technical Field The invention belongs to the technical field of forestry remote sensing and computer vision, and particularly relates to an unmanned aerial vehicle remote sensing blade accurate detection method and system based on rotation perception Gaussian encoding. Background The requirements for accurate and continuous acquisition of plant phenotype information for forestry breeding and ecological monitoring are increasingly increased, and leaf-level characters such as areas, forms and structures of leaves serving as key organs of photosynthesis and transpiration have important significance for growth evaluation and breeding decision. The existing research is gradually expanded from macroscopic level such as single wood segmentation (ITS) and the like to fine granularity direction such as blade instance segmentation (LIS) and the like so as to realize identification and segmentation of single leaves in a single tree crown and further support precise quantization of parameters such as leaf area, length, width, morphological index and the like. Along with the development of low-altitude close-range remote sensing of unmanned aerial vehicles, the development of forest fine phenotype based on multi-view RGB images becomes possible, but blade segmentation under a forest stand scene is significantly more complex than that of agricultural crop scenes, namely, the forest is tall and big, a canopy is closed, a shooting angle is limited, and background interference is strong, so that the model faces higher requirements in terms of boundary, texture and instance separability. Specifically, forestry blade example segmentation usually suffers from the following difficulties that (1) dynamic illumination drastic change caused by canopy shadow and backlight weakens boundary and texture legibility, (2) obvious scale difference caused by flight height, imaging distance and resolution change, (3) attention dispersion caused by complex background interference of weeds, branches, adjacent blades and the like, (4) blade overlapping and shielding are serious, example boundary is difficult to judge, (5) shape diversity such as leaf shape, leaf margin, color and the like caused by different tree species and different development stages is further increased, and the method for preparing the forestry blade example segmentation has the following disadvantages From an algorithm development perspective, the leaf instance segmentation underwent evolution from semantic segmentation+post-processing to end-to-end instance segmentation to a query-based transform segmentation framework. In recent years, query type convertors such as Mask2 force promote segmentation consistency under complex scenes through Mask classification and multi-scale interaction, and are used for plant leaf instance segmentation tasks. However, the existing method is still generally insufficient in terms of dense small targets, strong rotation deformation, blocking adhesion and illumination disturbance of forest stands, namely firstly, the capability of geometric explicit modeling of the direction, the scale and the like of a blade by conventional position coding is insufficient, the attention alignment is unstable under the condition of rotation and bending, boundary breaking, leakage and adhesion misclassification are easy to occur, secondly, the strong blocking and dense areas lack stable central anchor points and effective instance-level geometric characterization, so that query initialization and iteration updating are insufficient, and instance separability is insufficient, thirdly, in the stage of high-resolution detail recovery, semantic features and geometric information are not tightly coupled, and the capability of separating boundary restoration and mutual adhesion areas is limited. The problems enable high-precision and strong robust example detection and segmentation of the blade to be achieved under the multi-view forestry scene of the unmanned aerial vehicle, and obvious technical bottlenecks still exist. Disclosure of Invention The invention aims to solve the defects of the prior art, and provides an unmanned aerial vehicle remote sensing blade accurate detection method and system based on rotation perception Gaussian coding, by introducing rotatable elliptic Gaussian geometric representation between a pixel domain and a query domain, and the high-precision and strong robust detection and segmentation of the blade instance under the complex stand scene are realized by combining the image condition modulation, the center significant guidance, the dynamic position query and the Gaussian alignment feature fusion mechanism. In order to achieve the above purpose, the present invention provides the following technical solutions: an unmanned aerial vehicle remote sensing blade accurate detection method based on rotation percep