Search

CN-121982548-A - Crop disease early detection method and system based on multispectral image fusion

CN121982548ACN 121982548 ACN121982548 ACN 121982548ACN-121982548-A

Abstract

The invention discloses a crop disease early detection method and a crop disease early detection system based on multispectral image fusion, which belong to the technical field of agricultural intelligent monitoring and image processing, the method comprises a multispectral image self-adaptive registration fusion module, a spectrum space feature collaborative extraction module, a multiscale cross attention disease detection module and a confidence driving boundary optimization module, and a high-quality multispectral fusion image is obtained through a depth feature-based self-adaptive registration and frequency domain decomposition fusion strategy; the method realizes multi-mode feature characterization through the collaborative extraction of the spectral vegetation index and the spatial texture feature, realizes multi-scale disease detection through a cross enhancement mechanism of channel attention and spatial attention, realizes boundary refinement through conditional random field optimization and morphological operation, and has the identification accuracy of more than 85% in three days of attack in early detection of rice blast and early blight of tomatoes.

Inventors

  • LUO FANGLING
  • ZHOU CHANGXIU
  • LUO HUA
  • Xiao Qisi

Assignees

  • 攀枝花彝秀农业有限公司

Dates

Publication Date
20260505
Application Date
20260206

Claims (10)

  1. 1. The crop disease early detection method based on multispectral image fusion is characterized by comprising the following steps of: A multi-spectral image self-adaptive registration fusion step, namely, carrying out self-adaptive registration based on depth features on a synchronously acquired visible light image and near infrared image, adopting a twin convolution network to extract depth feature descriptors of two paths of images respectively, and calculating space transformation parameters through feature matching to realize pixel level alignment; A spectrum space feature collaborative extraction step, namely calculating a normalized difference vegetation index and a chlorophyll absorption ratio index based on the multispectral fusion image to serve as spectrum vegetation index features; A multi-scale cross attention disease detection step, namely inputting the spectrum space joint feature vector into a multi-scale feature pyramid network to generate feature graphs on different scale levels; calculating channel attention weight and space attention weight for each scale feature map respectively, multiplying the channel attention weight and the space attention weight element by element to generate a cross attention map, and multiplying the cross attention map and an original feature map element by element to obtain attention enhancement features; The method comprises a confidence coefficient driving boundary optimization step, a detection confidence coefficient feedback step, a multispectral image self-adaptive registration fusion step and a near-infrared band fusion weight increase step, wherein the confidence coefficient driving boundary optimization step is used for constructing a conditional random field energy function based on the position confidence coefficient graph to conduct boundary refinement on an initial detection result, morphological operation is adopted to eliminate isolated noise points and fill cavities in lesions, and the detection confidence coefficient is fed back to the multispectral image self-adaptive registration fusion step when the detection confidence coefficient is lower than a preset threshold value.
  2. 2. The method for early detection of crop diseases based on multispectral image fusion according to claim 1, wherein in the adaptive registration based on depth features, the twin convolution network comprises 5 convolution layers, convolution kernels are sequentially set to 7×7, 5×5, 3×3, and the number of channels is sequentially set to 32, 64, 128, and 256, 256-dimensional feature descriptors are output from the network, matching point pairs are found by calculating cosine similarity between the descriptors, a random sampling consistency algorithm is adopted to estimate a spatial transformation matrix, and when the number of the matching point pairs is not less than 16 pairs and the inner point proportion is not less than 0.6, the registration result is considered to be reliable.
  3. 3. The early detection method for crop diseases based on multispectral image fusion, which is disclosed in claim 1, is characterized in that the number of decomposition layers of the Laplacian pyramid is set to be 4, the fusion weight of the low-frequency base layer is determined by local variance calculated by pixel values in an 11×11 window with the current pixel as a center, a higher fusion weight is obtained in a region with a larger local variance, the fusion weight of the high-frequency detail layer is determined by comparing gradient energy of two wave bands at the same position, and the gradient energy is obtained by square root of square sum after calculating horizontal gradient and vertical gradient by a Sobel operator.
  4. 4. The early detection method for crop diseases based on multispectral image fusion of claim 1, wherein the normalized difference vegetation index is obtained by calculating the difference between the reflectivity of a near infrared band and the reflectivity of a red light band by dividing the difference by the sum of the reflectivity of the near infrared band and the reflectivity of the red light band, wherein the near infrared band is selected to be 850nm, the red light band is selected to be 680nm, and the chlorophyll absorption ratio index is obtained by calculating the ratio of the reflectivity of the near infrared band to the reflectivity of a red side band, wherein the red side band is selected to be 730nm.
  5. 5. The early detection method for crop diseases based on multispectral image fusion of claim 1, wherein the lightweight convolutional network comprises four feature extraction stages, each stage comprises a deep convolutional layer, a batch normalization layer and an activation layer, the number of channels is sequentially set to be 32, 64, 128 and 256, the size of an output feature map is one sixteenth of the size of an input image, and the number of channels of the spectral-spatial joint feature vector is the sum of the number of channels of spatial texture features and the number of channels of spectral vegetation index features.
  6. 6. The method for early detection of crop diseases based on multispectral image fusion of claim 1, wherein the multispectral feature pyramid network comprises four scale levels, the feature image sizes of each level are one fourth, one eighth, one sixteenth and one thirty half of the size of an input image respectively, the feature pyramid adopts a top-down structure, and a high-level feature image and a low-level feature image are subjected to transverse connection fusion through upsampling.
  7. 7. The method for early detection of crop diseases based on multispectral image fusion according to claim 1, wherein the channel attention weight is generated through global average pooling and an extrusion excitation structure of a full-connection layer, the full-connection layer compresses the channel number to be one sixteenth of the original channel number and then restores the channel number to the original channel number, and the spatial attention weight is generated through 7×7 convolution layers after the average value and the maximum value of the spatial attention weight are calculated along the channel dimension through a feature map.
  8. 8. The method for early detection of crop diseases based on multispectral image fusion of claim 1, wherein the conditional random field energy function comprises a unitary potential function and a binary potential function, the unitary potential function is determined by negative logarithmic transformation through pixel-level confidence of an initial detection result, the binary potential function is calculated through double Gaussian kernel according to spatial distance and color difference among pixels, the energy function optimization adopts a uniform field approximate iterative algorithm, and the iteration number is set to be 5.
  9. 9. The method for early detection of crop diseases based on multispectral image fusion of claim 1, wherein the morphological operation comprises an open operation and a closed operation, the open operation adopts 3X 3 rectangular structural elements to firstly corrode and then swell to eliminate isolated noise points, the closed operation adopts 5X 5 rectangular structural elements to firstly swell and then corrode to fill cavities in the lesions, the preset threshold is set to 0.7, and the weight adjustment step size is set to 0.05.
  10. 10. The early detection system for crop diseases based on multispectral image fusion is used for realizing the early detection method for crop diseases based on multispectral image fusion as claimed in any one of claims 1 to 9, and is characterized by comprising the following steps: The self-adaptive registration fusion module of the multispectral image is used for executing self-adaptive registration based on depth characteristics on the synchronously acquired visible light image and near infrared image, adopting a twin convolution network to respectively extract depth characteristic descriptors of two paths of images, and calculating space transformation parameters through characteristic matching to realize pixel level alignment; The spectrum space feature collaborative extraction module is used for calculating a normalized difference vegetation index and a chlorophyll absorption ratio index based on the multispectral fusion image to serve as spectrum vegetation index features; The multi-scale cross attention disease detection module is used for inputting the spectrum space joint feature vector into a multi-scale feature pyramid network to generate feature graphs on different scale levels, respectively calculating channel attention weights and space attention weights for the feature graphs, multiplying the channel attention weights and the space attention weights element by element to generate cross attention force diagram, multiplying the cross attention force diagram and an original feature graph element by element to obtain attention enhancement features, and fusing the multi-scale attention feature to output an initial disease detection result and a position confidence map; The confidence coefficient driving boundary optimization module is used for carrying out boundary refinement on an initial detection result based on the position confidence coefficient map construction condition random field energy function, eliminating isolated noise points by adopting morphological operation and filling cavities in lesions, feeding detection confidence coefficient back to the multispectral image self-adaptive registration fusion module, and increasing the fusion weight of a near infrared band when the detection confidence coefficient is lower than a preset threshold value.

Description

Crop disease early detection method and system based on multispectral image fusion Technical Field The invention relates to the technical field of agricultural intelligent monitoring and image processing, in particular to a crop disease early detection method and a crop disease early detection system based on multispectral image fusion. Background Crop diseases are important factors influencing agricultural production, and early detection and timely control of the diseases are of great significance for guaranteeing grain safety. The traditional manual inspection mode has the problems of low efficiency, strong subjectivity, difficult coverage of a large-area farmland and the like, and an automatic detection technology based on image analysis becomes an effective way for solving the problem. In the prior art, a crop disease detection method based on spectral imaging has a certain research basis. The Chinese patent application with publication number of CN112561883A discloses a method for reconstructing hyperspectral image of crop RGB image, which uses common camera and hyperspectral camera to collect image under the same environment, uses Mologold seven parameter model to make coordinate system conversion, adopts least square method to calculate conversion matrix from RGB image to hyperspectral image, and uses Hermite piecewise interpolation algorithm to implement spectral curve expansion fitting. The method can reconstruct hyperspectral images from RGB images, but has the following defects that firstly, the method is a spectrum reconstruction method essentially, feature extraction and network design identification are not carried out according to actual requirements of disease detection, secondly, the coordinate conversion process depends on preset mark points, the problem of complex deployment in actual field application exists, thirdly, the spectrum expansion method based on interpolation has limited capturing capability on fine spectrum difference of early disease spots, and fourthly, the method lacks space positioning and boundary segmentation capability of disease spot areas. In recent years, deep learning technology has made remarkable progress in the field of plant disease detection. Research shows that the combination of visible light and near infrared multispectral images can effectively improve disease identification precision, and a multiscale attention mechanism is beneficial to capturing weak characteristic response of early lesions. However, the existing method still has many challenges when processing early disease detection tasks, namely small early disease area and fuzzy boundary, symptoms in a visible light wave band are not obvious, global semantic information and local detail features are difficult to be considered in single-scale feature extraction, and the precision of the disease segmentation boundary is not high and is easy to be interfered by background noise. Therefore, how to design a detection method which can fully fuse multispectral information, effectively extract early disease features and realize high-precision disease spot positioning and segmentation becomes a key problem of the development of early detection technology of crop diseases. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a crop disease early detection method and a crop disease early detection system based on multispectral image fusion, which are used for constructing a complete closed loop detection system from image acquisition to disease positioning by performing multispectral image self-adaptive registration fusion, spectral space feature collaborative extraction, multiscale cross attention disease detection and confidence driving boundary optimization on deep coupling of four core modules, so as to realize high-precision identification and segmentation of early diseases. The crop disease early detection method based on multispectral image fusion comprises the following steps: And a multi-spectral image self-adaptive registration fusion step, namely performing depth feature-based self-adaptive registration on the synchronously acquired visible light image and near infrared image, eliminating spatial dislocation generated by dual-sensor acquisition, and fusing the registered dual-band image into a multi-spectral fusion image containing rich spectral information by adopting a frequency domain decomposition and gradient guidance fusion strategy. The method comprises the steps of spectrum space feature collaborative extraction, calculating spectrum vegetation index features reflecting the health state of crops based on multispectral fusion images, extracting space texture features by adopting a light convolution network, constructing spectrum space joint feature vectors through feature cascading, and realizing collaborative characterization of spectrum information and space information. The method comprises the steps of multi-scale cross attention disease detection, namel