CN-116843982-B - End-to-end hyperspectral image multi-classification camouflage target segmentation method and equipment

CN116843982BCN 116843982 BCN116843982 BCN 116843982BCN-116843982-B

Abstract

The invention relates to an end-to-end hyperspectral image multi-classification camouflage target segmentation method and equipment, and belongs to the technical field of target detection. The method comprises the steps of constructing a hyperspectral image dataset containing a camouflage target, performing iterative training on an initial camouflage target segmentation model constructed in advance by utilizing the dataset to obtain a converged camouflage target segmentation model, performing band information extraction and feature fusion focusing on an input image by the camouflage target segmentation model, performing camouflage target classification prediction based on a feature map after feature fusion focusing, inputting the hyperspectral image to be detected into the camouflage target segmentation model, and recognizing to obtain a multi-classification camouflage target segmentation result. The method solves the problem that the hyperspectral image camouflage target detection capability and accuracy are not high in the prior art.

Inventors

DENG YAO
YAN CHAO
WANG ZHENGWEI
LIU ZHIGANG
LI ZHU

Assignees

四川九洲电器集团有限责任公司

Dates

Publication Date: 20260508
Application Date: 20230726

Claims (8)

1. The end-to-end hyperspectral image multi-classification camouflage target segmentation method is characterized by comprising the following steps of: constructing a hyperspectral image dataset containing camouflage targets; The method comprises the steps of carrying out iterative training on an initial camouflage target segmentation model constructed in advance by utilizing a data set to obtain a converged camouflage target segmentation model, wherein the camouflage target segmentation model is used for carrying out band information extraction and feature fusion focusing on an input image and carrying out camouflage target classification prediction based on feature images after feature fusion focusing, the camouflage target segmentation model comprises a band information extraction network, a backbone network and a feature optimization network, wherein the band information extraction network is used for carrying out band information extraction and dimension reduction on the input hyperspectral image, the backbone network is used for carrying out size compression and feature extraction on feature images output by the band information extraction network in different scales to obtain backbone feature images in different sizes, and the feature optimization network is used for carrying out feature fusion and focusing based on the backbone feature images in different sizes through a feature fusion focusing module and carrying out classification prediction to obtain camouflage target segmentation results; the feature fusion focusing module comprises a deconvolution module, a first channel information fusion module, a second channel information fusion module, a third channel information fusion module and a space information processing module; the deconvolution module is arranged in series with the first channel information fusion module, and performs image size amplification and band information extraction fusion on a higher-order feature image in the two input feature images to obtain a first band information feature image; the second channel information fusion module is used for extracting and fusing wave band information of a low-order feature map in the two input feature maps to obtain a second wave band information feature map; the third channel information fusion module and the spatial information processing module are arranged in parallel and are respectively used for carrying out wave band information extraction and spatial information extraction on the wave band information fusion feature map to obtain a third wave band information feature map and a spatial information feature map; Inputting the hyperspectral image to be detected into the camouflage target segmentation model, and identifying to obtain a multi-classification camouflage target segmentation result.
2. The end-to-end hyperspectral image multi-classification camouflage target segmentation method according to claim 1, wherein the band information extraction network comprises a space pooling layer, a 1 xk convolution layer and a 1 x1 convolution layer which are sequentially arranged, and the band information extraction network is used for extracting and reducing the dimension of band information of an input hyperspectral image and comprises the following steps: carrying out pooling operation and convolution operation on an input hyperspectral image sequentially through a space pooling layer and a1 xk convolution layer to extract wave band information; Performing corresponding channel multiplication operation on an image obtained by extracting band information and the input hyperspectral image; And inputting the multiplied images into the 1 multiplied by 1 convolution layer to perform linear mixing of channel information, so as to obtain the output of the band information extraction network.
3. The end-to-end hyperspectral image multi-classification camouflage target segmentation method according to claim 1 is characterized in that n+1 backbone feature graphs are obtained through the backbone network, the feature optimization network comprises n feature fusion focusing modules, wherein, The nth feature fusion focusing module is used for carrying out feature fusion based on the (n+1) th backbone feature map and the nth backbone feature map; The nth-ith feature fusion focusing module is used for carrying out feature fusion on the feature image output by the nth-i+1th feature fusion focusing module and the nth-i backbone feature image, wherein i is an integer which is more than or equal to 1 and less than or equal to i < n; And carrying out classification prediction based on the feature map output by the 1 st feature fusion focusing module to obtain the camouflage target segmentation result.
4. The end-to-end hyperspectral image multi-classification camouflage target segmentation method according to claim 1, wherein the first channel information fusion module, the second channel information fusion module and the third channel information fusion module comprise a space pooling layer and a1 xk convolution layer which are sequentially arranged, and are used for sequentially carrying out space pooling operation and convolution operation on a feature map of an input channel information fusion module, and multiplying the input feature map with a feature map output by the 1 xk convolution layer to obtain a wave band information feature map.
5. The end-to-end hyperspectral image multi-classification camouflage target segmentation method according to claim 1, wherein the spatial information processing module comprises a 1×1 convolution layer and a sigmoid activation layer, and is used for processing spatial information on a feature map input into the spatial information processing module through linear convolution operation and activation operation in sequence, and adding the feature map output by the activation layer and the feature map input into the spatial information processing module through add operation to obtain a spatial information feature map.
6. The end-to-end hyperspectral image multi-class camouflage object segmentation method of claim 1, wherein the backbone network is a ResNet-152-based feature compression and extraction network.
7. The end-to-end hyperspectral image multi-classification camouflage object segmentation method of claim 1, wherein the constructing a hyperspectral camouflage object dataset comprises: Collecting a hyperspectral image containing a camouflage target; clipping the dimension of the hyperspectral image to h w C, wherein h, w and c are the height, width and spectral channel number of the image respectively; and correspondingly generating a three-channel pseudo-color image for each hyperspectral image, marking the camouflage target according to the segmentation task based on the pseudo-color image, and constructing a hyperspectral image data set containing the camouflage target based on the marked pseudo-color image.
8. An electronic device comprising at least one processor and at least one memory communicatively coupled to the processor; The memory stores instructions executable by the processor for performing the end-to-end hyperspectral image multi-class camouflage object segmentation method of any one of claims 1-7.

Description

End-to-end hyperspectral image multi-classification camouflage target segmentation method and equipment Technical Field The invention relates to the technical field of target detection, in particular to an end-to-end hyperspectral image multi-classification camouflage target segmentation method and equipment. Background Multi-class camouflage destination identification is a very challenging visual task, especially for hyperspectral data. Hyperspectral images generally have tens to hundreds of continuous spectrum bands within a specific band range, and have rich spectrum information. The high band resolution provides extremely effective discrimination information for distinguishing different materials of different substances, so that camouflage targets and backgrounds which are not easily distinguished in the visible light band can be distinguished on a hyperspectral image. But the high correlation and low variability between adjacent bands present challenges for a large spectrum, large data size, and much redundant information. Therefore, before processing the hyperspectral image related task, the hyperspectral image needs to be subjected to dimension reduction processing. On the other hand, multi-class camouflage recognition tasks based on hyperspectral images require that the network model be able to extract rich, important features from space and spectrum. Especially for multi-class camouflage target recognition tasks, the quality of the extracted features is the key of the task. Disclosure of Invention In view of the above analysis, the invention aims to provide an end-to-end hyperspectral image multi-classification camouflage target segmentation method and equipment, so as to solve the problem that the hyperspectral image camouflage target detection capability and precision are not high in the prior art. The aim of the invention is mainly realized by the following technical scheme: The invention provides an end-to-end hyperspectral image multi-classification camouflage target segmentation method, which comprises the following steps: constructing a hyperspectral image dataset containing camouflage targets; Performing iterative training on an initial camouflage target segmentation model constructed in advance by using the data set to obtain a converged camouflage target segmentation model, wherein the camouflage target segmentation model is used for extracting wave band information and focusing characteristic fusion of an input image and performing camouflage target classification prediction based on a characteristic map after the characteristic fusion focusing; Inputting the hyperspectral image to be detected into the camouflage target segmentation model, and identifying to obtain a multi-classification camouflage target segmentation result. Further, the camouflage target segmentation model comprises a wave band information extraction network, a backbone network and a characteristic optimization network; The band information extraction network is used for extracting band information and reducing dimension of the input hyperspectral image; The backbone network is used for carrying out size compression and feature extraction on a plurality of different sizes on the feature images output by the band information extraction network to obtain a plurality of backbone feature images with different sizes; The feature optimization network is used for carrying out feature fusion and focusing based on the backbone feature graphs with different sizes, and obtaining a camouflage target segmentation result through classified prediction. Further, the band information extraction network comprises a space pooling layer, a1 xk convolution layer and a1 x 1 convolution layer which are sequentially arranged, and the band information extraction network is used for extracting band information and reducing dimension of an input hyperspectral image and comprises the following steps: carrying out pooling operation and convolution operation on an input hyperspectral image sequentially through a space pooling layer and a1 xk convolution layer to extract wave band information; Performing corresponding channel multiplication operation on an image obtained by extracting band information and the input hyperspectral image; And inputting the multiplied images into the 1 multiplied by 1 convolution layer to perform linear mixing of channel information, so as to obtain the output of the band information extraction network. Further, n+1 backbone feature graphs are obtained through the backbone network, the feature optimization network comprises n feature fusion focusing modules, wherein, The nth feature fusion focusing module is used for carrying out feature fusion based on the (n+1) th backbone feature map and the nth backbone feature map; The nth-ith feature fusion focusing module is used for carrying out feature fusion on the feature image output by the nth-i+1th feature fusion focusing module and the nth-i backbone feature image, wherei