CN-116580221-B - High-resolution SAR image scene classification method based on multi-prototype learning

Abstract

The invention discloses a high-resolution SAR image scene classification method based on multi-prototype learning, which aims at the problems of few available SAR images and few high-quality labeling SAR images. In addition, the invention also provides a method for learning multiple prototypes, and because the single prototype of each category cannot well represent the category, the classification of the query set is guided by generating multiple prototypes of each category through a support set of small sample learning division. The data set of the invention adopts SAR image scene classification small sample data set. Experiments show that the high-resolution SAR image scene classification method based on the multi-prototype learning improves the accuracy of high-resolution SAR image scene classification and can effectively overcome interference caused by speckle noise.

Inventors

• ZHAO ZHIQIANG
• TONG YUHUI
• JIA MENG
• SHI WEIWEI
• YU HAO

Assignees

• 西安理工大学

Dates

Publication Date

20260505

Application Date

20230621

Claims (2)

1. 1. The high-resolution SAR image scene classification method based on the multi-prototype learning is characterized by comprising the following specific operation steps: Step 1, acquiring a small sample data set of SAR image scene classification, and dividing the data set into a training set, a verification set and a test set according to the classification; Step 2, randomly extracting a plurality of small sample data blocks from a training set according to categories to serve as a supporting set, randomly extracting a plurality of sample data from residual samples of the supporting set to serve as a query set, and preprocessing images of the supporting set and the query set; In the step 2, following the setting of small sample learning N-way k-shot, firstly randomly selecting N classes from a training set, then randomly selecting k samples in the N classes, taking the N multiplied by k samples as a data block of a supporting set, randomly selecting the samples from the N classes to form other data blocks in the same way, forming a supporting set by the data blocks, randomly selecting the samples from the rest samples in the N classes to form a query set, forming a training task by a group of supporting sets and the query set, then preprocessing the samples of the supporting sets and the query set, scaling the size of an image to 84 multiplied by 84, and then normalizing the image, wherein the image is a single-channel gray image, the average value is set to be 0.485, and the variance is set to be 0.229; Step 3, extracting features from a plurality of small sample data blocks of the preprocessed support set through a feature extractor, obtaining prototypes of each class through all features of each class in each data block, and integrating a plurality of prototypes of a plurality of data blocks; The feature extractor in the step 3 is composed of 4 convolution blocks, an 84×84 support set image is input into the feature extractor, the first convolution block input channel of the feature extractor is 1, the output channel is 64, the data is normalized through a convolution layer with the convolution kernel size of 3×3 and filling 1, then through a block normalization layer, the data is finally subjected to the operation of maximum pooling through an activation function relu layer, the pooling kernel size is 2×2, the rest three convolution blocks are completely consistent with the first convolution block except the input channel of 64, the output result is a feature map with the size of 5×5 and the channel number of 64 after the data is subjected to 4 convolution layers, and finally the output feature map is flattened into one line; When the support set extracts features through the feature extractor, the prototype of each class is obtained by averaging each class of each data block according to the formula (1), the prototype meaning exists as a representation of each class, then the prototypes of the same class of different data blocks are integrated together, so as to obtain the multiple prototypes of each class in the N classes, and the calculation formula of the prototypes is defined as: (1) Wherein, the Representing a prototype of class c, Is the i-th sample of the c-type samples, A sample of class c is represented and, Representing the feature extractor, k represents the total number of class c samples Step 4, extracting features from the query set image, then projecting the features on the prototypes of each class according to the extracted features of the query set, and classifying the features according to the projection distance from the features of the query set to the prototypes; in step 4, the query set image is also preprocessed, namely the size of the image is scaled to 84 multiplied by 84, then the image is standardized, and the average value is set to 0.485 and the variance is set to 0.229 because the image is a single-channel gray image; After preprocessing the query set, extracting features of the query set through a feature extractor, enabling the support set and the query set to share the same feature extractor, then projecting the obtained query sample features on the multiple prototypes of the N categories according to a formula (2), wherein a projection formula from the query sample features to the multiple prototypes is defined as follows: (2) is a query sample In a polytype The projection onto the surface of the lens, Is the average of all samples of class c, then calculate the distance between the projections of the query feature onto the N class prototypes according to equation (3), query the image to the prototypes Is defined as: (3) Finally, obtaining the probability that the query samples respectively belong to the N categories according to the softmax function of the formula (4), and querying the samples The probability formula assigned to class c is defined as: (4) the category with the highest probability is judged as the classification result.
2. 2. The method for classifying the high-resolution SAR image scenes based on the multi-prototype learning according to claim 1 is characterized in that in the step 1, a small sample data set of SAR image scene classification is obtained, the image of the data set is a single-channel gray level image, 12 classes are set in the data set, the data set is divided into 6 classes of training sets, 3 classes of verification sets and 3 classes of test sets according to the classes, the training sets, the verification sets and the test sets are not intersected, each image is provided with a corresponding class label, and the size of each image is 256×256 pixels.

Description

High-resolution SAR image scene classification method based on multi-prototype learning Technical Field The invention belongs to the field of mechanical manufacturing equipment, and particularly relates to a high-resolution SAR image scene classification method based on multi-prototype learning. Background Synthetic aperture radar (SYNTHETIC APERTURE RADAR, SAR) is an all-weather, all-day, long-distance, active earth observation system, which has been widely used in various fields of military and civilian use, mainly including automatic target recognition, crop growth prediction, vegetation monitoring, natural disaster risk assessment, marine environment monitoring, etc. SAR imaging technology is developed from early low space-time resolution, single-band and single-polarization to high space-time resolution, multi-polarization, multi-band, multi-mode, interference, polarized interference, tomography and the like, so that the observation capability of ground targets is improved dramatically, and observation means and observation data are enriched greatly. Aiming at the acquired massive high-resolution SAR images, development of SAR image interpretation technology is urgently needed to mine knowledge in the images, and theoretical support is provided for various important applications. Scene classification is one of important directions of SAR image interpretation, and is widely applied to the fields of city planning, ship identification, marine pollution monitoring, earth resource exploration and the like. Because SAR is active echo imaging, the method has the characteristics of relatively long imaging distance and complex imaging structure, serious speckle noise exists in SAR images, and effective distinction of different scenes is extremely difficult. Two key steps of the SAR image scene classification technology are 1. Key features of the representation image are extracted from the SAR image. 2. And designing a proper classifier according to the learned key features. This step of extracting key features of the image is particularly important. Early conventional methods mainly rely on feature extraction methods based on low-level manual features, including statistical features, texture features, structural features, and the like, and mainly include SCALE INVARIANT Feature Transform (SIFT) methods. Next, a method for extracting middle layer features appears, and the obtained low layer features are encoded to obtain middle layer features with stronger discriminant, mainly including methods such as Bag Of Word (BOW). With the development of deep learning, the deep neural network can be utilized to adaptively learn the high-level image features with stronger discriminant, and an end-to-end solution is adopted, so that the limitation of manually designing the features in the past is overcome. Common deep neural networks are Convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), deep Belief Networks (DBNs), generation countermeasure networks (GANs), and the like. The deep neural network needs a large number of high-quality marked images to train to an ideal effect, and because SAR image marking needs more expert knowledge, different person marked image results have deviation, so that the high-quality marked SAR images are very few, and the existing deep learning method cannot be directly applied to SAR image scene classification. Therefore, the prior art needs a method for classifying the SAR image scene to solve the problem of few high-quality labeling samples, and can overcome speckle noise to extract more discriminative features. Disclosure of Invention The invention aims to provide a high-resolution SAR image scene classification method based on multi-prototype learning, which solves the problem of few SAR image labeling samples by using small sample learning, and combines the multi-prototype learning to realize SAR image scene classification. The technical scheme adopted by the invention is that the high-resolution SAR image scene classification method based on the multi-prototype learning comprises the following specific operation steps: Step 1, acquiring a small sample data set of SAR image scene classification, and dividing the data set into a training set, a verification set and a test set according to the classification. And step 2, randomly extracting a plurality of small sample data blocks from the training set according to the category to serve as a supporting set, randomly extracting a plurality of sample data from the remaining samples of the supporting set class to serve as a query set, and preprocessing images of the supporting set and the query set. And 3, extracting the characteristics of the preprocessed small sample data blocks of the support set through a characteristic extractor, obtaining prototypes of the classes through all the characteristics of each class in each data block, and integrating a plurality of prototypes of the data blocks. And 4, extracting features from