CN-121982400-A - Cross-scene hyperspectral image classification method based on open-domain adaptation

CN121982400ACN 121982400 ACN121982400 ACN 121982400ACN-121982400-A

Abstract

The invention discloses a cross-scene hyperspectral image classification method based on open-set domain adaptation, which comprises the steps of firstly obtaining source domain data and target domain data, obtaining alignment features of the source domain data and the target domain data through an alignment encoder, obtaining intrinsic features of the target domain data through an intrinsic encoder, obtaining a classification prediction result of the source domain data through a classifier by the source domain alignment features, inputting the alignment features of the target domain data and the intrinsic features of the target domain data into an open-set recognition module, recognizing known classes and unknown classes in the target domain data, constructing a total loss function by adopting the alignment features of the source domain data and the target domain data, the classification prediction result of the source domain data and real class labels of the source domain data, training a network model through the minimum total loss function, and inputting the known classes of the target domain data output by the open-set recognition module into a trained model to obtain the prediction class of the known classes. The invention realizes the classification of the known class and the identification of the unknown class of the target image by designing the open-domain adaptation method.

Inventors

WANG MINGHUA
LIU YIWEN
ZHAO XIN

Assignees

南开大学

Dates

Publication Date: 20260505
Application Date: 20260123

Claims (7)

1. The cross-scene hyperspectral image classification method based on open-domain adaptation is characterized by comprising the following steps of: s1, acquiring source domain data and target domain data, wherein the source domain data is a hyperspectral image with a real type tag acquired from a satellite, and the target domain data is a hyperspectral image of a target area acquired from the satellite; S2, inputting the source domain data and the target domain data into an alignment encoder to respectively obtain source domain alignment features and target domain alignment features; S3, inputting the target domain data into an intrinsic encoder to obtain target domain intrinsic characteristics; s4, inputting the source domain alignment features into a classifier to obtain a classification prediction result of the source domain data; s5, inputting the target domain alignment features and the target domain intrinsic features into an open set identification module, and identifying known classes and unknown classes in target domain data; s6, constructing a total loss function by adopting the source domain alignment feature, the target domain alignment feature, the classification prediction result of the source domain data obtained from the S4 and the real class label of the source domain data obtained from the S1, and training a network model constructed by the S2-S5 by minimizing the total loss function; S7, inputting target domain data to be classified into a network model which is trained, and obtaining a prediction class of a known class.
2. The method for classifying hyperspectral images across scenes based on open-domain adaptation according to claim 1, wherein the alignment encoder in step S2 is constructed as follows: S2.1, extracting spectrum alignment features, namely respectively inputting a hyperspectral image of the source domain data and a hyperspectral image of the target domain data into three-dimensional convolution layers with the size of 7 multiplied by 1, adding the features after the first layer convolution as residual errors with the features after the three-layer convolution, inputting the added features into one layer of three-dimensional convolution layers to obtain spectrum alignment features of the source domain data and spectrum alignment features of the target domain data, and obtaining the weighted and enhanced source domain spectrum alignment features and target domain spectrum alignment features through a channel attention mechanism; S2.2, extracting spatial alignment features, namely respectively inputting the hyperspectral image of the source domain data and the hyperspectral image of the target domain data into three-dimensional convolution layers with the convolution kernel size of 1 multiplied by 3, inputting the features after the first layer convolution into one three-dimensional convolution layer with the convolution kernel size of 1 multiplied by 1 as residual errors, adding the residual errors and the features after the three-layer convolution into one three-dimensional convolution layer to obtain the spatial alignment features of the source domain data and the spatial alignment features of the target domain data, and obtaining the weighted and enhanced spatial alignment features of the source domain and the target domain through a spatial attention mechanism; s2.3, the alignment features are fused, namely the weighted and enhanced source domain spectrum alignment features and the weighted and enhanced source domain space alignment features are spliced to obtain source domain alignment features, and meanwhile, the weighted and enhanced target domain spectrum alignment features and the weighted and enhanced target domain space alignment features are spliced to obtain target domain alignment features.
3. The method for classifying the hyperspectral image across scenes based on open-domain adaptation according to claim 1, wherein the step of constructing the evidence encoder in the step S3 is as follows: S3.1, spectral intrinsic characteristics are extracted, namely a hyperspectral image of target domain data is input into a three-dimensional convolution layer with the size of 7 multiplied by 1, then the characteristics after the convolution of a first layer are taken as residual errors to be added with the characteristics after the convolution of three layers, the added characteristics are input into a three-dimensional convolution layer to obtain spectral intrinsic characteristics of the target domain data, and then weighted and enhanced target domain spectral intrinsic characteristics are obtained through a channel attention mechanism; S3.2, extracting space intrinsic characteristics, namely inputting a hyperspectral image of the target domain data into three-dimensional convolution layers with the convolution kernel sizes of 1 multiplied by 3, inputting the characteristics of the first layer of convolution into one three-dimensional convolution layer with the convolution kernel sizes of 1 multiplied by 1 as residual errors, adding the residual errors and the characteristics of the three layers of convolution, inputting the added residual errors into one three-dimensional convolution layer to obtain space intrinsic characteristics of the target domain data, and obtaining weighted and enhanced space intrinsic characteristics of the target domain through a space attention mechanism; and S3.3, the intrinsic characteristics are fused, namely the weighted and enhanced target domain spectrum intrinsic characteristics and the weighted and enhanced target domain space intrinsic characteristics are spliced to obtain the intrinsic characteristics of the target domain.
4. The method for classifying the hyperspectral image across the scene based on the open-domain adaptation according to claim 1, wherein the classifier in the step S4 is composed of a full-connection layer with 256 neurons, a batch normalization layer, a LeakyReLU activation function layer and a full-connection layer with 3 neurons which are connected in sequence.
5. The method for classifying the hyperspectral image across scenes based on open-set domain adaptation according to claim 1, wherein the step of constructing the open-set identification module in the step S5 is as follows: S5.1, calculating similarity measurement, namely calculating cosine similarity of the alignment feature of the target domain and the intrinsic feature of the target domain, and calculating square of the obtained cosine similarity to obtain a consistency score, wherein the consistency score is expressed as: ; Wherein the method comprises the steps of For target domain hyperspectral images The first of (3) A number of samples of the sample were taken, Representing target domain alignment features The first of (3) The number of alignment features is selected such that, Representing target domain intrinsic features The first of (3) The characteristic of each of the features is that, Representing a consistency score; S5.2, known-unknown class distinction, namely constructing a Gaussian mixture model containing two Gaussian components, inputting consistency scores of all target domain hyperspectral images into the Gaussian mixture model to obtain the average value of the two Gaussian components, and judging the class by comparing the average value of the two Gaussian components, wherein the target domain hyperspectral image belonging to the high-average component is an unknown class, and the target domain hyperspectral image belonging to the low-average component is a known class.
6. The method for classifying hyperspectral images across scenes based on open-domain adaptation according to claim 2, wherein the step of constructing the loss function in step S6 is as follows: S6.1, adopting cross entropy loss as classification loss, inputting the real label of the source domain data obtained in S1 and the classification prediction result of the source domain data obtained in S4 into the cross entropy loss to obtain the classification loss, wherein the classification loss is expressed as: ; Wherein the method comprises the steps of Representing a loss of classification, Representing the number of known class categories, A real tag representing the source domain data, A classification prediction result representing source domain data; S6.2, calculating the spectrum domain adaptation loss of the weighted enhanced source domain spectrum alignment feature and the weighted enhanced target domain spectrum alignment feature by adopting a maximum mean difference measurement method as domain adaptation loss, simultaneously calculating the space domain adaptation loss of the weighted enhanced source domain space alignment feature and the weighted enhanced target domain space alignment feature, and adding the spectrum domain adaptation loss and the space domain adaptation loss to obtain domain adaptation loss, wherein the domain adaptation loss is expressed as: ; Wherein the method comprises the steps of Representing a loss of domain adaptation, Indicating a loss of adaptation of the spectral domain, Representing the loss of adaptation of the spatial domain, And Respectively representing the weighted enhanced source domain spectral alignment features and the target domain spectral alignment features, And Respectively representing the weighted enhanced source domain space alignment feature and the target domain space alignment feature; s6.3, adding the classification loss and the domain adaptation loss to obtain a total loss function, wherein the total loss function is expressed as: ; Wherein the method comprises the steps of Representing the total loss function of the device, The weighting coefficients of the loss terms are adapted for the domain.
7. The method is characterized in that the step S7 is specifically that a trained network model comprises an alignment encoder, an intrinsic encoder, an open set identification module and a classifier, target domain hyperspectral images to be classified are respectively input into the trained alignment encoder and the trained intrinsic encoder to obtain target domain alignment features and target domain intrinsic features, the obtained target domain alignment features and target domain intrinsic features are input into the open set identification module to obtain known classes and unknown classes of target domain hyperspectral images, and finally the known classes of the target domain hyperspectral images are input into the trained classifier to obtain predicted classes of the known classes of the target domain hyperspectral images.

Description

Cross-scene hyperspectral image classification method based on open-domain adaptation Technical Field The invention belongs to the technical field of cross-scene hyperspectral image classification, and particularly relates to a cross-scene hyperspectral image classification method based on open-domain adaptation. Background The hyperspectral image is used as one of important technologies in the remote sensing field, has spectrum information and space information, and is widely applied to multiple fields such as precise agriculture, environment monitoring, mineral identification and the like. Hyperspectral image classification is one of the core tasks of hyperspectral remote sensing applications, the main purpose of which is to assign each pixel a class based on its spectral and spatial characteristics. The precondition of the traditional hyperspectral image classification method is that the training sample and the test sample are identical in data distribution and share the same class set. However, in practical applications, hyperspectral images acquired at different times and different places often have distribution differences due to the influence of multiple factors such as sensor parameters, imaging time, atmospheric conditions and the like. The domain adaptation method aims at reducing the distribution difference of the source domain and the target domain, and learns domain invariant features insensitive to domain changes, and is an effective method for solving the problems. Most of the existing cross-scene hyperspectral image classification methods are closed-set domain adaptation methods, namely, the classification spaces of a source domain and a target domain are assumed to be completely consistent. However, unknown categories in the source domain data that have not been present tend to be present in the target domain. If the closed-set domain adaptation method is directly adopted, unknown samples can be wrongly classified into known categories, so that the classification accuracy is reduced. Aiming at the problems, the invention provides a cross-scene hyperspectral image classification method based on open-domain adaptation. In the hyperspectral data, the spectral characteristics and the spatial characteristics are not only independent, but also different in contribution degree to the final classification result, and the spectral domain adaptation loss and the spatial domain adaptation loss between the source domain data and the target domain data are respectively constructed. In order to save calculation resources and training cost, an intrinsic encoder and an encoder are respectively constructed, and unknown categories are distinguished while known category classification results are obtained in a single-stage training mode. Disclosure of Invention The invention aims to provide a cross-scene hyperspectral image classification method based on open-domain adaptation, which is used for solving the problems that in the cross-scene hyperspectral image classification, the source domain and the target domain are not consistent in distribution, and unknown categories exist in the target domain. The technical scheme adopted by the invention is as follows: A cross-scene hyperspectral image classification method based on open-domain adaptation comprises the following steps: s1, acquiring source domain data and target domain data, wherein the source domain data is a hyperspectral image with a real type tag acquired from a satellite, and the target domain data is a hyperspectral image of a target area acquired from the satellite; S2, inputting the source domain data and the target domain data into an alignment encoder to respectively obtain source domain alignment features and target domain alignment features; S3, inputting the target domain data into an intrinsic encoder to obtain target domain intrinsic characteristics; s4, inputting the source domain alignment features into a classifier to obtain a classification prediction result of the source domain data; s5, inputting the target domain alignment features and the target domain intrinsic features into an open set identification module, and identifying known classes and unknown classes in target domain data; s6, constructing a total loss function by adopting the source domain alignment feature, the target domain alignment feature, the classification prediction result of the source domain data obtained from the S4 and the real class label of the source domain data obtained from the S1, and training a network model constructed by the S2-S5 by minimizing the total loss function; S7, inputting target domain data to be classified into a network model which is trained, and obtaining a prediction class of a known class. Further, the alignment encoder construction step in the step S2 is as follows: S2.1, extracting spectrum alignment features, namely respectively inputting a hyperspectral image of the source domain data and a hyperspectral image of the target doma