CN-122024069-A - Weight adjustment-based polarization self-attention feature enhancement method and system

CN122024069ACN 122024069 ACN122024069 ACN 122024069ACN-122024069-A

Abstract

The invention discloses a polarization self-attention characteristic enhancement method and a polarization self-attention characteristic enhancement system based on weight adjustment, comprising the following steps of preprocessing an original characteristic diagram to obtain a preprocessed characteristic diagram; the method comprises the steps of respectively modeling a preprocessing feature map and applying corresponding attention to generate a channel polarization weighted feature map and a space polarization weighted feature map, generating two groups of channel dimension non-mutually exclusive dynamic adjustment weights through global average pooling, feature transformation and activation splitting based on an original feature map, carrying out weighted fusion on the channel polarization weighted feature map and the space polarization weighted feature map by utilizing the two groups of channel dimension non-mutually exclusive dynamic adjustment weights, and superposing the original feature map to obtain a final feature map. The invention effectively solves the problems of limited receptive field, mutual exclusion of channel and space weight distribution and the like in the existing polarization self-attention mechanism.

Inventors

HE YAWEN
HU CHAOPENG
DENG JINXIU
ZHANG ZHIHUA
MA XIANGYU
DING PENGHUI
ZHAO YINGYUE
LI ZHIGANG
ZHANG JIUYAN
ZHAO QIAN

Assignees

中国石油大学(华东)
青岛市勘察测绘研究院
青岛中科蓝迪信息科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260227

Claims (10)

1. A method for enhancing a polarized self-attention feature based on weight adjustment, comprising: acquiring an original feature map, and preprocessing the original feature map to obtain a preprocessed feature map; modeling the spatial context dependence of the preprocessing feature map and applying channel attention to obtain a channel polarization weighted feature map; meanwhile, carrying out channel context dependent modeling on the preprocessing feature map and applying spatial attention to obtain a spatial polarization weighted feature map; Based on the original feature map, generating two groups of channel dimension non-mutually exclusive dynamic adjustment weights through global average pooling, feature transformation and activation splitting; and carrying out weighted fusion on the channel polarization weighted feature map and the space polarization weighted feature map by using the two groups of channel dimension non-mutually exclusive dynamic adjustment weights, and superposing the original feature map to obtain a final feature map.
2. The method for enhancing the polarization self-attention feature based on weight adjustment of claim 1, wherein the original feature map is preprocessed by a depth separable hole convolution structure, the depth separable hole convolution structure is composed of a grouping hole convolution layer, a batch normalization layer and a ReLU activation function layer, wherein the hole rate of the grouping hole convolution layer is larger than 1, and the grouping number is equal to the channel number of the feature map.
3. The method for enhancing the polarization self-attention feature based on weight adjustment according to claim 1, wherein the step of obtaining the channel polarization weighted feature map is characterized in that firstly, a value vector and a query vector are extracted through a convolution layer based on a preprocessing feature map, then matrix multiplication calculation is carried out on the value vector and the query vector along a space dimension to obtain a channel descriptor based on global space information, the channel descriptor is mapped back to the channel dimension, a channel attention mask is generated through a Sigmoid activation function, and finally the channel attention mask is applied to the preprocessing feature map to obtain the channel polarization weighted feature map.
4. The method for enhancing polarization self-attention feature based on weight adjustment as recited in claim 1, wherein the step of obtaining the spatial polarization weighted feature map is to extract channel context query vectors through global average pooling based on the preprocessing feature map, extract value vectors through convolution layers, perform self-attention computation along channel dimensions by utilizing the query vectors and the value vectors, generate a spatial attention mask through a Sigmoid activation function, and apply the spatial attention mask to the preprocessing feature map to obtain the spatial polarization weighted feature map.
5. The polarization self-attention feature enhancement method based on weight adjustment according to claim 1, wherein global descriptors are obtained by global average pooling of original feature graphs, the global descriptors sequentially pass through a channel compressed convolution layer and a channel 2C output convolution layer to output dual-channel weights, and the dual-channel weights are split into two groups of dynamic adjustment weights based on the dual-channel weights by utilizing a Sigmoid activation function, wherein the two groups of dynamic adjustment weights are respectively channel branch dynamic adjustment weights and space branch dynamic adjustment weights.
6. The method for enhancing polarization self-attention characteristics based on weight adjustment according to claim 5, wherein the two sets of dynamic adjustment weights have the same channel dimension and are non-mutually exclusive in the channel dimension, and the two sets of dynamic adjustment weights have the following calculation formulas: Wherein, the The weights are dynamically adjusted for the channel branches, The weights are dynamically adjusted for the spatial branches, For a convolutional layer with an output channel of 2C, For a convolutional layer of channel compression, To global average pooling of the original feature map.
7. The weight adjustment based polarized self-attention feature enhancement method of claim 1, wherein the final feature map is represented as: ; Wherein, the The feature map is weighted for the polarization of the channel, For the spatial polarization weighting profile, For element-level multiplication to occur after broadcast, Is the original feature map.
8. A weight adjustment based polarized self-attention feature enhancement system comprising the following modules: the preprocessing module is configured to acquire an original feature map, and preprocess the original feature map to acquire a preprocessed feature map; The channel module and the space module are configured to perform space context dependent modeling on the preprocessing feature map and apply channel attention to obtain a channel polarization weighted feature map; The channel specificity gating module is configured to generate two groups of channel dimension non-mutually exclusive dynamic adjustment weights through global average pooling, feature transformation and activation splitting based on an original feature map; And the dynamic self-adaptive fusion module is configured to carry out weighted fusion on the channel polarization weighted feature map and the space polarization weighted feature map by utilizing the two groups of channel dimension non-mutually exclusive dynamic adjustment weights, and superimpose the original feature map to obtain a final feature map.
9. A computer readable storage medium having stored thereon a program, which when executed by a processor performs the steps in the weight adjustment based polarization self-attention feature enhancement method as claimed in any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a program stored on the memory and executable on the processor, wherein the processor implements the steps of the weight adjustment based polarized self-attention feature enhancement method of any one of claims 1-7 when the program is executed by the processor.

Description

Weight adjustment-based polarization self-attention feature enhancement method and system Technical Field The invention relates to the technical field of computer vision and deep learning, in particular to a polarization self-attention characteristic enhancement method and system based on weight adjustment. Background In the fields of computer vision and deep learning, a self-attention mechanism is used as a key component for improving the feature expression capability of a model, and is widely applied to various tasks such as semantic segmentation. The core logic of the method is to accurately capture the association information among the features by modeling the global dependency relationship of the feature map, so as to optimize the understanding and identifying capability of the model to the complex scene. Among them, polarized self-attention mechanism (PSA) is a very representative implementation manner, and the mechanism innovatively decomposes self-attention into two parallel paths of channel polarization and spatial polarization, and refines the context information of channel dimension and spatial dimension respectively, so as to provide effective support for model performance improvement. In the prior art, the application of the polarized self-attention mechanism mainly relies on the following technical scheme that firstly, an input feature map is directly received and respectively sent into a channel polarization branch and a space polarization branch, the channel polarization branch extracts the feature dependency relationship of a space dimension through global information related to self-attention calculation mining channels by feature mapping, and finally, a fixed fusion strategy (such as direct series connection, parallel connection and the like) is adopted to integrate the output features of the two branches to form a final enhancement feature for a subsequent task. However, the prior art has a number of disadvantages in research: The prior art has the defects that the prior PSA mechanism has a lack of flexibility in a fixed fusion mode, and cannot adaptively adjust the emphasis degree of channel information and space information according to the dynamic requirements of a deep learning model in different network levels and in the face of different input data, so that the feature fusion effect is limited; Secondly, the receptive field is insufficient, namely the original PSA mechanism directly calculates the input characteristics and does not perform local context preprocessing on the characteristics of the input data. For complex scenes such as plots with changeable scales in remote sensing images, enough neighborhood characteristics are difficult to capture, and the segmentation result is easy to break; Thirdly, weight distribution mutual exclusion is carried out, a Softmax normalization method is often adopted for traditional dynamic weight generation, the method forces the sum of channel and space weights to be 1, a constraint relation of zero and game is formed, and the capability of the model for fully utilizing two characteristics to enhance advantages is limited; Fourthly, the training stability is poor, in a deep network architecture, the existing polarization self-attention mechanism lacks effective residual structure design, gradient loss is easy to cause, and the training convergence speed and the final performance stability of the model are affected. Disclosure of Invention In order to solve the problems, the invention provides a polarization self-attention characteristic enhancement method and a polarization self-attention characteristic enhancement system based on weight adjustment. In order to achieve the above purpose, the present invention adopts the following technical scheme: In a first aspect, the present invention provides a method for enhancing polarized self-attention features based on weight adjustment, comprising: acquiring an original feature map, and preprocessing the original feature map to obtain a preprocessed feature map; modeling the spatial context dependence of the preprocessing feature map and applying channel attention to obtain a channel polarization weighted feature map; meanwhile, carrying out channel context dependent modeling on the preprocessing feature map and applying spatial attention to obtain a spatial polarization weighted feature map; Based on the original feature map, generating two groups of channel dimension non-mutually exclusive dynamic adjustment weights through global average pooling, feature transformation and activation splitting; and carrying out weighted fusion on the channel polarization weighted feature map and the space polarization weighted feature map by using the two groups of channel dimension non-mutually exclusive dynamic adjustment weights, and superposing the original feature map to obtain a final feature map. According to a further technical scheme, the original feature map is preprocessed by adopting a depth separable h