CN-120707528-B - Deep learning-based gallium nitride radio frequency device defect detection method and system

Abstract

The invention provides a gallium nitride radio frequency device defect detection method and a gallium nitride radio frequency device defect detection system based on deep learning, which relate to the technical field of gallium nitride devices and comprise the steps of generating and preprocessing a multispectral image data set; the method comprises the steps of executing channel and space two-dimensional attention calculation, carrying out feature optimization by combining deformable convolution and residual connection, constructing a multi-scale feature pyramid, constructing a multi-level classification tree based on feature similarity to carry out fine-grained defect classification, and evaluating the influence degree of defects on device performance by using a deep neural network model. The invention can accurately identify the defect position and type of the gallium nitride radio frequency device, quantitatively evaluate the performance influence of the defect and improve the detection precision and efficiency.

Inventors

• LIU YANGZHOU

Assignees

• 中科苏州微电子产业技术研究院

Dates

Publication Date

20260505

Application Date

20250620

Claims (9)

1. 1. The defect detection method of the gallium nitride radio frequency device based on deep learning is characterized by comprising the following steps of: Generating a multispectral image dataset based on image data of a gallium nitride radio frequency device, performing image registration, image enhancement and noise removal on the multispectral image dataset, and generating a preprocessed standardized multispectral image; Performing channel and space two-dimensional attention calculation on the standardized multispectral image to obtain feature weights, performing alignment fusion on different scale features by adopting deformable convolution, performing feature recalibration by combining a residual error connection mechanism, and outputting optimized feature representation; Constructing a multi-scale feature pyramid by the optimized feature representation, carrying out feature fusion, classifying and distinguishing the fused features to obtain a preliminary defect region, constructing a multi-level classification tree based on the feature similarity of the preliminary defect region, and setting a self-adaptive classification threshold value at each level of the multi-level classification tree to classify the fine-granularity defects; And inputting the fine-granularity defect classification result into a deep neural network model, classifying the detected defects based on the mapping relation of the defect morphological characteristics, the spectrum characteristics and the device performance parameters, evaluating the influence degree of the detected defects on the device performance, and generating a detection report containing the defect position, the defect type, the defect size and the defect performance influence degree.
2. 2. The method of claim 1, wherein performing channel and spatial two-dimensional attention calculations on the normalized multispectral image to obtain feature weights, performing alignment fusion on features of different scales using deformable convolution, performing feature recalibration in combination with a residual connection mechanism, and outputting an optimized feature representation comprises: performing global average pooling operation and global maximum pooling operation on an input feature map to obtain a channel description vector, inputting the channel description vector into a shared multi-layer perceptron network to perform nonlinear feature transformation, and fusing through a gating mechanism to generate a channel attention weight; applying a multidirectional Sobel operator to the input feature map to perform edge gradient extraction, generating edge enhancement features based on self-adaptive threshold segmentation, and generating spatial attention weights after fusing the edge enhancement features with multi-scale features of the input feature map; Performing multi-scale pooling operation on the input feature map to construct a feature pyramid structure, performing feature alignment on each scale feature in the feature pyramid structure through a deformable convolution network, and performing weighted fusion on the aligned multi-scale features based on a self-adaptive weight coefficient to obtain multi-scale fusion features; The channel attention weight and the input feature map are multiplied element by element to obtain a channel calibration feature, the channel calibration feature and the space attention weight are multiplied element by element to obtain a space calibration feature, the space calibration feature and the multi-scale fusion feature are subjected to feature fusion to obtain a fusion feature, the input feature map and the fusion feature are subjected to residual connection and addition, and a recalibration feature is output.
3. 3. The method of claim 2, wherein applying a multi-directional Sobel operator to the input feature map for edge gradient extraction, generating edge enhancement features based on adaptive threshold segmentation, the edge enhancement features fused with multi-scale features of the input feature map to generate spatial attention weights comprises: Respectively applying Sobel operators in the horizontal direction, the vertical direction, the forty-five degree direction and the one hundred and thirty-five degree direction to the input feature map to carry out convolution operation to obtain a multi-directional edge gradient map, and carrying out amplitude superposition and direction coding on the multi-directional edge gradient map to generate an edge response feature map; calculating an adaptive threshold based on the edge response feature map, wherein the adaptive threshold is determined by the mean value, standard deviation and a learnable weight parameter of the feature map, and performing dynamic threshold segmentation on the edge response feature map according to the adaptive threshold to obtain an edge enhancement feature map; Inputting the edge enhancement feature map into a convolution layer for feature transformation, generating an edge attention weight map by the feature map after the feature transformation through a sigmoid function, and carrying out self-adaptive fusion on the edge attention weight map and an original space attention map to obtain a fusion attention map; Performing multi-scale decomposition on the input feature map to obtain feature maps with different scales, adaptively weighting the feature maps with the learning weights, and performing feature reconstruction with the fusion attention map to generate a boundary enhancement feature map; And carrying out residual connection on the boundary enhancement feature map and the edge enhancement feature map, controlling the contribution degree of residual features through a learnable enhancement coefficient, and outputting the spatial attention weight of fusion boundary perception.
4. 4. The method of claim 1, wherein constructing a multi-scale feature pyramid from the optimized feature representation and performing feature fusion, classifying and discriminating the fused features to obtain a preliminary defect region, constructing a multi-level classification tree based on feature similarity of the preliminary defect region, and setting an adaptive classification threshold at each level of the multi-level classification tree to perform fine-grained defect classification comprises: Constructing a multi-scale feature pyramid by the optimized feature representation through self-adaptive pooling operation, carrying out up-sampling and down-sampling fusion on feature graphs of adjacent scale layers in the multi-scale feature pyramid, and carrying out self-adaptive weighted combination on the fused multi-scale features based on the learnable weights to obtain fusion features; Performing primary defect identification on the fusion features through a classification discrimination network, dividing a primary defect region according to a comparison result of a probability value output by the classification discrimination network and a preset discrimination threshold value, and extracting depth feature representation of the primary defect region; Calculating a similarity matrix between depth feature representations of the preliminary defect region, and constructing a hierarchical classification tree based on the similarity matrix, wherein each level of the hierarchical classification tree corresponds to defect category division with different granularities; And calculating classification confidence coefficient at each level of the hierarchical classification tree, setting an adaptive classification threshold according to historical statistical distribution of the classification confidence coefficient, and comparing the depth feature representation with the corresponding adaptive classification threshold at each level to realize fine-granularity defect classification.
5. 5. The method of claim 4, wherein calculating classification confidence at each level of the hierarchical classification tree, setting an adaptive classification threshold based on historical statistical distribution of the classification confidence, comparing the depth feature representation at each level to a corresponding adaptive classification threshold, and achieving fine-grained defect classification comprises: Calculating classification confidence in each level of the hierarchical classification tree, wherein the classification confidence is obtained by calculating Euclidean distance between depth feature representation and feature centers of each level of the hierarchical classification tree and dividing the Euclidean distance by feature space standard deviation; A statistical distribution model is built for the historical classification confidence of each level, the statistical distribution model is converted into a continuous probability density function through Gaussian kernel density estimation, and an adaptive classification threshold is updated in an exponential moving average mode based on the value of the continuous probability density function at a preset dividing point and the classification threshold at the previous moment of the level; carrying out feature transformation on the depth feature representation through a learnable nonlinear mapping function at each level of the hierarchical classification tree, calculating a difference value between the transformed feature representation and an adaptive classification threshold of the level, and determining whether to enter the next level classification according to the difference value; And calculating cosine similarity between the depth feature representation and each subdivision class template based on the class template library of the hierarchy, determining the class with the highest cosine similarity as a classification result of the hierarchy, and updating the confidence coefficient of the classification result to historical statistical distribution for subsequent threshold dynamic adjustment.
6. 6. The method of claim 1, wherein inputting the result of the fine-grained defect classification into a deep neural network model, the deep neural network model classifying the detected defects and evaluating their impact on device performance based on a mapping of defect morphology features, spectral features, and device performance parameters, the generating a detection report including defect location, type, size, and performance impact level comprising: extracting morphological characteristics, spectral characteristics and size information of a defect region from the fine-granularity defect classification result, carrying out characteristic fusion on the morphological characteristics, the spectral characteristics and the size information through a multi-layer characteristic fusion network to obtain defect characteristic vectors, and extracting electrical performance parameters from device test data to construct performance index vectors; Inputting the defect characteristic vector and the performance index vector into a depth mapping network, wherein the depth mapping network establishes a corresponding relation between defect characteristics and device performance through multi-layer nonlinear transformation, and optimizes parameters of the depth mapping network based on historical sample data to obtain a performance mapping model; Calculating the influence weight of each defect area on the device performance by using the performance mapping model, combining the influence weight with the position coordinates, type labels and size parameters of the defects to generate defect feature descriptors, and quantitatively analyzing the defect feature descriptors through a performance evaluation network; and generating a normalized detection report according to the quantitative analysis result, wherein the detection report comprises a spatial distribution diagram of the defect, characteristic description information and a performance influence evaluation result.
7. 7. A deep learning-based gallium nitride radio frequency device defect detection system for implementing the method of any of the preceding claims 1-6, comprising: the first unit is used for generating a multispectral image dataset based on the image data of the gallium nitride radio frequency device, performing image registration, image enhancement and noise removal on the multispectral image dataset, and generating a preprocessed standardized multispectral image; The second unit is used for carrying out channel and space two-dimensional attention calculation on the standardized multispectral image to obtain feature weights, carrying out alignment fusion on features of different scales by adopting deformable convolution, carrying out feature recalibration by combining a residual error connection mechanism, and outputting optimized feature representation; the third unit is used for constructing a multi-scale feature pyramid from the optimized feature representation, carrying out feature fusion, carrying out classification discrimination on the fused features to obtain a preliminary defect region, constructing a multi-level classification tree based on the feature similarity of the preliminary defect region, and setting an adaptive classification threshold value at each level of the multi-level classification tree to carry out fine-granularity defect classification; And the fourth unit is used for inputting the fine-granularity defect classification result into a deep neural network model, classifying the detected defects based on the mapping relation of the defect morphological characteristics, the spectrum characteristics and the device performance parameters, evaluating the influence degree of the detected defects on the device performance, and generating a detection report containing the defect position, the defect type, the defect size and the defect performance influence degree.
8. 8. An electronic device, comprising: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 6.
9. 9. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 6.

Description

Deep learning-based gallium nitride radio frequency device defect detection method and system Technical Field The invention relates to gallium nitride device technology, in particular to a gallium nitride radio frequency device defect detection method and system based on deep learning. Background Gallium nitride radio frequency devices are widely used in the fields of high-frequency communication, radar and the like due to excellent electrical properties and thermal stability. In the gallium nitride device manufacturing process, various defects are easy to generate due to the influence of factors such as process, materials and the like, and the defects can obviously influence the performance and the reliability of the device. Therefore, developing a high-efficiency accurate defect detection method has important significance for guaranteeing the quality and performance of the gallium nitride radio-frequency device. The traditional gallium nitride device defect detection mainly relies on manual observation and a simple image processing algorithm, and with the development of a deep learning technology, a defect detection method based on deep learning gradually becomes a research hot spot. The existing detection method mainly adopts a single spectrum image for analysis, and uses a conventional convolutional neural network architecture for feature extraction and classification. The characteristic information of the defects of the gallium nitride device is difficult to comprehensively reflect by a single spectrum image, so that certain hidden defects are difficult to accurately identify, and the comprehensiveness and accuracy of detection are limited. The traditional convolutional neural network structure has limited capability of extracting characteristics of irregular shapes and multi-scale defects, cannot effectively capture detailed characteristics of the defects, and particularly has poor performance when processing the defects with complex shapes and large size differences. The existing defect detection method only focuses on the identification and classification of defects, lacks in-depth analysis of the correlation between the defects and the device performance, and cannot accurately evaluate the actual influence degree of the defects on the device performance, so that the guiding value of the detection result in actual production and application is limited. Disclosure of Invention The embodiment of the invention provides a gallium nitride radio frequency device defect detection method and system based on deep learning, which can solve the problems in the prior art. In a first aspect of an embodiment of the present invention, a method for detecting defects of a gallium nitride radio frequency device based on deep learning is provided, including: Generating a multispectral image dataset based on image data of a gallium nitride radio frequency device, performing image registration, image enhancement and noise removal on the multispectral image dataset, and generating a preprocessed standardized multispectral image; Performing channel and space two-dimensional attention calculation on the standardized multispectral image to obtain feature weights, performing alignment fusion on different scale features by adopting deformable convolution, performing feature recalibration by combining a residual error connection mechanism, and outputting optimized feature representation; Constructing a multi-scale feature pyramid by the optimized feature representation, carrying out feature fusion, classifying and distinguishing the fused features to obtain a preliminary defect region, constructing a multi-level classification tree based on the feature similarity of the preliminary defect region, and setting a self-adaptive classification threshold value at each level of the multi-level classification tree to classify the fine-granularity defects; And inputting the fine-granularity defect classification result into a deep neural network model, classifying the detected defects based on the mapping relation of the defect morphological characteristics, the spectrum characteristics and the device performance parameters, evaluating the influence degree of the detected defects on the device performance, and generating a detection report containing the defect position, the defect type, the defect size and the defect performance influence degree. Performing channel and space two-dimensional attention calculation on the standardized multispectral image to obtain feature weights, performing alignment fusion on different scale features by adopting deformable convolution, performing feature recalibration by combining a residual error connection mechanism, and outputting optimized feature representations, wherein the feature representation comprises the following steps: performing global average pooling operation and global maximum pooling operation on an input feature map to obtain a channel description vector, inputting the channel descr