Search

CN-121998967-A - Chip surface defect detection method and system based on deep learning and computer readable medium

CN121998967ACN 121998967 ACN121998967 ACN 121998967ACN-121998967-A

Abstract

The invention discloses a chip surface defect detection method and system based on deep learning and a computer readable medium, wherein the method comprises the following steps of obtaining a bright field image and a dark field image of a chip, fusing the bright field image and the dark field image to generate a fused image, roughly positioning a chip area in the fused image to obtain an external rectangular frame of the chip area, precisely positioning the edge of the chip in the external rectangular frame to obtain a precise area of the chip, dividing the precise area into a plurality of grid image blocks with overlapping areas, carrying out defect detection on each grid image block to obtain a local defect detection result, mapping all the local defect detection results back to an original image coordinate system, and outputting chip surface defect information after merging. The invention can realize high-precision and high-robustness automatic detection of the chip surface defects, reduce manual intervention and improve the configuration efficiency and detection capability of the AOI system.

Inventors

  • WANG SHAOLONG

Assignees

  • 上海铭沣科技股份有限公司

Dates

Publication Date
20260508
Application Date
20260306

Claims (10)

  1. 1. The chip surface defect detection method based on deep learning is characterized by comprising the following steps of: Acquiring a bright field image and a dark field image of a chip; Fusing the bright field image and the dark field image to generate a fused image; coarsely positioning a chip area in the fusion image to obtain an external rectangular frame of the chip area; In the circumscribed rectangular frame, accurately positioning the edge of the chip to obtain an accurate area of the chip; Dividing the precise area into a plurality of grid image blocks with overlapping areas; performing defect detection on each grid image block to obtain a local defect detection result; And mapping all the local defect detection results back to the original image coordinate system, and outputting the chip surface defect information after merging.
  2. 2. The method for detecting surface defects of chips based on deep learning of claim 1, wherein said bright field image and dark field image are fused to generate a fused image, specifically, a bright field image is taken as a first channel, a dark field image is taken as a second channel, and a difference image of the bright field image and the dark field image is taken as a third channel, and a three-channel fused image is generated by combining.
  3. 3. The method for detecting the chip surface defects based on the deep learning, as claimed in claim 1, is characterized in that the chip area in the fusion image is roughly positioned, specifically, a trained deep learning target detection model is used for detecting the chip area in the fusion image, and an circumscribed rectangular frame of the chip is output, and the trained deep learning target detection model has robustness to illumination change.
  4. 4. The method for detecting surface defects of a chip based on deep learning as claimed in claim 1, wherein the accurate positioning of the chip edge in the circumscribed rectangular frame is specifically: a plurality of one-dimensional section lines are put in the circumscribed rectangular frame along the vertical direction of the chip edge, gray jump points on each section line are detected, and gray gradient maximum points are used as edge points to form an edge point set; And fitting a straight line to the edge point set by using a least square method to obtain four accurate edges of the chip, and obtaining an accurate area of the chip through intersection points of the four edges.
  5. 5. The method for detecting surface defects of chips based on deep learning according to claim 1, wherein the grid size and the overlapping rate of the grid image blocks are configurable to ensure that overlapping areas exist between adjacent grid image blocks.
  6. 6. The method for detecting surface defects of a chip based on deep learning of claim 1, wherein the defect detection is performed on each grid image block to obtain a local defect detection result, specifically, a trained deep learning target detection model is used for performing defect identification on each grid image block, and defect types, positions and confidence are output.
  7. 7. The method for detecting surface defects of chips based on deep learning as defined in claim 1, wherein all local defect detection results are mapped back to an original image coordinate system, and are combined as follows: According to the position of the grid in the original image, converting the defect coordinates in the grid into original image coordinates; and carrying out non-maximum value inhibition on detection results of all grids, and merging and overlapping detection frames.
  8. 8. The method for detecting surface defects of a chip according to claim 1, wherein the chip surface defect information comprises one or more combinations of defect types, defect positions, defect numbers and defect visualization profiles.
  9. 9. A deep learning-based chip surface defect detection system, comprising: the acquisition module is used for acquiring bright field images and dark field images of the chip; the fusion module is used for fusing the bright field image and the dark field image to generate a fusion image; the coarse positioning module is used for performing coarse positioning on the chip area in the fusion image to obtain an external rectangular frame of the chip area; The fine positioning module is used for accurately positioning the edge of the chip in the external rectangular frame to obtain an accurate area of the chip; The region dividing module is used for dividing the precise region into a plurality of grid image blocks with overlapping areas; The detection module is used for carrying out defect detection on each grid image block to obtain a local defect detection result; and the output module is used for mapping all the local defect detection results back to the original image coordinate system, and outputting the chip surface defect information after merging.
  10. 10. A computer readable medium having non-volatile program code executable by a processor, the program code causing the processor to run the deep learning based chip surface defect detection method of any of claims 1-8.

Description

Chip surface defect detection method and system based on deep learning and computer readable medium Technical Field The present invention relates to the field of chip surface detection technologies, and in particular, to a method and a system for detecting a chip surface defect based on deep learning, and a computer readable medium. Background With the continuous improvement of semiconductor packaging processes, chip surface defects (such as edge chipping, cracks, pits, bubbles, scratches, foreign matters, etc.) have a more remarkable effect on yield and reliability. Automatic Optical Inspection (AOI) is used as the most core quality inspection mode of the package end, and the inspection precision, stability and speed directly determine the yield and productivity of the production line. Especially in high-density packaging, ultrathin chips and high-precision manufacturing processes, surface defects often have the characteristics of small scale, weak texture, low contrast and various forms, and bring great challenges to the traditional AOI detection algorithm. At present, a white coaxial light combined with annular light mode is commonly adopted in industrial sites to acquire a chip surface image. Although the lighting mode can provide uniform illumination, the problems of non-appearing characteristics, lost texture, overlapping reflection interference and the like often occur when the lighting mode faces weak reflection, light-colored micro defects or abnormal with extremely low contrast with the background, so that the defects are difficult to fully identify by a deep learning model and a traditional algorithm. In addition, the same defect often has larger difference in performance under different angles and different illumination, so that the model is difficult to generalize, and the omission factor is higher. In terms of image processing algorithms, the traditional method depends on edge detection, threshold segmentation, connected domain analysis and other means. However, in the field of semiconductor packaging, due to the interference of chamfering, smearing, reflective textures and the like on the edge of a chip, stable and accurate chip boundary positioning cannot be obtained by simply using a traditional algorithm, and small-size defect areas cannot be effectively extracted. In recent years, the deep learning technology is widely applied to industrial visual detection, but for a complex scene of strong reflective background, small-size defect and weak texture feature on the surface of a chip, the following defects still exist by directly using a deep learning model: 1. weak texture defects are difficult to learn by the model, resulting in missed detection. 2. The small target has extremely low duty ratio, insufficient model training and unstable detection performance. 3. The input image is too large resulting in slow reasoning speed and insufficient local detail attention by the model. 4. The lack of accurate positioning of the chip area leads to an over-wide reasoning range and more noise interference. 5. Traditional single model reasoning speed is limited, and high-speed production line requirements are difficult to meet. Therefore, the prior art still has non-negligible defects in the aspects of weak texture extraction, small target detection, edge fine positioning, real-time reasoning efficiency and the like. Disclosure of Invention According to a first aspect of the embodiment of the present invention, there is provided a chip surface defect detection method based on deep learning, including the steps of: Acquiring a bright field image and a dark field image of a chip; Fusing the bright field image and the dark field image to generate a fused image; coarsely positioning a chip area in the fusion image to obtain an external rectangular frame of the chip area; in the circumscribed rectangular frame, accurately positioning the edge of the chip to obtain an accurate area of the chip; Dividing the precise area into a plurality of grid image blocks with overlapping areas; performing defect detection on each grid image block to obtain a local defect detection result; And mapping all the local defect detection results back to the original image coordinate system, and outputting the chip surface defect information after merging. And further, fusing the bright field image and the dark field image to generate a fused image, namely combining the bright field image serving as a first channel, the dark field image serving as a second channel and a difference image of the bright field image and the dark field image serving as a third channel to generate a three-channel fused image. The method comprises the steps of obtaining a fusion image, obtaining a chip area in the fusion image, and carrying out rough positioning on the chip area in the fusion image, wherein the rough positioning is specifically carried out by using a trained deep learning target detection model to detect the chip area in the fusion image