CN-122023327-A - Chip defect detection system and method based on depth of field fusion and deep learning

CN122023327ACN 122023327 ACN122023327 ACN 122023327ACN-122023327-A

Abstract

The invention relates to a chip defect detection system and method based on depth of field fusion and deep learning, wherein the system comprises a control host, an image acquisition unit and a motion control unit, wherein the image acquisition unit adopts a liquid camera, and rapidly zooms through a voltage control liquid lens to acquire a plurality of high-definition images with different focal lengths under the same chip visual field, the control host synthesizes the images into a panoramic deep clear image by using a depth of field fusion algorithm, expands a sample set by using a data enhancement technology, and finally uses a trained YOLO v5s model to realize rapid and precise identification and positioning of chip defects.

Inventors

WEI CHANGZHOU
TANG XIA
SHI XUBO
HU ZIHANG
FENG YUYAN

Assignees

无锡职业技术大学

Dates

Publication Date: 20260512
Application Date: 20260129

Claims (7)

1. The chip defect detection system based on depth of field fusion and deep learning is characterized by comprising a control host, an image acquisition unit and a motion control unit; The control host executes system control and data processing; The image acquisition unit is connected with the control host, and performs image data acquisition by adopting a liquid camera, and the liquid camera receives the control host instruction to acquire high-resolution images with different focal lengths of the same chip to be detected through voltage control of the liquid lens; the motion control unit is connected with the control host and the image acquisition unit and is used for driving the image acquisition unit to perform three-dimensional space positioning corresponding to the chip to be detected.
2. The depth of field fusion and deep learning based chip defect detection system according to claim 1, wherein the control host comprises an image processing module, a data enhancement module and a deep learning detection module; The image processing module receives images with different focal lengths of the same chip to be detected, which are acquired by the image acquisition unit, and processes the images by adopting a depth of field fusion method to generate a panoramic deep image containing clear information of all focusing planes; The data enhancement module performs image rotation, image translation, image scaling and image overturning on the panoramic deep image to expand a data set; And the deep learning detection module loads and runs a YOLO v5s neural network model, trains the image processed by the data enhancement module, and identifies and locates the chip defects of the panoramic deep image.
3. The depth of field fusion and deep learning based chip defect detection system according to claim 2, wherein the depth of field fusion method comprises the steps of: Performing definition evaluation on images with different focal lengths of the same chip to be tested to obtain definition scores of pixels or areas in the images; Selecting the pixel or area with the highest score by comparing the definition scores of the multiple images at the same position; And synthesizing all the selected highest definition pixels or areas to obtain a panoramic deep image and outputting the panoramic deep image.
4. The depth of field fusion and deep learning based chip defect detection system of claim 2, wherein the YOLO v5s neural network model comprises an input layer, a backbone network layer, a neck layer and a head layer; The input layer carries out preprocessing, mosaic data enhancement, self-adaptive anchor frame calculation and self-adaptive image scaling on an input image, and the size of the input image is 640 multiplied by 640 pixels; The main network layer comprises a Focus structure and a CSPDARKNET structure, convolves and slices the input image, and extracts characteristics; The neck layer carries out multi-scale fusion on the images output by the trunk network layer; The header layer outputs an image classification result.
5. The system for detecting the chip defects based on depth of field fusion and deep learning according to claim 1, wherein the motion control unit comprises an XYZ three-axis motion platform and a motion control card, the XYZ three-axis motion platform carries and drives the liquid lens to position the chip to be detected, the motion control card is connected with the XYZ three-axis motion platform and a control host, and the motion control card receives instructions output by the control host and drives the XYZ three-axis motion platform to precisely move.
6. A depth of field fusion and deep learning based chip defect detection method, to which the detection system according to any one of claims 1 to 5 is applied, comprising the steps of: s1, adjusting the relative positions of an image acquisition unit and a chip to be tested through the motion control unit, and carrying out initial focusing and positioning; S2, controlling the liquid camera to acquire a plurality of high-resolution images of the chip to be detected in the same view field and with different focal lengths; s3, performing depth of field fusion processing on the high-resolution image to generate a panoramic deep image; s301, respectively calculating the definition value of each local area or pixel in each input image by adopting a definition evaluation function; S302, constructing a blank output image matrix with the same size as an input image, traversing definition values of the same area or pixels of all the input images, and selecting a pixel value corresponding to the highest definition value as a value of a pixel point corresponding to the output image; S303, traversing all pixel positions to complete image synthesis, and generating and outputting a panoramic deep image; s4, carrying out data enhancement processing on the panoramic deep image to obtain an extended training data set; s5, training the training data set by using a YOLO v5S neural network model to obtain a defect detection model; s6, inputting the panoramic deep image generated by the chip to be tested after being processed in the step S2 and the step S3 into a trained defect detection model, and outputting the type and position information of the chip defect.
7. The method for detecting chip defects based on depth fusion and depth learning according to claim 6, wherein the panoramic deep image is data enhanced by image rotation, image translation, image scaling and image flipping.

Description

Chip defect detection system and method based on depth of field fusion and deep learning Technical Field The invention relates to the technical field of industrial detection, in particular to a chip defect detection system and method based on depth of field fusion and deep learning. Background The chip is used as a core component of modern electronic equipment, has complex manufacturing process and fine structure, is extremely easy to generate various appearance defects such as pin deficiency, poor solder balls, broken or wound gold wires, surface scratch, unclear marks and the like due to process fluctuation, material defects or environmental interference in the production process, and directly influences the electrical performance, reliability and service life of the chip, so that key links of product yield and quality are ensured in the chip manufacturing process during defect detection. The traditional chip defect detection mainly depends on manual visual inspection, an operator observes the appearance of the chip through a microscope or a magnifying glass and judges whether the defect exists or not through experience, the method is high in subjectivity, detection results are easily influenced by factors such as personnel experience, fatigue degree and attention, the consistency is poor, manual efficiency is low, the large-scale and high-speed production rhythm is difficult to deal with, and as the chip size is continuously reduced and the integration level is continuously improved, the manual detection cannot meet the harsh requirements of the industry on detection precision, speed and stability. In order to replace manual work, automatic optical detection technology based on machine vision is rapidly developed, traditional image processing and pattern recognition algorithms are adopted, for example, defect discrimination is carried out by combining edge detection, template matching and feature extraction (such as SIFT and HOG features) with a classifier, the methods improve the automation level to a certain extent, but the performance of the method is seriously dependent on the quality of pre-designed manual features and strict control on imaging conditions such as illumination, contrast, background and the like, and the traditional method has limited feature expression capacity, so that the generalization capability of detection is weak, the false detection rate and the omission factor are higher, and the algorithm debugging is complex and has poor adaptability to different product lines; In recent years, the deep learning technology, especially the target detection algorithm (such as fast R-CNN, YOLO, SSD and the like) based on a convolutional neural network, can automatically learn the characteristic with more discriminant force from a large amount of data, however, the chip surface is not an ideal two-dimensional plane, the depth of field of a common optical lens is limited when imaging is carried out under a single focal length, and it is difficult to enable characteristic areas with different heights to be focused clearly at the same time in the same image, the photographed image tends to have clear partial areas and fuzzy partial areas, so that key detail information is seriously lost, the extraction and recognition precision of the defect characteristics by a subsequent deep learning model are directly influenced, in addition, in the industrial actual production, especially for a new product or a high-yield production line, the number of samples with defect labels which can be collected is very rare (small samples), the deep learning method usually needs to train massive labeling data to achieve the ideal performance, and the accuracy of the model is extremely easy to be fit too much under a small sample scene, so that the accuracy of the model in practical application is reduced. Disclosure of Invention In view of the above, the invention provides a system and a method for detecting chip defects based on depth of field fusion and depth learning, which solve the problem of low chip detection precision caused by insufficient depth of field and small sample size in the prior art, and realize high-resolution full-resolution imaging, high-precision identification under small samples and high-speed real-time detection of chips. In order to achieve the above purpose, the invention provides a chip defect detection system based on depth of field fusion and deep learning, which comprises a control host, an image acquisition unit and a motion control unit; The control host executes system control and data processing; The image acquisition unit is connected with the control host, and performs image data acquisition by adopting a liquid camera, and the liquid camera receives the control host instruction to acquire high-resolution images with different focal lengths of the same chip to be detected through voltage control of the liquid lens; the motion control unit is connected with the control host and the