EP-4737889-A1 - DEFECT INSPECTION DEVICE, DEFECT INSPECTION METHOD, AND DEFECT INSPECTION PROGRAM

EP4737889A1EP 4737889 A1EP4737889 A1EP 4737889A1EP-4737889-A1

Abstract

A defect inspection device includes an image recognition processing unit configured to determine attributes of respective regions within an inspection target by performing image recognition processing on captured image data obtained by photographing the inspection target, an extracting unit configured to extract, from the captured image data, partial image data including a region having a specific attribute among the respective regions within the inspection target, a predicting unit configured to predict, by inputting the partial image data into a trained defect detection model, a probability that the partial image data includes a defect, and a display control unit configured to superimpose and display, at a position from which the partial image data has been extracted in the captured image data, an indicator having a display mode corresponding to the predicted probability.

Inventors

OKUNO, Katsuki
NOMA, HIROKAZU
HIRAHARA, Mana
TAKEMOTO, Shimpei

Assignees

Resonac Corporation

Dates

Publication Date: 20260506
Application Date: 20240625

Claims (13)

A defect inspection device comprising: an image recognition processing unit configured to determine attributes of respective regions within an inspection target by performing image recognition processing on captured image data obtained by photographing the inspection target; an extracting unit configured to extract, from the captured image data, partial image data including a region having a specific attribute among the respective regions within the inspection target; a predicting unit configured to predict, by inputting the partial image data into a trained defect detection model, a probability that the partial image data includes a defect; and a display control unit configured to superimpose and display, at a position from which the partial image data has been extracted in the captured image data, an indicator having a display mode corresponding to the predicted probability.
The defect inspection device according to claim 1, wherein the region having the specific attribute is a region having an attribute in which occurrence of a defect is predicted.
The defect inspection device according to claim 1 or 2, wherein: the image recognition processing unit is configured to determine a region having a first attribute in which occurrence of a defect is predicted in the inspection target and regions having a second attribute in which occurrence of a defect is not predicted in the inspection target; and the extracting unit is configured to calculate a position of extraction from which the partial image data including the region having the first attribute is extracted, based on an arrangement of the regions having the second attribute.
The defect inspection device according to claim 3, wherein: the regions having the second attribute are arranged in a lattice in the inspection target; and the extracting unit is configured to: generate the lattice based on center points of the regions having the second attribute; calculate a position of a vertical-direction midpoint, a position of a horizontal-direction midpoint, or a position of a diagonal direction midpoint between lattice points, as the position of extraction; and extract, as the partial image data, rectangular regions centered on the calculated position of extraction within the captured image data.
The defect inspection device according to claim 4, wherein the extracting unit is configured to extract square regions of uniform size centered on respective positions of extraction as the partial image data.
The defect inspection device according to claim 4 or 5, wherein: the inspection target is a semiconductor package substrate; the region having the first attribute is a solder resist region; and the regions having the second attribute are solder joint regions.
The defect inspection device according to any one of claims 3 to 6, wherein the image recognition processing unit is configured to determine, for each pixel, the first attribute and the second attribute using a semantic segmentation model.
The defect inspection device according to any one of claims 1 to 7, wherein the display control unit is configured to superimpose and display, at a position where the partial image data having the predicted probability equal to or greater than a predetermined threshold has been extracted, a rectangular frame having a size corresponding to the extracted partial image data and indicating a defect.
The defect inspection device according to any one of claims 1 to 8, wherein the display control unit is configured to superimpose and display, at a position where the partial image data has been extracted, the predicted probability and a rectangular frame having a size corresponding to the extracted partial image data.
The defect inspection device according to any one of claims 1 to 9, further comprising: an aggregation unit configured to determine presence or absence of a defect based on the predicted probability with respect to each said partial image data and to aggregate determination results; and a calculation unit configured to perform statistical processing based on the determination results to calculate a statistical value, wherein the display control unit is configured to superimpose and display, on the captured image data, a result of aggregation and the calculated statistical value.
The defect inspection device according to any one of claims 1 to 10, wherein the predicting unit is configured to predict the probability that the partial image data includes a defect, by inputting the partial image data into the trained defect detection model trained based on training data including partial image data and information indicating types of defects.
A defect inspection method performed by a computer, comprising: determining attributes of respective regions within an inspection target by performing image recognition processing on captured image data obtained by photographing the inspection target; extracting, from the captured image data, partial image data including a region having a specific attribute among the respective regions within the inspection target; predicting, by inputting the partial image data into a trained defect detection model, a probability that the partial image data includes a defect; and superimposing and displaying, at a position from which the partial image data has been extracted in the captured image data, an indicator having a display mode corresponding to the predicted probability.
A defect inspection program configured to cause a computer to perform a method, the method comprising: determining attributes of respective regions within an inspection target by performing image recognition processing on captured image data obtained by photographing the inspection target; extracting, from the captured image data, partial image data including a region having a specific attribute among the respective regions within the inspection target; predicting, by inputting the partial image data into a trained defect detection model, a probability that the partial image data includes a defect; and superimposing and displaying, at a position from which the partial image data has been extracted in the captured image data, an indicator having a display mode corresponding to the predicted probability.

Description

TECHNICAL FIELD The disclosures herein relate to defect inspection devices, defect inspection methods, and defect inspection programs. BACKGROUND ART There has been known a defect inspection device which performs object detection processing on captured image data obtained by capturing an image of an inspection target to detect defects included in the inspection target. According to the defect inspection device, for example, a more minute defect can be detected by increasing resolution of the captured image data. CITATION LIST PATENT LITERATURE Patent Literature 1: Japanese Patent No. 7270314 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION Conversely, when the amount of data per sheet of captured image data is increased by increasing the resolution, a processing load of the defect inspection device increases. Therefore, from the perspective of reducing the processing load, it is possible to divide and process the captured image data. Further, in the case where an inspection target is a structure formed by a plurality of types of members, it may be considered that a minute defect can be detected with high accuracy if object detection processing is performed separately for each type of member, for example, rather than collectively for the entire captured image data. The present disclosure aims to reduce a processing load when detecting a defect and to improve detection accuracy. MEANS FOR SOLVING PROBLEMS A defect inspection device according to aspect 1 of the present disclosure includes: an image recognition processing unit configured to determine attributes of respective regions within an inspection target by performing image recognition processing on captured image data obtained by photographing the inspection target;an extracting unit configured to extract, from the captured image data, partial image data including a region having a specific attribute among the respective regions within the inspection target;a predicting unit configured to predict, by inputting the partial image data into a trained defect detection model, a probability that the partial image data includes a defect; anda display control unit configured to superimpose and display, at a position from which the partial image data has been extracted in the captured image data, an indicator having a display mode corresponding to the predicted probability. Aspect 2 of the present disclosure is the defect inspection device according to aspect 1, wherein the region having the specific attribute is a region having an attribute in which occurrence of a defect is predicted. Aspect 3 of the present disclosure is the defect inspection device according to aspect 1 or 2, wherein: the image recognition processing unit is configured to determine a region having a first attribute in which occurrence of a defect is predicted in the inspection target and regions having a second attribute in which occurrence of a defect is not predicted in the inspection target; andthe extracting unit is configured to calculate a position of extraction from which the partial image data including the region having the first attribute is extracted, based on an arrangement of the regions having the second attribute. Aspect 4 of the present disclosure is the defect inspection device according to aspect 3, wherein: the regions having the second attribute are arranged in a lattice in the inspection target; andthe extracting unit is configured to: generate the lattice based on center points of the regions having the second attribute;calculate a position of a vertical-direction midpoint, a position of a horizontal-direction midpoint, or a position of a diagonal direction midpoint between lattice points, as the position of extraction; andextract, as the partial image data, rectangular regions centered on the calculated position of extraction within the captured image data. Aspect 5 of the present disclosure is the defect inspection device according to aspect 4, wherein the extracting unit is configured to extract square regions of uniform size centered on respective positions of extraction as the partial image data. Aspect 6 of the present disclosure is the defect inspection device according to aspect 4 or 5, wherein: the inspection target is a semiconductor package substrate;the region having the first attribute is a solder resist region; andthe regions having the second attribute are solder joint regions. Aspect 7 of the present disclosure is the defect inspection device according to any one of aspects 3 to 6, wherein the image recognition processing unit is configured to determine, for each pixel, the first attribute and the second attribute using a semantic segmentation model. Aspect 8 of the present disclosure is the defect inspection device according to any one of aspects 1 to 7, wherein the display control unit is configured to superimpose and display, at a position where the partial image data having the predicted probability equal to or greater than a predetermi