Search

JP-7857111-B2 - Image processing device, defect detection system, image processing method, and image processing program

JP7857111B2JP 7857111 B2JP7857111 B2JP 7857111B2JP-7857111-B2

Inventors

  • 篠田 雅文

Assignees

  • レーザーテック株式会社

Dates

Publication Date
20260512
Application Date
20220210

Claims (8)

  1. An image acquisition unit that acquires an RGB color image of the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the region within the converted image, A first color conversion unit converts pixels within the set range of HSV values in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold. Equipped with, A learning process is performed to generate a defect detection model by training a deep learning model using the converted image, which is created by processing the image including the defect region with the color space conversion unit and the first color conversion unit to convert the defect region having different brightnesses into a specific fixed color, as training data. A detection process is performed by inputting the converted image, which is processed by the color space conversion unit and the first color conversion unit, onto the RGB color image of the sample, into the trained defect detection model to detect defects in the image. An image processing apparatus characterized by performing the following.
  2. The image processing apparatus according to claim 1, further comprising: a second color conversion unit that converts pixels outside the range of the HSV values set for each defective region in the image into second color information such that the color contrast value with the first color information is greater than or equal to an arbitrarily given non-zero threshold.
  3. The image processing apparatus according to claim 1, characterized in that it outputs an RGB color image of the image in which the defect was detected in the detection process.
  4. The aforementioned sample is a wafer or a mask. The image processing apparatus according to claim 1.
  5. An imaging optical system for imaging the sample, An image processing device that processes the image captured by the aforementioned imaging optical system, Equipped with, The aforementioned image processing device is An image acquisition unit that acquires an RGB color image of the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the region within the converted image, A first color conversion unit converts pixels within the set range of HSV values in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold. Equipped with, A learning process is performed to generate a defect detection model by training a deep learning model using the converted image, which is created by processing the image including the defect region with the color space conversion unit and the first color conversion unit to convert the defect region having different brightnesses into a specific fixed color, as training data. A detection process is performed by inputting the processed image obtained from the RGB color image of the sample, processed by the color space conversion unit and the first color conversion unit, into the trained defect detection model to detect defects in the image. A defect detection system characterized by performing the following.
  6. The defect detection system according to claim 5, further comprising a second color conversion unit that converts pixels in the defective region of the image that are outside the range of the set HSV values into second color information whose color contrast value with the first color information is greater than or equal to an arbitrarily given non-zero threshold.
  7. We acquire an RGB color image of the sample, The acquired image is converted from RGB color space to HSV color space. For the regions within the converted image, set the range of each HSV value, The process includes converting pixels within the set range of HSV values in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold, A learning process to generate a defect detection model by training a deep learning model using a converted image obtained by converting the image including the defect region to the HSV color space and the first color information, wherein the converted image has the defect region having different brightness levels assigned to a specific fixed color, and the trained model is obtained by training the converted image as training data. A detection process is performed by inputting the converted image, which is obtained by converting the RGB color image of the sample to the color space and the first color information, into the trained defect detection model to detect defects in the image, An image processing method that performs this operation.
  8. We acquire an RGB color image of the sample, The acquired image is converted from RGB color space to HSV color space. For the regions within the converted image, set the range of each HSV value, The process includes converting pixels within the set range of HSV values in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold, A learning process to generate a defect detection model by training a deep learning model using a converted image obtained by converting the image including the defect region to the HSV color space and the first color information, wherein the converted image has the defect region having different brightness levels assigned to a specific fixed color, and the trained model is obtained by training the converted image as training data. A detection process is performed by inputting the converted image, which is obtained by converting the RGB color image of the sample to the color space and the first color information, into the trained defect detection model to detect defects in the image, An image processing program that instructs a computer to execute commands to perform an action.

Description

This invention relates to an image processing apparatus, a defect detection system, an image processing method, and an image processing program. With the miniaturization of semiconductor process nodes, there is an urgent need for even higher sensitivity in the inspection of wafers, masks, and other components. For example, die-to-die inspection and mask-to-mask inspection are known methods for inspecting foreign objects in masks. In these inspections, techniques are being researched and developed to detect defects using machine learning, such as deep learning, based on images recognized as defects. Japanese Patent Publication No. 2021-124746Japanese Patent Publication No. 2021-149952 This is a block diagram illustrating an image processing device according to Embodiment 1.This is a diagram illustrating the image processing according to Embodiment 1.This diagram illustrates the configuration of the defect detection system according to Embodiment 2.This diagram illustrates the image color conversion process according to Embodiment 2.This is a flowchart illustrating the image processing method according to Embodiment 2.This is a conceptual diagram illustrating the defect detection method according to Embodiment 2.This diagram illustrates the overall operation of the defect detection system according to Embodiment 2.This diagram illustrates the configuration of the defect detection device according to Embodiment 2. The specific configuration of this embodiment will be described below with reference to the drawings. The following description illustrates preferred embodiments of the present invention, and the scope of the present invention is not limited to these embodiments. In the following description, components denoted by the same reference numerals indicate substantially equivalent components. <Embodiment 1> Figure 1 is a block diagram illustrating an image processing apparatus according to Embodiment 1. Figure 2 is a diagram illustrating the image processing according to Embodiment 1. The image processing device 30 may be used to generate training data for machine learning. The image processing device 30 may be implemented by a computer equipped with a GPU (Graphics Processing Unit) and memory, etc. In a specific example, as shown in Figure 1, the image processing device 30 includes an image acquisition unit 31, a color space conversion unit 32, a range setting unit 33, and a first color conversion unit 34. The image acquisition unit 31 acquires an RGB color image of the region containing the defect area of the sample. For example, in a semiconductor wafer inspection process, it acquires one or more color images of semiconductor wafers recognized as defective. The acquired image may be, for example, an image of the entire wafer, or an image of only a portion of the wafer. The color space conversion unit 32 converts the acquired image from RGB color space to HSV color space. While it is possible to determine the hue and brightness within an RGB image, determining the range of colors from light to dark, or from dark to bright, is difficult with RGB images due to the discrete coding. Therefore, the color space conversion unit 32 is used to convert the acquired RGB color image into HSV image format. The range setting unit 33 sets the range for each HSV value in the defective region of the image converted to the HSV color space. For example, in some embodiments, the operator can set the range (upper and lower limits) for each HSV value by operating the control unit of the image processing device 30 (e.g., an input device such as a mouse or keyboard). It is desirable to set the range (upper and lower limits) for all of H, S, and V, rather than just one of them. Alternatively, in other embodiments, a predetermined range (plus/minus threshold) for HSV may be automatically set based on each HSV value (e.g., maximum value, minimum value, average value, etc.) within the defective region of the acquired image. The first color conversion unit 34 converts pixels within the set range of HSV values into first color information such that the color contrast with respect to the background color (for example, any single color if there are multiple background colors) is greater than or equal to a threshold. As used herein, "color contrast" refers to the difference in brightness or sharpness, and represents the difference between the DOI and the HSV values of the background color. Specifically, the color contrast value is: H DOI -H background , S DOI -S background , V DOI -V background This can be defined by: If all HSV values of a pixel in an image fall within the set upper and lower HSV limits, the image is converted to primary color information. Conversely, if any of the HSV values of a pixel in an image fall outside the set upper and lower HSV limits, the image is not converted to primary color information. Therefore, as shown in Figure 2, specific defect areas in the image are converted to specific colors (e.g., primary co