EP-4736109-A1 - COMPUTER-IMPLEMENTED METHOD AND SYSTEM FOR GENERATING A DEFECTIVE IMAGE OF A MACHINE THROUGH A DEFECT MASK

Abstract

According to the innovative method a defective image of a portion of a machine is generated through at least the steps of: receiving (240) an non-defective image of the portion of the machine, receiving (250) a segmentation image mask corresponding to a defect, receiving (260) defect coordinates corresponding to a position of the defect, generating (270) a masked image by blanking an area of the non-defective image, and using a conditional Generative Adversarial Network for generating (280) the defective image; notably, it is the masked image that is input to the conditional Generative Adversarial Network as image input and not the whole non-defective image; notably, both the segmentation image mask and the defect coordinates are input to the conditional Generative Adversarial Network as condition input and not simply a label.

Inventors

• VALVANO, Gabriele
• PANIZZA, Andrea
• GRAZIANO, Antonino

Assignees

• NUOVO PIGNONE TECNOLOGIE - S.R.L.

Dates

Publication Date

20260506

Application Date

20240624

Claims (17)

1. 1. A computer-implemented method (200) for generating a defective image of a portion of a machine, wherein the defective image shows the portion of the machine containing a defect, wherein the method comprises the steps of: c) receiving (240) an non-defective image of the portion of the machine, wherein the non-defective image shows the portion of the machine containing no defect, wherein the non-defective image has a first bidimensional size, d) receiving (250) a segmentation image mask corresponding to a defect, wherein the segmentation image mask defines a bounding area, wherein the bounding area has a second bidimensional size, the second bidimensional size being smaller than the first bidimensional size, e) receiving (260) defect coordinates corresponding to a position of the defect, f) generating (270) a masked image by blanking an image area of the non-defective image, wherein said image area corresponds to said bounding area and has a size equal to the second bidimensional size, wherein said image area is at said position of the defect, and g) using a conditional Generative Adversarial Network for generating (280) the defective image, wherein the masked image is input to the conditional Generative Adversarial Network as image input, wherein the segmentation image mask and the defect coordinates are input to the conditional Generative Adversarial Network as condition input.
2. 2. The method of claim 1, wherein the segmentation image mask is a line-shape mask or an area-shape mask or a pixel-wise mask.
3. 3. The method of claim 1, wherein each pixel of the segmentation image mask has a digital value selected between only a first digital value and a second digital value.
4. 4. The method of claim 1, wherein the segmentation image mask is associated to at least one label, the at least one label providing information regarding the defect.
5. 5. The method of claim 4, wherein the at least one label corresponds to a type of defect, the type of defect being in particular selected from “crack”, “erosion”, “oxidation”, “discoloration”, and “spallation” .
6. 6. The method of claim 4, wherein the at least one label corresponds to a cause of defect.
7. 7. The method of claim 4, wherein the at least one label corresponds to a dimensional indication, in particular defect length and/or defect area and/or bounding area length and/or bounding area width.
8. 8. The method of claim 7, wherein the non-defective image of step c is associated to a dimensional indication, and wherein the image area to be blanked at step f takes into account the dimensional indication of the segmentation image mask and the dimensional indication of the non-defective image.
9. 9. The method of claim 1, wherein the bounding area is a square or a rectangle or a circle or an ellipse.
10. 10. The method of claim 1, wherein the bounding area is determined automatically by a computer program.
11. 11. The method of claim 1, wherein, at step f, the conditional Generative Adversarial Network generates an artificial image having the first bidimensional size, afterwards a partial image having the second bidimensional size and corresponding to said bounding area is derived from the artificial image, and afterwards the partial image is located at said image area of the masked image thus creating the defective image.
12. 12. The method of claim 1, wherein, at step f, the conditional Generative Adversarial Network generates a defect image having the second bidimensional size and corresponding to said bounding area, and afterwards the defect image is located at said image area of the masked image thus creating the defective image.
13. 13. The method of claim 1, wherein the conditional Generative Adversarial Network is prepared through the training steps (220) of al) receiving a first plurality of images of machines similar to the machine and a corresponding first plurality of segmentation image masks, the images of the first plurality being defective images, and training the conditional Generative Adversarial Network through said first pluralities, and a2) receiving a second plurality of images of machines similar to the machine and a second plurality of segmentation image masks, the images of the second plurality being non-defective images, and training the conditional Generative Adversarial Network through said second pluralities.
14. 14. The method of claim 1, wherein the segmentation image mask derives from the step (230) of bl) receiving mask data from a computer program, wherein the computer program is configured to output real defective images through a user interface and to input mask data drawn by a user through the user interface.
15. 15. The method of claim 1, wherein the segmentation image mask derives from the step of: b2) receiving mask data from a computer program, wherein the computer program is configured to process real defective images based on predetermined rules and to generate mask data from the processed real defective images.
16. 16. The method of claim 1, wherein the conditional Generative Adversarial Network of step e) comprises only one generator being neural -network based and only one discriminator being neural -network based, wherein the generator is trained, and wherein the discriminator is trained.
17. 17. A computer-based system configured to carry out the defective image generation method according any of claims from 1 to 16.

Description

TITLE Computer-implemented method and system for generating a defective image of a machine through a defect mask DESCRIPTION TECHNICAL FIELD [0001] The subject matter disclosed herein relates to a computer-implemented method for generating a defective image of a portion of a machine, in particular a turbomachine, through a defect mask. BACKGROUND ART [0002] Artificial Intelligence (usually abbreviated as “Al”) may be used for example in order to identify defects in manufactured machines or parts thereof. A machine or a part containing a defect may have been manufactured since a short time, for example few seconds or minutes; in this case, the defect is typically due to a manufacturing problem. Alternatively, a machine or a part containing a defect may have been manufactured since a long time, for example days, weeks, months or years, and may have been used in operation or stored in warehouse after being manufactured; in this case, the defect is typically due to its use in operation and/or its storage in warehouse. [0003] A “defect” in a machine or part may be defined as a deviation in the value of one of its features that affects the correct operation or performance of the machine or part, i.e. that leads to operation or performance not in accordance with predefined limits; the refence for evaluating the deviation is the “rated value”, i.e. the value corresponding to the design of the machine or part. A “defect” in a machine or part may also be defined as a deviation in the value of one of its features not in accordance with predefined limits. For example, a “crack” in e.g. a part of a machine is a defect as a surface of the part having a crack does not correspond to the surface according to its design, i.e. its ideal surface. [0004] Some kinds of defect may be identified simply by watching e.g. a part of a machine or by watching an image, e.g. a photograph, of the part. In the case of images, visibility of a defect may depend on one or more factors, for example the observation point and the illuminating light. [0005] Other kinds of defects cannot be seen, and may require measurements by non-destructive techniques and/or invasive examinations. [0006] Assuming that a defect in e.g. a part of a machine is visible, a captured image of this part shows the defect in that the captured image is different from an image of the same part that contains no defect. In the following, the expression “defective image” will be used to mean an image that shows, in a more or less evident way, a defect in a machine or a part of a machine, and the expression “non-defective image” will be used to mean an image that does not show a defect. [0007] In general, it is difficult to identify defects based on images even for a so-called “subject matter expert” as differences in images may not be evident and/or may be caused by other reasons, i.e. not defects, and more difficult for a computer-based system. Therefore, nowadays, systems based on Artificial Intelligence are sometimes used for automatically identifying defects in machines or parts starting from images, in particular for quality tests after manufacturing. [0008] Such a system based on Artificial Intelligence usually requires three main phases: 1) data collection, 2) data annotation, and 3) optimization of an Al model. For example, in order to detect cracks in industrial combustor chambers, one must first collect a large amount of photos of industrial combustor chambers containing cracks; then, one must involve one or more experts to annotate these photos, i.e. mark the cracks in the photos; finally, one can use the annotated photos to optimize an Al model in order to perform automated crack detection, in other words, as well known in the field of Al, the system “learns” how to perform a task, before performing the task, directly from data provided to the system during a so-called “training phase”. [0009] Unfortunately, collecting a large annotated dataset is not only timeconsuming but also requires experts. [0010] Therefore, it is desirable to reduce the temporal bottleneck introduced by data collection and annotation as much as possible. [0011] From the article of Rajhans Singh et al. entitled “Generative Adversarial Networks for Synthetic Defect Generation in Assembly and Test Manufacturing" in the 31 st Annual SEMI ASM Conference held in 2020, there are known Artificial Intelligence-based synthetic defect generation techniques to augment the training image sets for CNNs-based defect detection and classification systems to be used in semiconductor manufacturing processes of e.g. “chips” or “ICs”; Generative Adversarial Networks (GANs) are described to create various modes of the defects which are difficult to create manually; they declare that the output of their adapted GANs are images of realistic- looking defects for a wide variety of common manufacturing defects including foreign material, misplaced epoxy, scratches, and die chipping defects among others. According