CN-115147517-B - Method for generating flow chart of defect detection algorithm, defect detection method and system

Abstract

The embodiment of the invention provides a method and a system for generating a flow chart of a defect detection algorithm, an electronic device and a storage medium. The method for generating the flow chart of the defect detection algorithm comprises a naming step, a generating step and a displaying step, wherein the naming step is used for naming at least one folder based on a preset naming rule, each folder is used for storing training sample images of models corresponding to the folders, each model is used for detecting target information in images to be identified, the name of each folder comprises model category identification and/or target identification of the corresponding model, the generating step is used for responding to confirmation operation of a user, and the flow chart of the defect detection algorithm is generated and displayed based on the model category identification and/or the target identification in the name of each folder, and the flow chart shows operation paths of steps of the defect detection algorithm. The technical scheme is simple and easy to implement, and has lower requirements on users, so that the scheme has strong applicability and better user experience.

Inventors

• DAI JIE
• KUAI DUOJIE
• SUN XIN

Assignees

• 苏州镁伽科技有限公司
• 苏州镁伽科技有限公司

Dates

Publication Date

20260421

Application Date

20220630

Priority Date

20220630

Claims (17)

1. 1. A method of generating a flowchart of a defect detection algorithm, the method comprising: Naming at least one folder based on a preset naming rule, wherein each folder is used for storing training sample images of corresponding models, different folders correspond to different models, each model is used for detecting target information in the images to be identified, the name of each folder comprises model category identifiers and/or target identifiers of the corresponding models, each folder comprises a first type folder and a second type folder, each first type folder is used for storing training sample images of the corresponding first models, each second type folder is used for storing training sample images of the corresponding second models, each first model is used for identifying defect areas of at least one defect in the images to be identified, each second model is used for identifying target areas configured for at least one specific defect in the images to be identified, and each target identifier comprises a type identifier of the defect identified by the corresponding first model or an identifier of the target area of the specific defect identified by the corresponding second model; A generation step of generating and displaying a flow chart of a defect detection algorithm based on the model category identification and/or the target identification in the name of each folder in response to a confirmation operation of a user, wherein the flow chart shows a running path of steps of the defect detection algorithm.
2. 2. The method of generating a flowchart of a defect detection algorithm according to claim 1, wherein, The generating step comprises the following steps: In response to a user's validation operation, generating and displaying a flowchart of a defect detection algorithm based on model category identifications in names of the first category folders and/or the target identifications, the steps of the defect detection algorithm including defect detection steps performed using the first model.
3. 3. The method of generating a flowchart of a defect detection algorithm according to claim 2, wherein, The generating step includes: And generating and displaying the flow chart based on model category identifiers and/or the target identifiers in respective names of the first-type folder and the second-type folder in response to the confirmation operation, wherein the step of the defect detection algorithm further comprises a target area detection step and a post-processing step which are executed by using the second model, and in the post-processing step, a defect detection result of the image to be identified is determined based on the defect area detected in the defect detection step, the target area detected in the target area detection step and the configuration relation between the defect area and the target area.
4. 4. A method of generating a flow chart of a defect detection algorithm according to claim 3, wherein the names of the second class folder or the first class folder further comprise configuration relation identifications of the configuration relations; the generating step comprises the following steps: reading the names of the first-type folders and the names of the second-type folders in response to the confirmation operation; acquiring the configuration relation between the target area of each specific defect and the defect area of the defect based on the configuration relation identification in the names of the first type of folders or the second type of folders; And generating and displaying a flow chart of the defect detection algorithm based on the names of the first class folders, the names of the second class folders and the configuration relation.
5. 5. A method of generating a flowchart of a defect detection algorithm according to claim 3, wherein prior to the generating step, the method further comprises: Acquiring a configuration relation between a target area of each specific defect preset by a user and a defect area of the defect; the generating step comprises the following steps: and generating and displaying a flow chart of the defect detection algorithm based on the names of the first type folders, the names of the second type folders and the configuration relation in response to the confirmation operation.
6. 6. A method of generating a flow chart of a defect detection algorithm according to claim 3, wherein the number of the first model and/or the second model is plural.
7. 7. A method of generating a flowchart of a defect detection algorithm according to claim 3, wherein the first model comprises a target detection model and a semantic segmentation model, and/or The classes of the second model include an object detection model and a semantic segmentation model, The object detection model is used for identifying a defect area or an object area with a first morphological feature in the image to be identified, and the semantic segmentation model is used for identifying a defect area or an object area with a second morphological feature in the image to be identified.
8. 8. The method of generating a flowchart of a defect detection algorithm according to any one of claims 1 to 7, wherein the folders include third-class folders each for storing training sample images of a third model corresponding thereto, each third model for detecting whether the image to be identified is an abnormal condition, wherein the target information includes abnormal condition information; the generating step comprises the following steps: In response to the validation operation, the flowchart is generated and displayed based at least on model category identifiers and/or target identifiers in names of the third class folder, wherein the steps of the defect detection algorithm include an anomaly detection step performed using the third model.
9. 9. A method of generating a flowchart of a defect detection algorithm according to claim 8 when dependent on claim 2, wherein the generating step comprises: Generating and displaying the flow chart based on at least model category identifiers and/or target identifiers in respective names of the first class folder and the third class folder in response to the confirmation operation, wherein the step of the defect detection algorithm further includes a post-processing step in which a defect detection result of the image to be identified is determined based on a defect region detected by the defect detection step and a result of whether the image determined by the abnormality detection step is an abnormal condition; and determining that the defect detection result of the image to be identified is that a new type of defect area exists on the image to be identified under the condition that the image to be identified is determined to be in an abnormal condition by the abnormality detection step and that the defect area does not exist in the image to be identified by the defect detection step.
10. 10. The method of generating a flowchart of a defect detection algorithm according to claim 8, wherein the third model comprises an anomaly detection model and/or a classification model, wherein the training sample image of the anomaly detection model comprises only normal images, and wherein the training sample image of the classification model comprises annotated normal images and annotated anomaly images.
11. 11. The method for generating a flowchart of the defect detection algorithm according to any one of claims 1 to 7, wherein the number of folders is plural, the folders including a parent folder and child folders in each of the parent folders, wherein, Model categories of models corresponding to child folders under each parent folder are the same; the models corresponding to the child folders under different parent folders are different in model category, Wherein the name of each parent folder includes a model category identifier and the name of the child folder includes a target identifier.
12. 12. The method of generating a flowchart of a defect detection algorithm according to any one of claims 1 to 7, wherein prior to the naming step, the method further comprises: creating the folder in response to a user's creation operation, and/or And deleting the folder in response to a deleting operation of the user.
13. 13. A method of defect detection, the method comprising: Storing training sample images of models corresponding to the folders named based on preset naming rules, wherein different folders correspond to different models, each model is used for detecting target information in an image to be identified, the name of each folder comprises model category identifiers and/or target identifiers of the corresponding model, each folder comprises a first type folder and a second type folder, each first type folder is used for storing training sample images of the first model corresponding to the first type folder, each second type folder is used for storing training sample images of the second model corresponding to the second type folder, each first model is used for identifying defect areas of at least one defect in the image to be identified, each second model is used for identifying target areas configured for at least one specific defect in the image to be identified, and the target identifiers comprise types of defects identified by the corresponding first model or identifiers of target areas of specific defects identified by the corresponding second model; Automatically training the model based on the training sample image and model category identifiers and/or target identifiers in the names of each folder in response to a storage confirmation operation of a user to obtain a trained model; Using the trained model, performing a defect detection algorithm based on a travel path shown by a flowchart generated by the flowchart generated method of the flowchart of the defect detection algorithm as claimed in any one of claims 1 to 12.
14. 14. A system for generating a flow chart of a defect detection algorithm, comprising: The system comprises a naming module, a naming module and a processing module, wherein the naming module is used for naming at least one folder based on a preset naming rule, each folder is used for storing training sample images of corresponding models, different folders correspond to different models, each model is used for detecting target information in an image to be identified, the name of each folder comprises model category identifiers and/or target identifiers of the corresponding models, the folder comprises a first type folder and a second type folder, each first type folder is used for storing training sample images of corresponding first models, each second type folder is used for storing training sample images of corresponding second models, each first model is used for identifying defect areas of at least one defect in the image to be identified, and each second model is used for identifying target areas configured for at least one specific defect in the image to be identified, and the target identifiers comprise types of defects identified by the corresponding first models or identifiers of target areas of specific defects identified by the corresponding second models; And the generation module is used for responding to the confirmation operation of the user, generating and displaying a flow chart of a defect detection algorithm based on the model category identification and/or the target identification in the name of each folder, wherein the flow chart shows the running path of the steps of the defect detection algorithm.
15. 15. A defect detection system, comprising: The system comprises a storage module, a storage module and a target identification module, wherein the storage module is used for storing training sample images of models corresponding to the files named based on preset naming rules, the different folders correspond to different models, each model is used for detecting target information in an image to be identified, the name of each folder comprises model category identification and/or target identification of the corresponding model, the folders comprise a first type folder and a second type folder, each first type folder is used for storing training sample images of the first model corresponding to the first type folder, each second type folder is used for storing training sample images of the second model corresponding to the first type folder, each first model is used for identifying defect areas of at least one defect in the image to be identified, and each second model is used for identifying target areas configured for at least one specific defect in the image to be identified, and the target identification comprises identification of the type of the defect identified by the corresponding first model or identification of the target area of the specific defect identified by the corresponding second model; The training module is used for responding to the storage confirmation operation of a user, and training the model automatically based on the training sample image and the model category identification and/or the target identification in the name of each folder so as to acquire a trained model; detection module for executing a defect detection algorithm based on a running path shown in a flowchart generated by a method for generating a flowchart of the defect detection algorithm according to any one of claims 1 to 12, using the trained model.
16. 16. An electronic device comprising a display for displaying a user interface, a processor and a memory, wherein the memory has stored therein computer program instructions which, when executed by the processor, are adapted to carry out the method of generating a flow chart of a defect detection algorithm according to any one of claims 1 to 12 and/or the method of defect detection according to claim 13.
17. 17. A storage medium having stored thereon program instructions for executing, when executed, the method of generating a flowchart of a defect detection algorithm according to any one of claims 1 to 12 and/or the method of defect detection according to claim 13.

Description

Method for generating flow chart of defect detection algorithm, defect detection method and system Technical Field The present invention relates to the field of automatic detection, and in particular, to a method and system for generating a flowchart of a defect detection algorithm, an electronic device, and a storage medium. Background In recent years, computer vision has been widely used in the field of automatic detection. For example, computer vision techniques are applied to detect defects in critical components of some electronic devices to facilitate timely repair. However, in practical applications, since defects of some targets may have many different forms, a general detection algorithm cannot be used to identify all defects. Taking defect inspection of wafers as an example. Because of the components with different types or numbers on the wafers with different types, the imaged wafers are complex and changeable, and defects of different wafers cannot be identified by adopting a general defect detection algorithm flow. Therefore, different defect detection algorithm flows need to be built according to different wafers. The prior art generally adopts operations of module dragging, wire connection and the like performed by a user to build a defect detection algorithm flow. This requires the user to drag different modules into the process sequentially to combine into a final algorithm process. However, this method requires the user to understand a certain operation rule and have a certain quality and expertise, so that the user can perform operations such as correct dragging and connecting to generate a desired flow, which has a limitation, and on the other hand, requires the user to perform more complicated operations, which results in poor user experience. Disclosure of Invention The present invention has been made in view of the above-described problems. According to one aspect of the present invention, a method of generating a flowchart of a defect detection algorithm is provided. The method comprises a naming step of naming at least one folder based on a preset naming rule, wherein each folder is used for storing training sample images of models corresponding to the folders, each model is used for detecting target information in images to be identified, the name of each folder comprises model category identifiers and/or target identifiers of the corresponding models, a generating step of responding to confirmation operation of a user and generating and displaying a flow chart of a defect detection algorithm based on the model category identifiers and/or the target identifiers in the names of each folder, and the flow chart shows operation paths of steps of the defect detection algorithm. The folders comprise first-type folders, wherein each first-type folder is used for storing training sample images of first models corresponding to the first-type folders, each first model is used for identifying defect areas of at least one defect in an image to be identified, target information comprises information of the defect areas, target identification comprises type identification of the defect identified by the corresponding first model, the generating step comprises the steps of responding to a confirmation operation of a user, generating and displaying a flow chart of a defect detection algorithm based on model type identification and/or target identification in names of the first-type folders, and the step of the defect detection algorithm comprises a defect detection step executed by using the first model. The method further comprises the step of generating and displaying a flow chart based on model category identifiers and/or object identifiers in respective names of the first and second types of folders in response to a confirmation operation, wherein the step of defect detection algorithm further comprises a target area detection step and a post-processing step performed by using the second models, and in the post-processing step, a defect detection result of the image to be identified is determined based on the defect area detected by the defect detection step, the target area detected by the target area detection step and the configuration relation between the defect area and the target area. Illustratively, the names of the second class folder or the first class folder further include a configuration relationship identifier of the configuration relationship; the generating step comprises the steps of responding to the confirming operation, reading the names of the first type of folders and the names of the second type of folders, acquiring the configuration relation between the target area of each specific defect and the defect area of the defect based on the configuration relation identification in the names of the first type of folders or the second type of folders, and generating and displaying a flow chart of a defect detection algorithm based on the names of the first type of folders, the