CN-121982711-A - Feature labeling method, device, medium and product based on vision

Abstract

A feature labeling method, device, medium and product based on vision relates to the field of image processing. The method comprises the steps of calculating the confidence coefficient of target features of an image to be detected through a feature recognition model, determining the image to be detected as an image to be calibrated if the confidence coefficient is smaller than a preset confidence coefficient threshold value, extracting a first feature sequence of a first preset number from a historical labeling database, obtaining a labeling instruction of a quality inspector for the image to be calibrated, labeling the image to be detected as a defect type corresponding to a second feature sequence if the labeling instruction indicates that the target features are newly added features of the second feature sequence, adding the image to be detected to the tail end of the second feature sequence to obtain a third feature sequence, and performing incremental training on the feature recognition model to obtain an updated feature recognition model which is used for image recognition and labeling. By implementing the technical scheme provided by the application, the robustness of the feature labeling can be improved.

Inventors

• XIA YULONG
• XIA JING

Assignees

• 苏州辰瓴信息技术有限公司

Dates

Publication Date

20260505

Application Date

20251224

Claims (10)

1. 1. A vision-based feature labeling method applied to an electrical control system, wherein the electrical control system at least comprises a feature recognition model and a history labeling database, the method comprising: Calculating the confidence coefficient of the target feature of the image to be detected through the feature recognition model; If the confidence coefficient is smaller than a preset confidence coefficient threshold value, determining that the image to be detected is an image to be calibrated; Extracting a first preset number of first feature sequences from the history labeling database, wherein the history labeling database comprises a plurality of feature sample sets, each feature sample set corresponds to a different defect type, and the first feature sequences are used for recording an evolution stage of the defect type corresponding to the feature sample set; Acquiring a labeling instruction of a quality inspector on the image to be calibrated, wherein the labeling instruction is used for indicating whether the target feature is a newly added feature of a second feature sequence, and the second feature sequence is any one of the first feature sequences; If the labeling instruction indicates that the target feature is a new feature of the second feature sequence, labeling the image to be detected as a defect type corresponding to the second feature sequence, and adding the image to be detected to the tail end of the second feature sequence to obtain a third feature sequence; And performing incremental training on the feature recognition model based on the third feature sequence to obtain an updated feature recognition model, wherein the updated feature recognition model is used for recognition and labeling of the image.
2. 2. The method of claim 1, wherein each set of feature samples comprises a plurality of historical image samples, the computing confidence of a target feature of an image to be detected by the feature recognition model comprising: Extracting a target feature vector of the target feature of the image to be detected; Extracting feature vectors of all the historical image samples to be used as standard feature vectors corresponding to the historical image samples; Calculating the vector similarity of the target feature vector and each standard feature vector to obtain a similarity set corresponding to each feature sample set; and taking the highest vector similarity in each similarity set as the confidence.
3. 3. The method of claim 2, wherein extracting a first predetermined number of first feature sequences in the history annotation database comprises: taking the highest vector similarity in the similarity set as the standard similarity corresponding to each defect type; ranking the standard similarity, and selecting the historical image samples corresponding to the first preset number of standard similarity according to the sequence from high to low to obtain the first preset number of standard image samples; and extracting a second preset number of historical image samples taking each standard image sample as a center to obtain a first characteristic sequence with the first preset number.
4. 4. A method according to claim 3, wherein the incremental training of the feature recognition model based on the third feature sequence to obtain an updated feature recognition model comprises: Taking the image to be detected as an anchor point sample, and taking all historical image samples except the image to be detected in the third characteristic sequence as positive samples of the second preset number; selecting other characteristic sample sets different from the defect types corresponding to the positive samples from the history labeling database; Randomly extracting the second preset number of historical image samples from the other characteristic sample sets to serve as the second preset number of negative samples; inputting the anchor point sample, each positive sample and each negative sample into the feature recognition model; Extracting a first feature vector of the anchor point sample, a plurality of second feature vectors corresponding to each positive sample and a plurality of third feature vectors corresponding to each negative sample, and constructing a training ternary set, wherein the training ternary set comprises a plurality of training ternary, and each training ternary comprises one second feature vector, one third feature vector and one first feature vector; and based on an optimization algorithm, minimizing the vector distance between the anchor point sample and each positive sample in each training ternary, and maximizing the vector distance between the anchor point sample and each negative sample until the feature recognition model reaches a preset training round, so as to obtain the updated feature recognition model.
5. 5. The method of any one of claims 1-4, wherein the electrical control system further comprises a human-machine interaction interface for responding to operation of the quality inspector, the method further comprising, prior to the obtaining of labeling instructions of the quality inspector for the image to be calibrated: Generating a visual interface based on the image to be detected and each first feature sequence, wherein the visual interface is used for displaying target feature contrast information of the image to be detected and each first feature sequence; And displaying the visual interface on the man-machine interaction interface, wherein the man-machine interaction interface further comprises an instruction generating component responding to the labeling instruction.
6. 6. The method of any one of claims 1-4, wherein after the acquiring labeling instructions of the quality inspector for the image to be calibrated, the method further comprises: If the labeling instruction indicates that the target feature is not the newly added feature of any second feature sequence, extracting a feature recognition result of the labeling instruction; Based on the feature recognition result, carrying out feature labeling on the image to be detected to obtain a labeled image to be detected; and performing incremental training on the feature recognition model based on the marked image to be detected and the feature recognition result to obtain an updated feature recognition model.
7. 7. The method according to any one of claims 1-4, further comprising: And if the confidence coefficient is larger than or equal to the preset confidence threshold value, marking the features of the image to be detected based on the feature recognition result of the feature recognition model on the feature of the image to be detected.
8. 8. A vision-based labeling apparatus comprising one or more processors and memory coupled with the one or more processors, the memory to store computer program code comprising computer instructions that the one or more processors invoke to cause the vision-based labeling apparatus to perform the method of any of claims 1-7.
9. 9. A computer-readable storage medium comprising instructions that, when run on a vision-based labeling device, cause the vision-based labeling device to perform the method of any of claims 1-7.
10. 10. A computer program product, characterized in that the computer program product, when run on a vision-based labeling device, causes the vision-based labeling device to perform the method of any of claims 1-7.

Description

Feature labeling method, device, medium and product based on vision Technical Field The application relates to the field of image processing, in particular to a vision-based feature labeling method, equipment, medium and product. Background Electrical control systems are increasingly integrating high precision machine vision systems in precision manufacturing for automated surface defect detection, particularly in the production lines of high-reflective curved surface parts such as mobile phone metal midframes, high-end automotive panels, and the like. After stamping, CNC processing or polishing, the shape and visual characteristics of the defects such as fine scratches, pits or orange peel on the surface of the parts are extremely unstable, and the parts are seriously dependent on the on-site illumination angle and the fine material difference among parts. The traditional visual detection method which relies on a static off-line model cannot meet the production requirement of high yield when facing the defect characteristics of dynamic change. In order to solve the limitations, the existing active learning-based detection method introduces manual labeling and fine-tuning the model when the model encounters low-confidence samples, but has inherent limitations when processing the dynamic defects, each labeled sample is generally regarded as an independent individual, the model lacks cognition of the relevance between the defect history morphology and the current morphology, and the knowledge updating process has a memory coverage effect. The isolated learning mode enables the model to form excessive sensitivity aiming at the specific form after learning the characteristics of the defect in a certain evolution stage, so that harmless optical noise (such as instantaneous glare) which is common on a curved surface and has similar forms is easily misjudged as the defect, the misjudgment rate is increased, the situation of overnormal is avoided, and the robustness of the characteristic marking based on visual identification is reduced. Disclosure of Invention The embodiment of the application provides a feature labeling method, device, medium and product based on vision, which are used for solving the technical problem of how to improve the robustness of feature labeling based on vision identification. The technical scheme of the embodiment of the application is realized as follows: In a first aspect, an embodiment of the present application provides a vision-based feature labeling method, which is applied to an electrical control system, where the electrical control system at least includes a feature recognition model and a history labeling database, and includes: Calculating the confidence coefficient of the target feature of the image to be detected through the feature recognition model; If the confidence coefficient is smaller than a preset confidence coefficient threshold value, determining that the image to be detected is an image to be calibrated; Extracting a first preset number of first feature sequences from the history labeling database, wherein the history labeling database comprises a plurality of feature sample sets, each feature sample set corresponds to a different defect type, and the first feature sequences are used for recording an evolution stage of the defect type corresponding to the feature sample set; Acquiring a labeling instruction of a quality inspector on the image to be calibrated, wherein the labeling instruction is used for indicating whether the target feature is a newly added feature of a second feature sequence, and the second feature sequence is any one of the first feature sequences; If the labeling instruction indicates that the target feature is a new feature of the second feature sequence, labeling the image to be detected as a defect type corresponding to the second feature sequence, and adding the image to be detected to the tail end of the second feature sequence to obtain a third feature sequence; And performing incremental training on the feature recognition model based on the third feature sequence to obtain an updated feature recognition model, wherein the updated feature recognition model is used for recognition and labeling of the image. Optionally, each feature sample set includes a plurality of historical image samples, the calculating of the confidence coefficient of the target feature of the image to be detected through the feature recognition model includes extracting target feature vectors of the target feature of the image to be detected, extracting feature vectors of all the historical image samples as standard feature vectors corresponding to the historical image samples, calculating vector similarity between the target feature vectors and the standard feature vectors to obtain similarity sets corresponding to the feature sample sets respectively, and taking the highest vector similarity in the similarity sets as the confidence coefficient. Optionally, the