CN-121998935-A - Metal weld joint hintable segmentation method and system based on semantic and prototype matching fusion

CN121998935ACN 121998935 ACN121998935 ACN 121998935ACN-121998935-A

Abstract

The invention belongs to the technical field, and particularly relates to a metal weld joint hintable segmentation method and system based on semantic and prototype matching fusion. The method comprises the steps of obtaining a query image of the surface of a metal material to be segmented, inputting the query image into a trained semantic branch network for forward reasoning, outputting a two-class semantic log-score difference graph of a weld joint foreground and a background and a query feature graph, processing the query feature graph and the two-class semantic log-score difference graph by adopting a trained core clue network to obtain a foreground core probability graph, generating a core point prompt or a core mask prompt according to the foreground core probability graph, and inputting the core point prompt or the core mask prompt into a prompt segmentation model according to the query image, namely outputting a weld joint pixel level segmentation mask. The method can reduce pixel-level labeling requirements and improve cross-working condition robustness, and is suitable for on-line detection and positioning of the metal weld joints.

Inventors

XIE JINGMING
YANG ZIHENG
LIU MOYUN
Dai Haotian
CHEN YONG

Assignees

华中科技大学

Dates

Publication Date: 20260508
Application Date: 20260123

Claims (10)

1. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion is characterized by comprising the following steps of: s1, acquiring a query image of the surface of a metal material to be segmented; S2, inputting the query image into a trained semantic branch network for forward reasoning, and outputting a two-class semantic log-number difference map of a welding seam foreground and a background and a query feature map; S3, processing the query feature map and the two-class semantic log-difference map by adopting a trained core clue network to obtain a foreground core probability map, and generating a core point prompt or a core mask prompt according to the foreground core probability map; s4, inputting the core point prompt or the core mask prompt and the query image into a prompt segmentation model, namely outputting a weld joint pixel level segmentation mask.
2. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 1 is characterized in that in step S2, the semantic branch network is a Mask2Former semantic segmentation network, resNet-101 is adopted as an encoder, and the query feature map is a C4 layer feature map of the encoder; The two-classification semantic log-number difference map is a difference map of a weld joint foreground channel score and a background channel score output by the semantic branch network.
3. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 2 is characterized in that training of the semantic branch network comprises the steps of collecting sample images of the surface of a metal material, constructing a labeling sample set containing pixel-level weld joint mask labels, training the semantic branch network by using the labeling sample set, and carrying out end-to-end updating on parameters of the semantic branch network in an iterative mode to minimize a semantic segmentation loss function.
4. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 3, wherein in step S3, the core clue network comprises a prototype matching branch and a consistency fusion module, the prototype matching branch is used for performing learning mapping on the query feature map and performing similarity matching with a weld joint prototype vector to obtain a similarity map, a prototype branch log probability map is formed through temperature scaling and bias calibration, and the consistency fusion module is used for performing consistency fusion on the two classification semantic log difference map and the prototype branch log probability map to obtain a foreground core probability map.
5. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 4, wherein the acquisition mode of the weld joint prototype vector comprises the steps of collecting a plurality of sample images of the surfaces of metal materials of different types, and carrying out pixel-level weld joint mask labeling on each sample image to construct a support set consisting of a support image and a corresponding pixel-level weld joint mask label; Freezing encoder parameters of the trained semantic branch network, extracting a plurality of support feature images from the support images of the support set by utilizing the frozen encoder, carrying out mask guide aggregation on the plurality of support feature images in a weld joint coverage area according to the pixel-level weld joint mask mark to obtain a plurality of support prototype vectors, and carrying out aggregation and normalization on the plurality of support prototype vectors to obtain the weld joint prototype vectors and fixedly storing the weld joint prototype vectors.
6. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 4, wherein the training of the core clue network comprises training the core clue network based on a query feature map of a labeling sample set, a two-class semantic log-difference map and a pixel-level weld joint mask label; The method comprises the steps of determining a difficult negative sample in a background area marked by a welding seam mask, selecting a pixel with the largest response as the difficult negative sample according to at least one of semantic branch network output, a prototype branch probability map and a foreground core probability map, and simultaneously restraining the prototype branch probability map and the foreground core probability map to ensure that the prototype branch logarithmic probability map provides a judging basis for generating the foreground core probability map; the prototype branch probability map is obtained by mapping a prototype branch logarithmic probability map through a nonlinear activation function.
7. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 1 is characterized in that in step S3, when core point hinting is adopted, top K pixel positions with the largest probability value are selected from the foreground core probability map as a point hinting set, minimum distance constraint is set to avoid excessive concentration of hinting points, and when core mask hinting is adopted, highest response pixels are selected from the foreground core probability map according to a preset proportion and binary masks are formed to serve as mask hints.
8. The metal weld joint hintable segmentation method based on semantic and prototype matching fusion according to claim 4, wherein the similarity matching adopts cosine similarity or normalized dot product similarity, and the hintable segmentation model is a SAM model.
9. A metal weld hintable segmentation system based on semantic and prototype matching fusion, comprising: An image acquisition unit for acquiring a query image of the surface of the metal material to be segmented; The semantic branch unit is used for inputting the query image into a trained semantic branch network to perform forward reasoning and outputting a two-class semantic log-number difference map of a welding seam foreground and a background and a query feature map; the core clue network unit is used for processing the query feature map and the two-class semantic log-number difference map by adopting a trained core clue network to obtain a foreground core probability map; The prompt generation unit is used for generating a core point prompt or a core mask prompt according to the foreground core probability map; And the hinting and dividing unit is used for inputting the core point hinting or the core mask hinting and the query image into a hinting and dividing model, namely outputting a welding seam pixel level dividing mask.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1-8.

Description

Metal weld joint hintable segmentation method and system based on semantic and prototype matching fusion Technical Field The invention belongs to the technical field of industrial visual detection, and particularly relates to a metal weld joint hintable segmentation method and system based on semantic and prototype matching fusion. Background The laser cutting of the metal material needs to position the welding line in advance, but the welding line usually presents an elongated structure and is interfered by reflection, oxidation, processing scratch and the like of the metal surface, and the boundary is unclear and pseudo-texture is easy to appear. The problems of few samples and cross working conditions are often faced in actual production, and the data volume is limited by high labeling cost, so that the generalization capability of the traditional full-supervision segmentation method is insufficient when the complex textures are handled. In the prior art, a deep learning scheme (such as patent CN 116765635B) generally depends on sufficient training data, has limited anti-interference capability on reflection and pseudo-texture, is easily affected by positioning drift based on a scheme of a traditional image algorithm (such as patent CN105427295 a), needs to frequently adjust parameters to adapt to different batches of pipes, and is difficult to meet flexible production requirements. In addition, although the general hint type segmentation model (such as SAM) has a strong generalization, it cannot rely on manual interaction in online detection. If an effective automatic prompt mechanism is lacking, the model is extremely easy to be misled by anti-facula or scratch. Therefore, a technical scheme capable of automatically generating a high-precision foreground core prompt under the condition of few samples and driving a prompt model to stably output a weld mask is needed. Disclosure of Invention The invention aims to provide a metal weld joint hintable segmentation method and a system based on semantic and prototype matching fusion, which can reduce pixel-level labeling requirements and improve cross-working condition robustness. In order to achieve the above purpose, the invention provides a metal weld joint hintable segmentation method based on semantic and prototype matching fusion, which comprises the following steps: s1, acquiring a query image of the surface of a metal material to be segmented; S2, inputting the query image into a trained semantic branch network for forward reasoning, and outputting a two-class semantic log-number difference map of a welding seam foreground and a background and a query feature map; S3, processing the query feature map and the two-class semantic log-difference map by adopting a trained core clue network to obtain a foreground core probability map, and generating a core point prompt or a core mask prompt according to the foreground core probability map; s4, inputting the core point prompt or the core mask prompt and the query image into a prompt segmentation model, namely outputting a weld joint pixel level segmentation mask. In step S2, the semantic branch network is a Mask2Former semantic segmentation network, resNet-101 is adopted as an encoder, and the query feature map is a C4 layer feature map of the encoder; The two-classification semantic log-number difference map is a difference map of a weld joint foreground channel score and a background channel score output by the semantic branch network. Further, training of the semantic branch network comprises the steps of collecting sample images of the surface of a metal material, constructing a labeling sample set containing pixel-level weld mask labels, training the semantic branch network by using the labeling sample set, and carrying out end-to-end updating on parameters of the semantic branch network in an iterative mode to minimize a semantic segmentation loss function. In step S3, the core clue network includes a prototype matching branch and a consistency fusion module, where the prototype matching branch is used for performing learning mapping on the query feature map, performing similarity matching with a weld prototype vector to obtain a similarity map, and performing temperature scaling and bias calibration to form a prototype branch log probability map, and the consistency fusion module is used for performing consistency fusion on the two-classification semantic log difference map and the prototype branch log probability map to obtain a foreground core probability map. Further, the acquisition mode of the weld prototype vector comprises the steps of collecting a plurality of sample images of the surfaces of metal materials of different types, and carrying out pixel-level weld mask labeling on each sample image to construct a support set consisting of a support image and a corresponding pixel-level weld mask label; Freezing encoder parameters of the trained semantic branch network, extracting a plurality of s