Search

CN-121997129-A - Electronic component detection system and method

CN121997129ACN 121997129 ACN121997129 ACN 121997129ACN-121997129-A

Abstract

The invention discloses an electronic component detection system and method, comprising a multi-mode data acquisition module, an image enhancement and generation module, a multi-mode feature fusion and decision module, a result display and control module, wherein the image enhancement and generation module is electrically connected with the multi-mode data acquisition module, the multi-mode feature fusion and decision module is electrically connected with the image enhancement and generation module and the multi-mode data acquisition module, and the result display and control module is electrically connected with the multi-mode feature fusion and decision module, so that the working condition of poor data quality or temporary failure of a single sensor is effectively treated through an image enhancement and cross-mode generation technology, the continuity of a detection flow and the integrity of decision information are ensured, and a final report not only contains defect classification and position, but also records complete detection decision paths and key data, thereby facilitating quality tracing, model optimization and manual review.

Inventors

  • LIU ZHENLONG
  • XUE WEINA

Assignees

  • 浙江工业大学

Dates

Publication Date
20260508
Application Date
20260119

Claims (9)

  1. 1. The electronic component detection system is characterized by comprising a multi-mode data acquisition module, an image enhancement and generation module, a multi-mode feature fusion and decision module, a result display and control module, The multi-mode data acquisition module is used for synchronously or asynchronously acquiring at least two different types of data of the electronic component to be detected, wherein the types of the data comprise optical image data, electric signal waveform data and internal structure imaging data; The image enhancement and generation module is electrically connected with the multi-mode data acquisition module and is used for carrying out quality evaluation on the acquired image data and enhancing or generating image data with poor quality or missing image data based on other mode data; The multi-mode feature fusion and decision module is electrically connected with the image enhancement and generation module and the multi-mode data acquisition module and is used for extracting and fusing features from different mode data, performing defect detection and classification based on the fused features and dynamically adjusting a detection strategy; The result display and control module is electrically connected with the multi-mode feature fusion and decision module and is used for displaying detection results, providing an interactive interface and controlling the system to execute fine rechecking actions.
  2. 2. The electronic component inspection system of claim 1, wherein the image enhancement and generation module comprises a quality assessment unit, a data enhancement unit, and a cross-modality generation unit, wherein, A quality evaluation unit for evaluating sharpness, contrast or integrity of the image data; the data enhancement unit is used for enhancing the image data with poor quality based on bilateral filtering, edge enhancement and detail fusion algorithm; And the cross-mode generation unit is used for generating a characteristic diagram or image of the target mode by utilizing the data of other available modes when the image data of the specific mode is missing based on the generation countermeasure network architecture.
  3. 3. The electronic component inspection system of claim 1, wherein the multi-modal feature fusion and decision module comprises a feature extraction unit, a graph fusion unit, and an adaptive decision unit, wherein, The feature extraction unit is used for respectively extracting feature vectors or feature graphs of different modal data by adopting a deep convolutional neural network; The image fusion unit is used for constructing characteristic nodes, component entity nodes and preset defect concept nodes of different modes into different patterns, and carrying out message transmission and characteristic aggregation through a different pattern neural network to generate a global fusion characteristic representation; And the self-adaptive decision unit is used for modeling the detection flow as a sequence decision process, dynamically selecting subsequent operations based on the global fusion characteristic representation and the current detection context, wherein the operations comprise performing another mode detection, focusing on specific area fine scanning, performing defect classification or requesting external intervention.
  4. 4. The electronic component inspection system of claim 1, wherein the adaptive decision unit is trained using a reinforcement learning algorithm, and wherein the state space comprises the global fusion feature representation, the history of executed inspection actions, and the identified suspicious region information, and wherein the reward function is designed to encourage accurate classification and penalize unnecessary resource consumption and erroneous decisions.
  5. 5. The electronic component inspection system of claims 1-4, wherein the multi-modality data acquisition module comprises at least two of a high resolution optical camera, an oscilloscope or dedicated test equipment for acquiring electrical signal parameters, and an X-ray imager or an ultrasound scanning microscope.
  6. 6. An electronic component inspection method applied to the electronic component inspection system according to any one of claims 1 to 5, characterized by comprising the steps of: The method comprises the steps of S1, acquiring at least two different types of data of an electronic component to be detected through the multi-mode data acquisition module, S2, evaluating the data acquired in the step S1 through the image enhancement and generation module, enhancing or generating cross-mode image data with poor quality or missing, S3, extracting the characteristics of all the mode data through the multi-mode characteristic fusion and decision module, carrying out deep fusion to obtain global characteristic representation, carrying out dynamic decision on the next detection action, S4, repeating part or all of the steps S1 to S3 until the self-adaptive decision unit makes final defect classification decision or triggers manual review, S5, outputting a visual report containing defect types, positions, confidence and detection path trace back through the result display and control module, and controlling an executor to carry out review on a specific area.
  7. 7. The method for detecting electronic components according to claim 6, wherein in step S2, the enhancing the image data specifically includes: bilateral filtering is carried out on the image so as to smooth the non-edge area; Performing edge extraction and marking by using a Roberts operator; And in the non-edge region, calculating the local variance and selecting the pixel value of the region with the maximum variance in a plurality of images for fusion so as to improve the detail definition.
  8. 8. The method for detecting electronic components according to claim 6, wherein in step S3, the next detection action based on the dynamic decision is specifically implemented by inputting the current global fusion feature, the executed action sequence and the environmental state as the state into a pre-trained reinforcement learning strategy network, outputting probability distribution of executing different detection actions by the network, and selecting and executing the actions according to the probability distribution.
  9. 9. The method for detecting electronic components according to claim 6, wherein in step S3, the depth fusion is performed, specifically, an iso-composition including modal feature nodes, component entity nodes and defect concept nodes is constructed, node features are updated by using an iso-composition neural network through a relationship-specific message passing and aggregation mechanism, and final features of the component entity nodes are represented as the global fusion features.

Description

Electronic component detection system and method Technical Field The present invention relates to the field of electronic components, and in particular, to a system and a method for detecting electronic components. Background Under the trend of high integration and miniaturization of electronic products, quality detection of electronic components is of great importance. The traditional detection method mostly depends on a single mode, such as manual visual inspection or single type automatic optical detection, and has the problems of low efficiency, strong subjectivity, insufficient detection capability for internal and electrical defects and the like. Although multi-mode detection systems adopting various sensors exist in the prior art, the problems still exist that 1, quality of data of each mode is uneven or even missing, and overall judgment reliability is affected, 2, multi-mode information fusion mode is simple, complex association and complementarity among modes cannot be deeply mined, 3, a detection flow is fixed and stiff, strategies cannot be dynamically adjusted according to current detected clues, efficiency is low when the complex or atypical defects are faced, or high-cost detection means are excessively used for pursuing reliability, and therefore, an electronic component detection system and an electronic component detection method capable of intelligently processing data incompleteness, realizing depth feature fusion and adaptively optimizing the detection flow are needed at present. Disclosure of Invention The present invention aims to solve at least one of the technical problems in the related art to some extent. Therefore, the invention aims to provide an electronic component detection system and an electronic component detection method. In order to achieve the above objective, the present invention provides an electronic component detection system and method, which includes a multi-modal data acquisition module, an image enhancement and generation module, a multi-modal feature fusion and decision module, and a result display and control module, wherein the multi-modal data acquisition module is used for synchronously or asynchronously acquiring at least two different types of data of an electronic component to be detected, the types of the data include optical image data, electric signal waveform data and internal structure imaging data, the image enhancement and generation module is electrically connected with the multi-modal data acquisition module, and is used for performing quality assessment on the acquired image data, enhancing or generating image data with poor quality or missing based on other modal data, the multi-modal feature fusion and decision module is electrically connected with the image enhancement and generation module and the multi-modal data acquisition module, and is used for extracting and fusing features from different modal data, executing defect detection and classification based on the fusion features, and dynamically adjusting a detection strategy, and the result display and control module is electrically connected with the multi-modal feature fusion and decision module, and is used for displaying a detection result, providing an interaction interface and controlling the system to execute a fine detection action. In addition, the system and the method for detecting the electronic component provided by the invention can also have the following additional technical characteristics: the image enhancement and generation module comprises a quality assessment unit, a data enhancement unit and a cross-mode generation unit, wherein the quality assessment unit is used for assessing the definition, contrast or integrity of image data, the data enhancement unit is used for enhancing the image data with poor quality based on bilateral filtering, edge enhancement and detail fusion algorithms, and the cross-mode generation unit is used for generating a characteristic diagram or image of a target mode by utilizing data of other available modes when the image data of a specific mode is absent based on a generation countermeasure network architecture. The multi-mode feature fusion and decision module comprises a feature extraction unit, a graph fusion unit and an adaptive decision unit, wherein the feature extraction unit is used for respectively extracting feature vectors or feature graphs of different mode data by adopting a deep convolutional neural network, the graph fusion unit is used for constructing feature nodes, component entity nodes and preset defect concept nodes of different modes into different patterns, and carrying out message transmission and feature aggregation through the different pattern neural network to generate global fusion feature representation, the adaptive decision unit is used for modeling a detection flow as a sequence decision process, and dynamically selecting subsequent operations based on the global fusion feature representation and the curr