Search

CN-121977749-A - Non-contact liquid leakage detection system based on infrared thermal imaging and visible light vision for semiconductor factory

CN121977749ACN 121977749 ACN121977749 ACN 121977749ACN-121977749-A

Abstract

The invention discloses a non-contact type liquid leakage detection system based on infrared thermal imaging and visible light vision for a semiconductor factory, which comprises an image acquisition unit, a detection unit and a detection unit, wherein the image acquisition unit comprises an infrared thermal imaging module and a visible light imaging module which are fixedly arranged on the same imaging installation structure, the infrared thermal imaging module is used for continuously acquiring an infrared thermal image sequence of a monitoring area to capture temperature distribution information.

Inventors

  • QIU MINGYU
  • HUANG WENJUN
  • FAN SHAOFEI

Assignees

  • 北京怡联科技有限公司

Dates

Publication Date
20260505
Application Date
20260127

Claims (10)

  1. 1. A non-contact liquid leakage detection system for a semiconductor factory based on infrared thermal imaging and visible light vision, comprising: The system comprises an image acquisition unit, a monitoring unit and a target area monitoring unit, wherein the image acquisition unit comprises an infrared thermal imaging module and a visible light imaging module which are fixedly installed on the same imaging installation structure, the infrared thermal imaging module is used for continuously acquiring an infrared thermal image sequence of the monitoring area to capture temperature distribution information, the visible light imaging module is used for synchronously acquiring a visible light image sequence of the monitoring area to capture surface visual state information, and the image acquisition unit is configured to be installed above or at the side of a chemical pipeline, a valve or an equipment interface in a semiconductor factory to realize non-contact three-dimensional coverage monitoring of the target area; The data processing and analyzing unit is in communication connection with the image acquisition unit and is used for receiving and processing the infrared thermal image sequence and the visible light image sequence, and the data processing and analyzing unit comprises: The image registration and fusion subunit is used for carrying out pixel-level registration on the infrared thermal image and the visible light image under the same time stamp to generate space-time synchronous multi-mode fusion image data; A feature extraction and anomaly detection subunit that extracts, based on a pre-trained deep learning model, multi-modal features related to liquid leakage from the multi-modal fusion image data, the multi-modal features including at least local temperature anomaly features, temperature gradient variation features, and surface gloss variation features, liquid diffuse reflection features, and liquid trace morphology profile features that are separated out by the infrared thermal image sequence; The decision subunit is used for comprehensively judging whether the chemical leakage occurs or not through the classifier based on the extracted multi-mode characteristics, and generating a detection result comprising leakage probability, suspected position and leakage range; And the alarm and output unit is connected with the data processing and analyzing unit and is used for triggering a multi-stage alarm signal according to a detection result when the decision subunit judges that leakage occurs, and outputting the multi-mode fusion image data and the leakage area identification information to a monitoring terminal.
  2. 2. The system of claim 1, wherein the imaging mounting structure in the image acquisition unit is an active optical synchronization mount comprising: the rigid mounting base is used for being fixed on a wall, a ceiling or an equipment bracket of a factory building; The universal adjusting mechanism is arranged on the rigid mounting base and used for finely adjusting the pitch angle and the horizontal angle of the infrared thermal imaging module and the visible light imaging module so as to ensure that the optical axes of the infrared thermal imaging module and the visible light imaging module are intersected with each other in the center of the monitoring area, and the field of view range is overlapped to the greatest extent; and the synchronous trigger circuit is embedded in the bracket and is used for receiving a synchronous acquisition instruction from the data processing and analyzing unit, generating a synchronous electric signal and triggering the infrared thermal imaging module and the visible light imaging module to acquire images at the same time, so that each frame of infrared thermal image and each frame of visible light image are ensured to have accurate consistent time stamps.
  3. 3. The system of claim 2, wherein the specific method for performing image registration by the image registration and fusion subunit comprises: Extracting feature points, namely extracting scale-invariant feature transform (SIFT) feature points or acceleration robust feature (SURF) feature points from an infrared thermal image and a visible light image which are acquired at the same moment respectively; a feature matching step of matching feature points extracted from two images by using a random sample consensus algorithm (RANSAC) to eliminate mismatching point pairs and estimating a homography matrix describing a pixel coordinate transformation relationship between the two images; the image transformation and alignment step is that the homography matrix is utilized to carry out perspective transformation on the visible light image, so that the visible light image and the infrared thermal image are precisely aligned in space, and a registered visible light image is generated; And a data fusion step of carrying out channel superposition on the color channel information of the registered visible light image and the temperature information matrix of the infrared thermal image to generate the multi-mode fusion image data.
  4. 4. The system of claim 1, wherein the pre-trained deep learning model in the feature extraction and anomaly detection subunit is a dual stream convolutional neural network comprising: An infrared characteristic extraction branch takes the infrared thermal image sequence as input, and the network structure of the infrared thermal image sequence comprises a plurality of convolution layers and pooling layers and is specially used for learning and outputting space-time temperature characteristics related to leakage; The visible light characteristic extraction branch takes the visible light image sequence as input, and the network structure of the visible light characteristic extraction branch also comprises a plurality of convolution layers and pooling layers, which are specially used for learning and outputting surface visual texture characteristics related to leakage; the feature fusion layer is positioned at the tail ends of the two branches and is used for splicing or weighting and fusing the space-time temperature features and the surface visual texture features to form a comprehensive multi-modal feature vector; And the anomaly detection layer is used for receiving the multi-mode feature vector and outputting a preliminary judgment indicating whether leakage exists or not and a corresponding feature map, wherein the feature map is used for highlighting an anomaly region in the image.
  5. 5. The system of claim 4, wherein the classifier employed by the decision subunit is a Support Vector Machine (SVM) or a random forest model, the decision process comprising: Receiving a multimodal feature vector output from the anomaly detection layer; calculating the matching degree of the current feature vector and the leakage category based on the decision boundary obtained by training the historical leakage event data, and outputting a leakage confidence score; comparing the leakage confidence score with a preset first threshold value and a second threshold value, wherein the first threshold value is lower than the second threshold value; when the confidence score is below the first threshold, determining that there is no leakage; When the confidence score is between the first threshold value and the second threshold value, judging that the leakage is suspected, and generating a low-level early warning signal to prompt monitoring staff to pay attention to observation; And when the confidence score is higher than the second threshold value, judging that leakage is confirmed, and triggering a high-level alarm signal.
  6. 6. The system of claim 1, wherein the system further comprises an environment adaptive compensation unit comprising: the environment sensor group is used for collecting background environment parameters of the monitored area in real time, wherein the parameters at least comprise environment temperature, environment humidity, environment illumination intensity and background vibration frequency; The compensation model is used for pre-storing compensation coefficients of an infrared thermal image temperature reference value and a visible light image brightness reference value under different environment parameters; the environment self-adaptive compensation unit is configured to receive real-time data of the environment sensor group and call the compensation model to correct original image data acquired by the image acquisition unit in real time so as to eliminate interference caused by environment fluctuation on feature extraction.
  7. 7. The system of claim 6, wherein the correction of the infrared thermal image by the environmental adaptive compensation unit is specifically: according to the real-time collected ambient temperature, dynamically adjusting the temperature display range of the infrared thermal image, and compensating the apparent temperature value to an equivalent temperature value under the standard reference ambient temperature; according to the environmental humidity collected in real time, compensating the infrared radiation attenuation caused by water vapor in the air, and restoring the real temperature distribution of the surface of the object; and the image blurring caused by background vibration is compensated by software through a digital image stabilizing algorithm, so that the image definition is improved.
  8. 8. The system of claim 1, wherein the multi-level alarm signal implemented by the alarm and output unit comprises: the audible and visual alarm is arranged at a key position on the site and used for sending out strong audible and visual alarm when triggering high-level alarm; The network alarm message is pushed to a computer terminal and mobile equipment of operation and maintenance personnel through an internal network of a factory, wherein the alarm message comprises a leakage point position screenshot, leakage confidence and time information; And the system linkage interface is used for sending a linkage control signal to a central control system of the factory to automatically trigger safety measures of associated equipment when the occurrence of leakage is confirmed, wherein the safety measures comprise, but are not limited to, closing an upstream chemical supply valve, starting an emergency ventilation system and activating an area isolation warning lamp.
  9. 9. The system of claim 1, wherein the system further incorporates a digital management platform that provides: The leakage event database is used for storing complete data of each alarm event, including an original multi-mode image sequence, processed characteristic data, an alarm level, a treatment record and a post analysis report; The visual man-machine interaction interface is used for displaying the layout of monitoring points of the whole factory in a two-dimensional plan view or a three-dimensional model view, displaying the states (normal, early warning and alarming) of all the monitoring points in real time and supporting the backtracking and data analysis of historical leakage events; and the model self-learning module is used for performing incremental training on the deep learning model in the feature extraction and anomaly detection subunit according to the newly-generated leakage event sample and feedback marks of operation and maintenance personnel on false alarm and missing alarm, and continuously optimizing the detection performance of the model.
  10. 10. The system of claim 1, wherein the system is configured to cooperate with an existing contact leak detection string system of a semiconductor factory to form a redundant detection; When the non-contact detection system and the contact detection rope system trigger an alarm, confirming that the alarm is a high-reliability leakage event; when only the non-contact detection system triggers an alarm and the contact system does not alarm, the system marks a special event needing to be verified manually and records the mode for optimizing an algorithm; When only the contact type detection rope system alarms and the non-contact type system does not detect an abnormality, a system self-checking flow is triggered, and whether the image acquisition unit is shielded or fails is checked.

Description

Non-contact liquid leakage detection system based on infrared thermal imaging and visible light vision for semiconductor factory Technical Field The invention relates to the technical field of non-contact detection, in particular to a non-contact liquid leakage detection system based on infrared thermal imaging and visible light vision for a semiconductor factory. Background The invention relates to the technical field of industrial safety monitoring and machine vision, in particular to a non-contact liquid leakage detection system based on infrared thermal imaging and visible light vision for a semiconductor factory. In the semiconductor manufacturing process, various high purity, highly corrosive chemicals (such as acid-base, solvents, specialty gases, etc.) are widely used. These chemicals are typically transported and handled through complex piping networks, valves, pumps, and reaction equipment. Any minor leakage can have serious consequences including damage to expensive precision equipment, significant economic loss from contamination of batches of products, triggering environmental pollution events, and even threatening personnel safety. At present, a commonly used leak detection scheme for semiconductor factories is a contact type leak detection rope. The technical scheme has the core principle that a special detection rope is laid in a region (such as the lower part of equipment and the line of a pipeline) where leakage is likely to occur, and when chemicals leak and contact with the detection rope, the electrical characteristics (such as resistance and capacitance) of the detection rope are changed, so that an alarm system is triggered. However, this mainstream contact detection technique has a number of inherent drawbacks: The contact detection results in low reliability and short service life, namely the detection rope must be in direct contact with chemicals to respond, and the highly corrosive chemicals can rapidly degrade the detection rope material, so that the performance of the detection rope material is deteriorated, misinformation or failure is caused, frequent replacement is required, the maintenance cost is high, and the reliability is difficult to ensure. The monitoring coverage is limited, and a blind area exists, namely the detection rope can only protect a linear path which is paved physically, can not effectively cover non-paved areas such as equipment interfaces, valve sides, complex pipe galleries and the like, and a large number of monitoring blind areas exist. Is insensitive to micro-leakage and initial leakage, and a certain amount of liquid is usually required to be accumulated to be reliably detected, so that early and slow leakage is difficult to find in time, and the optimal treatment time is missed. The system can only provide alarm signals, can not provide images, sizes and diffusion conditions of leakage points, and is difficult for operation and maintenance personnel to quickly and accurately position leakage sources, so that emergency treatment time is prolonged. To overcome the limitations of contact detection, the industry has also explored non-contact solutions, such as visual detection based on a single visible camera or a single thermal infrared imager. However, the environment of a semiconductor factory is complex (such as illumination change of a clean room, environmental temperature fluctuation, reflection of equipment surfaces and the like), and a single vision technology is easily affected by the environment and has extremely high false alarm rate. While the single infrared thermal imaging can detect abnormal temperature, the normal heat source, sunlight irradiation and the like of the equipment can be misjudged as leakage, and the cross-validation capability is lacking. Therefore, there is an urgent need in the art for a non-contact leakage detection solution that can adapt to the complex environment of a semiconductor factory, is sensitive and reliable in detection, and can provide visual leakage information. Disclosure of Invention The invention aims to provide a non-contact liquid leakage detection system based on infrared thermal imaging and visible light vision for a semiconductor factory, which aims to solve the defects in the prior art. In order to achieve the above object, the present invention provides the following technical solutions: a non-contact liquid leakage detection system for a semiconductor factory based on infrared thermal imaging and visible light vision, comprising: The system comprises an image acquisition unit, a monitoring unit and a target area monitoring unit, wherein the image acquisition unit comprises an infrared thermal imaging module and a visible light imaging module which are fixedly installed on the same imaging installation structure, the infrared thermal imaging module is used for continuously acquiring an infrared thermal image sequence of the monitoring area to capture temperature distribution information, the visib