CN-122023289-A - Continuous casting red billet surface defect online identification method and device based on image sensing

CN122023289ACN 122023289 ACN122023289 ACN 122023289ACN-122023289-A

Abstract

The invention relates to the technical field of defect detection, in particular to an image sensing-based continuous casting red billet surface defect online identification method and device, wherein the identification method comprises the steps of acquiring a medium-wave infrared thermal image, a short-wave infrared image and a three-dimensional contour map of the red billet surface in real time based on a multi-mode sensor unit and executing preprocessing operation; and performing time synchronization operation on the preprocessed intermediate wave infrared thermal image, the short wave infrared image and the three-dimensional contour image to obtain time synchronized images, establishing a perspective transformation model to obtain a first perspective transformation matrix and a second perspective transformation matrix, constructing a registration error correction factor, and optimizing by adopting a gradient descent iterative algorithm to obtain the images after spatial registration. The method is based on double accurate calibration of time and space, provides a reliable data base for subsequent multi-mode feature fusion and defect identification, and remarkably improves the accuracy of defect position and size measurement.

Inventors

ZHOU TENG
WU ZHENZHONG
SHU MEILIANG
DUAN YULI
ZHOU YAMING
WANG PENG
HAN JIPENG
FENG CHAOSHENG
ZHANG HONGYU

Assignees

安徽富凯特材有限公司

Dates

Publication Date: 20260512
Application Date: 20260114

Claims (9)

1. The continuous casting red billet surface defect online identification method based on image sensing is characterized by comprising the following steps of: s1, acquiring a medium wave infrared thermal image, a short wave infrared image and a three-dimensional contour map of the surface of an infrared blank in real time based on a multi-mode sensor unit, and executing preprocessing operation; S2, performing time synchronization operation on the preprocessed intermediate wave infrared thermal image, the short wave infrared image and the three-dimensional contour image to obtain a time synchronized image, establishing a perspective transformation model to obtain a first perspective transformation matrix and a second perspective transformation matrix, constructing a registration error correction factor, optimizing by adopting a gradient descent iterative algorithm to obtain a space registered image, and constructing multi-mode data comprising the time synchronized image and the space registered image; s3, constructing a multi-mode fusion defect identification model, constructing a multi-mode sample data set containing a plurality of historical multi-mode data, and training the multi-mode fusion defect identification model; s4, inputting the multi-mode data into a trained multi-mode fusion defect recognition model to output the type, position, size and severity level of the surface defects of the red blank.
2. The method for on-line identification of surface defects of a continuous casting red billet according to claim 1, wherein in step S1, the step of performing the pretreatment operation is as follows: S11, collecting an output gray scale image of the medium-wave thermal infrared imager under a corresponding radiation scene by taking a low-temperature blackbody radiation source and a high-temperature blackbody radiation source as calibration references, and calculating response gain of each pixel Offset of The calculation formula is as follows: In the above-mentioned method, the step of, 、 When the low-temperature blackbody radiation source and the high-temperature blackbody radiation source are respectively calibrated The gray value at the pixel is a function of the gray value at the pixel, 、 Respectively outputting voltage signals of the medium wave thermal infrared imager under the corresponding radiation scene; s12, according to the gain Offset of Correcting the intermediate wave infrared thermal image to obtain a corrected intermediate wave infrared thermal image, wherein the expression is as follows: In the above-mentioned method, the step of, Is in the middle wave infrared thermal image The gray value at the pixel is a function of the gray value at the pixel, In the corrected intermediate wave infrared thermal image Gray values at the pixels; S13, repairing pixels with gray values exceeding the pixel threshold value of the medium wave infrared thermal imager in the corrected medium wave infrared thermal image by adopting a neighborhood mean value interpolation method to obtain a repaired medium wave infrared thermal image, and converting the repaired medium wave infrared thermal image into red blank surface temperature field data based on the gray body Planck law to obtain a preprocessed medium wave infrared thermal image ; S14, performing convolution operation on the short-wave infrared image by adopting 5X 5 Gaussian collation to obtain a denoised short-wave infrared image, and performing contrast-limited self-adaptive histogram equalization operation on the denoised short-wave infrared image to obtain a preprocessed short-wave infrared image ; S15, carrying out statistical outlier filtering and voxel grid downsampling operation on the three-dimensional contour map to obtain a preprocessed three-dimensional contour map 。
3. The method for on-line identification of surface defects of a continuous casting red billet according to claim 1, wherein in step S2, the step of time synchronizing operation is as follows: s211, using three-dimensional contour map Defining a medium wave infrared thermal image based on the time stamp of (a) Is the timestamp of (a) Short wave infrared image Is the timestamp of (a) Three-dimensional profile Is the timestamp of Calculating a medium wave infrared thermal image And three-dimensional profile Time stamp difference of (2) Short wave infrared map And three-dimensional profile Time stamp difference of (2) The calculation formula is as follows: S212, setting a time stamp threshold value and judging a time stamp difference value Difference from the time stamp Whether both are less than the timestamp threshold: if yes, it is determined that synchronization is valid and the process proceeds to step S221; if not, judging that the synchronization is invalid and entering step S213; s213, according to the diameter of the roller way Encoder pulse frequency Calculating the moving speed of the red blank The calculation formula is as follows: In the above-mentioned method, the step of, The number of pulses per revolution of the encoder; s214, performing medium wave infrared thermal imaging according to the edge of the red blank Is of the pixel offset of (2) And short wave infrared map Pixel offset in (a) Pixel equivalent Calculating a medium wave infrared thermal image And three-dimensional profile Is of the acquired area deviation amount of (1) Short wave infrared map And three-dimensional profile Is of the acquired area deviation amount of (1) The calculation formula is as follows: S215, according to the deviation of the acquisition area And the acquisition area deviation amount And the red blank moving speed Compensation time of medium wave infrared thermal image graph is calculated by ratio of (2) And compensation time of short wave infrared image ; S216, medium wave infrared thermal image And short wave infrared map Setting trigger delay of trigger signal of next frame, wherein the trigger delay time is compensation time And compensation time 。
4. The method for on-line identification of surface defects of a continuous casting red billet according to claim 1, wherein in step S2, the step of the spatial registration operation is as follows: S221, selecting S characteristic points of edges at two sides of the red blank as registration anchor points, and performing three-dimensional profile drawing In acquiring space coordinates In the medium wave infrared diagram Acquiring pixel coordinates of corresponding registration anchor points In short wave infrared image Acquiring pixel coordinates of corresponding registration anchor points ; S222, performing medium wave infrared image And short wave infrared map Are all matched with the three-dimensional profile Registering and establishing a perspective transformation model, wherein the expression is as follows: In the above-mentioned method, the step of, And A first perspective transformation matrix and a second perspective transformation matrix respectively; s223, establishing a plurality of equation sets through the coordinates of the registration anchor points, and solving a matrix by adopting a least square method Sum matrix ; S224, red blank moving speed based on encoder output Calculating the displacement of red blanks between adjacent frames And to a matrix Sum matrix Performing translation correction to obtain an initial registration matrix Initial registration matrix The expression is: In the above-mentioned method, the step of, In order to translate the transformation matrix, Is the spacing of adjacent frames; S225, using three-dimensional contour map Taking a coordinate system of (2) as a reference, and taking a medium wave infrared image Through an initial registration matrix Short wave infrared image Through an initial registration matrix Mapping to the corresponding positions under the reference coordinate system respectively to obtain an image after preliminary registration; S226, extracting a plurality of feature points in the preliminarily registered image, calculating a feature point registration error and an average error, and constructing a registration error correction factor For initial registration matrix Initial registration matrix Correcting, optimizing by adopting a gradient descent iterative algorithm and outputting an optimized registration matrix Registration matrix ; S227, repeating step S225 to obtain a spatially registered image.
5. The method for online identification of surface defects of a continuous casting red blank according to claim 4, wherein in step S226, the specific implementation steps are as follows: S2261, randomly selecting a plurality of uniformly distributed characteristic points from the preliminarily registered images to obtain a three-dimensional contour map The space coordinates of each feature point are used as reference coordinates, and the registration error between the registered space coordinates and the reference coordinates is calculated The calculation formula is as follows: In the above-mentioned method, the step of, For the coordinates of the feature points after registration, Is a three-dimensional profile Is defined by the reference coordinates of (a); s2262, registration error according to a plurality of feature points Dividing the sum by the number of feature points to obtain an average error ; S2263, setting registration error threshold and average error threshold, and judging registration error of single feature point Whether or not it is less than the registration error threshold and the average error Whether less than the average error threshold: if yes, judging that the registration is normal and outputting an optimized registration matrix Registration matrix ; If not, determining that the registration is abnormal and proceeding to step S2264; s2264 based on medium wave infrared thermal image Surface temperature of extraction And red blank moving speed Construction of registration error correction factors The expression is: S2265, correcting factor according to registration error For initial registration matrix Initial registration matrix Performing scaling correction to obtain a primarily corrected registration matrix Registration matrix ; S2266, registering matrix by adopting gradient descent iterative algorithm Registration matrix Performing a minimizing operation until the registration error after iteration And average error Respectively smaller than the registration error threshold and the average error threshold, and outputting the currently optimized registration matrix Registration matrix The iterative formula is: In the above-mentioned method, the step of, In order for the rate of learning to be high, For loss function with respect to matrix Is a gradient of (a).
6. The online continuous casting red billet surface defect identification method according to claim 1, wherein the multi-modal fusion defect identification model comprises a feature extraction module, a cross-modal feature fusion module and a classification regression module: the feature extraction module adopts a three-branch parallel structure, and the three-branch parallel structure is respectively two lightweight convolutional neural networks and one PointNet ++ network; The cross-modal feature fusion module adopts an attention weighted fusion mechanism; The classification regression module comprises a defect type classification sub-module, a position coordinate regression sub-module, a size parameter calculation sub-module and a severity level evaluation sub-module.
7. The online identification method of surface defects of a continuous casting red billet according to claim 1, wherein in step S3, the step of training the multi-modal fusion defect identification model to obtain a trained multi-modal fusion defect identification model is as follows: s31, adopting random horizontal overturning, rotation, scaling operation and Gaussian noise adding operation on the multi-modal sample data set to obtain the multi-modal sample data set with enhanced data; S32, dividing the multi-mode sample data set after data enhancement into a training set, a verification set and a test set according to the proportion of 7:2:1; S33, adopting a weighted sum of the cross entropy loss function and the L1 loss function as a total loss function, and setting a weight coefficient; And S33, selecting AdamW an optimizer, and training the multi-mode fusion defect recognition model by adopting a cosine annealing learning rate scheduling strategy to obtain a trained multi-mode fusion defect recognition model.
8. The method for on-line identification of surface defects of continuous casting red billets according to claim 1, wherein the multi-mode sample data set comprises continuous casting red billets multi-mode image data of different working conditions and different defect types, and each sample comprises a defect type label, a center coordinate of a defect under a reference coordinate system, a defect boundary frame coordinate, a defect actual size and a severity grade label.
9. An apparatus for applying the continuous casting red billet surface defect online identification method as claimed in any one of claims 1 to 8, comprising: The multi-mode sensor unit consists of a short-wave infrared high Wen Xiangji (300), a medium-wave infrared thermal imager (200) and a line laser scanner (400), wherein the short-wave infrared camera (300) is used for acquiring the image information of the surface of the red blank, the medium-wave infrared thermal imager (200) is used for acquiring the temperature field distribution information of the surface of the red blank, and the line laser scanner (400) is used for acquiring the three-dimensional profile information of the surface of the red blank; The protection cooling unit consists of an integrated water-cooling protection box body (100) and a composite cleaning component, wherein the multi-mode sensor unit is installed in the integrated water-cooling protection box body (100), a cooling water channel (101) for cooling the multi-mode sensor unit is installed in the integrated water-cooling protection box body (100), the composite cleaning component comprises an air curtain (102) and a scraping plate (103) which are arranged in front of a lens of the multi-mode sensor unit, the air curtain (102) is used for forming a positive pressure air curtain to prevent dust and water vapor from adhering, and the scraping plate (103) is used for removing stubborn dirt possibly adhered.

Description

Continuous casting red billet surface defect online identification method and device based on image sensing Technical Field The invention relates to the technical field of defect detection, in particular to an online identification method and device for continuous casting red billet surface defects based on image sensing. Background The continuous casting red billet is used as a core intermediate product in the steel production flow, and the surface quality of the continuous casting red billet directly determines the processing efficiency of the subsequent rolling, forging and other procedures, and the mechanical property and the service life of the final product. However, the red billet is easily affected by multiple factors such as vibration of a crystallizer, uneven secondary cooling, abrasion of a roller way, purity of molten steel and the like in the continuous casting process, and surface defects such as cracks, slag inclusion, scars, pits and the like are generated. The method for detecting the defects on the surface of the red blank can only acquire single dimensional information of the surface of the red blank, cannot comprehensively characterize complex characteristics such as morphology, depth, temperature distribution and the like of the defects, and cannot fully identify the fine defects and hidden defects, and if multi-dimensional information is acquired by adopting a multi-source sensor, the multi-mode data acquisition area is misplaced due to the fact that the actual moving speed of the red blank is not considered, and the mounting positions of different sensors are different, the problem of coordinate deviation caused by the difference of viewing angles exists, and further the problems of asynchronous time and low spatial registration precision of the multi-source data are caused, so that the method is difficult to adapt to the defect detection requirements under different working conditions, and the key information such as the position, the size, the serious grade and the like of the defects cannot be accurately output, and the follow-up quality tracing and the process optimization are difficult to support. Disclosure of Invention Aiming at the defects of the prior art, the invention provides an image sensing-based continuous casting red billet surface defect online identification method and device, which solve the technical problems that the existing identification method is insufficient in identification capability of fine defects and hidden defects, and the multi-source data is asynchronous in time and low in spatial registration precision. In order to solve the technical problems, the invention provides the following technical scheme that the method for identifying the surface defects of the continuous casting red billet on line based on image sensing comprises the following steps: s1, acquiring a medium wave infrared thermal image, a short wave infrared image and a three-dimensional contour map of the surface of an infrared blank in real time based on a multi-mode sensor unit, and executing preprocessing operation; S2, performing time synchronization operation on the preprocessed intermediate wave infrared thermal image, the short wave infrared image and the three-dimensional contour image to obtain a time synchronized image, establishing a perspective transformation model to obtain a first perspective transformation matrix and a second perspective transformation matrix, constructing a registration error correction factor, optimizing by adopting a gradient descent iterative algorithm to obtain a space registered image, and constructing multi-mode data comprising the time synchronized image and the space registered image; s3, constructing a multi-mode fusion defect identification model, constructing a multi-mode sample data set containing a plurality of historical multi-mode data, and training the multi-mode fusion defect identification model; s4, inputting the multi-mode data into a trained multi-mode fusion defect recognition model to output the type, position, size and severity level of the surface defects of the red blank. Preferably, in step S1, the step of performing the preprocessing operation is as follows: S11, collecting an output gray scale image of the medium-wave thermal infrared imager under a corresponding radiation scene by taking a low-temperature blackbody radiation source and a high-temperature blackbody radiation source as calibration references, and calculating response gain of each pixel Offset ofThe calculation formula is as follows: In the above-mentioned method, the step of, 、When the low-temperature blackbody radiation source and the high-temperature blackbody radiation source are respectively calibratedThe gray value at the pixel is a function of the gray value at the pixel,、Respectively outputting voltage signals of the medium wave thermal infrared imager under the corresponding radiation scene; s12, according to the gain Offset ofCorrecting the intermediate