CN-116823645-B - Industrial text image patching method based on feature recognition mechanism

CN116823645BCN 116823645 BCN116823645 BCN 116823645BCN-116823645-B

Abstract

The invention belongs to the field of text recognition and image restoration in an industrial production process, and discloses an industrial text image restoration method based on a feature recognition mechanism. The method provided by the invention solves the problem of data waste caused by the influence of the defective image on the plate blank number identification system in the steel plate production process. By the image patching method based on the feature recognition mechanism, the slab number recognition system can obtain high-quality training and reasoning data, so that the slab number recognition accuracy is greatly improved. And the improvement of the identification accuracy of the identification plate blank number is benefited, and steel production enterprises can realize accurate steel material tracking.

Inventors

WANG XUEFEI
WANG ZHAODONG
ZHANG TIAN

Assignees

东北大学
沈阳建筑大学

Dates

Publication Date: 20260505
Application Date: 20230606

Claims (7)

1. The industrial text image patching method based on the feature recognition is characterized by comprising the following steps of: Step 1, acquiring industrial text image data from a plate blank number recognition system, and preparing a defect training data set and a defect test data set, wherein content information and position information of a text area of the industrial text image data are added into a data tag; step 2, defect area positioning is carried out on the images to be repaired in the defect training data set, and a defect area in the images to be repaired is obtained and a difference matrix is generated through calculation; step 3, performing feature recognition once through the difference matrix obtained in the step 2, reducing the range of the defect area and filling the feature value V i ; Step 4, performing fusion operation on the feature map group F to obtain a feature map F m , and mapping F m into a feature vector sequence through a double-layer full-connection convolution layer ; Step 5, based on the two-way long-short term memory network, the feature vector sequence obtained in step 4 is compared Re-sequencing to obtain a new sequence, and decoding the new sequence through an attention mechanism to obtain a weight alpha and a characteristic value V h of each sequence element; And 6, performing secondary feature recognition on the feature value V h through the weight alpha, outputting text content, comparing the text content with the text region content information and the position information in the data tag in the step 1, and obtaining a reconstructed image after 2 times of feature recognition according to a comparison result or obtaining a reconstructed image after 2n times of feature recognition after n times of iteration steps 2 to 6, wherein the reconstructed image is an industrial text image after final repair.
2. The industrial text image inpainting method based on feature recognition according to claim 1, wherein the step 1 is specifically: And acquiring industrial text image data from a plate blank number recognition system, collecting defective text images which cannot be recognized by an OCR system, and simultaneously inputting a binary pixel mask to superimpose perfect text images, wherein the text area is different from the target repair area.
3. The method for repairing industrial text image based on feature recognition according to claim 2, wherein the step 2 specifically comprises: Obtaining defect area information in an image to be repaired in a data set by traversing g layers of local convolution layers, wherein the defect area information comprises the shape, the size, the position and the hole away from the boundary of the image to be repaired of an abnormal pixel area: ; Defining the difference matrix between the updated mask and the input mask as the defect area needing to be inferred in the defect image, and finally processing the defect area by a 1-layer fully-connected network layer and an activation function, wherein the processed difference matrix is used for primary feature recognition.
4. The industrial text image inpainting method based on feature recognition as set forth in claim 3, wherein said step 3 is specifically: The difference matrix is input into a knowledge consistency attention module to calculate the feature value of the mask area updated in the step 2 Then, repeating the step 2 and the step 3, filling the characteristic value v i calculated by the knowledge consistency attention module in a difference matrix, pooling to obtain a characteristic diagram f pool until the step 2 does not detect the defect area in the diagram any more, and marking the characteristic diagram group obtained by circulation as 。
5. The method for industrial text image inpainting based on feature recognition as set forth in claim 4, wherein said step 4 is specifically: The feature map group F is fused to obtain a feature map F m , and F m is mapped into a feature vector sequence through a double-layer full-connection convolution layer and recorded as Wherein each feature vector Arranged laterally from left to right at the location of the original signature F m .
6. The industrial text image inpainting method based on feature recognition according to claim 1, wherein the step 5 is specifically: Use of two-way long and short term memory network module pairs Calculating to obtain a new sequence Internal degree of association with H ; The calculation process is expressed as: wherein the alignment factor is obtained by scoring each element in the new sequence H by an attention function and then normalizing A weighted sum of the new sequences H; After the new sequence H and the internal association degree of the new sequence H are obtained, the attention function is utilized to carry out secondary decoding on the new sequence H, and the characteristic value V h of each sequence element H is obtained.
7. The method for industrial text image inpainting based on feature recognition as set forth in claim 6, wherein said step 6 is specifically: Outputting the characteristic value V h obtained in the step 5 as text content according to the weight alpha, comparing the text content with text region content information and text information in the data tag in the step 1, and carrying out self-adaptive fusion on the low-layer high-resolution characteristic and the high-layer semantic characteristic of the characteristic image group F to obtain a reconstructed image when the text content and the text information are equal, wherein the self-adaptive fusion uses an FPN algorithm, carrying out n times of iteration on the steps 2 to 6 to obtain the reconstructed image when the text content and the text region content information are unequal, and iterating to a loss function Below a set threshold, wherein the required loss function is iterated The expression is as follows: ; Wherein λ is a learnable parameter, style penalty Loss of perception And predicting loss The respective expressions are as follows: ; Wherein, the For the true value in the step 1 tag, A vector representing all parameters in the feature recognition mechanism, The sequence of feature vectors, Representing the i-th characteristic diagram after pooling, H, W, C represents the length and width and channel number of the input picture respectively.

Description

Industrial text image patching method based on feature recognition mechanism Technical Field The invention relates to the field of text recognition and image restoration in an industrial production process, in particular to an industrial text image restoration method based on a feature recognition mechanism. Background In steel production, the slab number is a unique identifier that is tracked throughout the production process. The identification of the ID of the steel plate or the plate blank number is the basis of intelligent tracking of the steel, and the information-based material tracking of large-scale steel enterprises depends on a plate blank number identification system. In a similar text recognition task, researchers have developed many sophisticated methods that can be categorized into traditional methods based on character features and recognition methods based on convolutional neural networks. The latter has good application effect, but depends on a large amount of quality data. However, image data of a character recognition task is easily damaged due to the limitation of printing performance of a jet printing apparatus and the interference of a production environment, resulting in a large amount of data waste, and it is difficult to obtain sufficient quality data. For example, specular reflection light causes a character to be covered on the surface of a steel sheet, and when the character content is covered, the board number recognition effect is poor. So far, in industrial production, especially in the steel production industry, there has been no study on text image repair. The defective image data in the current production environment greatly reduces the image quality of an input plate blank number identification system, increases the identification difficulty, reduces the plate blank number identification accuracy, and is difficult to realize accurate tracking of steel. Disclosure of Invention In view of the foregoing, it is necessary to provide an industrial text image repair method based on a feature recognition mechanism, which can repair defective text images and provide high-quality data for a plate blank number recognition system. The technical scheme of the invention is as follows, an industrial text image patching method based on a feature recognition mechanism comprises the following steps: Step 1, acquiring industrial text image data from a plate blank number recognition system, and preparing a defect training data set and a defect test data set, wherein content information and position information of a text area of the industrial text image data are added into a data tag; step 2, defect area positioning is carried out on the images to be repaired in the defect training data set, and a defect area in the images to be repaired is obtained and a difference matrix is generated through calculation; step 3, performing feature recognition once through the difference matrix obtained in the step 2, reducing the range of the defect area and filling the feature value V i; Step 4, performing fusion operation on the feature map group F to obtain a feature map F m, and mapping F m into a feature vector sequence through a double-layer full-connection convolution layer Step 5, based on the two-way long-short term memory network, the feature vector sequence obtained in step 4 is comparedRe-sequencing to obtain a new sequence, and decoding the new sequence through an attention mechanism to obtain a weight alpha and a characteristic value V h of each sequence element; And 6, performing secondary feature recognition on the feature value V h through the weight alpha, outputting text content, comparing the text content with the text region content information and the position information in the data tag in the step 1, and obtaining a reconstructed image after 2 times of feature recognition according to a comparison result or obtaining a reconstructed image after 2n times of feature recognition after n times of iteration steps 2 to 6, wherein the reconstructed image is an industrial text image after final repair. The step 1 specifically comprises the following steps: And acquiring industrial text image data from a plate blank number recognition system, collecting defective text images which cannot be recognized by an OCR system, and simultaneously inputting a binary pixel mask to superimpose perfect text images, wherein the text area is different from the target repair area. The defective text image which cannot be recognized by the OCR system is the defective text image which cannot be recognized by the plate blank number recognition system due to over-strong illumination, partial shielding and the like, the defective text image is I da epsilon R, the perfect text image is I gt epsilon R, the binary pixel mask is M epsilon {0,1}, and the superimposed image is I in epsilon R, wherein I in＝Igt epsilon M. And randomly fusing the I in and the I da to obtain a defect training data set and a defect testing