CN-115359393-B - Image screen display abnormality identification method based on weak supervision learning

CN115359393BCN 115359393 BCN115359393 BCN 115359393BCN-115359393-B

Abstract

The invention relates to the field of computer vision and machine learning, and discloses an image screen pattern abnormality identification method based on weak supervision learning, which comprises the steps of initializing parameters in a screen pattern abnormality identification network and screen pattern feature generation processing; sampling from video data to obtain a first image set, performing screen pattern feature generation processing on the first image set to obtain a second image set and tag data, and performing weak supervision training on the basis of the first image set and the second image set to obtain a screen pattern recognition model. The invention uses a data simulation mode, reduces the acquisition difficulty of training samples for the video pattern recognition of the monitoring, provides a great convenience condition for the application of a machine learning method in pattern recognition, reduces the requirement of a model on the training data amount by using a weak supervision mode, ensures the accuracy of model recognition, and solves the actual requirement of abnormal recognition of the video pattern recognition of the monitoring by using an optimal cost performance scheme.

Inventors

NIE HUI
CHEN LI
YANG XIAOBO
LI JUN

Assignees

武汉东智科技股份有限公司

Dates

Publication Date: 20260505
Application Date: 20220816

Claims (7)

1. The image screen display abnormality identification method based on weak supervision learning is characterized by comprising the following steps of: s1, initializing parameters in a screen pattern abnormality identification network and screen pattern feature generation processing; s2, sampling from video data to obtain a first image set, and performing screen-pattern feature generation processing on the first image set to obtain a second image set and tag data; s3, performing weak supervision training based on the first image set and the second image set to obtain a pattern recognition model; the step S2 specifically includes: s21, collecting video data, and randomly selecting a certain number of video files as video data to be processed; S22, each video file in the video data to be processed is processed at time intervals Image sampling is carried out to obtain a corresponding video image set, and a first image set is constructed according to the obtained video image set; S23, processing the first image set according to a pattern space domain generation method or a pattern time domain generation method to obtain an intermediate pattern image; S24, processing the screen-pattern area of the middle screen-pattern image based on a random patch algorithm to obtain a second image set; s25, automatically labeling the first image set and the second image set based on data characteristics to obtain the tag data; the method for generating the pattern space domain specifically comprises the following steps: let the image to be processed be I, its height is H, carry out the flower screen processing to I: Wherein the method comprises the steps of Representing pixel values at the original non-panel image coordinates (x, y), Representing the pixel values at the coordinates (x, y) of the intermediate screen image obtained by the processing, Representing a method of vertical transfer of color patches, Representing a method of compression of the tonal region, Take a value between 0 and 1; The pattern time domain generation specifically comprises the following steps: acquiring non-splash screen images 、 Wherein the image And Centralizing adjacent images for the video image; For images 、 Graying to obtain corresponding gray image 、 ; Calculating a difference matrix between gray scale images ; Obtaining a panel of the splash screen area according to the difference matrix and the set threshold value T of the splash screen area ; According to the flower screen mask M and the image Obtaining a target screen image : The pattern screen area is 。
2. The method for identifying the abnormal image screen pattern based on weak supervised learning as set forth in claim 1, wherein the color block vertical transfer method specifically comprises: 。
3. The method for identifying abnormal image patterns based on weak supervised learning according to claim 1, wherein the method for compressing the tone area is specifically as follows: Wherein, the 、、 Representation of Is used for the three channel values of RGB, Representing the compression coefficient.
4. The method for identifying the abnormal image screen based on the weak supervised learning as set forth in claim 1, wherein the random plaque algorithm specifically comprises the following steps: setting the plaque size P and the plaque quantity = Wherein H, W is the height and width of the middle screen image, and m is a sparse coefficient; generating a plaque matrix sequence: Wherein, the Is that A matrix of all 1's elements, Is that Random integers in the range; Generating a random coordinate sequence with the number of coordinates of : Initializing plaque mask Wherein Initial as Matrix of elements of all 0' s Adding the plaque matrix sequence into the plaque mask by taking the coordinates in the coordinate sequence as the coordinates of the upper left corner of the plaque in the plaque mask, and updating 。
5. The method for identifying the abnormal image screen based on the weak supervised learning as set forth in claim 4, wherein the intermediate screen image is processed according to a random patch algorithm, and the calculation formula is as follows: Wherein, the In order to obtain the final image of the screen, And the middle screen image is obtained.
6. The method for identifying abnormal image patterns based on weak supervised learning as set forth in claim 1, wherein the step S3 specifically includes: S31, constructing a training data set based on the first image set, the second image set and corresponding label data; S32, sending the training data set into a screen recognition network for training, verifying the obtained screen recognition model, and obtaining a verification index; s33, adjusting the adjustable parameters of the screen generating process in the step S2 according to the verification index; s34, obtaining a new first image set and a new second image set according to the step S2; S35, repeating the steps S31 to S34 until the model reaches the required index.
7. The method for identifying abnormal image screen according to claim 6, wherein the adjusting adjustable parameters specifically comprises: adjusting the value range of the parameter h in the pattern space domain generation method or adjusting the time interval of image acquisition in the pattern time domain generation method 。

Description

Image screen display abnormality identification method based on weak supervision learning Technical Field The invention relates to the field of computer vision and machine learning, in particular to an image screen anomaly identification method based on weak supervision learning. Background Because of factors such as network transmission performance limitation, functional defects of encoding and decoding equipment, or misoperation of personnel, some frames of video can be damaged in the transmission and storage processes, and the video can be displayed as an image abnormality (screen display) after decoding and displaying. The image anomalies can affect the use efficiency of the public safety video monitoring system, jeopardize the construction of a social security and control system, and seriously even affect the public security of the society and the forensic work of important criminal cases. Weak supervised learning is a branch of the machine learning field that uses limited, noisy, or inaccurately annotated data for training of model parameters, as compared to traditional supervised learning. In the practice of the splash screen image recognition, if fully supervised machine learning is desired, a large amount of tagged splash screen image data is required. Such data acquisition effort often requires high labor costs, resulting in fully supervised panel recognition training often being difficult to achieve. It is hoped that a model achieving practical and accurate can be trained by using a small amount of sample data and low cost, and weak supervision learning is the current preferred scheme. Disclosure of Invention Aiming at the problems of difficult data annotation, multiple data types, miscellaneous data annotation and the like in video screen-display detection, the invention provides an image screen-display anomaly identification method based on weak supervised learning. In order to solve the technical problems, the invention provides an image screen display abnormality identification method based on weak supervision learning, which comprises the following steps: s1, initializing parameters in a screen pattern abnormality identification network and screen pattern feature generation processing; s2, sampling from video data to obtain a first image set, and performing screen-pattern feature generation processing on the first image set to obtain a second image set and tag data; s3, performing weak supervision training based on the first image set and the second image set to obtain a pattern recognition model; Further, the step S2 specifically includes: s21, collecting video data, and randomly selecting a certain number of video files as video data to be processed; S22, performing image sampling on each video file in the video data to be processed at a time interval t v to obtain a corresponding video image set, and constructing a first image set according to the obtained video image set; S23, processing the first image set according to a pattern space domain generation method or a pattern time domain generation method to obtain an intermediate pattern image; S24, processing the screen area of the middle screen image based on a random patch algorithm to obtain a second image set. And S25, automatically labeling the first image set and the second image set based on the data characteristics to obtain the tag data. Further, the method for generating the flower screen airspace specifically comprises the following steps: let the image to be processed be I, its height be H. Carrying out screen display processing on the I: Wherein I (x, y) represents a pixel value at an original non-screen image coordinate (x, y), I * (x, y) represents a pixel value at an intermediate screen image coordinate (x, y) obtained by processing, f 1 represents a color block vertical transfer method, f 2 represents a tone region compression method, and k h takes a value between 0 and 1. Further, the color block vertical transfer method specifically comprises the following steps: f1(I(x,y),h)=I(h,y) Further, the tone region compression method specifically includes: where r xy、gxy、bxy represents the RGB three channel value of I (x, y) and p represents the compression coefficient. Further, the generating of the splash screen time domain specifically includes: Acquiring a non-screen image I 1、I2, wherein images I 1 and I 2 are adjacent images in the video image set; Graying the image I 1、I2 to obtain a corresponding gray image G 1、G2; Calculating a difference matrix delta between gray images; Δ=|G2-G1| obtaining a screen area mask M according to the difference matrix and a set screen area threshold T; obtaining a target flower screen image I * according to the flower screen mask M and the image I 2: The screen region is { (x, y) |m (x, y) =1 }. Further, the random plaque algorithm specifically comprises the following steps: setting the plaque size P and the plaque quantity Wherein H, W is the height and width of the middle screen image, an