CN-122024140-A - LED screen content safety early warning method and system based on multi-mode fusion

CN122024140ACN 122024140 ACN122024140 ACN 122024140ACN-122024140-A

Abstract

The invention discloses a multi-mode fusion-based LED screen content safety pre-warning method and system. The method comprises the steps of cooperatively collecting video, spectrum and attitude information through a plurality of sensors, realizing geometric correction and dynamic image stabilization of a shot picture based on physical light field modeling, dynamically fusing multi-mode features such as contours, colors, motions and the like by utilizing a dual-channel residual error network, adopting a hierarchical decision mechanism, wherein the primary alarm response time is 35 milliseconds, the depth verification process is completed within 200 milliseconds, and linking a hardware execution module to complete closed loop response. The problems of high false alarm rate and poor real-time performance caused by illumination rapid change, camera shake and visual angle offset in an outdoor environment are solved, the contour omission ratio is reduced from 45% to 8% in a scene with low contrast in foggy days, the color distinguishing degree is improved by 3 times in a strong light environment, and the hardware response delay is reduced by 67%.

Inventors

LI LONG
LIU QINYUAN
LI MING

Assignees

上海尔控科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260202

Claims (10)

1. The LED screen content safety early warning method and system based on multi-mode fusion are applied to an LED screen monitoring system, and the system comprises a multi-sensor collaborative acquisition module, an image processing unit and a hardware execution module, and is characterized in that the method comprises the following steps executed by the image processing unit: Step S110, synchronously acquiring an original video stream, a real-time shooting video stream, environmental spectrum data and camera motion attitude data of an LED screen to be monitored through a multi-sensor collaborative acquisition module; step S120, based on the camera motion attitude data and the environmental spectrum data, performing preprocessing of geometric correction, dynamic blur suppression and physical light field compensation on the real-time shooting video stream to generate a stabilized video frame; Step S130, performing geometrical outline feature extraction, dynamic color histogram feature extraction and optical flow motion trail feature extraction on the original video stream and the stabilized video frame in parallel, and performing dynamic weight fusion on the extracted multi-mode features by using a dual-channel residual error network to generate fusion feature vectors; Step S140, executing a confidence-driven hierarchical security decision based on the fusion feature vector, wherein the hierarchical security decision includes primary alarm triggering, depth verification confirmation and final treatment instruction generation; step S150, according to the final disposal instruction, the hardware execution module is linked to execute alarm information release, screen brightness adjustment or power cut-off operation.
2. The LED screen content safety pre-warning method according to claim 1 is characterized in that the geometric correction is specifically that perspective transformation is carried out on each frame of image in a real-time shooting video stream according to a homography transformation matrix between a pre-calibrated camera and an LED screen, a picture under an inclined view angle is corrected to a virtual view angle opposite to the screen, the homography transformation matrix is a3×3 matrix, the homography transformation matrix is obtained in advance through a camera calibration method, and the perspective transformation is executed through a hardware acceleration unit.
3. The method for pre-warning the content of an LED screen according to claim 1, wherein the physical light field compensation is specifically implemented by using The method comprises the steps of theoretically decomposing an image into a reflection component and an illumination component, combining the environmental spectrum data and a pre-constructed screen material reflectivity database by the estimation of the illumination component, and carrying out gamma correction on the illumination component according to the incident light angle and the sensor spectrum response in the compensation process, wherein the correction coefficient is determined by the environmental light intensity and the screen material.
4. The LED screen content safety pre-warning method according to claim 1, wherein the geometric outline feature extraction adopts an improved Canny-Hough algorithm, and the improvement is that a self-adaptive double-threshold setting is performed, wherein a histogram of image gradient amplitude values is calculated, the gradient values are arranged in a descending order, a front part quantile is taken as a high threshold, and a rear part quantile is taken as a low threshold.
5. The LED screen content safety precaution method according to claim 1, wherein the dynamic color histogram feature extraction is performed in HSV color space, the tone H channel is non-uniformly quantized, fine interval quantization is adopted in a safety precaution key color zone, regular interval quantization is adopted in other areas, and the quantized two-dimensional histogram is calculated, wherein the dimensions are tone H and saturation S.
6. The LED screen content safety pre-warning method according to claim 1, wherein the dual-channel residual error network comprises two parallel feature extraction channels and a feature fusion module, wherein a first channel is used for processing original video frame features, a second channel is used for processing preprocessed stabilized video frame features and comprises a deformable convolution layer, and the feature fusion module dynamically generates weight coefficients for output feature graphs of the two channels by adopting an attention mechanism And (3) with The fusion formula is 。
7. The method of claim 1, wherein the hierarchical security decision is specifically that if the similarity of the fusion feature vectors of consecutive frames is lower than a preset threshold, a primary alarm is triggered and executed Response, if the trigger is accumulated for a plurality of times within the time window If the alarm is given, a depth verification process is started, the depth network is called to calculate the mahalanobis distance of the feature vector, and if the mahalanobis distance is larger than a threshold value, the content abnormality is confirmed and generated The instruction is finally handled.
8. The utility model provides a LED screen content safety precaution system based on multimode dynamic fusion which characterized in that, this system includes: A processor; a memory storing computer executable instructions; The multi-sensor collaborative acquisition module comprises an industrial camera, a multi-spectrum ambient light sensor array and an inertia measurement unit, and is used for synchronously acquiring an original video stream of an LED screen to be monitored, a real-time shooting video stream, ambient spectrum data and camera motion attitude data; The hardware execution linkage module comprises an alarm issuing unit, a PWM dimming unit and a magnetic latching relay power-off unit and is used for executing alarm information issuing, screen brightness adjustment or power-off operation according to the treatment instruction; the processor, when executing the computer-executable instructions, implements: based on the camera motion attitude data and the environmental spectrum data, performing preprocessing of geometric correction, dynamic blur suppression and physical light field compensation on the real-time shooting video stream to generate a stabilized video frame; Performing geometrical outline feature extraction, dynamic color histogram feature extraction and optical flow motion track feature extraction on the original video stream and the stabilized video frame in parallel, and performing dynamic weight fusion on the extracted multi-mode features by using a dual-channel residual error network to generate fusion feature vectors; Based on the fused feature vector, confidence-driven hierarchical security decisions are performed, including primary alarm triggering, depth validation, and final treatment instruction generation.
9. The LED screen content safety precaution system of claim 8, wherein the multispectral ambient light sensor array comprises 5 spatial sampling points, each sampling point is provided with a 6-channel spectral sensor, the spectral response range is 400nm to 1050nm, and the inertial measurement unit reads the three-axis angular velocity and three-axis acceleration data of the camera in real time at a sampling frequency not lower than 200 Hz.
10. The LED screen content safety precaution system of claim 8, wherein the alarm issuing unit automatically sends an alarm message and a telephone notification to a preset responsible person through the cloud communication module within a preset time after confirming the abnormality, and simultaneously generates SHA-256 hash values from event log data and uploads the SHA-256 hash values to the blockchain for certification, and the magnetic latching relay power-off unit completes physical power-off within 100ms after receiving a Level 3 instruction through the hardware watchdog circuit.

Description

LED screen content safety early warning method and system based on multi-mode fusion Technical Field The invention belongs to the technical field of image data processing, and particularly relates to a multi-mode fusion-based LED screen content safety early warning method and system. Background With the rapid development of smart cities and digital media infrastructures, the carrier form of outdoor information transmission is continuously upgraded, wherein an outdoor LED screen is used as a core public information display and commercial advertisement release platform, and the content safety and the operation reliability of the outdoor LED screen are increasingly becoming key guarantees of public safety and commercial operation. The method ensures the accuracy and controllability of the display content of the screen, and has important significance for maintaining public order and commercial interests. Currently, content security monitoring of outdoor LED screens relies primarily on computer vision techniques. The existing scheme mostly adopts a core thought based on image comparison, and a specific technical path comprises two types of methods, namely pixel-level RGB difference detection, such as the scheme disclosed by CN109615022A, for judging whether the content is abnormal or not by comparing the real-time shot image with standard content pixel by pixel color, and feature matching mechanisms, such as a display monitoring system proposed by US10244563B2, for carrying out state judgment by extracting and matching the features of the contour, texture and the like of the screen content. The prior art scheme mainly has the following defects: 1. The existing method has single perception dimension, lacks accurate modeling on physical characteristics of the environment, and causes high false alarm rate and large chromatic aberration under severe illumination change and weather interference. The fundamental reasons are that on one hand, algorithms based on a single visual mode, such as pure RGB pixels or geometric features, are difficult to resist coupling interference of complex light fields and physical shielding, feature characterization is fragile, and on the other hand, a simple and non-physical illumination compensation model cannot consider key factors such as reflectivity of screen materials, incident angles of ambient light and the like, so that color judgment is seriously distorted. 2. The existing scheme has a significant delay bottleneck, and is difficult to meet the high real-time early warning requirement. The method is mainly characterized in that the traditional pure software image stabilizing algorithm is high in computational complexity, processing delay is usually more than 30ms and becomes a main delay source of the whole detection and response chain, meanwhile, when a camera or a screen moves at high speed, image blurring is serious, blurring radius is often more than 5 pixels, and difficulty and time consumption of feature extraction and matching are further increased. 3. The decision mechanism in the prior art is stiff and lacks a hierarchical verification process. The single threshold judgment is adopted generally, and the highest-level response is triggered immediately after the abnormality is detected, so that unnecessary service interruption is easy to be caused by false alarm, and mild early warning cannot be carried out at the early stage of the real abnormality, and a balance mechanism between speed and precision is lacked. 4. The existing scheme focuses on the algorithm level and lacks an end-to-end closed loop response and credibility guarantee mechanism. The method has the advantages that the response action depends on a general hardware module, the delay is high, the reliability is doubtful, all operation logs lack of tamper-proof evidence, when a security event occurs, the traceability and non-repudiation of the operation behavior cannot be ensured, and the severe requirements of the public security field on responsibility identification are difficult to meet. Disclosure of Invention In order to overcome the defects of the prior art, the invention provides the LED screen content safety early warning method and system based on multi-mode fusion, which are characterized in that a robust perception front end is constructed through multi-sensor collaborative collection and environment physical modeling, a dynamic weight fusion and grading decision mechanism is adopted to realize high-precision and low-delay safety judgment, and finally, a linkage hardware execution module forms a reliable disposal closed loop, so that the problems that the prior art cannot meet the real-time reliable early warning requirement due to high false alarm rate, invalid feature matching and large response delay caused by outdoor environment dynamic interference are solved. In order to achieve the above purpose, the present invention provides the following technical solutions: The LED screen content safety