CN-115619638-B - Dangerous behavior identification method, system and related equipment based on super-resolution reconstruction

CN115619638BCN 115619638 BCN115619638 BCN 115619638BCN-115619638-B

Abstract

The invention discloses a dangerous behavior identification method, a dangerous behavior identification system and related equipment based on super-resolution reconstruction, wherein the method comprises the steps of obtaining continuous multi-frame images to be identified, performing super-resolution reconstruction on the continuous multi-frame images to be identified according to a trained super-resolution reconstruction model to obtain continuous multi-frame super-resolution images, wherein the resolution of the super-resolution images is higher than that of the images to be identified, performing dangerous behavior identification through a trained behavior identification model according to the super-resolution images, and obtaining target behavior categories corresponding to the images to be identified, wherein the target behavior categories comprise one or more of a plurality of preset behavior categories, and the preset behavior categories comprise no danger, abnormal protection and climbing. Compared with the prior art, the dangerous behavior recognition method and device are beneficial to improving accuracy of dangerous behavior recognition.

Inventors

YANG ZHILE
WU CHENGKE
LIU XIANGFEI
GUO YUANJUN
WANG YAO
FENG WEI

Assignees

深圳先进技术研究院

Dates

Publication Date: 20260512
Application Date: 20220927

Claims (8)

1. A dangerous behavior identification method based on super-resolution reconstruction, the method comprising: acquiring continuous multi-frame images to be identified; Performing super-resolution reconstruction on the continuous multi-frame image to be identified according to the trained super-resolution reconstruction model to obtain continuous multi-frame super-resolution images, wherein the resolution of the super-resolution images is higher than that of the image to be identified; Inputting the continuous multi-frame super-resolution images into a trained behavior recognition model; setting region weight values for each image sub-region of each frame of super-resolution image in the trained behavior recognition model, wherein the region weight values of the same image sub-region corresponding to each frame of super-resolution image in the continuous multi-frame super-resolution images are the same, the region weight values of the image sub-regions are determined according to the variation of pixel values of the image sub-regions in each frame of super-resolution image, the region weight values corresponding to the image sub-regions with large variation of pixel values are larger than the region weight values corresponding to the image sub-regions with small variation of pixel values; matching each behavior in the target behavior class with a preset dangerous behavior class table, acquiring all corresponding matched dangerous classes, taking the matched dangerous class with the highest dangerous class as the target dangerous class corresponding to the target behavior class, and carrying out dangerous alarm according to the target dangerous class.
2. The dangerous behavior recognition method based on super-resolution reconstruction according to claim 1, wherein the performing super-resolution reconstruction on the continuous multi-frame image to be recognized according to the trained super-resolution reconstruction model to obtain the continuous multi-frame super-resolution image comprises: Inputting each frame of the image to be identified into the trained super-resolution reconstruction model in sequence, and respectively carrying out super-resolution reconstruction on each frame of the image to be identified through the super-resolution reconstruction model so as to respectively acquire one frame of super-resolution image corresponding to each frame of the image to be identified; The image to be identified is an image obtained by shooting a target area through a camera, and the super-resolution reconstruction model carries out super-resolution reconstruction in an interpolation mode.
3. The dangerous behavior recognition method based on super-resolution reconstruction according to claim 2, wherein the super-resolution reconstruction model is trained in advance according to the following steps: Inputting a low-resolution training image in first training data into the super-resolution reconstruction model, and carrying out pixel point interpolation according to the super-resolution reconstruction model to generate a high-resolution reconstruction image corresponding to the low-resolution training image, wherein the first training data comprises a plurality of first training image groups, each first training image group comprises a low-resolution training image and a high-resolution training image, one low-resolution training image corresponds to one high-resolution training image, and the resolution of the low-resolution training image is lower than that of the high-resolution training image; And adjusting model parameters of the super-resolution reconstruction model according to the high-resolution training image corresponding to the low-resolution training image and the high-resolution reconstruction image corresponding to the low-resolution training image, and continuously executing the step of inputting the low-resolution training image in the first training data into the super-resolution reconstruction model until a first preset training condition is met, so as to obtain the trained super-resolution reconstruction model.
4. The dangerous behavior recognition method based on super-resolution reconstruction of claim 3, wherein the high-resolution training image is an image obtained by photographing with a high-definition camera, the low-resolution training image corresponding to the high-resolution training image is an image obtained by photographing with a low-definition camera a photographing region corresponding to the high-resolution training image, and the resolution of the high-definition camera is higher than that of the low-definition camera.
5. The dangerous behavior recognition method based on super-resolution reconstruction of claim 3, wherein the high-resolution training image is an image obtained by capturing with a high-definition camera, and the low-resolution training image corresponding to the high-resolution training image is an image obtained by processing the high-resolution training image based on a preset interference mode, wherein the preset interference mode includes at least one of jitter and blur processing.
6. The dangerous behavior recognition method based on super-resolution reconstruction according to claim 1, wherein the behavior recognition model is trained in advance according to the following steps: inputting continuous multi-frame training super-resolution images in second training data into the behavior recognition model, setting training area weight values for all training image subregions of each frame of training super-resolution images through the behavior recognition model, carrying out feature recognition on the continuous multi-frame training super-resolution images through the behavior recognition model according to the training area weight values to obtain training feature vectors, and carrying out dangerous behavior classification on the training feature vectors to obtain training target behavior categories corresponding to the training super-resolution images, wherein the second training data comprises a plurality of groups of second training image groups, and each group of second training image groups comprises a group of continuous multi-frame training super-resolution images and training actual dangerous behavior categories corresponding to the training super-resolution images; and adjusting model parameters of the behavior recognition model according to training target behavior categories corresponding to the continuous multi-frame training super-resolution images and training actual dangerous behavior categories corresponding to the continuous multi-frame images to be recognized, and continuously executing the step of inputting the continuous multi-frame training super-resolution images in the second training data into the behavior recognition model until a preset second training condition is met, so as to obtain the trained behavior recognition model.
7. A dangerous behavior recognition system based on super-resolution reconstruction, the system comprising: The image to be identified acquisition module is used for acquiring continuous multi-frame images to be identified; The super-resolution reconstruction module is used for performing super-resolution reconstruction on the continuous multi-frame image to be identified according to the trained super-resolution reconstruction model so as to obtain continuous multi-frame super-resolution images, wherein the resolution of the super-resolution images is higher than that of the image to be identified; The dangerous behavior recognition module is used for inputting the continuous multi-frame super-resolution images into a trained behavior recognition model; setting region weight values for each image sub-region of each frame of super-resolution image in the trained behavior recognition model, wherein the region weight values of the same image sub-region corresponding to each frame of super-resolution image in the continuous multi-frame super-resolution images are the same, the region weight values of the image sub-regions are determined according to the variation of pixel values of the image sub-regions in each frame of super-resolution image, the region weight values corresponding to the image sub-regions with large variation of pixel values are larger than the region weight values corresponding to the image sub-regions with small variation of pixel values; The system is further used for matching each behavior in the target behavior category with a preset dangerous behavior class table, obtaining all corresponding matched dangerous classes, taking the matched dangerous class with the highest dangerous class as the target dangerous class corresponding to the target behavior category, and carrying out dangerous alarm according to the target dangerous class.
8. An intelligent terminal, characterized in that it comprises a memory, a processor and a dangerous behavior recognition program based on super-resolution reconstruction stored on the memory and executable on the processor, the dangerous behavior recognition program based on super-resolution reconstruction realizing the steps of the dangerous behavior recognition method based on super-resolution reconstruction according to any one of claims 1-6 when being executed by the processor.

Description

Dangerous behavior identification method, system and related equipment based on super-resolution reconstruction Technical Field The invention relates to the technical field of image processing, in particular to a dangerous behavior identification method, a dangerous behavior identification system and related equipment based on super-resolution reconstruction. Background With the development of science and technology, especially the development of image acquisition devices such as cameras, schemes for monitoring dangerous behaviors based on images acquired by the cameras or objects to be monitored (such as processing personnel and equipment operators) are getting more and more attention. In the prior art, analysis is generally performed directly according to a frame of image acquired by a monitoring camera to determine whether dangerous behavior exists. The problem in the prior art is that the resolution of the monitoring camera is generally lower, the resolution of the image acquired by corresponding acquisition is also lower, and clear identification is difficult to realize when the image acquired by the acquisition is identified and analyzed, so that the accuracy of dangerous behavior identification is not improved. Accordingly, there is a need for improvement and development in the art. Disclosure of Invention The invention mainly aims to provide a dangerous behavior identification method, a dangerous behavior identification system and related equipment based on super-resolution reconstruction, and aims to solve the problem that in the prior art, a scheme for directly analyzing according to a frame of image acquired by a monitoring camera to determine whether dangerous behaviors exist is unfavorable for improving the accuracy of dangerous behavior identification. In order to achieve the above object, a first aspect of the present invention provides a dangerous behavior identification method based on super-resolution reconstruction, wherein the dangerous behavior identification method based on super-resolution reconstruction includes: acquiring continuous multi-frame images to be identified; Performing super-resolution reconstruction on the continuous multi-frame image to be identified according to the trained super-resolution reconstruction model to obtain continuous multi-frame super-resolution images, wherein the resolution of the super-resolution images is higher than that of the image to be identified; And carrying out dangerous behavior recognition through a trained behavior recognition model according to the super-resolution image, and obtaining a target behavior class corresponding to the image to be recognized, wherein the target behavior class comprises one or more of a plurality of preset behavior classes, and the plurality of preset behavior classes comprise no danger, abnormal protection tools and climbing. Optionally, the performing super-resolution reconstruction on the continuous multi-frame image to be identified according to the trained super-resolution reconstruction model to obtain continuous multi-frame super-resolution images includes: Inputting each frame of the image to be identified into the trained super-resolution reconstruction model in sequence, and respectively carrying out super-resolution reconstruction on each frame of the image to be identified through the super-resolution reconstruction model so as to respectively acquire one frame of super-resolution image corresponding to each frame of the image to be identified; the image to be identified is an image obtained by shooting a target area through a camera, and the super-resolution reconstruction model carries out super-resolution reconstruction in an interpolation mode. Optionally, the super-resolution reconstruction model is trained in advance according to the following steps: Inputting a low-resolution training image in first training data into the super-resolution reconstruction model, and carrying out pixel point interpolation according to the super-resolution reconstruction model to generate a high-resolution reconstruction image corresponding to the low-resolution training image, wherein the first training data comprises a plurality of first training image groups, each first training image group comprises a low-resolution training image and a high-resolution training image, one low-resolution training image corresponds to one high-resolution training image, and the resolution of the low-resolution training image is lower than that of the high-resolution training image; And adjusting model parameters of the super-resolution reconstruction model according to the high-resolution training image corresponding to the low-resolution training image and the high-resolution reconstruction image corresponding to the low-resolution training image, and continuously executing the step of inputting the low-resolution training image in the first training data into the super-resolution reconstruction model until a first preset train