Search

CN-121600652-B - Day and night double-mode smoke detection method and device

CN121600652BCN 121600652 BCN121600652 BCN 121600652BCN-121600652-B

Abstract

The invention discloses a day and night dual-mode smoke detection method and device, wherein the method comprises the steps of determining an acquisition mode according to ambient illumination conditions; the method comprises the steps of acquiring a color image sequence acquired by a visible light camera, inputting the color image sequence acquired by the visible light camera into a first smoke and fire detection model to judge whether a fire event exists or not, inputting a gray image sequence acquired by a near infrared light supplementing camera into the first smoke and fire detection model if the acquisition mode is a daytime mode, inputting an acquired second detection result into a physical perception convolution time sequence model, and generating a time sequence correction coefficient of each frame of gray image by the physical perception convolution time sequence model to screen the gray image if the self-adaptive fractional Fourier transform characteristic of a brightness characteristic sequence of a suspected smoke and fire region of each frame of gray image in an acquired third detection result accords with the smoke and fire spectrum mode and the edge jitter quantity meets the smoke and fire dynamic characteristic. The invention can adapt to all-weather environment, and has high robustness and low false alarm rate.

Inventors

  • ZHANG HAIBIN
  • MENG FEI
  • He Bichang

Assignees

  • 浙江清华长三角研究院

Dates

Publication Date
20260508
Application Date
20260122

Claims (11)

  1. 1. A diurnal dual-mode smoke detection method, comprising: Determining an acquisition mode according to the ambient illumination condition at the current moment; If the acquisition mode is a daytime mode, inputting a color image sequence acquired by a visible light camera into a first smoke and fire detection model to obtain a first detection result, and judging whether a fire event exists according to the first detection result, wherein the first smoke and fire detection model is of a convolutional neural network structure; If the acquisition mode is a night mode, a gray image sequence acquired by a near infrared light supplementing camera is input into a first smoke detection model to obtain a second detection result, the second detection result is input into a physical perception convolution time sequence model to obtain a third detection result, and if the self-adaptive fractional Fourier transform characteristic of the brightness characteristic sequence of a suspected smoke area of each frame of gray image in the third detection result accords with the smoke spectrum mode and the edge jitter quantity meets the smoke dynamic characteristic, a fire event is determined to exist; the physical perception convolution time sequence model combines the space feature extraction capacity of the convolution neural network and the time sequence modeling advantage of LSTM; The physical perception convolution time sequence model comprises a linking module, a calibration module, a fusion module, a screening module and a third detection result, wherein the linking module is used for cutting out suspected smoke and fire areas of each frame of gray image based on the boundary frames of each frame of gray image and constructing a brightness characteristic sequence of a continuous frame; The calibration module is used for calculating a brightness characteristic change matrix according to a brightness characteristic matrix of a current time frame and a brightness characteristic matrix of a previous time frame in a brightness characteristic sequence, calculating a brightness characteristic change spectrum according to the brightness characteristic change matrix, calculating a main frequency component of the brightness characteristic change spectrum, calculating a pyrotechnic physical consistency score according to the main frequency component and a preset frequency band range of pyrotechnic flickering, and generating a time sequence correction coefficient of each frame of gray image through gating modulation by utilizing the pyrotechnic physical consistency score.
  2. 2. The method of claim 1, wherein the first detection result comprises a bounding box and a confidence level for each frame of color image; judging whether a fire event exists according to the first detection result, including: And determining that a fire event exists when the confidence level of the continuous preset number of the color images exceeds a first threshold value and the shape and the size of the boundary frame of the continuous preset number of the color images meet preset requirements.
  3. 3. The method of claim 1, wherein the calibration module comprises: the physical frequency characteristic extraction module is used for generating a smoke and fire physical consistency score according to the brightness characteristic sequence; The gating modulation module is used for obtaining a basic input door and a basic forgetting door without physical constraint based on convolution operation, and modulating the physical constraint of the basic input door and the basic forgetting door by utilizing the physical consistency score of smoke and fire to obtain a modulation input door and a modulation forgetting door; The cell state updating module is used for updating the cell state of the current time frame according to the modulation input gate, the modulation forgetting gate, the hidden state of the previous time frame, the brightness characteristic matrix of the current time frame and the cell state of the previous time frame; The hidden state output module is used for calculating an output gate according to the brightness characteristic matrix of the current time frame and the hidden state of the previous time frame, and calculating the hidden state of the current time frame according to the output gate and the cell state of the current time frame; And the mapping module is used for mapping the hidden state of the current time frame into a time sequence correction coefficient of the current time frame.
  4. 4. The method as recited in claim 1, further comprising: extracting deep features of a brightness feature sequence of a suspected smoke and fire region of each frame of gray image in the second detection result through a one-dimensional convolution time sequence coding network to obtain a time sequence feature vector; calculating the optimal order of the fractional Fourier transform based on the time sequence feature vector; performing self-adaptive fractional Fourier transform on the brightness characteristic sequence based on the optimal order to obtain a frequency domain signal; Extracting self-adaptive fractional Fourier transform characteristics from the frequency domain signals, wherein the self-adaptive fractional Fourier transform characteristics comprise fractional energy, fractional spectrum entropy, spectrum bandwidth and main peak value; And comparing the self-adaptive fractional Fourier transform characteristic with a corresponding threshold range, and judging whether the self-adaptive fractional Fourier transform characteristic accords with a pyrotechnic frequency spectrum mode.
  5. 5. The method as recited in claim 1, further comprising: Extracting an edge intensity matrix of a suspected smoke and fire area of any two continuous frames of gray images, and calculating an edge differential matrix based on a Sobel operator; Screening the edge differential matrix according to a preset minimum differential threshold value to obtain a screened edge differential matrix; Performing global statistics on the screened edge differential matrix to obtain the edge jitter amount; and judging whether the edge shaking quantity meets the pyrotechnic dynamic characteristics according to a preset shaking threshold value.
  6. 6. The method as recited in claim 1, further comprising: Obtaining an original real sample set, a fireless night scene image and a firework template image, wherein the original real sample set comprises a firework color image sample and a gray image sample; according to the original real sample set, the fireless night scene image and the firework template image, obtaining an expanded sample set through image enhancement and synthesis; training a first detection model based on the extended sample set to obtain a trained first detection model; training a physical perception convolution time sequence model based on the trained first detection model to obtain a trained physical perception convolution time sequence model; And carrying out series connection and then combined training on the trained first detection model and the trained physical perception convolution time sequence model.
  7. 7. The method of claim 6, wherein enhancing and synthesizing the single frame image in the original real sample set from the fireless night scene image and the pyrotechnic template image to obtain the extended sample set comprises: enhancing the single frame image in the original real sample set to obtain an enhanced image, wherein the enhancement comprises at least one of geometric transformation enhancement, geometric transformation enhancement and infrared imitation enhancement; Adding dynamic brightness disturbance to the firework template image, and then performing image synthesis with the fireless night scene image to obtain a multi-scene fusion generated image and marking; Based on a preset text prompt and image constraint, an open source diffusion model is used, and a diffusion model generated image is obtained and marked; screening a representative firework image from a single frame image in an original real sample set, generating a scene text label, combining the single frame image and the scene text label, generating a continuous frame sequence, performing time sequence smooth correction on the continuous frame sequence to obtain a synthesized firework video, and performing frame disassembly on the synthesized firework video to obtain a firework video synthesized image; and combining and de-duplicating the original real sample set, the enhanced image, the multi-scene fusion generated image, the diffusion model generated image and the pyrotechnic video synthesized image to form an extended sample set.
  8. 8. A diurnal dual-mode smoke detection apparatus, comprising: the acquisition mode determining module is used for determining an acquisition mode according to the ambient illumination condition at the current moment; The system comprises a daytime detection module, a first smoke detection module and a second smoke detection module, wherein the daytime detection module is used for inputting a color image sequence acquired by a visible light camera into the first smoke detection module to obtain a first detection result, and judging whether a fire event exists according to the first detection result, wherein the first smoke detection module is of a convolutional neural network structure; The night detection module is used for inputting a gray image sequence acquired by the near infrared light supplementing camera into the first smoke and fire detection model to obtain a second detection result, and inputting the second detection result into the physical perception convolution time sequence model to obtain a third detection result if the acquisition mode is the night mode; if the self-adaptive fractional Fourier transform characteristic of the brightness characteristic sequence of the suspected smoke and fire area of each frame of gray image in the third detection result accords with the smoke and fire spectrum mode and the edge jitter quantity meets the smoke and fire dynamic characteristic, determining that a fire event exists; the physical perception convolution time sequence model combines the space feature extraction capacity of the convolution neural network and the time sequence modeling advantage of LSTM; The physical perception convolution time sequence model comprises a linking module, a calibration module, a fusion module, a screening module and a third detection result, wherein the linking module is used for cutting out suspected smoke and fire areas of each frame of gray image based on the boundary frames of each frame of gray image and constructing a brightness characteristic sequence of a continuous frame; The calibration module is used for calculating a brightness characteristic change matrix according to a brightness characteristic matrix of a current time frame and a brightness characteristic matrix of a previous time frame in a brightness characteristic sequence, calculating a brightness characteristic change spectrum according to the brightness characteristic change matrix, calculating a main frequency component of the brightness characteristic change spectrum, calculating a pyrotechnic physical consistency score according to the main frequency component and a preset frequency band range of pyrotechnic flickering, and generating a time sequence correction coefficient of each frame of gray image through gating modulation by utilizing the pyrotechnic physical consistency score.
  9. 9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 7 when executing the computer program.
  10. 10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any of claims 1 to 7.
  11. 11. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the method of any of claims 1 to 7.

Description

Day and night double-mode smoke detection method and device Technical Field The invention relates to the technical field of safe production data processing and monitoring, in particular to a day and night dual-mode smoke and fire detection method and device. Background This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section. The existing video-based smoke detection algorithm has a plurality of defects in a complex environment. Firstly, the traditional visible light smoke detection algorithm relies on smoke color and motion characteristics, and under the condition of low illumination at night, the detection performance is rapidly deteriorated due to the obvious reduction of imaging quality. Some systems use infrared or thermal imaging devices to cope with night scenes, but such dedicated devices are costly, complex to deploy and maintain, and limit their wide application. In night monitoring, the visible light camera usually needs to image by means of infrared light supplement, and the obtained gray level image lacks color characteristics, so that bright light sources such as car lights and fluorescent lamps are easily misjudged as smoke and fire, and misinformation is caused. Finally, the deep learning smoke and fire detection model relies on large-scale high-quality data training, and in reality, night fire images are scarce, and the coverage of a public data set is insufficient, so that the generalization capability of the existing model under a day-night alternating scene is weaker. Therefore, development of a new method for detecting smoke and fire, which is adaptable to all-weather environments, has high robustness and low false alarm rate, is needed. Disclosure of Invention The embodiment of the invention provides a day and night dual-mode smoke detection method which can adapt to all-weather environments, has high robustness and low false alarm rate and comprises the following steps: Determining an acquisition mode according to the ambient illumination condition at the current moment; If the acquisition mode is a daytime mode, inputting a color image sequence acquired by a visible light camera into a first smoke and fire detection model to obtain a first detection result, and judging whether a fire event exists according to the first detection result, wherein the first smoke and fire detection model is of a convolutional neural network structure; The method comprises the steps of acquiring a gray image sequence by a near infrared light supplementing camera, inputting the gray image sequence acquired by the near infrared light supplementing camera into a first smoke detection model to obtain a second detection result, inputting the second detection result into a physical perception convolution time sequence model to obtain a third detection result, and screening the gray image by the time sequence correction coefficient to obtain a third detection result if the self-adaptive fractional Fourier transform characteristic of the brightness characteristic sequence of the suspected smoke area of each frame of gray image in the third detection result accords with the smoke spectrum mode and the edge jitter quantity meets the smoke dynamic characteristic, and determining that a fire event exists. The embodiment of the invention provides a day and night dual-mode smoke detection device which can adapt to all-weather environments and has high robustness and low false alarm rate, and the device comprises: the acquisition mode determining module is used for determining an acquisition mode according to the ambient illumination condition at the current moment; The system comprises a daytime detection module, a first smoke detection module and a second smoke detection module, wherein the daytime detection module is used for inputting a color image sequence acquired by a visible light camera into the first smoke detection module to obtain a first detection result, and judging whether a fire event exists according to the first detection result, wherein the first smoke detection module is of a convolutional neural network structure; The night detection module is used for inputting a gray image sequence acquired by the near infrared light supplementing camera into the first smoke detection model to obtain a second detection result, inputting the second detection result into the physical perception convolution time sequence model to obtain a third detection result, and if the self-adaptive fractional Fourier transform characteristic of the brightness characteristic sequence of the suspected smoke region of each frame of gray image in the third detection result accords with the smoke spectrum mode and the edge jitter quantity meets the smoke dynamic characteristic, determining that a fire event exists, wherein the second detection result comprises a boundary frame and