CN-121982612-A - Low-illumination moving object identification method based on laser illumination and event stream

CN121982612ACN 121982612 ACN121982612 ACN 121982612ACN-121982612-A

Abstract

The invention provides a low-illumination moving target identification method based on laser illumination and event stream, which achieves a better low-illumination target identification effect by combining static picture identification and dynamic event data identification under laser active illumination. The method comprises the specific steps of S1, converting laser illumination video data into event stream data, S2, training a static target detection model by using a single-stage convolutional neural network, S3, training a dynamic target detection model by using a pulse neural network, and S4, integrating and deploying the model.

Inventors

WU XIANYU
Lin Rennan
XU ZHIXIANG
HUANG FENG

Assignees

福州大学

Dates

Publication Date: 20260505
Application Date: 20260128

Claims (10)

1. The low-illumination moving object identification method based on laser illumination and event stream is characterized by comprising the following steps of: s1, converting laser illumination video data into event stream data; s2, training a static target detection model by using a single-stage convolutional neural network; S3, training a dynamic target detection model by using a pulse neural network; and S4, integrating and deploying the static target detection model and the dynamic target detection model.
2. The method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 1, wherein the specific steps of S1 are as follows: s11, inserting frames of laser illumination video data to construct a high-frame-rate video; S12, carrying out logarithmic brightness modeling on the inserted frame video sequence; And S13, converting the video brightness information into event stream data by using a trained convolutional neural network.
3. The method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 2, wherein in the log brightness modeling of the interpolated video sequence, the event triggering condition is as follows: Wherein, the Representing pixels At the moment of time Is used for the control of the brightness of the light, Representing a time interval of the pixel point since a last trigger event; A threshold value is issued for the event, For polarity of event, i.e. event data versus continuous logarithmic brightness signal In response to a change in the number of pixels At the time of The logarithmic brightness change amplitude of (a) exceeds the threshold since the last trigger event for that pixel When an event Will be triggered.
4. The method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 1, wherein the specific flow of S2 is as follows: s21, preprocessing a static laser illumination image data set; s22, sending the preprocessed data into a convolutional neural network to train a static target detection model; the single-stage convolutional neural network for training the static target detection model adopts Yolov network architecture.
5. The method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 1, wherein the specific flow of S3 is as follows: S31, preprocessing the event stream data set obtained in the step S1; s32, sending the preprocessed event stream data set into a plurality of feature extraction modules consisting of a downsampling layer and a convolution-based pulse neural network module Conv-based SNN Block to extract bottom layer detail features; S33, inputting the bottom-layer detail features obtained in the S32 into a pulse neural network module based on a transducer to extract high-layer semantic information; S34, constructing a pyramid multi-scale feature fusion neck network, and carrying out cross-scale feature fusion on the obtained feature map; S35, designing a joint loss function training dynamic target detection model.
6. The method for identifying the low-illumination moving target based on the laser illumination and the event stream according to claim 1, wherein a pulse neural network module leakage integration-pulse release model LIF dynamic equation is adopted as follows: Where t is the time step, and, Is the membrane potential, which will be the timing information of the previous moment Spatial information with current time Accumulating; Representing a step function when When equal to 1, otherwise equal to 0, if Exceeding the burst threshold The impulse neuron emits an impulse After pulse emission, membrane potential Will decay to , Is the attenuation coefficient.
7. The method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 6, wherein the calculation process of the convolution-based impulse neural network module is as follows: Wherein, the Representing the model input data after the preprocessing, Representing the output data after the split convolution residual connection, Representing the output of a convolution-based impulse neural network module Conv-based SNN Block, T representing the time step, C representing the number of channels, and H×W representing the spatial resolution; Representing a separable convolution module with an inverse separable convolution of 7 x 7 convolution kernel for capturing spatial features with global receptive field and adding a 3 x 3 depth convolution to further fuse local spatial features, the overall computational flow of which is represented as: Wherein, the And (3) with Representing a depth-separable convolution, And (3) with A point-by-point convolution is represented, Activating a function layer for impulse neurons; The whole calculation flow of the channel mixing module is expressed as follows: 。
8. the method for identifying a low-illumination moving object based on laser illumination and event stream according to claim 1, wherein the pulse neural network module based on a transducer comprises a pulse-driven self-attention module and a channel feedforward network, and the overall calculation process is as follows: Wherein, the Representing a reparameterizable convolution employing a 3 x 3 convolution kernel, the module is composed of a multi-branch structure during the training phase and may be equivalently a single convolution structure during the reasoning phase to reduce computational overhead. In the form of a channel feed-forward module, And Is a weight matrix in the channel feed forward module.
9. The method for identifying a low-illumination moving target based on laser illumination and event stream according to claim 1, wherein the training dynamic target detection model adopts a joint loss function as follows: Wherein, the Is a classification cross entropy loss for calculating a classification loss, Is the loss of the full cross-over ratio, Is the loss of the distributed focal point, Weights are the penalty functions.
10. The method for identifying the low-illumination moving target based on the laser illumination and the event stream according to claim 1, wherein the specific flow of the step S4 is as follows: S41, deploying a static target detection model and a dynamic target detection model on shooting equipment with a near infrared laser illumination function; S42, when the shooting equipment operates, laser illumination is started, and the image and the event data are respectively input into a corresponding model for real-time reasoning detection.

Description

Low-illumination moving object identification method based on laser illumination and event stream Technical Field The invention relates to the technical field of computer vision and photoelectric imaging, in particular to a low-illumination moving target identification method based on laser illumination and event stream. Background The target identification technology under the low-illumination condition has wide application prospect in scenes such as security monitoring, unmanned, frontier defense inspection and the like. However, in a low-illumination environment, the image often has the problems of serious noise interference, low contrast, missing detail information and the like, and the performance of the target detection and recognition algorithm is obviously affected. How to realize efficient and accurate target recognition under the low-illumination condition has become one of the key technologies for the development of a machine vision system, and has important significance for improving the safety, the automation level and the execution capacity of night tasks of the system. The traditional method improves the image quality through an image enhancement algorithm and then detects the image. However, the enhancement method usually damages the texture structure of the original image, generates artifacts, amplifies noise, and causes blurring of image details, thereby affecting the accuracy of the subsequent recognition algorithm. Particularly in a dynamic target recognition scene, the motion of the target may cause a smear or a blurring effect, so that the detection difficulty is further increased, and the application effect of the existing method in a practical complex scene is limited. Disclosure of Invention Therefore, the invention aims to provide a low-illumination moving target identification method based on laser illumination and event stream, which effectively enhances the imaging quality of a camera under low illumination and realizes high-precision identification of a static target. In order to achieve the purpose, the invention adopts the following technical scheme that the low-illumination moving object identification method based on laser illumination and event stream comprises the following steps: s1, converting laser illumination video data into event stream data; s2, training a static target detection model by using a single-stage convolutional neural network; S3, training a dynamic target detection model by using a pulse neural network; and S4, integrating and deploying the static target detection model and the dynamic target detection model. In a preferred embodiment, the specific steps of S1 are as follows: s11, inserting frames of laser illumination video data to construct a high-frame-rate video; S12, carrying out logarithmic brightness modeling on the inserted frame video sequence; And S13, converting the video brightness information into event stream data by using a trained convolutional neural network. In a preferred embodiment, the log brightness modeling of the interpolated video sequence is performed under the following event trigger conditions: Wherein, the Representing pixelsAt the moment of timeIs used for the control of the brightness of the light,Representing a time interval of the pixel point since a last trigger event; A threshold value is issued for the event, For polarity of event, i.e. event data versus continuous logarithmic brightness signalIn response to a change in the number of pixelsAt the time ofThe logarithmic brightness change amplitude of (a) exceeds the threshold since the last trigger event for that pixelWhen an eventWill be triggered. In a preferred embodiment, the specific flow of S2 is as follows: s21, preprocessing a static laser illumination image data set; s22, sending the preprocessed data into a convolutional neural network to train a static target detection model; the single-stage convolutional neural network for training the static target detection model adopts Yolov network architecture. In a preferred embodiment, the specific flow of S3 is as follows: S31, preprocessing the event stream data set obtained in the step S1; s32, sending the preprocessed event stream data set into a plurality of feature extraction modules consisting of a downsampling layer and a convolution-based pulse neural network module Conv-based SNN Block to extract bottom layer detail features; S33, inputting the bottom-layer detail features obtained in the S32 into a pulse neural network module based on a transducer to extract high-layer semantic information; S34, constructing a pyramid multi-scale feature fusion neck network, and carrying out cross-scale feature fusion on the obtained feature map; S35, designing a joint loss function training dynamic target detection model. In a preferred embodiment, the impulse neural network module leakage integration-impulse release model LIF dynamic equation is used as follows: Where t is the time step, and, Is the membrane potential