CN-121982103-A - Fire point distance measurement positioning method based on fusion of camera image and three-dimensional sparse point cloud

CN121982103ACN 121982103 ACN121982103 ACN 121982103ACN-121982103-A

Abstract

The invention discloses a fire point distance measurement and positioning method based on fusion of a camera image and a three-dimensional sparse point cloud, which specifically comprises the steps of early stage mountain fire scene high resolution event image and sparse point cloud combined acquisition and preprocessing; the method comprises the steps of carrying out space registration on an event camera image and a sparse point cloud, mapping the newly acquired point cloud to an event camera coordinate system and projecting the newly acquired point cloud to an event image plane to form a projection point set, carrying out local dense three-dimensional reconstruction on fire points based on event-point cloud fusion, carrying out fire point identification and three-dimensional ranging positioning on the dense point cloud, adopting sparse depth constraint and smooth regular complement depth guided by event edges and carrying out back projection to generate local dense point cloud, and realizing high-precision three-dimensional reconstruction on early fire point volumes, fire wire boundaries and space relations between the fire points and surrounding ground surfaces/vegetation.

Inventors

KUANG JUNWEI
CHENG YUTING
ZHANG LINGHAO
DENG CHUANG
LI LIN
LI SHENGJIE
XIAO DONGHUA
XIANG SIYU
LI YAQIANG

Assignees

国网四川省电力公司泸州供电公司

Dates

Publication Date: 20260505
Application Date: 20260130
Priority Date: 20251217

Claims (12)

1. The fire point distance measurement and positioning method based on fusion of the camera image and the three-dimensional sparse point cloud is characterized by comprising the following specific steps of: Constructing a combined observation system consisting of a high-resolution event camera and a three-dimensional sparse point cloud sensor, fixedly installing the two types of sensors on a watchtower, an unmanned aerial vehicle or a vehicle-mounted platform, realizing cooperative acquisition through a unified triggering and time synchronization module, and carrying out intensity normalization and noise suppression treatment on each acquired frame of event image; Carrying out spatial registration on the event camera image subjected to intensity normalization and noise suppression processing and the sparse point cloud by adopting an unsupervised learning framework, automatically solving external parameters between the two sensors under the condition of lacking a pixel-three-dimensional point true value corresponding relation, and generating a pixel-three-dimensional point corresponding relation; Mapping the newly acquired point cloud to an event camera coordinate system and projecting the newly acquired point cloud to an event image plane by using external parameters obtained through training to form a registered projection point set; Performing depth map construction and cross-modal feature fusion on the sparse point cloud under an event camera coordinate system, and realizing dense three-dimensional reconstruction of a local area of a fire point on the basis of the constructed and fused sparse point cloud; After the reconstruction of the local dense three-dimensional point cloud of the fire points is completed, the automatic identification and the three-dimensional ranging positioning of the fire point areas are realized on the premise of not depending on manual labeling.
2. The fire point distance measurement and positioning method based on the fusion of the camera image and the three-dimensional sparse point cloud according to claim 1, wherein the intensity normalization and noise suppression processing is performed on each acquired frame of event image, specifically, on one frame of event image Calculate it in the image domain Mean of the upper And standard deviation of And (3) performing standardization treatment: , Wherein, the The normalized event image; , Wherein, the Is the standard deviation of the corresponding standard deviation, For the total number of pixels of the image, To prevent a minute positive number with zero denominator.
3. The fire point distance measurement and positioning method based on fusion of camera images and three-dimensional sparse point clouds according to claim 1, wherein the specific steps of spatially registering event camera images and sparse point clouds by using an unsupervised learning framework are as follows: Step 1, given the current external parameters In the case of (a), a point cloud sensor coordinate system Lower three-dimensional point Mapping to an event camera coordinate system C through rigid transformation to obtain: , Wherein, the In order to rotate the matrix is rotated, In order to translate the vector of the vector, Is the first Three-dimensional coordinates of the points in an event camera coordinate system; step 2, utilizing an internal reference matrix of the event camera Can be used for Projecting to an event image plane to obtain pixel coordinates, wherein the expression is as follows: , Wherein, the Is the first Pixel locations of the individual points in the event image coordinate system, Is of depth and depth A related scaling factor; From normalized event images Extracting structural features to obtain an edge response graph based on gradient The expression is: , Wherein the method comprises the steps of For the pixel coordinates in the event image, And (3) with Respectively shown in And (3) with Gradient in direction; Step 3, constructing distance transformation or cost field on event image Encoding the distance from each pixel to the nearest structure edge as a scalar for measuring the spatial deviation between the projection point and the real structure; on the basis, defining the structural consistency loss of a single frame, wherein the expression is as follows: , Wherein, the Representation and the first The number of valid projection points in the point cloud that the frame event image matches, Is the projection point Characterizing the distance from the proxel to the edge of the nearest event at a value on the distance field; Step 4, recording the point at the first position Projection in a frame is In the first place Projection in a frame is The multi-frame re-projection consistency loss expression is: , Wherein, the For the number of static points where a correspondence can be established between two frames, Representing a binary norm; step 5, constructing an overall loss function of unsupervised spatial registration of the event camera image and the sparse point cloud, wherein the calculation formula is as follows: , Wherein alpha, beta, gamma and eta are non-negative weight coefficients, In order to achieve a loss of structural consistency, For a multi-frame re-projection consistency penalty, In order to distribute/exchange the information loss, Is a regularization term for the extrinsic parameters.
4. The fire point distance measurement and positioning method based on the fusion of the camera image and the three-dimensional sparse point cloud as claimed in claim 1, wherein the specific process of the depth map construction is that the obtained external parameters are utilized , ) And the internal reference matrix K is used for integrating three-dimensional points in the coordinate system L of the point cloud sensor Projecting into the k-frame event image plane to obtain pixel coordinates And its depth in the camera coordinate system In the event image domain In, constructing a sparse depth map with pixel coordinates as an index And validity mask The expression is: 。
5. the fire point distance measurement positioning method based on the fusion of the camera image and the three-dimensional sparse point cloud according to claim 1, wherein the cross-modal feature fusion adopts a dual-branch coding structure to extract event images and sparse depth features respectively, and feature level fusion is performed at a pixel level to obtain three-dimensional reconstruction guide features.
6. The fire point distance measurement positioning method based on fusion of camera images and three-dimensional sparse point clouds according to claim 1, wherein dense three-dimensional reconstruction of the local fire point areas is specifically implemented by constructing a depth complement decoding network, and a dense depth map on the whole image domain is predicted by taking fusion features, sparse depth and masks as inputs.
7. The fire point distance measurement and positioning method based on fusion of camera images and three-dimensional sparse point clouds according to claim 1, wherein the specific process of automatic identification of the fire point area is as follows: Three-dimensional fire point anomaly identification is carried out by referring to the thought of local density deviation, and the reconstructed fire point local dense point cloud is expressed as follows under an event camera coordinate system C: , Wherein, the Is the first Coordinates of the three-dimensional points in the event camera coordinate system, Points that are dense point clouds; based on the fusion feature extraction result, constructing a feature vector containing geometric and event information, wherein the expression is as follows: , Wherein, the Representation points Is used to determine the local geometry of the (c) device, Representing the intensity or characteristics of the event resulting from the projection of the event image, Is the overall feature dimension.
8. The fire point ranging and positioning method based on fusion of camera images and three-dimensional sparse point clouds according to claim 1, wherein the unsupervised anomaly detection framework in S5 is specifically characterized by feature set For input, estimating the local density of each point in the three-dimensional feature space and calculating an anomaly score, taking the local density deviation metric based on K nearest neighbors as an example, recording the point K-neighbor set of (2) For Euclidean distance, then The expression of the distance of a point to its K-nearest neighbor is: , defining the local reachable distance, and the expression is: , Estimating a point Is defined by the expression: , Wherein the method comprises the steps of To measure the number of adjacent points A degree of density deviation relative to its neighborhood; defining an anomaly score, the expression being: 。
9. the fire ranging and locating method based on fusion of camera images and three-dimensional sparse point clouds according to claim 8, wherein when said anomaly score When the local density of the point is close to the neighboring point, the point can be regarded as a normal background point When the density of the position of the point is obviously higher than 1, the local density of the position of the point is obviously lower than that of the neighborhood, and the abnormal point which belongs to 'local sparse and abrupt' can be regarded as a fire point or flame boundary candidate.
10. The fire point distance measurement and positioning method based on fusion of camera images and three-dimensional sparse point clouds according to claim 8, wherein based on anomaly scores, three-dimensional anomaly regions of fire points are automatically extracted from dense point clouds through threshold segmentation and spatial connectivity analysis, and anomaly thresholds are set as The preliminary outlier set may be defined as: , And at Performing three-dimensional connected domain segmentation and clustering on the obtained three-dimensional connected domain to obtain a plurality of spatially independent fire point candidate clusters, wherein for each fire point candidate cluster The geometric center or the highest point is used as a fire point representative point, and the three-dimensional position expression is as follows: , Wherein, the For the number of points in the q-th fire cluster, I.e. the three-dimensional representation of the location of the fire in the event camera coordinate system.
11. A computer system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of the fire ranging positioning method based on fusion of camera images with a three-dimensional sparse point cloud according to any one of claims 1 to 10.
12. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the fire ranging positioning method based on fusion of camera images with a three-dimensional sparse point cloud according to any of claims 1 to 10.

Description

Fire point distance measurement positioning method based on fusion of camera image and three-dimensional sparse point cloud Technical Field The invention belongs to the technical field of fire monitoring and early warning, and particularly relates to a fire point distance measurement and positioning method based on fusion of a camera image and a three-dimensional sparse point cloud. Background At present, early stage mountain fire monitoring for forest and mountain areas is mainly carried out by adopting a fixed observation tower, an unmanned plane or a visible light camera and an infrared thermal imaging device carried by satellites on engineering, and automatic warning is realized by recognizing smoke pillars, flames or temperature anomalies through two-dimensional images or thermal infrared images, but the method is mostly stopped at a two-dimensional plane detection level, high-precision three-dimensional ranging and space positioning of fire points are difficult to provide, the precision is easily influenced by topography fluctuation, smoke shielding and imaging geometric errors, the continuous three-dimensional characterization capability of small-scale early fire points is insufficient, meanwhile, a laser radar and camera fusion technology has been widely used for three-dimensional target detection and environment perception in the fields of automatic driving and the like, the defect of insufficient point cloud details and lack of depth information of images is effectively overcome by jointly encoding sparse point cloud and two-dimensional image feature pairs Ji Binglian, but related researches are mainly oriented to rigid target scenes such as vehicles, and a three-dimensional reconstruction and ranging model applicable to the high-dynamic, intense-deformation and small-scale targets such as flames are not established. In recent years, the event camera has the advantages of high time resolution, high dynamic range, low delay and the like, the existing work begins to explore flame detection data sets and recognition algorithms based on the event camera, but most of the work is still limited to two-dimensional detection or fire presence/absence judgment, the high-resolution event camera image and the three-dimensional sparse point cloud are not yet subjected to depth fusion, and the high-resolution event camera image and the three-dimensional sparse point cloud are used for dense three-dimensional reconstruction and high-precision ranging and positioning of early stage mountain fire points, so that a fire point ranging and positioning method for fusion of the high-resolution event camera image and the three-dimensional sparse point cloud for early stage mountain fire scenes is necessary to be provided. Disclosure of Invention The invention provides a fire point distance measurement positioning method based on fusion of a camera image and a three-dimensional sparse point cloud, and aims to construct an abnormal score based on neighborhood density deviation, extract a fire point candidate region of dense point cloud and improve judgment accuracy of fire points, distances and the like. In order to achieve the above effects, the following technical scheme is adopted: a fire point distance measurement and positioning method based on fusion of a camera image and a three-dimensional sparse point cloud comprises the following specific steps: Constructing a combined observation system consisting of a high-resolution event camera and a three-dimensional sparse point cloud sensor, fixedly installing the two types of sensors on a watchtower, an unmanned aerial vehicle or a vehicle-mounted platform, realizing cooperative acquisition through a unified triggering and time synchronization module, and carrying out intensity normalization and noise suppression treatment on each acquired frame of event image; Carrying out spatial registration on the event camera image subjected to intensity normalization and noise suppression processing and the sparse point cloud by adopting an unsupervised learning framework, automatically solving external parameters between the two sensors under the condition of lacking a pixel-three-dimensional point true value corresponding relation, and generating a pixel-three-dimensional point corresponding relation; Mapping the newly acquired point cloud to an event camera coordinate system and projecting the newly acquired point cloud to an event image plane by using external parameters obtained through training to form a registered projection point set; Performing depth map construction and cross-modal feature fusion on the sparse point cloud under an event camera coordinate system, and realizing dense three-dimensional reconstruction of a local area of a fire point on the basis of the constructed and fused sparse point cloud; After the reconstruction of the local dense three-dimensional point cloud of the fire points is completed, the automatic identification and the three-dimensional ranging