CN-121982199-A - Three-dimensional Gaussian sputtering power line reconstruction method and system based on catenary physical constraint

CN121982199ACN 121982199 ACN121982199 ACN 121982199ACN-121982199-A

Abstract

The invention provides a three-dimensional Gaussian sputtering power line reconstruction method and system based on catenary physical constraint, and relates to the technical field of computer vision and intelligent power grid inspection. The method comprises the steps of firstly obtaining a power line multi-view image acquired by monocular vision of a live working robot, extracting a power line mask by utilizing an improved Unet segmentation network, obtaining a pose of the camera and an initial sparse point cloud by combining a motion restoration Structure (SFM) algorithm, then fitting catenary equation parameters to construct a three-dimensional theoretical track model of the power line, further guiding a three-dimensional Gaussian ball to carry out physical initialization seeding along the track, finally constructing a loss function containing physical geometric constraint, and carrying out iterative optimization on the three-dimensional Gaussian model. The invention integrates the catenary prior into the whole initialization and optimization process, effectively eliminates break points and noise points caused by weak textures and strong light reflection of the power line, ensures the spatial continuity and geometric accuracy of the reconstructed model, and provides reliable three-dimensional data support for autonomous operation of the robot.

Inventors

YAN JUNHUA
LU CHAOYANG
ZHANG YIN
ZHANG HAOTIAN
ZHANG JIAZHUO
WANG DAWEI

Assignees

南京航空航天大学

Dates

Publication Date: 20260505
Application Date: 20251231

Claims (9)

1. The three-dimensional Gaussian sputtering power line reconstruction method based on catenary physical constraint is characterized by comprising the following steps of: S1, acquiring a monocular Image sequence Image1 acquired aiming at a power line inspection scene, and constructing a power line Image sample library based on the monocular Image; S2, carrying out semantic segmentation processing on the Image sequence Image1, and extracting a power line mask Image2, and solving the Image sequence Image1 by utilizing a motion restoration structure algorithm SfM to obtain camera pose parameters Pose and initial sparse point cloud Data1 of a scene; S3, performing physical parameter fitting based on the initial sparse point cloud Data1, calculating catenary equation parameters, and constructing a three-dimensional theoretical space trajectory Curve1 of the power line; s4, generating a three-dimensional Gaussian ball along the track space position by taking the three-dimensional theoretical space track Curve1 as a center, and finishing physical priori initialization of a three-dimensional Gaussian sputtering model; S5, constructing a loss function containing a catenary physical constraint term, performing iterative training on the initialized three-dimensional Gaussian sputtering Model by using the Image sequence Image1 and the camera pose Pose1, and obtaining a final three-dimensional reconstruction Model1 of the power line when training loss converges; S6, analyzing the space geometric parameters of the power line according to the power line three-dimensional reconstruction Model1, and realizing reconstruction of the three-dimensional form of the power line.
2. The three-dimensional gaussian sputtering power line reconstruction method based on catenary physical constraints according to claim 1, wherein the S2 is specifically: S21, preprocessing an Image by acquiring a monocular Image of a diversified scene containing a power line, and then constructing a power line Image dataset Image-dataset; S22, marking a power line region in an image_open of a single Image in the power line Image dataset by using Image marking software, setting the power line region as a label for representing a power line type, and storing a marking file under the same path as the Image to generate a marking file in json format containing the power line type; s23, converting the Labeling files into binary mask format files required by training in batches, and constructing a semantic segmentation Labeling data set Labelin1_dataset containing original image-mask pairs; s24, constructing a U-Net network based on an encoder-decoder structure, embedding a convolution block attention CBAM module in a bottleneck layer Bottleneck of the U-Net network, wherein the CBAM module is composed of channel attention submodules connected in series And a spatial attention submodule Composition, characteristic diagram The arithmetic logic processed by the module is as follows: ; ; Wherein, the In order to input the feature map, The characteristics are weighted for the channel attention, In order to achieve a final output characteristic, Representing element-by-element multiplication; S25, dividing the semantic segmentation Labeling data set Labelim1_dataset into a training set and a testing set, and constructing a binary cross entropy loss And Dice coefficient loss Weighted composition mixing loss function The method is used for solving the problem of unbalance of the samples with extremely small power line pixel duty ratio, and the formula is as follows: ; ; Wherein, the In order to balance the weight coefficient(s), Is the first The predicted probability value for each pixel, First, the The true label value of the individual pixels, For the total number of pixels, Is a smooth term; S26, carrying out iterative training on the U-Net network by utilizing the training set and the mixing loss function, and when the intersection ratio IoU index of the testing set reaches a preset threshold, storing an optimal model weight; S27, extracting key points and descriptors of each frame of Image by using a SIFT algorithm according to the Image sequence Image1, performing feature matching by using a nearest neighbor ratio method, and removing mismatching point pairs by using a RANSAC algorithm; S28, selecting an image pair with a wider base line for initialization, and calculating an essential matrix by utilizing epipolar geometric constraint to restore an initial pose; S29, constructing a re-projection error minimization objective function, and carrying out joint nonlinear optimization on the pose parameters Pose of all cameras and the space coordinates of the initial sparse point cloud Data1, wherein the objective function is specifically as follows: ; Wherein, the For the total weight of the projection errors, For the total number of images, Is the total number of three-dimensional points; for visibility indication, when the camera Three-dimensional points are observed Taking 1 if not, taking 0 if not; Is a three-dimensional point In a camera Observing pixel coordinates thereon; for camera projection model, three-dimensional points are formed Projecting to a pixel coordinate system, and outputting final camera pose parameters Pose and initial sparse point cloud Data1 after optimization convergence.
3. The three-dimensional gaussian sputtering power line reconstruction method based on catenary physical constraints according to claim 1, wherein the S3 specifically is: S31, traversing each three-dimensional point in the initial sparse point cloud Data1, projecting the three-dimensional Points to a power line mask Image2, reserving three-dimensional Points with projection coordinates in an effective mask area, and forming a power line candidate point set Points1 consistent with Image semantics by the Points; S32, adopting a random sampling consistency algorithm RANSAC to process the power line candidate point set Points1, counting the number of internal Points falling in the range of a Plane model threshold in the iterative process, selecting a Plane containing the most internal Points as an optimal vertical Plane1, and rejecting Points which are more than a preset threshold from the Plane as outliers to obtain an optimized internal point set; s33, analyzing the space geometrical property of the vertical Plane1 under a world coordinate system, and constructing a local orthogonal basis vector group describing the power line morphology, wherein the method specifically comprises the following steps of: Calculating geometric centroid of inner point set as reference origin Defining the normal vector of the vertical Plane1 as The gravity direction unit vector is Constructing horizontal unit vector in the power line span direction by vector cross operation Constructing a vertical unit vector perpendicular to the span direction Thereby establishing a free point Base vector A defined local geometric reference frame; s34 concentrating the inner points Decomposing to the local geometric reference system to obtain scalar coordinates Wherein Constructing a catenary equation model based on decomposed scalar coordinates Solving the optimal shape parameters by using a nonlinear regression equation Location parameters Obtaining a curve equation in a two-dimensional plane; s34, based on the solved equation parameters and the local orthogonal basis vectors Setting sampling step length in the span range of the power line, and directly constructing a three-dimensional theoretical space track Curve1 of the power line by vector linear combination, wherein any point on the track The three-dimensional coordinate calculation formula of (2) is as follows: ; Wherein, the Is along the horizontal vector The direction independent variable samples.
4. The three-dimensional gaussian sputtering power line reconstruction method based on catenary physical constraints according to claim 1, wherein the S4 is specifically: S41 is based on the three-dimensional trajectory equation obtained in the step S3 Calculating total arc length of curve, setting linear sampling density along curve path, and calculating parameters Uniformly sampling in the definition domain of (2) to generate a series of discrete three-dimensional coordinate point sets, and taking each coordinate point in the three-dimensional coordinate point sets as an initial position mean parameter of an independent three-dimensional Gaussian sphere Initializing to obtain a power line Gao Sidian cloud set consisting of a plurality of three-dimensional Gaussian balls; S42 traversing each three-dimensional Gaussian ball in the power line Gao Sidian cloud set according to the three-dimensional Gaussian ball position mean value Corresponding curve parameters for the three-dimensional trajectory equation Determining parameters of interest And normalized to obtain a unit tangent vector of the point at the sample Constructing a vector of the cuts Local rotation matrix for X-axis main axis direction And rotates the matrix Conversion into unit rotation quaternions As an initial rotation parameter of the three-dimensional Gaussian ball, the main axis direction of the Gaussian ball is strictly aligned with the local space trend of the power line; S43 is an initial scale parameter imparting anisotropy to each three-dimensional Gaussian sphere in the set Specifically defining a scale vector Wherein For a longitudinal dimension along the tangential vector direction, For radial dimension perpendicular to tangential vector plane, and set constraint conditions Enabling the initialized three-dimensional Gaussian balls to take on an elongated elliptic geometrical form extending tangentially along a power line; S44 initial opacity parameter of all three-dimensional Gaussian spheres And setting the initial three-dimensional Gaussian spherical harmonic coefficient SH to be zero or setting the initial three-dimensional Gaussian spherical harmonic coefficient SH according to the color mean value of the projection image to finish the physical guidance initialization construction of the three-dimensional Gaussian sputtering model.
5. The three-dimensional gaussian sputtering power line reconstruction method based on catenary physical constraints according to claim 1, wherein the S5 specifically is: S51, projecting the three-dimensional Gaussian sphere to a two-dimensional image plane by using a micro rasterizer to obtain a rendered image of the current view angle And calculates the image and the real image Loss of photometric consistency between; S52, constructing physical constraint loss based on geometric distance field For constraining the centre of a gaussian sphere of a power line not deviating from the theoretical catenary trajectory; S53 defines the total loss function Wherein In order to achieve a loss of luminosity, In order to achieve a loss of structural similarity, Is a physical constraint loss; s54 calculating physical constraint loss Specifically, calculating the average value of the shortest Euclidean distance from the center position of each three-dimensional Gaussian ball belonging to the power line class to the three-dimensional theoretical space trajectory Curve1 in the current iteration step; S55, updating the position, rotation, scaling and spherical harmonic coefficient attributes of the three-dimensional Gaussian sphere by using a back propagation algorithm, and finely adjusting the catenary equation parameters according to the current Gaussian sphere distribution every preset iteration step number, so that the physical model can dynamically adapt to the fine adjustment of the three-dimensional Gaussian distribution until the total loss function converges; S56, stopping the iterative optimization process when the total loss function converges, outputting a three-dimensional Gaussian scene representation which is finally optimized, reserving geometric attributes and appearance attributes of each three-dimensional Gaussian sphere after convergence, and forming the series of discrete anisotropic Gaussian spheres with continuous rendering capability into the power line three-dimensional reconstruction Model1 with catenary morphological characteristics.
6. The three-dimensional gaussian sputtering power line reconstruction method based on catenary physical constraints according to claim 1, wherein the S6 is specifically: S61, extracting all Gaussian sphere center coordinates in a final converged power line three-dimensional reconstruction Model1, constructing a power line three-dimensional point cloud set, calculating a first main direction of the power line three-dimensional point cloud set by using a principal component analysis PCA (principal component analysis) method, taking the first main direction as an overall space trend vector of the power line, identifying end point coordinates of the point cloud set at two trend ends, and calculating Euclidean distance between the two end points to determine a span parameter of the power line; S62, projecting the three-dimensional point cloud set of the power line into a vertical plane formed by a gravity direction vector and a space trend vector, calculating the vertical distance from the lowest point of the point cloud to the connecting line of two end points, namely the maximum sag height, in the two-dimensional projection plane, and checking whether sag parameters meet safety operation standards by utilizing a state equation in combination with known environment temperature and weather conditions; S63, equidistant slice sampling is carried out on the three-dimensional reconstruction Model1 along the span direction of the power line, the connection line of the gravity center of the Gaussian ball position in each slice section is calculated, a smooth center skeleton line is generated, the coordinates of the center skeleton line are compared with truth value data measured by a laser radar or a total station, the geometric accuracy of the reconstruction Model is evaluated, and finally the power line automatic reconstruction report containing the visual view, sag, span and trend geometric parameters of the three-dimensional Model is output.
7. A three-dimensional gaussian sputtering power line reconstruction system based on catenary physical constraints, the system comprising: The data acquisition module is configured to acquire a monocular Image sequence Image1 shot for a power line inspection scene; The Data preprocessing module is configured to perform semantic segmentation on the Image sequence Image1 based on a deep learning model to extract a power line mask Image2, and calculate pose parameters Pose of a camera and initial sparse point cloud Data1 of a scene by utilizing a motion restoration structure algorithm; the physical modeling module is configured to execute catenary parameter regression based on the initial sparse point cloud Data1 to construct a three-dimensional theoretical space trajectory Curve1 of the power line conforming to gravity constraint; the physical guide initialization module is configured to generate an initialized three-dimensional Gaussian ball along the three-dimensional theoretical space trajectory Curve1, endow the initialized three-dimensional Gaussian ball with anisotropic geometric form and finish physical priori initialization of the three-dimensional Gaussian sputtering model; The Model joint optimization module is configured to construct a mixed objective function containing luminosity consistency loss and catenary geometric constraint loss, and the initialized Model is subjected to iterative training and parameter updating by utilizing the Image sequence Image1 and the camera pose Pose1 to output a power line three-dimensional reconstruction Model1; And the parameter analysis module is configured to analyze and output the space geometric parameters of the power line based on the power line three-dimensional reconstruction Model 1.
8. The three-dimensional Gaussian sputtering power line reconstruction system based on catenary physical constraints according to claim 7, wherein the data preprocessing module comprises a data set construction unit, a labeling processing unit, a model training unit, a mask reasoning unit and a pose resolving unit; The data set constructing unit is used for acquiring a public data set and a self-building robot visual angle data set and constructing a power line Image data set Image-dataset; the marking processing unit is used for carrying out pixel-level marking on the power line area in the power line image data set by using an image marking tool, generating a marking file and converting the marking file into a mask format; The model training unit is used for dividing the data set into a training set and a testing set, training an improved U-Net model embedded into the CBAM module by using the training set, and storing weights when IoU of the testing set reaches a set threshold; the mask reasoning unit is used for loading the saved weight, dividing the Image sequence Image1 and outputting a power line mask Image2; the pose resolving unit is used for processing the Image sequence Image1 by using an incremental SfM algorithm and outputting sparse point cloud Data1 and a camera pose Pose.
9. The three-dimensional Gaussian sputtering power line reconstruction system based on catenary physical constraint according to claim 7, wherein the physical guidance initialization module and the model joint optimization module comprise a physical seeding unit, a geometric initialization unit, a loss calculation unit and a parameter updating unit; the physical sowing unit is used for setting sampling density according to the total arc length of the catenary track Curve1 and uniformly generating the initial center position of the three-dimensional Gaussian ball along the track; the geometric initialization unit is used for calculating tangential vectors of each point on the track, constructing a rotation quaternion to enable a Gaussian sphere main axis to be aligned with the tangential direction, and endowing the Gaussian sphere slender elliptic anisotropic scale attribute; The loss calculation unit is used for calculating the shortest Euclidean distance from the Gaussian sphere center belonging to the power line class to the theoretical track Curve1 in real time in training iteration, and taking the shortest Euclidean distance as a physical constraint loss term; The parameter updating unit is used for constructing a total loss function containing a physical constraint term, updating the position, rotation, scaling and spherical harmonic coefficient attributes of the Gaussian sphere by using a back propagation algorithm and a gradient descent strategy, and outputting a converged power line three-dimensional Model1.

Description

Three-dimensional Gaussian sputtering power line reconstruction method and system based on catenary physical constraint Technical Field The invention relates to the technical field of computer vision and intelligent power grid inspection, in particular to a three-dimensional Gaussian sputtering power line reconstruction method based on catenary physical constraint. Background The power line is a core component of the power grid system, and the safe operation of the power line is directly related to national folk life. With the development of smart power grids, the adoption of live working robots to replace manual high-voltage line inspection and maintenance has become a necessary trend. The premise of robot operation is that the surrounding environment can be accurately perceived, and particularly a high-precision three-dimensional model of a power line is obtained. Traditional power line reconstruction mainly relies on a laser radar (LiDAR), but equipment is expensive and large in size, and is difficult to adapt to a lightweight robot platform. Traditional three-dimensional reconstruction methods (such as MVS) based on pure vision are limited by sparse texture and slender structure of the power line, and often lead to fracture and deletion of reconstruction results. In recent years, although the three-dimensional gaussian sputtering (3 DGS) technology has excellent rendering effect, the technology is data-driven in nature, a large number of floating noise points deviating from the true position are easy to generate in a weak texture area, and the geometric accuracy cannot meet the operation requirement of a robot. Aiming at the conditions of weak texture and strong reflection of a power line image in a natural scene, the invention provides a three-dimensional Gaussian sputtering power line reconstruction method and system based on catenary physical constraint in order to improve the spatial continuity and geometric accuracy of reconstruction. Disclosure of Invention Aiming at the defects of the prior art, the invention aims to provide a three-dimensional Gaussian sputtering power line reconstruction method based on catenary physical constraint, which combines semantic segmentation, a motion recovery structure, physical model fitting and a nerve rendering technology to construct an integrated reconstruction frame. Aiming at a monocular image sequence acquired by a robot, firstly extracting a power line region through an improved semantic segmentation model, combining SFM to obtain initial information, then fitting a catenary physical equation by using sparse point cloud to obtain a theoretical track, further guiding 3DGS to initialize by using the track, introducing physical distance constraint in an optimization process, and forcing the point cloud to converge on a real physical form. Compared with the traditional pure vision method, the method can effectively eliminate fracture and artifacts and realize high-precision three-dimensional reconstruction of the power line. The three-dimensional Gaussian sputtering power line reconstruction method based on catenary physical constraint comprises the following steps of: S1, acquiring a monocular Image sequence Image1 acquired aiming at a power line inspection scene, and constructing a power line Image sample library based on the monocular Image; S2, carrying out semantic segmentation processing on the Image sequence Image1, and extracting a power line mask Image2, and solving the Image sequence Image1 by utilizing a motion restoration structure algorithm SfM to obtain camera pose parameters Pose and initial sparse point cloud Data1 of a scene; S3, performing physical parameter fitting based on the initial sparse point cloud Data1, calculating catenary equation parameters, and constructing a three-dimensional theoretical space trajectory Curve1 of the power line; s4, generating a three-dimensional Gaussian ball along the track space position by taking the three-dimensional theoretical space track Curve1 as a center, and finishing physical priori initialization of a three-dimensional Gaussian sputtering model; S5, constructing a loss function containing a catenary physical constraint term, performing iterative training on the initialized three-dimensional Gaussian sputtering Model by using the Image sequence Image1 and the camera pose Pose1, and obtaining a final three-dimensional reconstruction Model1 of the power line when training loss converges; S6, analyzing the space geometric parameters of the power line according to the power line three-dimensional reconstruction Model1, and realizing reconstruction of the three-dimensional form of the power line. Further, the S2 specifically is: S21, preprocessing an Image by acquiring a monocular Image of a diversified scene containing a power line, and then constructing a power line Image dataset Image-dataset; S22, marking a power line region in an image_open of a single Image in the power line Image dataset by using I