Search

CN-115457120-B - Absolute position sensing method and system under GPS refusing condition

CN115457120BCN 115457120 BCN115457120 BCN 115457120BCN-115457120-B

Abstract

The invention discloses an absolute position sensing method and system under a GPS rejection condition, wherein the method comprises the following steps of A, conducting 360-degree annular splicing on images shot by rotating a specific angle through a camera, splicing discrete images into a panoramic image, B, extracting an astronomical line from a natural panoramic image formed in the step A based on a Deeplabv & lt+ & gt improved semantic segmentation method, C, forming a rendering image by adopting OpenGL visual DEM data, conducting edge detection on the rendering image by using a Canny operator, and D, sensing the position of the user by using VGG16 as a natural image and an image astronomical line feature encoder and combining a twin neural network to output similarity between the two types of astronomical lines. Aiming at the difficult problem that the self-position is required to be perceived on the ground under the conditions of military application of GPS refusing conditions and the like, the invention develops the research of determining the self-position and realizes the aim of absolute positioning which does not depend on GPS and initial position and has no error increasing with time.

Inventors

  • TANG JIN
  • YANG ZIRONG
  • GUO FAN
  • WU ZHIHU
  • GAO YAN
  • GONG CHENG
  • PAN ZHIBIN
  • LI WEICHAO
  • CHEN JIANTANG

Assignees

  • 中南大学

Dates

Publication Date
20260505
Application Date
20210521

Claims (9)

  1. 1. A method of absolute position sensing under GPS rejection conditions, the method comprising the steps of: step A, performing 360-degree annular stitching on discrete natural images acquired by a vehicle-mounted camera to form a natural panoramic image; Step B, extracting a natural image panoramic astronomical line from a natural panoramic image by adopting a semantic segmentation network based on Deeplabv & lt+ & gt improvement; step C, obtaining DEM data of a position to-be-perceived area based on a remote sensing center, forming a rendering graph by using OpenGL (open graphics library), and performing edge detection on the rendering graph by using a Canny operator to output a panoramic astronomical line of a DEM image; adopting a migration learning method, adopting VGG16 as a feature encoder of a twin neural network, and utilizing the twin neural network to output the similarity between a natural image panoramic astronomical line and a DEM image panoramic astronomical line as a matching result so as to perceive the position of the user according to the astronomical line matching result; The specific processing procedure of the step D is as follows: Step D1, extracting features of a astronomical line; Two astronomical line images are input into the twin neural network by adopting VGG16 as a feature encoder of the twin neural network, and the input is mapped to a new space through the feature encoder of the VGG16 respectively to form the representation of the astronomical line images in the new space, namely the encoded astronomical line features; step D2, establishing a DEM discrete feature database; each DEM image resize is 224 x 3 in size, then feature coding is carried out on each panoramic image in the DEM through a twin neural network, a 7 x 512 feature map is obtained, and the feature map is offline stored in a csv file; Step D3, feature matching; The natural panoramic image resize is input into a twin neural network after being 224 x 3 standard size, a feature map F Final is obtained through calculation, the size is 7 x 512, a DEM offline feature database is traversed, euclidean distance between the feature map F Final and each feature map in the DEM discrete feature database is calculated, and a DEM point with the highest similarity is selected as a locating point.
  2. 2. The method according to claim 1, wherein the specific processing procedure of step a is as follows: A1, shooting at intervals of 40 degrees each time on a scout car by using a spherical camera at fixed view points, and rotating for one circle to obtain 9 natural images; and A2, respectively projecting the natural images to be spliced to the same coordinate system, wherein the cylindrical projection formula is as follows: ; Wherein, the Representing the original coordinates of the pixel point in the natural image, Representing coordinates after cylindrical projection, (width, height) represents the width and height of the natural image, respectively, and f represents the focal length of the camera; a3, identifying characteristic points between adjacent natural images by using a SIFT operator, and then carrying out characteristic matching on 9 acquired natural images, wherein the images subjected to characteristic matching are used as images to be spliced; And step A4, respectively decomposing the images to be spliced onto different spatial frequency bands by establishing a Laplacian pyramid, and respectively merging and fusing the images on each spatial frequency layer to obtain a natural panoramic image.
  3. 3. The absolute position sensing method under the condition of GPS rejection according to claim 2, wherein the method is characterized in that the image to be spliced is decomposed onto different spatial frequency bands respectively by establishing a Laplacian pyramid, and merging and fusing are carried out on each spatial frequency layer respectively, so as to obtain a natural panoramic image, and the specific steps are as follows: A41, establishing a Gaussian pyramid of the image to be spliced; Step A42, subtracting the extended image of the upper layer after up-sampling and Gaussian convolution by utilizing each layer of image of the Gaussian pyramid to obtain LP, and combining the same layers of the LP in the overlapping area by adopting a weighted average method, wherein the LP represents the Laplacian pyramid; a43, expanding the combined LP from the top layer, and adding the expanded image and the combined LP of the next layer to obtain a fusion image of the next layer; step A44, completing image fusion by layer recursion to obtain a preliminary natural panoramic image; Step A45, separating the preliminary natural panoramic image obtained by splicing the step A44 from the middle to form two images p1 and p2; Step A46, setting p2 as the head of the second stitching, setting p1 as the tail, adopting a SIFT operator to obtain the feature of the overlapping part of the p2 and p1 images, and matching to obtain a new image with the matched features as an image to be stitched; And A47, decomposing the images to be spliced onto different spatial frequency bands respectively by establishing a Laplacian pyramid, merging and fusing the images on each spatial frequency layer respectively to obtain a natural panoramic image, realizing seamless fusion of the images p2 and p1, and removing head-to-tail overlapping of the images so as to obtain a final natural image splicing result.
  4. 4. The absolute position sensing method under the GPS rejection condition according to claim 1, wherein the specific processing procedure of the step B is that step B1 is that a trunk network ResNet-101 in Deeplabv & lt+ & gt semantic segmentation network is replaced by a GhostNet network, the GhostNet network is formed by stacking 101 GhostBottleNeck modules in total, a natural panoramic image is input into the Deeplabv & lt+ & gt improved semantic segmentation network, and a segmentation feature map F nature of the natural image is obtained; step B2, enhancing the characteristics; Information enhancement is carried out on the segmentation feature map F nature by utilizing a CCAM module, so that an enhancement feature map FM Out is obtained; The method comprises the steps that a CCAM module is adopted to divide a feature map F nature , feature information of each channel is compressed into a feature value through global average pooling and global maximum pooling respectively, so that a global average pooling feature map FM GA and a global maximum pooling feature map FM GM are obtained, and the feature maps FM s are obtained through channel splicing; The method comprises the steps of inputting a feature map FM s into a Conv-BN-ReLU structure, carrying out dimension reduction and feature extraction on the input feature map FM s by Conv through convolution, carrying out dimension increase by utilizing a linear layer to obtain a feature map FM Ex , decomposing the feature map FM Ex into a linear global average pooling feature map FM EGA and a linear global maximum pooling feature map FM EGM according to channels, adding FM EGA and FM EGM pixel by pixel, activating by sigmoid to obtain a final channel feature map FM Channel , and multiplying the channel feature map FM Channel and a segmentation feature map F nature according to channels to obtain an output enhancement feature map FM Out .
  5. 5. The method for absolute position sensing under GPS rejection according to claim 4, wherein the initial feature map FM Ini of the astronomical line is obtained by performing pooling and convolution operations after noise removal on the enhanced feature map FM Out by using the regional attention module; The regional attention module consists of a Mask branch and a main line branch, wherein the output end of the Mask branch and the output end of the main line branch are added; The main line branch directly transmits an input image to an output end; The Mask branch adopts a U-Net structure, namely an encoder is used for downsampling an input image, and a decoder is used for upsampling a feature map to gradually restore a feature scale; The specific process is as follows: Step B3, the enhancement feature map FM Out is subjected to one-time Maxpooling and BottleNeck downsampling to obtain a feature map After one more downsampling by Maxpooling and BottleNeck, a feature map is obtained ; Step B4, for the characteristic diagram Performing one BottleNeck up-sampling to obtain a characteristic diagram ; Step B5, feature map And feature map After channel splicing, performing up-sampling twice by using BottleNeck and bilinear interpolation to obtain a feature map FM Up ; Step B6, after the feature map FM Up is convolved to reduce the feature dimension, a single-channel region probability feature map FM Single is output, the probability value is normalized to be between 0 and 1 by using sigmoid, the single-channel region probability feature map FM Single is used as region attention information and is multiplied with a Trunk branch point by point according to a channel and then added, and the feature map FM Area subjected to region enhancement is output, wherein the specific calculation formula is as follows: H(x)=(x+f(I(x)))×I(x); Wherein H (x) is a probability feature map output by the CCAM module, f (I (x)) is an output feature map of the Mask branch, and I (x) is a feature map input by the CCAM module; And B7, after passing through GhostNet networks, a CCAM module and a CAAM attention mechanism, obtaining an initial characteristic diagram FM Ini of the astronomical line through convolution and pooling operation.
  6. 6. The absolute position sensing method under the GPS rejection condition according to claim 5, wherein an SCPA module is used for preprocessing an acquired natural image to obtain a feature map FM Binary , the feature map FM Binary is used for fusing with an initial feature map FM Ini , the fused result is subjected to two RCU operations and 1 x 1Conv convolution operations, and then up-sampling is carried out to obtain a final astronomical semantic segmentation feature map; setting the threshold value of the final astronomical semantic segmentation feature map as 0.95, namely setting the probability value to be 1 at points with the probability value greater than 0.95, and setting the probability value to be 0 at points with the probability value less than 0.95, so as to obtain a final output natural panoramic astronomical image; The RCU comprises four serially connected convolution kernels, wherein the sizes of the four convolution kernels are 3*1, 1*3, 3*1 and 1*3 in sequence, the convolution operation of the first three convolution kernels is normalized by BN (Batch Normalization) and then activated by Relu functions, the convolution operation of the last convolution kernel is normalized by BN and then overlapped with the input part of the RCU to obtain output, the acquired natural image is preprocessed by the SCPA module, namely, the input color image is firstly converted into a gray image, and then each sampling column in the gray image is processed by utilizing the characteristic that the gray value of a far area of a mountain area is larger than that of a near area to obtain the pixel gradient of the corresponding column: judging whether the difference between the upper gray level and the lower gray level, which is 2 from the current point, of each point in the sampling columns is greater than 0, if so, marking the point as 1, otherwise, marking the point as 0, and then finding out the maximum position of the pixel gradient from the continuous area marked as 1 as a candidate point so as to inhibit the rest non-maximum values; G m (y)=f(m·p,y+2)-f(m·p,y+2); ; (x m ,y k )＝arg max(G(y));(x m ,y k )∈Area(x m ,k) In the formula, Representing the position as Values in the gray scale map, p denotes a sampling interval; For gray gradient of the m-th sampling column position, g m (y) is a gradient label, and (x m ,y k ) is the maximum value of the gradient of the region.
  7. 7. The absolute position sensing method under the GPS rejection condition of claim 1, wherein when the semantic segmentation network based on Deeplabv3+ improvement is trained, the natural panoramic image obtained by splicing in the step A is adopted, and the astronomical line in the panoramic image is subjected to label making to obtain a training set, wherein the size of the training set is 150 panoramic natural images, and the size of the training set is 800 x 600; Training period is 150 epochs, training batchsize is set to 8, parameters of a semantic segmentation network based on Deeplabv & lt3+ & gt improvement are optimized by using a random gradient descent algorithm, initial learning rate is 0.001, learning rate is adjusted by using a cosine fire-down mode, momentum is set to 0.9, and weight descent WEIGHTDECAY is set to 0.0005; During training, the model branch of CAAM adopts binary intersection entropy as a loss function by carrying out loss calculation on a astronomical line feature map FM Out formed after GhostNet networks and a CCAM attention mechanism and regional tag information, wherein the specific formula is as follows: ; Where N is the total length of the region probability map after being developed into a vector, y is the region label, Is a predictive value for Mask branch.
  8. 8. The method according to claim 1, wherein the specific processing procedure of step C is as follows: step C1, reading and converting DEM data; Firstly, data are read and stored in a matrix form, coordinate information of DEM data is obtained by using a GetGeoTransform () method in GDALDATASET types of GDAL, the obtained information is stored in an array form, a group of key coefficients is obtained, and a formula is used for converting row and column numbers of the matrix with geographical coordinates: X=gt[0]+col*gt[1]+row*gt[2] Y=gt[3]+col*gt[4]+row*gt[5]; wherein col, row represent column number and row number respectively, gt [ i ] represents six key coefficients obtained, gt [0], gt [3] represents geographic coordinates at image coordinates (0, 0), gt [1], gt [5] represents resolution of x-axis and y-axis of image, gt [2], gt [4] represents image rotation coefficient, XY represents geographic coordinates; Step C2, generating a perspective view; Step C21, using frustum functions of vmath libraries in OpenGL, generating a perspective projection matrix by setting near plane rectangle, distance of far plane and distance of near plane, and replacing color information of points on the model with position information of the points on the model to generate a common perspective without depth information; Step C22, outputting the distance value from the fragment to the observation point to a color buffer memory to obtain a two-dimensional projection of a surface distance field taking the observation point as a datum point on a screen, namely a distance map, wherein the z value after projection conversion is shown in the following formula: ; Wherein, -f and-n are respectively distance between far and near planes; Step C23, converting the distance z ndc and the depth z eye into a linear relation to obtain: ; And C3, extracting a DEM (digital elevation model) astronomical line: Step C31, firstly, calculating first derivatives of the DEM image in the horizontal direction and the vertical direction by adopting a Sobel operator through a Canny algorithm, so as to obtain gradient diagrams of the DEM image in the horizontal direction and the vertical direction, and finally obtaining gradients and directions of boundaries of the two gradient diagrams, wherein the steps are as follows: ; Wherein, the And Gradient in x and y directions, edge_gradient and Angle denote Gradient and direction; Step C32 non-maximum suppression Judging the gradient of the pixel point and the gradient of two points in the front and back directions of the gradient of the point, and judging whether the point is a local maximum gradient point in the neighborhood of the point or not, and if the point is not the local maximum gradient point, eliminating the point; And step C33, adopting a method of double-threshold detection and hysteresis boundary, identifying points larger than the upper threshold boundary as strong edges, identifying points lower than the lower threshold boundary as non-edges, identifying points in the middle of the upper threshold boundary and the lower threshold boundary as weak edges, performing hysteresis tracking, namely identifying the weak edges connected with the strong edges as edges, and otherwise identifying the weak edges as non-edges.
  9. 9. An absolute position sensing system under GPS rejection conditions, comprising: The natural panoramic image acquisition and splicing unit is used for carrying out 360-degree annular splicing on the discrete natural images acquired by the vehicle-mounted camera to form a natural panoramic image; a natural image panoramic astronomical line extraction unit, which is used for extracting a natural image panoramic astronomical line from a natural panoramic image by adopting a semantic segmentation network based on Deeplabv3+ improvement; the DEM image panoramic astronomical line extraction unit is used for obtaining DEM data of a position area to be perceived based on a remote sensing center, forming a rendering graph by using OpenGL (open graphics library) and carrying out edge detection on the rendering graph by using a Canny operator to output a DEM image panoramic astronomical line; The position sensing unit adopts a migration learning method, adopts VGG16 as a feature encoder of a twin neural network, and utilizes the twin neural network to output the similarity between a panoramic astronomical line of a natural image and a panoramic astronomical line of a DEM image as a matching result, so that the position of the user is sensed according to the matching result of the astronomical line; The natural panoramic image acquisition and stitching unit, the natural image panoramic astronomical line extraction unit, the DEM image panoramic astronomical line extraction unit and the position sensing unit perform data processing by adopting the absolute position sensing method under the GPS rejection condition according to any one of claims 1 to 8.

Description

Absolute position sensing method and system under GPS refusing condition Technical Field The invention belongs to the field of image information processing, and particularly relates to an absolute position sensing method and system under a GPS refusing condition. Background In the prior art, satellite positioning technologies such as a GPS and a Beidou are generally used, but under certain specific conditions, the satellite positioning technologies cannot be used, for example, in a war state, electronic equipment such as a satellite is a primary hit target, once the electronic equipment is destroyed, the positioning cannot be performed, all weapon equipment, an airplane, a vehicle and the like cannot know the position of the electronic equipment, and in the wild state, satellite signals are weak due to interference, so that the satellite technologies such as the GPS cannot be used and cannot be positioned. Therefore, it is important to develop a technique capable of performing self-positioning without depending on GPS. Independent of a positioning technology of satellites such as a GPS (global positioning system), li Yin et al (patent publication No. CN 109579841A) disclose a vehicle-mounted fire-fighting high-load rotor unmanned aerial vehicle high-precision positioning method under the GPS rejection condition, and a pull rope type displacement sensor and an axle angle encoder are used for carrying out high-precision positioning on the vehicle-mounted unmanned aerial vehicle, but the positioning is only used for positioning the height of the unmanned aerial vehicle and not the position of the unmanned aerial vehicle, wang Weiping et al (patent publication No. CN 110068335A) disclose a unmanned aerial vehicle cluster real-time positioning method and system under the GPS rejection environment, and the common characteristics are triangulated by detecting and matching the captured images with each other to form sparse reconstruction, so that a global map which can be accessed by all unmanned aerial vehicles is generated. In recent years, in the absolute positioning of the GPS rejection condition, most of the positioning is performed based on the natural line, and the natural line has unique, stable and other characteristics, and is not changed in the natural for a long period of time, and the natural line in each place has almost different characteristics, so that the positioning can be well used as the positioning characteristics. Currently Tzeng et al propose an image vision positioning system based on DEM and astronomical lines in a desert environment (in The 18th International Conferenceon Information Fusion,2013, volume 3) that does not use any metadata such as GPS readings, camera focal length or field of view, uses only publicly available Digital Elevation Models (DEM) to quickly and accurately position photographs in The desert, but with low accuracy. In this context, it is important to study a method for absolute position location based on information provided only by natural environment, such as the astronomical line, with high accuracy and without depending on GPS information. Disclosure of Invention The invention aims to solve the technical problem of providing an absolute position sensing method and system under the GPS rejection condition, which are used for searching and matching an astronomical line by adopting natural inherent astronomical line characteristics and combining with DEM (digital elevation data), so that the absolute position positioning under the GPS rejection condition is realized, and the absolute position positioning is integrated with the system to form a complete system. The technical scheme adopted by the invention is as follows: a method of absolute position awareness under GPS rejection conditions, the method comprising the steps of: step A, performing 360-degree annular stitching on discrete natural images acquired by a vehicle-mounted camera to form a natural panoramic image; Step B, extracting a natural image panoramic astronomical line from a natural panoramic image by adopting a semantic segmentation network based on Deeplabv & lt+ & gt improvement; step C, obtaining DEM data of a position to-be-perceived area based on a remote sensing center, forming a rendering graph by using OpenGL (open graphics library), and performing edge detection on the rendering graph by using a Canny operator to output a panoramic astronomical line of a DEM image; And D, adopting a migration learning method, adopting VGG16 as a feature encoder of a twin neural network, and utilizing the twin neural network to output the similarity between the panoramic astronomical line of the natural image and the panoramic astronomical line of the DEM image as a matching result, so as to perceive the self position according to the matching result of the astronomical line. Further, the specific processing procedure of the step A is as follows: A1, shooting at intervals of 40 degrees each time on a scout ca