CN-120765555-B - Rock and soil deformation measuring method and remote monitoring system based on computer vision

CN120765555BCN 120765555 BCN120765555 BCN 120765555BCN-120765555-B

Abstract

The invention discloses a rock-soil deformation measurement method and a remote monitoring system based on computer vision, wherein the deformation measurement method comprises the steps of constructing a target data set, training a model, calibrating an image, selecting monitoring points, determining target matching and searching parameters, searching whole pixels, searching sub-pixels, converting real coordinates and calculating displacement, integrating the advantages of deep learning rapid positioning and DIC accurate searching, solving the problems that the traditional DIC method fails in severe environment and is difficult to track transient large deformation represented by surface subsidence, and maintaining monitoring continuity under the condition of temporary shielding; the remote monitoring system consists of an image acquisition module, an image transmission module and a deformation calculation and visualization display module, realizes the on-line monitoring and management of the deformation of the rock-soil structure through deformation calculation software and a visualization management platform, reduces the cost and the deployment difficulty of deformation monitoring, improves the accuracy and the efficiency of deformation calculation, has strong universality and is easy for engineering popularization.

Inventors

LI YUANHAI
XU XIAOHUA
HE WEIGUO
WEI LIYUAN
YANG SHUO
FAN GUOGANG
LIU QINGFANG
ZHAO WANYONG

Assignees

中国矿业大学
中铁第六勘察设计院集团有限公司

Dates

Publication Date: 20260508
Application Date: 20250623

Claims (10)

1. The rock and soil deformation measuring method based on computer vision is characterized by comprising the following steps of: Step 1, constructing a target data set, namely acquiring a target image under indoor simulation and actual scenes, constructing a minimum external rectangle to mark the target image, taking an inner ring number of the target as a class label thereof, completing the construction of the target data set through data expansion, and dividing the target data set into a training set and a verification set according to a preset proportion; Training a target data set by using a deep-learning target detection model, and performing joint optimization by using cross entropy loss and DFL+ CIOU loss aiming at target classification and positioning to obtain a target recognition model; Step 3, image calibration and monitoring point selection, namely, in a group of time sequence deformation images containing targets, taking a first image in the group of time sequence images as a reference image, taking any other image as a displacement analysis image, determining the monitoring point and the calibration point of each target in the reference image, setting a plane Z=0 under a world coordinate system on the plane of the target, and calculating a unit change matrix H of the plane of each target through pixel coordinates and world coordinates of 4 calibration points; Step 4, target matching and searching parameters are determined, namely, a minimum circumscribed rectangle and a category of a target in a time sequence deformation image are obtained by utilizing a target recognition model, the target in a reference image and the target in a displacement analysis image are matched according to the category, if the target i is successfully matched, a search frame range (X min : X max, Y min : Y max ) for DIC calculation and a square matching frame size W of the target i in the displacement analysis image are dynamically calculated according to the minimum circumscribed rectangle, otherwise, the target i is considered to be lost or temporarily blocked when the displacement analysis image is shot; Step 5, searching the whole pixels, namely if the target i in the step 4 is successfully matched, constructing a square template matching frame u with the side length W by taking a monitoring point of the target i in a reference image as a center, constructing a search frame with the range of (X min : X max, Y min : Y max ) in a displacement analysis image, constructing a retrieval matching frame v with the side length W in the displacement analysis image by taking any pixel point in the search frame as a center, and calculating correlation coefficients R of the two matching frames; Step 6, sub-pixel searching, namely acquiring coordinates (X, Y) of nine points in an eight-neighborhood pixel matrix of the pixel point based on the whole pixel position coordinates of the target i monitoring point obtained in the step 5 in the displacement analysis image, constructing a three-dimensional data point set based on the nine points (X, Y, R), fitting a secondary phase Guan Qumian by adopting the data point set, and obtaining sub-pixel coordinates of the target i monitoring point by solving extreme points of a curved surface; Step 7, converting real coordinates, namely obtaining a homography transformation matrix H corresponding to each target plane according to the step 3, and converting sub-pixel coordinates of the monitoring points of the target i into world coordinates; And 8, calculating displacement, namely calculating the real displacement value of the target i according to the world coordinates in the step 7.
2. The rock and soil deformation measuring method based on computer vision is characterized in that in the step 1, a target image is designed into a circle and divided into an outer ring (2) and an inner ring, the inner ring is a target body and comprises a background (1), lines (3), a calibration circle (4), dots (5) and numerals (6), wherein the area of the background (1) is equal to that of the target body, the dots (5) are monitoring points and are arranged at the center point of the target body, the numerals (6) are positioned at the center of the target body and used for distinguishing multiple targets, four lines (3) are arranged in total and penetrate through the dots and are just the diameter of the target body and used for increasing the texture complexity of the background (1), the calibration circle (4) comprises four lines and is used for calibrating a target plane, and the circle centers of the calibration points are positioned at two ends of the two vertically crossed lines.
3. The rock and soil deformation measuring method based on computer vision according to claim 2, wherein in the step 3, the pixel positions of the calibration points are obtained through two methods, when the radius of a calibration circle in a time sequence deformation image is more than or equal to 10piexls, the range of four calibration circles is manually constructed, then the circle centers of the four calibration circles are detected by adopting a Hough circle detection method, so that the pixel coordinates of the four calibration points are obtained, and when the radius of a blue calibration circle is less than 10piexls, the circle centers of the calibration circles are manually selected as the calibration points.
4. The method of claim 1, wherein in step 4, when the reference image is successfully matched with the target i in the displacement analysis image, the range (Xmin: xmax, ymin: ymax) of the search box in the displacement analysis image and the size W of the square matching box adapted to the targets with different sizes are dynamically calculated, as shown in formula (1): ; Wherein (X, Y) is the coordinate of the upper left corner of the minimum circumscribed rectangular frame, W is the width of the minimum circumscribed rectangular frame, h is the height of the minimum circumscribed rectangular frame, n is a scaling factor, X min , X max respectively represents two boundaries of the search frame in the X direction of the displacement analysis image, Y min , Y max respectively represents two boundaries of the search frame in the Y direction of the displacement analysis image, and W is the side length of the square matching frame; In step 5, the correlation coefficient R of the two matching boxes is calculated, as shown in formula (2): ; wherein R is a correlation coefficient; The average gray value of the template matching frame; the method comprises the steps of searching an average gray value of a matching frame, wherein u (x, y) is the average gray value of a template matching frame at a (x, y) point, and v (x, y) is the average gray value of the matching frame at the (x, y) point; in step 6, the equation of the secondary related surface is shown in formula (3): ; Wherein: For the correlation coefficient at any image coordinate point (x, y), ~ Is a curved surface coefficient; In step 7, the subpixel coordinates are converted into world coordinates by formula (4): ; wherein (X, Y) is the position of any point in the plane of the target i under the image coordinate system, and (X, Y) is the coordinate of the Z=0 plane (the plane of the target i under the world coordinate system) projected by the point (X, Y) under the world coordinate system; S is a scaling factor; In step 8, the true displacement value is calculated by the formula (5): ; Wherein: The displacement of the monitoring point of the target i in the X direction at the moment of the displacement analysis image is obtained; The displacement of the monitoring point of the target i in the Y direction at the moment of the displacement analysis image is obtained; the total displacement of the target i monitoring point at the moment of the displacement analysis image is shown , ) The real coordinate value of the target i monitoring point in the reference image is shown , ) And analyzing the true coordinate value of the monitoring point of the target i in the image for displacement.
5. The method for measuring rock and soil deformation based on computer vision according to claim 4, wherein in step 8, if the target i is lost or temporarily blocked at the moment of displacement analysis image, the position of the target i at the moment of displacement analysis image is roughly estimated through a formula (6), then the true displacement value of the target i is calculated according to a formula (5), and an error log of a word of 'lost or temporarily blocked target i' is returned in programming until the target i in a subsequent image is successfully re-matched, and the original calculation process is restored; ; the Chinese medicine is:. The Chinese medicine is: (Chinese medicine) , ) The true coordinate value of the target i monitoring point in the displacement analysis image is calculated , ) The real coordinate value of the target i monitoring point in the previous image of the displacement analysis image is obtained; The average displacement speed of the target i in the X direction in the previous interval time is set as the average displacement speed of the target i in the X direction; the average displacement speed of the target i in the Y direction in the previous interval time is set as the average displacement speed of the target i in the Y direction; The time between the image and the previous image is analyzed for displacement.
6. A remote monitoring system for implementing the computer vision based rock deformation measurement method of any one of claims 1-5, comprising: the image acquisition module comprises a network camera, an illuminating lamp and a self-made target device, wherein the self-made target device is arranged in a deformation monitoring area, the network camera is arranged in a stable area far away from the deformation monitoring area, the target is shot in an alignment manner, and the visual field range of the camera covers the whole target arrangement range; The image transmission module comprises an intelligent gateway and a cloud server, wherein the intelligent gateway is directly connected with the network camera and the relay through a network cable and is connected with a network through two modes of a 4G card and the network cable; the intelligent gateway is positioned under a local area network, the cloud server is positioned under a public network, and the ports of a control network camera capture, focal length adjustment and a relay switch are mapped under the corresponding ports of the public network through a rapid reverse proxy (FRP) intranet penetration technology to establish a data transmission channel, wherein the intelligent gateway carries an FRP client program and can automatically start the FRP client program after being started and networked; The deformation calculation and visual display module comprises a deformation calculation server, deformation calculation software and a visual management platform, wherein the deformation calculation server is simultaneously provided with the deformation calculation software and the visual management platform, and respectively carries out on-line monitoring, visual display and early warning of deformation of the rock-soil structure.
7. The remote monitoring system according to claim 6, wherein the self-made targeting device is made of steel material, and is in the shape of a right triangular prism, the side surface is hollowed out, and the thickness of three complete surfaces of the remaining bottom surface, the vertical front surface and the inclined back surface is set to be 20mm, wherein the bottom surface comprises four anchor holes, the vertical front surface is vertical to the ground, and the target image is carved on the vertical front surface, and the vertical displacement represents the ground subsidence.
8. The remote monitoring system according to claim 6, wherein in the deformation calculation and visualization display module, a specific analysis flow of the deformation calculation and visualization display is as follows: The method comprises the steps of (1) controlling a network camera to shoot an initial image through a public network port after connection mapping to load the initial image into a software page, storing the image into a designated folder A, selecting a central point of each target in the initial image as a monitoring point through a mouse, adding a label to the monitoring point according to the number of the center of the target, setting an acquisition method of the position of the calibration point according to the size of a calibration circle in each target in the image, selecting and constructing a calibration circle range or directly selecting the calibration point according to the acquisition method, simultaneously automatically generating a txt-format data file for storing a calculation result for each monitoring point under the folder A by the software, automatically generating a data table with the same number as the monitoring point in a database, wherein the names of the data files and the data table are consistent with the labels of the monitoring points, then automatically grabbing an image every N seconds, and storing the image into the folder A, and then, searching all images in the folder A by the deformation calculation software, and adding the latest grabbed image into a to-be-analyzed list of the software; The method comprises the steps of (2) carrying out deformation analysis on a newly-added image through a deformation measurement algorithm built in deformation analysis software to obtain an X-direction displacement value, a Y-direction displacement value, a total displacement value and an error log of each target at the current moment, saving calculation results of each monitoring point into a corresponding data file under a folder A by the software, saving calculation results of each monitoring point into a corresponding data table in a database, and synchronously updating deformation curves built in the software; And 3) constructing a visual management platform by adopting a WebGIS, marking on an electronic map according to the real longitude and latitude coordinates of each monitoring point, simultaneously establishing connection with a data table corresponding to each monitoring point in a database, setting a deformation alarm threshold according to different engineering requirements, searching the database once at intervals of N seconds after the setting, loading the new data into the platform if the new data exists, visually displaying, and performing risk early warning according to the set displacement threshold.
9. The remote monitoring system according to claim 8, wherein in the process 1), before capturing an image, the relay is controlled to be turned on and turned off, after capturing the image, the relay is turned off and the illumination is turned off, the captured image is automatically generated according to the capturing time, the basic format of the image name is "year-month-day & hour-minute-second. Bmp", each data table in the database contains seven fields of an ID (int), a current time (datatime), a time interval (double) with a previous image, an X-direction displacement (double), a Y-direction displacement (double), a total displacement (double), and an error log (varchar), wherein "ID" represents the number of images currently analyzed, "the time interval of the previous image" is in seconds, "the X-direction displacement", "the Y-direction displacement", "the total displacement" are in mm, and "the error log" is error information occurring in the analysis process.
10. The remote monitoring system according to claim 8, wherein in the process 2), the deformation measurement algorithm adopts the rock deformation measurement method based on computer vision according to claim 1, immediately after the deformation of the current analysis image is calculated, the next image in the to-be-analyzed list is calculated, if no residual image exists in the to-be-analyzed list, the program is paused until the program is restarted after the new loading image in the to-be-analyzed list, after a new set of deformation data is obtained, the program respectively constructs a character string and a list according to six data of ' the number of the current analysis image ', ' the current time ', ' the interval time between the current image and the previous image ', ' the X direction displacement ', ' the Y direction displacement ', ' the total displacement ' and ' the error log ', and stores the character string and the list into a data file and a data table, wherein ' the error log ' only contains ' no ' and ' target loss or ' temporary occlusion ', and corresponds to the situation that the target matching is successful and not successful in the rock deformation measurement method based on computer vision respectively.

Description

Rock and soil deformation measuring method and remote monitoring system based on computer vision Technical Field The invention belongs to the technical field of geotechnical engineering deformation monitoring, and particularly relates to a geotechnical deformation measuring method and a remote monitoring system based on computer vision. Background In the engineering construction and ground disaster investigation process, the change degree of the rock-soil structure such as the earth surface, the slope and the like needs to be accurately observed, and the geographic information database is statistically updated, so that decision support is provided for engineering construction management and emergency rescue. The traditional rock-soil structure deformation monitoring mainly adopts a leveling technique, and the technique has the defects of high measurement precision, simple operation, large field work load, low speed, low coverage density, difficult accurate mastering of the spatial distribution characteristics of sedimentation and the like. The development of camera photosensitive elements has also driven the rapid advance of photogrammetry techniques. The shape, size, position, characteristics and interrelationships of the object to be photographed are obtained through digital processing by taking a picture of the object to be measured by using a photo obtained by an optical camera. The vision measurement technology is used as a non-contact modern optical measurement technology, has the characteristics of high measurement precision, long-distance multipoint, high frequency, full-field measurement and the like, and the main vision measurement algorithms at present comprise a feature point matching method, an optical flow estimation method, a digital image Correlation method (DIGITAL IMAGE corelation, DIC) and the like. The digital image correlation method searches the monitoring points through the correlation of the matching areas, and the resolution can be thinned into sub-image level through the difference method, so that the digital image correlation method has higher accuracy, adaptability and reliability. However, the current vision measurement method still has the limitation that the monitoring is invalid due to large environmental interference such as illumination, shielding, shadow, rain and fog, and the like, so that the stable monitoring cannot be maintained, the stability of long-time monitoring in the natural environment is difficult to ensure, most of monitoring researches only pay attention to the improvement of a monitoring algorithm, and a set of mature and reliable system is not formed. Disclosure of Invention The invention aims to provide a rock and soil deformation measuring method and a remote monitoring system based on computer vision, which can effectively solve the problems that the existing vision measuring method is greatly interfered by severe environment, is difficult to monitor due to instantaneous large deformation, has poor stability and is immature in system. In order to achieve the above purpose, the invention provides a rock and soil deformation measuring method based on computer vision, which comprises the following steps: Step 1, constructing a target data set, namely acquiring a target image under indoor simulation and actual scenes, constructing a minimum external rectangle to mark the target image, taking an inner ring number of the target as a class label thereof, completing the construction of the target data set through data expansion, and dividing the target data set into a training set and a verification set according to a preset proportion; Training a target data set by using a deep-learning target detection model, and performing joint optimization by using cross entropy loss and DFL+ CIOU loss aiming at target classification and positioning to obtain a target recognition model; Step 3, image calibration and monitoring point selection, namely, in a group of time sequence deformation images containing targets, taking a first image in the group of time sequence images as a reference image, taking any other image as a displacement analysis image, determining the monitoring point and the calibration point of each target in the reference image, setting a plane Z=0 under a world coordinate system on the plane of the target, and calculating a unit change matrix H of the plane of each target through 4 pixel coordinates and world coordinates passing through the calibration point; Step 4, target matching and searching parameters are determined, namely, a minimum circumscribed rectangle and a category of a target in a time sequence deformation image are obtained by utilizing a target recognition model, the target in a reference image and the target in a displacement analysis image are matched according to the category, if the target i is successfully matched, a search frame range (X min:Xmax,Ymin:Ymax) for DIC calculation and a square matching frame size W of the target i in