CN-121821417-B - Ammeter operation robot control method based on environment perception

CN121821417BCN 121821417 BCN121821417 BCN 121821417BCN-121821417-B

Abstract

The invention relates to the technical field of robot control and visual perception processing, and discloses an ammeter operation robot control method based on environment perception, which comprises the steps of acquiring an operation instruction, determining a target ammeter and an operation type according to the operation instruction, and acquiring image data of the target ammeter; and performing ammeter coarse detection on the image data to obtain two-dimensional positioning information of the ammeter, and calculating pixel deviation between the center of the detection frame and the image data center. By repeatedly adjusting the observation gesture according to the pixel deviation under the condition that the mobile platform keeps static and combining the dead-facing deviation to judge the dead-facing state, the dead-facing image data can be obtained under a stable visual angle, and the problems that the fluctuation of a positioning result is large and the stable alignment is difficult due to the movement of the platform and the change of the visual angle are solved.

Inventors

WANG XIANGWEI
QIN CHAO
ZHOU JIAFU
WANG FENG
LI ZHENHONG

Assignees

科曼智能科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260313

Claims (6)

1. The ammeter operation robot control method based on environment perception is characterized by comprising the following steps of S1, acquiring an operation instruction, determining a target ammeter and an operation type, and acquiring image data of the target ammeter; Step S2, performing rough ammeter detection on the image data to obtain a detection frame and two-dimensional positioning information of a target ammeter, and calculating pixel deviation between the center of the detection frame and an image data center; Under the condition that the mobile platform keeps static, adjusting the observation gesture, repeatedly collecting image data and roughly detecting until the pixel deviation is smaller than a pixel deviation threshold value, calculating the positive deviation based on the aspect ratio of the detection frame and the aspect ratio of the working surface of the target ammeter, and adjusting the positive deviation to be smaller than the positive deviation threshold value to obtain positive image data; Acquiring the width and the height of a detection frame pixel on the facing image data, respectively calculating a distance estimated value based on the width and a distance estimated value based on the height, and taking the absolute value of the difference value of the two as a distance estimated value error; if the error of the distance estimation value is larger than a preset threshold value, acquiring the image data, otherwise taking the arithmetic average of the two as the distance estimation value, and taking whether the distance estimation value falls into a preset operable distance interval or not as a criterion for controlling the movement of the mobile platform or not; step S3, after the right-facing position is reached, synchronously acquiring two paths of image data of complementary visual angles, performing instance segmentation on the surface area of the ammeter, performing depth estimation based on the segmentation result to obtain two paths of depth results, respectively generating a front view point cloud and a diagonal view point cloud, and transforming the front view point cloud and the diagonal view point cloud into the same coordinate system; Performing three-dimensional consistency screening in the overlapped surface area of the two point clouds to remove unreliable points, giving confidence weights to the three-dimensional points passing through the screening, carrying out weighted fusion according to the weights to obtain fusion point clouds, carrying out three-dimensional optimization on the fusion point clouds, outputting three-dimensional information of a target ammeter working surface, and obtaining grabbing accurate positions of the ammeter based on the three-dimensional information; And S4, generating an ammeter operation pose according to the grabbing accurate position and the operation instruction and executing ammeter operation actions.
2. The method for controlling an electric meter operation robot based on environmental awareness according to claim 1, wherein the performing electric meter rough detection on the image data in step S2 includes: the image data is input into a target detection model based on an attention mechanism for coarse detection to obtain a detection frame of the target ammeter, wherein the detection frame at least comprises an upper left corner pixel coordinate, a lower right corner pixel coordinate and a center pixel coordinate, and the center pixel coordinate is used for calculating pixel deviation between the center of the detection frame and an image data center.
3. An electricity meter work robot control method based on environmental awareness as set forth in claim 2 wherein said pixel deviation includes: The horizontal pixel deviation deltau and the vertical pixel deviation deltav, wherein deltau=uz-u 0, deltav=vz-v 0, uz is the center horizontal pixel coordinate of the detection frame, u0 is the center horizontal pixel coordinate of the image data center, vz is the center vertical pixel coordinate of the detection frame, and v0 is the center vertical pixel coordinate of the image data center; the pixel deviation threshold value is a proportional threshold value according to the image resolution, and the proportion is 1%; The threshold value of the dead-front deviation is 5%.
4. The method for controlling an electric meter operation robot based on environmental perception according to claim 1, wherein the distance estimation value Dw based on the width and the distance estimation value Dh based on the height are calculated by a pinhole imaging relationship, dw=fx×l/mL, dh=fy×g/mG, where fx is a camera horizontal focal length pixel value, fy is a camera vertical focal length pixel value, L is a target electric meter operation surface true width, G is a target electric meter operation surface true height, mL is a positive detection frame pixel width, and mG is a positive detection frame pixel height; And when E is smaller than or equal to a preset distance estimation value error threshold, D= (Dh+Dw)/2 is taken as a distance estimation value, and whether D falls into a preset operable distance interval is taken as a criterion for controlling the movement of the mobile platform.
5. The method for controlling an electric meter operation robot based on environmental awareness according to any one of claims 1 to 4, wherein the two paths of image data of complementary viewing angles in the step S3 are collected in a synchronous triggering manner, and the image data comprises opposite viewing angle image data and oblique side viewing angle image data; The acquisition of the oblique side view angle image data meets the constraint that an optical axis of an oblique side view angle passes through a common aiming point which is the same as a dead side view angle and keeps the same working distance with the dead side view angle, and the oblique side view angle is formed by deflection of 25 degrees and deflection of 10 degrees in a pitching way on the basis of the dead side view angle; the public aiming point is an aiming point corresponding to the center of a detection frame obtained by rough detection of the ammeter; The example segmentation uses a candidate region obtained by rough detection of the ammeter as a segmentation input region, and sub-pixel level refinement processing is carried out on the boundary of an example segmentation mask to obtain a segmentation result of the ammeter surface region.
6. The method for controlling an electric meter operation robot based on environmental awareness according to claim 1, wherein the depth estimation in step S3 is performed in an effective pixel area of an electric meter surface area to obtain two-way depth results and generate a front view point cloud and a diagonal view point cloud, respectively; after transforming the two point clouds into the same coordinate system, three-dimensional consistency screening is performed in the overlapped surface area of the two point clouds, specifically: Carrying out nearest neighbor matching on three-dimensional points in the overlapping area, judging the consistency degree according to the matching error, judging the three-dimensional points with the consistency degree lower than a preset consistency screening threshold value as unreliable points, and eliminating the unreliable points; Constructing point cloud confidence weights for the three-dimensional points which are screened by consistency according to the consistency degree, carrying out weighted fusion on the two paths of point clouds according to the point cloud confidence weights to obtain fusion point clouds, inputting the fusion point clouds into a three-dimensional information optimization model for three-dimensional optimization so as to output three-dimensional information of a target ammeter working surface and obtain a grabbing accurate position according to the three-dimensional information.

Description

Ammeter operation robot control method based on environment perception Technical Field The invention relates to the technical field of robot control and visual perception processing, in particular to an ammeter operation robot control method based on environment perception. Background The existing ammeter installation and operation and maintenance work gradually introduces an operation robot to reduce the risk of manual operation and improve the efficiency, the related technology generally combines the pose adjustment and visual perception results of a mobile platform to finish the positioning and operation alignment of a target ammeter, the visual perception side generally adopts modes such as target detection or region segmentation to obtain the ammeter surface region from an image, then combines camera imaging relation or depth information to estimate the target relative position, and accordingly generates an operation pose to drive and execute the action, however, the ammeter field environment often has the characteristics of narrow space, multiple shielding objects, surface reflection intensity, weak texture and the like, so that the image detection frame shakes, the region boundary is unstable and the depth estimation noise is increased, and further, the deviation or outlier is easy to appear between the distance estimation and the three-dimensional reconstruction result based on single visual angle or single observation, and the operation alignment precision and the operation stability are affected. In addition, the existing scheme is to adapt to targets at different positions, the mobile platform is often relied on to repeatedly move to obtain available visual angles or distance conditions, the observation coordinates are continuously updated in the moving process, positioning drift is easy to introduce and error accumulation is caused, and the operation conditions can be met only by repeated heuristic adjustment, so that the operation beat becomes long, and inconsistent observation results are easier to appear in a reflective shielding scene and can not be stably converged to executable operation pose. Therefore, there is still a need for an ammeter operation control method that can achieve stable alignment and reduce ineffective movement through convergence constraint and consistency criterion of perceived results in a complex environment, so as to improve reliability and efficiency of positioning and operation. Disclosure of Invention In order to solve the technical problems, the invention provides the following technical scheme: the ammeter operation robot control method based on environment awareness comprises the following steps: step S1, acquiring a working instruction, determining a target ammeter and a working type, and acquiring image data of the target ammeter; Step S2, performing rough ammeter detection on the image data to obtain a detection frame and two-dimensional positioning information of a target ammeter, and calculating pixel deviation between the center of the detection frame and an image data center; Under the condition that the mobile platform keeps static, adjusting the observation gesture, repeatedly collecting image data and roughly detecting until the pixel deviation is smaller than a pixel deviation threshold value, calculating the positive deviation based on the aspect ratio of the detection frame and the aspect ratio of the working surface of the target ammeter, and adjusting the positive deviation to be smaller than the positive deviation threshold value to obtain positive image data; Acquiring the width and the height of a detection frame pixel on the facing image data, respectively calculating a distance estimated value based on the width and a distance estimated value based on the height, and taking the absolute value of the difference value of the two as a distance estimated value error; if the error of the distance estimation value is larger than a preset threshold value, acquiring the image data, otherwise taking the arithmetic average of the two as the distance estimation value, and taking whether the distance estimation value falls into a preset operable distance interval or not as a criterion for controlling the movement of the mobile platform or not; step S3, after the right-facing position is reached, synchronously acquiring two paths of image data of complementary visual angles, performing instance segmentation on the surface area of the ammeter, performing depth estimation based on the segmentation result to obtain two paths of depth results, respectively generating a front view point cloud and a diagonal view point cloud, and transforming the front view point cloud and the diagonal view point cloud into the same coordinate system; Performing three-dimensional consistency screening in the overlapped surface area of the two point clouds to remove unreliable points, giving confidence weights to the three-dimensional points passing through the screening, carr