CN-116363353-B - Oil pipe detection method and device, electronic equipment and storage medium

CN116363353BCN 116363353 BCN116363353 BCN 116363353BCN-116363353-B

Abstract

The invention provides an oil pipe detection method, an oil pipe detection device, electronic equipment and a storage medium, and relates to the field of oil pipe detection, wherein the method comprises the steps of obtaining an oil pipe image to be detected, and extracting characteristics of the oil pipe image to be detected to obtain a corresponding characteristic map; the method comprises the steps of carrying out key point prediction on a characteristic map by using a preset key point detection model to predict the positions of a preset number of first key points representing the positions and the forms of the oil pipes in an image of the oil pipes to be detected, determining the positions and the forms of the oil pipes in the image of the oil pipes to be detected according to the positions of the first key points, detecting flexible targets such as the oil pipes in a key point detection mode, further effectively detecting the oil pipes in various postures in various environments, and improving the robustness of oil pipe detection.

Inventors

DONG BAOLEI
BAO HANBIN
ZHANG TONGSHUN

Assignees

济南博观智能科技有限公司

Dates

Publication Date: 20260505
Application Date: 20230310

Claims (10)

1. An oil pipe detection method, comprising: Acquiring an oil pipe image to be detected, and extracting features of the oil pipe image to be detected to obtain a corresponding feature map; Performing key point prediction on the characteristic map by using a preset key point detection model to predict the positions of a preset number of first key points representing the positions and the forms of the oil pipe in the image of the oil pipe to be detected; determining the position and the form of the oil pipe in the oil pipe image to be detected according to the position of the first key point; the training process of the preset key point detection model comprises the following steps: The method comprises the steps of obtaining a training oil pipe image, carrying out feature extraction on the training oil pipe image to obtain a corresponding training feature map, marking the training oil pipe image with the preset number of oil pipe key points, recording the positions of the oil pipe key points and the affinities among the oil pipe key points, carrying out key point prediction on the training feature map by using a first network branch of a preset key point detection model to predict the positions of second key points corresponding to the oil pipe key points in the training oil pipe image, and carrying out key point affinity prediction on the training feature map by using a second network branch of the preset key point detection model to predict the affinities among the second key points; the oil pipe key points comprise turning key points and fitting key points, the turning key points are marked at the starting positions of the oil pipe and the bending positions of which the bending angles are larger than a preset value, the fitting key points are marked between adjacent turning key points, and the second network branches of the preset key point detection model are used for carrying out key point affinity prediction on the training feature map so as to predict the affinities among the second key points, and the method comprises the following steps: The method comprises the steps of adding all second key points to a key point sequence according to the arrangement sequence of the oil pipe key points, setting the first second key point in the key point sequence as a key point to be calculated, determining the type of the oil pipe key point corresponding to the key point to be calculated, if the type is a turning key point, searching a next target second key point adjacent to the key point to be calculated and corresponding to the turning key point in the key point sequence, setting a vector from the key point to be calculated to the target second key point as an affinity value corresponding to the key point to be calculated, if the type is a fitting key point, determining that two target second key points adjacent to the key point to be calculated and corresponding to the turning key point are searched in the key point sequence, determining a straight line passing through the two target second key points, and setting the distance between the key point to be calculated and the straight line as the affinity value corresponding to the key point to be calculated.
2. The oil pipe detection method according to claim 1, wherein the performing the keypoint prediction on the feature map using a preset keypoint detection model includes: inputting the characteristic map to the preset key point detection model to generate a confidence map corresponding to each first key point by using the preset key point detection model, wherein the confidence map is used for recording the confidence degree that each pixel point in the oil pipe image to be detected is the first key point corresponding to the confidence map; And determining the position of each first key point in the oil pipe image to be detected according to the confidence map.
3. The oil pipe detection method according to claim 1 or 2, wherein the training process of the preset key point detection model further comprises: Generating a first loss value by using the positions of the second key points and the positions of the corresponding oil pipe key points, and generating a second loss value by using the affinities of the second key points and the affinities of the corresponding oil pipe key points; Generating a total loss value by using the first loss value and the second loss value, and judging whether the total loss value is larger than a preset threshold value or not; if yes, carrying out parameter updating on the preset key point detection model by using the total loss value, carrying out key point prediction on the training feature map by using a first network branch of the preset key point detection model based on the updated preset key point detection model, and carrying out key point affinity prediction on the training feature map by using a second network branch of the preset key point detection model; If not, training the preset key point detection model is completed.
4. The oil pipe detection method according to claim 1, wherein the performing, by using the second network branch of the preset keypoint detection model, the keypoint affinity prediction on the training feature map to predict the affinity between the second keypoints further comprises: when the calculation of the affinity value of the current key point to be calculated is completed, if the second key point which does not calculate the affinity value exists in the key point sequence, updating the key point to be calculated into the next second key point, and entering the step of determining the type of the oil pipe key point corresponding to the key point to be calculated; and if the fact that the second key points which do not calculate the affinity value do not exist in the key point sequence is determined, completing the affinity prediction of the key points.
5. The method for detecting an oil pipe according to claim 3, wherein generating the second loss value using the affinity between the second keypoints and the affinity between the corresponding oil pipe keypoints comprises: Setting a first second key point in the key point sequence as a key point to be processed, and determining the type of an oil pipe key point corresponding to the key point to be processed; If the type is a turning key point, determining a cosine included angle value between the vector corresponding to the key point to be processed and the vector corresponding to the oil pipe key point corresponding to the key point to be processed, and setting the cosine included angle value as an affinity loss value corresponding to the key point to be processed; If the type is a fitting key point, determining a distance loss value between the distance corresponding to the key point to be processed and the distance corresponding to the oil pipe key point corresponding to the key point to be processed, and setting the distance loss value as an affinity loss value corresponding to the key point to be processed; When the calculation of the affinity loss value of the current key point to be processed is completed, if the fact that a second key point which does not calculate the affinity loss value exists in the key point sequence is determined, updating the key point to be processed into a next second key point, and determining the type of the oil pipe key point corresponding to the key point to be processed; And if the fact that the second key points which do not calculate the affinity loss value do not exist in the key point sequence is determined, generating the second loss value by using the affinity loss values corresponding to all the second key points.
6. The oil pipe detection method according to claim 1, wherein the performing, by using the first network branch of the preset key point detection model, the key point prediction on the training feature map to predict a position of a second key point corresponding to each oil pipe key point in the training oil pipe image, and performing, by using the second network branch of the preset key point detection model, the key point affinity prediction on the training feature map to predict an affinity between each second key point, includes: setting the training feature map as a feature to be processed, and initializing the current stage to be 1; Inputting the feature to be processed into a first network branch corresponding to the preset key point detection model at the current stage to predict the position of each second key point in the training oil pipe image at the current stage, and inputting the feature to be processed into a second network branch corresponding to the preset key point detection model at the current stage to predict the affinity among each second key point at the current stage; generating a first sub-loss value by using the positions of the second key points and the positions of the corresponding oil pipe key points in the current stage, and generating a second sub-loss value by using the affinities of the second key points and the affinities of the corresponding oil pipe key points in the current stage; Fusing the training feature map with the positions of the second key points in the current stage and the affinities of the second key points in the current stage to obtain intermediate features; if the current stage value is determined to be smaller than the preset stage value, adding one to the current stage, setting the intermediate feature as the feature to be processed, and entering the step of inputting the feature to be processed into a first network branch corresponding to the preset key point detection model in the current stage; Correspondingly, the generating a first loss value by using the position of each second key point and the position of the corresponding oil pipe key point, and generating a second loss value by using the affinity between each second key point and the affinity between the corresponding oil pipe key points, includes: if the current stage value is determined to be equal to the preset stage value, calculating the first loss value by using a first sub-loss value of each stage, and calculating the second loss value by using a second sub-loss value of each stage.
7. An oil pipe detection device, characterized by comprising: the feature extraction module is used for acquiring an oil pipe image to be detected, and extracting features of the oil pipe image to be detected to obtain a corresponding feature map; The key point prediction module is used for predicting key points of the characteristic map by using a preset key point detection model so as to predict the positions of a preset number of first key points representing the positions and the forms of the oil pipe in the image of the oil pipe to be detected; The oil pipe determining module is used for determining the position and the form of the oil pipe in the oil pipe image to be detected according to the position of the first key point; the oil pipe detection device further comprises: The training oil pipe image feature extraction module is used for acquiring a training oil pipe image, and carrying out feature extraction on the training oil pipe image to obtain a corresponding training feature map, wherein the training oil pipe image is marked with a preset number of oil pipe key points, and the positions of the oil pipe key points and the affinities among the oil pipe key points are recorded; The training module is used for predicting key points of the training feature map by using a first network branch of the preset key point detection model so as to predict the positions of second key points corresponding to the key points of each oil pipe in the training oil pipe image, and predicting the affinity of the key points of the training feature map by using a second network branch of the preset key point detection model so as to predict the affinities among the second key points; The oil pipe key points comprise turning key points and fitting key points, the turning key points are marked at the starting positions of the oil pipe and the bending positions of which the bending angles are larger than a preset value, the fitting key points are marked between adjacent turning key points, and the training module comprises: The adding sub-module is used for adding all the second key points to the key point sequence according to the arrangement sequence of the key points of the oil pipe; the first type determining submodule is used for setting a first second key point in the key point sequence as a key point to be calculated and determining the type of an oil pipe key point corresponding to the key point to be calculated; The first affinity calculation sub-module is used for searching a next target second key point which is adjacent to the key point to be calculated and corresponds to the turning key point in the key point sequence if the type is the turning key point, and setting a vector from the key point to be calculated to the target second key point as an affinity value corresponding to the key point to be calculated; And the second affinity calculation submodule is used for determining to find two target second key points adjacent to the key point to be calculated and corresponding to the turning key point in the key point sequence if the type is the fitting key point, determining a straight line passing through the two target second key points, and setting the distance between the key point to be calculated and the straight line as an affinity value corresponding to the key point to be calculated.
8. The oil pipe detection apparatus of claim 7, wherein the keypoint prediction module comprises: The confidence map generation sub-module is used for inputting the characteristic map to the preset key point detection model so as to generate a confidence map corresponding to each preset key point by utilizing the preset key point detection model, wherein the confidence map is used for recording the confidence degree that each pixel point in the oil pipe image to be detected is a first key point corresponding to the confidence map; And the key point prediction sub-module is used for determining the position of each first key point in the oil pipe image to be detected according to the confidence map.
9. An electronic device, comprising: A memory for storing a computer program; A processor for implementing the tubing detection method according to any one of claims 1 to 6 when executing said computer program.
10. A storage medium having stored therein computer executable instructions which when loaded and executed by a processor implement the tubing detection method of any one of claims 1 to 6.

Description

Oil pipe detection method and device, electronic equipment and storage medium Technical Field The present invention relates to the field of oil pipe detection, and in particular, to an oil pipe detection method, an oil pipe detection device, an electronic device, and a storage medium. Background In petrochemical industry, in order to better standardize the execution of oil stabilizing and discharging operations by workers, it is very important to perform target recognition and gesture detection on oil pipes. In the related art, if the shape of the oil pipe is identified in a gradient calculation mode by adopting a traditional machine vision method, the identification process is easily influenced by complex background, if the oil pipe target is identified by adopting a target detection algorithm, the oil pipe is difficult to position by utilizing a target detection frame in consideration of different forms of the oil pipe, and meanwhile, the model is difficult to converge due to the complexity of the form of the oil pipe, so that no little difficulty is brought to model training. Disclosure of Invention The invention aims to provide an oil pipe detection method, an oil pipe detection device, electronic equipment and a storage medium, which can detect flexible targets such as oil pipes in a key point detection mode, can effectively detect oil pipes in various postures in various environments, and further can improve the robustness of oil pipe detection. In order to solve the technical problems, the invention provides an oil pipe detection method, which comprises the following steps: Acquiring an oil pipe image to be detected, and extracting features of the oil pipe image to be detected to obtain a corresponding feature map; Performing key point prediction on the characteristic map by using a preset key point detection model to predict the positions of a preset number of first key points representing the positions and the forms of the oil pipe in the image of the oil pipe to be detected; and determining the position and the shape of the oil pipe in the image of the oil pipe to be detected according to the position of the first key point. Optionally, the performing the keypoint prediction on the feature map by using a preset keypoint detection model includes: inputting the characteristic map to the preset key point detection model to generate a confidence map corresponding to each first key point by using the preset key point detection model, wherein the confidence map is used for recording the confidence degree that each pixel point in the oil pipe image to be detected is the first key point corresponding to the confidence map; And determining the position of each first key point in the oil pipe image to be detected according to the confidence map. Optionally, the training process of the preset keypoint detection model includes: acquiring a training oil pipe image, and carrying out feature extraction on the training oil pipe image to obtain a corresponding training feature map, wherein the training oil pipe image is marked with the preset number of oil pipe key points, and the positions of the oil pipe key points and the affinities among the oil pipe key points are recorded; Performing key point prediction on the training feature map by using a first network branch of the preset key point detection model to predict the position of a second key point corresponding to each oil pipe key point in the training oil pipe image, and performing key point affinity prediction on the training feature map by using a second network branch of the preset key point detection model to predict the affinity between the second key points; Generating a first loss value by using the positions of the second key points and the positions of the corresponding oil pipe key points, and generating a second loss value by using the affinities of the second key points and the affinities of the corresponding oil pipe key points; Generating a total loss value by using the first loss value and the second loss value, and judging whether the total loss value is larger than a preset threshold value or not; If yes, carrying out parameter updating on the prediction model by using the total loss value, carrying out key point prediction on the training feature map by entering the first network branch by using the preset key point detection model based on the updated prediction model, and carrying out a key point affinity prediction step on the training feature map by using the second network branch of the preset key point detection model; If not, training the preset key point detection model is completed. Optionally, the oil pipe key points include turning key points and fitting key points, the turning key points are marked at starting positions of the oil pipe and bending positions where bending angles are larger than a preset value, the fitting key points are marked between adjacent turning key points, and the second network branch of the preset key