CN-115485744-B - Rope strand unmanned following traction intelligent identification system and method based on machine vision

CN115485744BCN 115485744 BCN115485744 BCN 115485744BCN-115485744-B

Abstract

The invention discloses a machine vision-based intelligent rope strand unmanned trailing and pulling recognition system which comprises a data acquisition module, a data wireless transmission module, an edge calculation terminal, a rope strand attitude abnormal recognition module and an abnormal state early warning module, wherein the data acquisition module is used for acquiring front-view pictures and rear-view pictures of a puller and position information of the puller, the data wireless transmission module is used for transmitting data of the data acquisition module, the edge calculation terminal is used for receiving the front-view pictures and rear-view pictures of the puller and the position information of the puller, the edge calculation terminal also comprises a rope strand attitude abnormal recognition module and an abnormal state early warning module, the rope strand attitude abnormal recognition module is used for recognizing a rope strand abnormal state and warning through the abnormal state early warning module, and the receiving end is used for receiving abnormal warning information sent by the abnormal state early warning module. The invention also discloses an intelligent identifying method for unmanned trailing traction of the rope strands based on machine vision. The invention improves the intelligentized level of on-site strand erection, saves labor, improves traction efficiency, has strong operability, and is suitable for engineering sites.

Inventors

ZHANG YONGTAO
YANG JIANPING
LI MIAN
LI KUNYAO
DAI BAIHUA
ZHENG JIANXIN
XUE XIANKAI
LI HAO
YANG HUADONG
XIAO YAO
HU QINXIA
TIAN WEI
PAN GUILIN
HUANG CAN
ZHU HAO
PENG CHENGMING
WANG YONGWEI
LIU ZHIANG
CHEN YUAN

Assignees

中交第二航务工程局有限公司
中交公路长大桥建设国家工程研究中心有限公司

Dates

Publication Date: 20260508
Application Date: 20220804

Claims (5)

1. Rope strand unmanned trailing intelligent recognition system based on machine vision, its characterized in that includes: The data acquisition module comprises a front-view high-definition camera, a rear-view high-definition camera and a Beidou positioning terminal, wherein the front-view high-definition camera and the rear-view high-definition camera respectively acquire a front-view picture and a rear-view picture of the puller, and the Beidou positioning terminal is connected with the rear-view high-definition camera into a whole and is used for acquiring position information of the puller; The data wireless transmission module is used for data transmission of the data acquisition module; The edge computing terminal is used for receiving the front view picture and the rear view picture of the puller and the position information of the puller, which are transmitted by the data wireless transmission module, and also comprises a strand gesture abnormal recognition module and an abnormal state early warning module, wherein the strand gesture abnormal recognition module is used for recognizing the strand abnormal state through the front view picture and the rear view picture of the puller and warning through the abnormal state early warning module; the receiving end receives the abnormal alarm information sent by the abnormal state early-warning module; The abnormal alarm information comprises cable strand abnormal gesture categories, abnormal gesture picture screen shots and current position information of the puller; the edge computing terminal further comprises a winch control module which is used for controlling the start and stop of the winch, and the winch control module is used for controlling the stop of the winch after the abnormal state of the strand is identified by the strand gesture abnormal identification module; The identification method using the intelligent rope strand unmanned trailing traction identification system based on machine vision comprises the following steps: step one, installing an intelligent recognition system on a puller and carrying out communication connection with a background receiving end; step two, starting an intelligent recognition system and starting cable strand traction construction; Step three, a data acquisition module acquires a front view picture and a rear view picture of a puller through a front view high definition camera and a rear view high definition camera respectively, acquires position information of the puller through a Beidou positioning terminal, transmits data acquired by the data acquisition module to an edge computing terminal through a data wireless transmission module, and identifies a cable abnormal state through a cable strand gesture abnormal identification module after acquiring front view picture and rear view picture data of the puller, and transmits abnormal alarm information to a receiving end of a background through an abnormal state early warning module after the edge computing terminal identifies abnormality, and simultaneously controls a winch to stop through a winch control module; Step four, an operator at the receiving end checks the abnormal gesture screen shot through the background after receiving the abnormal alarm information, and determines whether the abnormal gesture screen shot is abnormal, if the abnormal gesture screen shot is abnormal, the operator rapidly reaches the position of the puller according to the current position information of the puller to process, after the processing is finished, the winch is controlled to start up to work through the winch control module, and if the operator confirms the abnormal alarm information which is wrong, the winch is controlled to continue to work through the winch control module; step five, repeating the step three and the step four until the whole traction construction of the cable strand to be hauled is completed; the specific method for identifying the abnormal state of the strand by the strand gesture abnormal identification module comprises the following steps: 3.1, carrying out semantic segmentation on the acquired strand gesture image to obtain a strand pixel semantic segmentation area and a roll pixel semantic segmentation area in the image; 3.2, extracting an circumscribed rectangle of the area by utilizing the semantic segmentation area of the riding pixel obtained in the step 3.1; 3.3, calculating an intersection region of the region and the circumscribed rectangle obtained in the step 3.2 by utilizing the strand pixel semantic segmentation region obtained in the step 3.1, and calculating geometric parameters of each region, wherein each parameter is as follows: (X w1 ,Y w1 ) -the coordinates of the leftmost point of the circumscribed rectangle of the pad; (X w2 ,Y w1 ) -the coordinates of the rightmost point of the circumscribed rectangle of the pad; (X rmin ,Y r1 ) -coordinates of the leftmost point of the strand area; (X rmax ,Y r2 ) -coordinates of the rightmost point of the strand area; w rmax -strand area maximum width value; 3.4, carrying out abnormality judgment based on geometric parameters of the supporting roller circumscribed rectangle and the strand area, wherein the method specifically comprises the following steps: ① If X w1 ＜X rmin and X rmax ＜X w2 and w rmax /( X w2 - X w1 ) is < threshold1, the system works normally and the winch is not stopped, wherein threshold1 is the threshold; ② If X w1 ＜X rmin , X rmax ＜X w2 and w rmax /( X w2 - X w1 ) is not less than threshold1, at the moment, the system works normally, the winch is not stopped, the system sends abnormal alarm information to a receiving end of the background through an abnormal state early warning module, and related operators determine the next action; ③ Removing roller: working condition 1, if X rmin ＜X w1 and X w1 ＜X rmax are adopted, the cable strand is separated from the supporting roller at the left side but is not completely separated, and at the moment, the abnormal alarm information is sent to a receiving end of the background through an abnormal state early warning module, and the next action is determined by related operators; 2, if X rmax ＜X w1 is adopted, the cable strand is completely separated from the riding roller at the left side, the abnormal state early warning module is used for sending the abnormal warning information to the receiving end of the background, and the winch is controlled to stop by the winch control module; Working condition 3, if X w2 ＜X rmax and X rmin ＜X w2 are adopted, the cable strand is separated from the supporting roller on the right side but is not completely separated, and at the moment, the abnormal state early warning module sends abnormal warning information to the receiving end of the background, and related operators determine the next action; And 4, if X w2 ＜X rmin is adopted, the cable strand is completely separated from the riding roller on the right side, the abnormal alarm information is sent to the receiving end of the background through the abnormal state early warning module, and meanwhile, the winch is controlled to stop through the winch control module.
2. The intelligent recognition system for unmanned strand following traction based on machine vision according to claim 1, wherein the specific steps of semantic segmentation of strand pose images in step 3.1 are as follows: 3.1.1, marking data, namely acquiring cable strand posture images acquired by a plurality of data acquisition modules, wherein the cable strand posture images comprise normal posture and abnormal posture images of cable strands, and marking the cable strand posture images by using a marking tool; 3.1.2, training the model, namely training the data by adopting a semantic segmentation model based on deep learning; 3.1.3, model reasoning, namely performing semantic segmentation on the newly input image by using the model trained by 3.1.2 to obtain a strand gesture semantic segmentation image, and obtaining a roll pixel semantic segmentation region and a strand pixel semantic segmentation region in the image.
3. The intelligent recognition system for unmanned strand following traction based on machine vision according to claim 1, wherein in the step 3.4, the threshold1 is 0.45.
4. The intelligent recognition system for unmanned cable strand following traction based on machine vision according to claim 2, wherein the step 3.1.2 adopts DeeplabV3+ model to train data, and a cable strand posture image semantic segmentation model is obtained.
5. The intelligent recognition system for unmanned cable-strand following traction based on machine vision according to claim 1, wherein the data wireless transmission module adopts a wireless AP and wireless network bridge fusion mode.

Description

Rope strand unmanned following traction intelligent identification system and method based on machine vision Technical Field The invention relates to the field of construction of suspension bridge strand erection. More particularly, the invention relates to a machine vision-based intelligent recognition system and method for unmanned strand following traction. Background In the existing construction of suspension bridge strand erection, a method of manually following a puller is generally adopted to monitor the strand traction erection state and judge abnormality, namely, whether the working state of the puller is normal or not is observed and judged on site, whether the erected strand just falls into a supporting roller or not is judged, whether scattered wires appear on the strand or not is judged, if the problem is found, an interphone is used for alarming and stopping the puller, and the fault is eliminated. The method requires special personnel to follow the monitoring on the catwalk, is time-consuming and labor-consuming, and has potential safety hazards in high-altitude personnel operation. The emerging image recognition technology based on machine vision has the advantages of non-contact, long distance, high precision, time and labor saving, real-time monitoring and the like, and is widely applied to the field of bridge construction, but the technology has not been applied to the construction case of strand traction state recognition so far. Disclosure of Invention An object of the present invention is to provide a system and a method for intelligent recognition of unmanned trailing traction of strands based on machine vision, so as to solve the above-mentioned problem of strand traction erection state recognition. To achieve these objects and other advantages and in accordance with the purpose of the invention, a machine vision-based intelligent recognition system for unmanned strand following traction is provided, comprising: The data acquisition module comprises a front-view high-definition camera, a rear-view high-definition camera and a Beidou positioning terminal, wherein the front-view high-definition camera and the rear-view high-definition camera respectively acquire a front-view picture and a rear-view picture of the puller, and the Beidou positioning terminal is connected with the rear-view high-definition camera into a whole and is used for acquiring position information of the puller; The data wireless transmission module is used for data transmission of the data acquisition module; The edge computing terminal is used for receiving the front view picture and the rear view picture of the puller and the position information of the puller, which are transmitted by the data wireless transmission module, and also comprises a strand gesture abnormal recognition module and an abnormal state early warning module, wherein the strand gesture abnormal recognition module is used for recognizing the strand abnormal state through the front view picture and the rear view picture of the puller and warning through the abnormal state early warning module; the receiving end receives the abnormal alarm information sent by the abnormal state early warning module. Preferably, the data wireless transmission module adopts a wireless AP and wireless network bridge fusion mode. Preferably, the abnormal alarm information comprises a cable strand abnormal gesture category, an abnormal gesture screen shot and current position information of the puller. Preferably, the edge computing terminal further comprises a winch control module for controlling the start and stop of the winch, and the winch control module is used for controlling the stop of the winch after the abnormal state of the strand is identified by the strand gesture abnormal identification module. The invention also provides a machine vision-based intelligent identifying method for unmanned trailing traction of the rope strands, which comprises the following steps: step one, installing an intelligent recognition system on a puller and carrying out communication connection with a background receiving end; step two, starting an intelligent recognition system and starting cable strand traction construction; Step three, a data acquisition module acquires a front view picture and a rear view picture of a puller through a front view high definition camera and a rear view high definition camera respectively, acquires position information of the puller through a Beidou positioning terminal, transmits data acquired by the data acquisition module to an edge computing terminal through a data wireless transmission module, and identifies a cable abnormal state through a cable strand gesture abnormal identification module after acquiring front view picture and rear view picture data of the puller, and transmits abnormal alarm information to a receiving end of a background through an abnormal state early warning module after the edge computing terminal identifies abnor