CN-121980807-A - Marine rocket erection support stable control method based on deep learning

CN121980807ACN 121980807 ACN121980807 ACN 121980807ACN-121980807-A

Abstract

The invention discloses a deep learning-based marine rocket erection support stability control method, which comprises the steps of generating a corresponding text prompt set, outputting a target detection result with a two-dimensional detection frame, inputting the target detection result into a SAM2, executing target level instance segmentation to generate a mask sequence, outputting an instance mask sequence with consistent time sequence to obtain a control level instance mask, extracting pose-speed information quintuple of a target relative to a lifting point and a support system, obtaining a support pose safety window, a lifting point track safety window and a collision cone safety window, outputting a support leg lifting adjustment instruction, a lifting point track correction instruction and a damping system action instruction, and executing the instruction to complete an erection support stability control operation. The invention realizes end-to-end and interpretable quantitative transfer between the sensing layer and the control layer, and improves the prospective response of the stability control system to sudden risk events and the accurate cutting capability of the safety window.

Inventors

TENG YAO
GONG QINGTAO
ZHANG SHUNING
HE SHILONG
HU XIN
LI KANGQIANG
SHEN KECHANG
GUO YANLI
HAN YANQING

Assignees

鲁东大学

Dates

Publication Date: 20260505
Application Date: 20260203

Claims (10)

1. The method for controlling the stability of the vertical support of the marine rocket based on deep learning is characterized by comprising the following steps: Collecting a multi-source image sequence of an erection operation site, and generating a corresponding text prompt set; Inputting GroundingDINO a text prompt set, executing open vocabulary target detection on the multi-source image sequence, and outputting a target detection result with a two-dimensional detection frame; Inputting a target detection result into the SAM2 to execute target level instance segmentation to generate a mask sequence; performing cross-frame mask matching on the mask sequence, outputting an instance mask sequence with consistent time sequence, and performing geometric regularization processing on the instance mask sequence with consistent time sequence to obtain a control level instance mask; performing space-time alignment on the control level instance mask and the inertia measurement unit data, the wave radar data and the ship automatic identification system data, and extracting pose-speed information quintuple of a target relative to a lifting point and a supporting system; constructing a dynamic risk field based on pose-speed information quintuple, generating a corresponding risk influence domain for each high-risk target, updating the risk influence domain in real time, mapping the dynamic risk field into a safety window boundary, and obtaining a support pose safety window, a lifting point track safety window and a collision cone safety window; And constructing a hard constraint and soft constraint set of the discrete dynamics model, solving the discrete dynamics model, outputting a supporting leg lifting adjustment instruction, a lifting point track correction instruction and a damping system action instruction, and executing the instruction to finish the stable control operation of the vertical support.
2. The deep learning-based marine rocket erection support stability control method according to claim 1, wherein the capturing a multi-source image sequence of an erection operation site and generating a corresponding text prompt set comprises: The method comprises the steps of collecting multi-source image information of an erection operation site, continuously obtaining the multi-source image information by imaging equipment arranged on a main hanging boat, an auxiliary boat and a site fixed platform, forming a multi-source image sequence, and generating a corresponding text prompt set based on the erection operation stage, wherein the text prompt set comprises a wave crest, a wave wall, a floater, a mooring cable, an auxiliary boat, a lifting hook, a sling, a tower barrel and a jacket open vocabulary.
3. A deep learning based marine rocket erection support stabilization control method according to claim 2, wherein the outputting of the target detection result with the two-dimensional detection frame comprises: inputting the text prompt set to GroundingDINO; Performing sea state structure prior analysis on the current frame multisource image according to the scene structure of the sea state target level segmentation, and generating horizon likelihood mapping, wave peak likelihood mapping and highlight foam suppression mapping; For each frame of multi-source image, calculating the corresponding prompt weight of each text prompt word according to the prior statistical characteristics of the sea state structure; Calculating basic semantic matching scores through multi-mode feature alignment for all candidate areas and text prompt words of each frame of multi-source image; Carrying out self-adaptive re-weighting processing on each basic semantic matching score based on sea state structure prior mapping to obtain sea state self-adaptive matching scores; extracting basic detection frame coordinates corresponding to each candidate region, selecting a text prompt word with the largest sea state self-adaptive matching score as a category index of the corresponding candidate region, and taking the largest sea state self-adaptive matching score as the detection confidence of the candidate region; calculating the intersection ratio of each pair of basic detection frames for all basic detection frames, carrying out exponential weighting according to the pixel displacement of the centroid of the detection frames in the horizontal and vertical directions and a directivity factor, traversing in sequence from high to low according to the detection confidence, removing redundant detection frames with high overlapping degree, and reserving a redundancy-removed detection frame set; And outputting a detection frame set after redundancy elimination of each frame, wherein each detection frame comprises a text prompt word, detection frame coordinates and detection confidence coefficient which are associated with the detection frame set, and the detection frame sets of all frames are summarized to be a target detection result.
4. A deep learning based marine rocket erection support stabilization control method according to claim 3, wherein said inputting the target detection result into SAM2 performs target level instance segmentation, comprising: each detection frame in the target detection result sequence is input into SAM2 as an input prompt, and a binary instance mask is generated in a corresponding frame image according to the pixel coordinate position of the detection frame; Counting the set of all the pixels of the binary instance mask as the area of the binary instance mask, extracting the boundary pixel set of the binary instance mask as an instance contour, and taking the number of the pixels of the instance contour as the perimeter; the area and the perimeter of each binary instance mask are used as the shape description quantity of the instance mask, and are spliced with the feature vector of the same target to form an extended feature vector after splicing; For each extended feature vector of the current frame and all extended feature vectors of the previous frame, respectively calculating the ratio of the pixel intersection set and the pixel union set of the binary instance mask as the intersection ratio, taking Euclidean distance between the extended feature vectors of the current frame and the previous frame as a penalty term, and taking weighted combination of the intersection ratio and the penalty term as the cross-frame matching similarity; According to the cross-frame matching similarity, a weighted Hungary matching algorithm is adopted to allocate an instance number, and if the maximum similarity is smaller than a preset threshold value, a new instance number is allocated to the current mask; And outputting an instance triplet by using a binary instance mask of each allocated instance number, and summarizing the instance triplet sets of all frame images to form a mask sequence with consistent time sequence.
5. The deep learning-based offshore rocket erection support stability control method according to claim 1, wherein the weighted hungarian matching algorithm comprises: Calculating the cross-frame matching similarity of each binary instance mask of the current frame and all binary instance masks of the previous frame respectively, forming a set of binary instance mask matching similarity matrixes by the cross-frame matching similarity of each binary instance mask of the current frame and each binary instance mask of the previous frame, searching a set of optimal allocation relations from the matching similarity matrixes, so that each binary instance mask of the current frame is allocated to the binary instance mask of the previous frame, and the global cross-frame matching similarity sum is maximized, and for each allocated pair of matches: if the cross-frame matching similarity of the current frame binary instance mask and the allocated previous frame binary instance mask is greater than or equal to a preset threshold value, inheriting and allocating the instance number of the previous frame binary instance mask for the current frame binary instance mask; If the cross-frame matching similarity is smaller than a preset threshold, a new instance number is allocated to the binary instance mask of the current frame, and the new instance number is different from all existing instance numbers.
6. A deep learning based marine rocket erection support stability control method according to claim 5, wherein said obtaining a control level instance mask comprises: Performing boundary smoothing processing on mask sequences with consistent sequence, performing smoothing operation on pixel boundary points of each instance mask by adopting sliding window filtering, curvature constraint or polynomial fitting, removing boundary saw teeth and micro disturbance, performing hole filling processing on the smoothed instance mask, detecting non-communicated hole areas in each instance mask, performing pixel filling on hole areas with areas smaller than a set threshold by adopting an area growing method, eliminating isolated hole pixels in the mask, performing outlier suppression processing on the instance mask filled with holes, and outputting a control level instance mask meeting the physical consistency requirement.
7. The deep learning-based marine rocket erection support stability control method according to claim 6, wherein the extracting pose-velocity information quintuple of the target relative to the lifting point and the support system comprises: the center pixel coordinates of each instance mask in the control level instance mask sequence are converted into three-dimensional space direction vectors under an image coordinate system; Mapping the three-dimensional space direction vector to a world coordinate system through a rotation matrix of a platform to obtain a direction vector of an instance mask direction under world coordinates; acquiring a nearest target distance set measured at the corresponding time of the current frame image, correspondingly combining the nearest target distance measured by the wave radar system with a direction vector under the world coordinate of each instance mask, and estimating the instantaneous position coordinate of the target under the world coordinate system; Acquiring a positioning result of the ship automatic identification system on an auxiliary ship target at the time corresponding to the current frame image, and adopting the positioning result of the ship automatic identification system to replace an instantaneous position coordinate for each mask example if the target class is an auxiliary ship; For mask examples of the same example number in the current frame and the previous frame, respectively calculating the change quantity of the target in the world coordinate positions of two continuous frames to obtain an instantaneous speed vector of the target between the two continuous frames, and subtracting the speed vector provided by the inertia measurement unit of the lifting point system to obtain a relative speed vector of the target relative to the lifting point; Outputting pose-speed information quintuple of each instance number under the current frame; the pose-speed information quintuple comprises an instance number, an instantaneous position coordinate of the object, a direction vector of the object under world coordinates, a relative speed vector of the object relative to a lifting point and a text prompt word class label corresponding to the instance, and is used for recording the unique identity, the spatial position, the orientation, the movement speed and the semantic class of each instance mask object.
8. The deep learning-based offshore rocket erection support stability control method of claim 7, wherein mapping the dynamic risk field into a safety window boundary to obtain a support attitude safety window, a lifting point track safety window and a collision cone safety window comprises: Inputting pose-speed information quintuple sets as sea state target states of the current frame, and performing linear prediction on the position change of the instance mask target in a future time window by utilizing real-time instantaneous position coordinates of each instance mask target and relative speed vectors relative to a lifting point system and combining a timestamp of the current frame and a specified prediction time window to obtain a predicted track of the instance mask target in the future time window; Based on the predicted track, the current position of the instance mask target and the relative speed vector relative to the lifting point system, constructing a risk cone area for each instance mask target according to risk opening angle parameters set by different text prompt word category labels; Calculating the minimum Euclidean distance between the real-time predicted track of the example mask target and the boundary point set of the surface of the support system, and carrying out exponential decay transformation on the minimum distance by adopting a risk decay coefficient related to semantic categories to obtain a risk intrusion index; weighting the risk cone areas corresponding to all the instance mask targets according to respective risk invasion indexes, and synthesizing the risk cone areas into a dynamic risk field of the current frame; cutting a control space of the erection support system according to the dynamic risk field, and supporting a posture safety window, a lifting point track safety window and a collision cone safety window.
9. The method for controlling the stability of the standing support of the marine rocket based on deep learning as claimed in claim 8, wherein the solving the model predictive control optimization problem outputs a supporting leg lifting adjustment command, a lifting point track correction command and a damping system action command, and the method comprises the following steps: Establishing a discrete dynamics model in a prediction step length and a prediction time domain, wherein the discrete dynamics model describes the system state at the current moment, multiplies and overlaps the control input vector through a group of system discrete dynamics matrixes, and predicts to obtain the system state at the next moment; constructing a hard constraint set and soft constraint loss; Taking soft constraint loss as an objective function, sequentially enumerating and calculating all possible value combinations of each step of control input vector in the whole prediction time domain under the established discrete dynamics model and all hard constraint conditions, and screening a control input vector sequence which enables the soft constraint loss objective function to be minimum by evaluating all feasible control input vector sequences one by one, wherein the control input vector sequence is taken as an optimal control input vector sequence; extracting a supporting leg lifting adjustment instruction, a lifting point track correction instruction and a damping system action instruction from an optimal control input vector sequence, wherein the supporting leg lifting adjustment instruction, the lifting point track correction instruction and the damping system action instruction correspond to the height adjustment of the supporting leg, the lifting point position fine adjustment and the output of the damping system action moment respectively; and respectively sending the supporting leg lifting adjustment instruction, the lifting point track correction instruction and the damping system action instruction to corresponding execution mechanisms, entering the next sampling period after the instruction execution is completed, and repeating the model prediction control flow to realize the erection supporting operation.
10. A deep learning based marine rocket erection support stability control method according to claim 9, wherein the hard constraint set comprises: The euler attitude angle of the support system must be within the range allowed by the support attitude safety window, the suspension point spatial position must be within the range allowed by the suspension point trajectory safety window, the suspension point spatial position must not enter the dangerous area defined by the collision cone safety window, and each component of all control input vectors must be within the upper and lower limits allowed by the physical actuator.

Description

Marine rocket erection support stable control method based on deep learning Technical Field The invention relates to the technical field of offshore rockets, in particular to a deep learning-based offshore rocket erection support stability control method. Background With the continuous development of the field of ocean engineering, offshore wind power and large-scale structure offshore installation, higher requirements on safety and intelligent stability control capability are put forward in the process of erection operation, and the traditional offshore erection stability control method mainly depends on wave height threshold judgment, single-point sensor alarm or offline safety rules. In general, real-time, fine-grained and interpretable identification and management of various dynamic targets on a job site cannot be performed only for overall environmental parameters or extremely sparse sensing data. Under complex sea conditions, the traditional method can not provide continuous information input of specific targets, specific geometries and specific mutual movements, so that the erection window judgment is too conservative or risk is underestimated, and the real-time stability control strategy of the support system and the hanging points is affected. The existing image target detection and segmentation method mostly adopts a fixed type detection or single frame segmentation mode, lacks the capability of opening scene oriented and target level dynamic understanding, and even if a deep learning detection segmentation model is developed in visual recognition in recent years, the example consistency, time sequence consistency and motion estimation of the detection and segmentation target are difficult to ensure in a marine complex scene, so that the 'perception-decision-control' disjoint between sea environment and mechanical control requirement is caused. Disclosure of Invention The invention aims to provide a deep learning-based offshore rocket erection support stable control method, which realizes end-to-end and interpretable quantitative transfer between a sensing layer and a control layer, and improves the prospective response of a stability control system to sudden risk events and the accurate cutting capability of a safety window. According to the embodiment of the invention, the method for controlling the stability of the standing support of the marine rocket based on deep learning comprises the following steps: Collecting a multi-source image sequence of an erection operation site, and generating a corresponding text prompt set; Inputting GroundingDINO a text prompt set, executing open vocabulary target detection on the multi-source image sequence, and outputting a target detection result with a two-dimensional detection frame; Inputting a target detection result into the SAM2 to execute target level instance segmentation to generate a mask sequence; performing cross-frame mask matching on the mask sequence, outputting an instance mask sequence with consistent time sequence, and performing geometric regularization processing on the instance mask sequence with consistent time sequence to obtain a control level instance mask; performing space-time alignment on the control level instance mask and the inertia measurement unit data, the wave radar data and the ship automatic identification system data, and extracting pose-speed information quintuple of a target relative to a lifting point and a supporting system; constructing a dynamic risk field based on pose-speed information quintuple, generating a corresponding risk influence domain for each high-risk target, updating the risk influence domain in real time, mapping the dynamic risk field into a safety window boundary, and obtaining a support pose safety window, a lifting point track safety window and a collision cone safety window; And constructing a hard constraint and soft constraint set of the discrete dynamics model, solving the discrete dynamics model, outputting a supporting leg lifting adjustment instruction, a lifting point track correction instruction and a damping system action instruction, and executing the instruction to finish the stable control operation of the vertical support. Optionally, the capturing a multi-source image sequence of the erection operation site and generating a corresponding text prompt set includes: The method comprises the steps of collecting multi-source image information of an erection operation site, continuously obtaining the multi-source image information by imaging equipment arranged on a main hanging boat, an auxiliary boat and a site fixed platform, forming a multi-source image sequence, and generating a corresponding text prompt set based on the erection operation stage, wherein the text prompt set comprises a wave crest, a wave wall, a floater, a mooring cable, an auxiliary boat, a lifting hook, a sling, a tower barrel and a jacket open vocabulary. Optionally, the outputting the target detection result wi