CN-121998820-A - Submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault tolerance matching

CN121998820ACN 121998820 ACN121998820 ACN 121998820ACN-121998820-A

Abstract

The application provides a submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault tolerance matching. The method comprises the steps of obtaining sea cable image pairs of adjacent frames, extracting original color channel information of each frame of image, carrying out channel enhancement processing on each frame of image to obtain a channel enhancement image, carrying out processing on the channel enhancement image based on a dual-domain decoupling enhancement network to obtain a final enhancement image, carrying out feature point matching on the final enhancement image pair based on a multi-level fault-tolerant matching network to obtain mutually matched effective feature point pairs, calculating a homography transformation matrix of the final enhancement image pair based on the effective feature point pairs, carrying out image stitching on the final enhancement image pair according to the homography transformation matrix, and carrying out color enhancement processing on the stitched image based on the original color channel information to obtain a final stitched image. The technical scheme of the application can effectively inhibit color distortion while improving the contrast and detail definition of the image.

Inventors

HU MENGLONG
WANG WEN
ZHANG XUETING
ZHU KAICHEN
WANG YUANMING
Zhou Yehan
ZHU YIWEI
ZHANG ZHIHENG
Wang Daining
GAO CHAOYING

Assignees

杭州电子科技大学

Dates

Publication Date: 20260508
Application Date: 20260410

Claims (9)

1. A submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault tolerance matching is characterized by comprising the following steps: Step S1, sea cable image pairs of adjacent frames are obtained, original color channel information of each frame image in the sea cable image pairs is extracted, and channel enhancement processing is carried out on each frame image to obtain channel enhancement images; Step S2, processing the channel enhanced image based on a preset double-domain decoupling enhanced network to obtain a final enhanced image, wherein the double-domain decoupling enhanced network comprises a feature extraction branch, a double-domain feature decoupling branch and a feature fusion branch, and the steps are as follows: the feature extraction branch is configured to perform multi-scale feature extraction on each frame of channel enhanced image to obtain a high-resolution spatial feature map and a low-resolution spatial feature map; The double-domain feature decoupling branch is configured to extract a spatial domain enhancement feature and a frequency domain related feature of the low-resolution spatial feature map, fuse the spatial domain enhancement feature and the frequency domain related feature to obtain a joint representation feature, correspondingly fuse a phase component and an amplitude component of the joint representation feature and a phase component and an amplitude component of the high-resolution spatial feature map, and recombine and transform the fused phase component and the fused amplitude component to obtain a frequency domain enhancement feature; The feature fusion branch is configured to cross-domain fuse the to-frequency domain enhancement feature with the spatial domain enhancement feature based on an attention strategy, and to perform multi-scale reconstruction based on the joint representation feature and the cross-domain fusion feature to obtain a final enhanced image; step S3, carrying out feature point matching on the final enhanced image pair based on a multi-level fault-tolerant matching network to obtain effective feature point pairs matched with each other; And S4, calculating a homography transformation matrix of the final enhanced image pair based on the effective feature point pair, performing image stitching on the final enhanced image pair according to the homography transformation matrix, and performing color enhancement processing on the stitched image based on the original color channel information to obtain a final stitched image.
2. The method according to claim 1, wherein said step S1 obtains a channel enhanced image by: Converting each frame of image into LAB color space, carrying out limit contrast self-adaptive histogram equalization on brightness channels, and carrying out linear stretching on the color channels; and converting the processed image back to the original RGB color space to obtain the channel enhanced image.
3. The method according to claim 1, wherein the two-domain feature decoupling branch of step S2 is configured to obtain the spatial domain enhancement feature and the frequency domain related feature by: Splitting the low-resolution spatial feature map along a channel, and carrying out feature interaction on the split feature map through an affine learning layer and a half-instance normalization block to obtain spatial domain enhancement features; And respectively carrying out convolution serialization operation on the frequency domain components of the low-resolution spatial feature map, and then converting the frequency domain components into a spatial domain to obtain the frequency domain related features.
4. The method according to claim 1, wherein the two-domain feature decoupling branch of step S2 is configured to obtain the fused phase component and the fused amplitude component by: Performing convolution sequence processing on the phase components of the high-resolution spatial feature map, and performing channel splicing on the phase components of the combined representation feature to obtain fused phase components; And fusing the amplitude components of the joint representation features with the amplitude components of the high-resolution spatial feature map through an affine learning strategy to obtain fused amplitude components.
5. The method according to claim 1, wherein the feature fusion branch of step S2 is configured to obtain the final enhanced image by: Convolving the joint representation feature to obtain an enhancement result of the joint representation feature; performing feature refinement and pixel rearrangement processing on the cross-domain fusion features to obtain an enhancement result of the cross-domain fusion features; And performing channel splicing and convolution processing on the enhancement result of the joint representation feature and the enhancement result of the cross-domain fusion feature to obtain the final enhancement image.
6. The method according to claim 1, wherein the two-domain decoupling enhancement network of step S2 is jointly optimized based on a multi-scale joint loss function, wherein the multi-scale joint loss function comprises at least two of the following losses: pixel level loss configured to constrain pixel differences between the final enhanced image and the sea-cable image; a structural loss configured as a structural similarity index between the final enhanced image and the submarine cable image; A perceptual penalty configured to constrain a perceived consistency between the final enhanced image and the submarine cable image.
7. The method of claim 1, wherein the multi-level fault tolerant matching network of step S3 comprises a trunk matching layer, a secondary matching layer, and a spam matching layer, wherein: the trunk matching layer is configured to perform feature matching on the final enhanced image pair based on a pre-trained LoFTR model; The secondary matching layer is configured to respectively extract key points of the final enhanced image of each frame based on a scale-invariant feature transform (SIFT) detector, wherein the key points comprise local gradient feature descriptors, and nearest neighbor searching is carried out on the local gradient feature descriptors of the final enhanced image pair based on a rapid nearest neighbor search library matcher; The spam matching layer is configured to extract corner points of the final enhanced image of each frame based on OFRB detectors respectively, wherein the corner points comprise BRIEF binary descriptors, and calculate the similarity of the final enhanced image pair based on a violence matcher and a Hamming distance.
8. The method according to claim 7, wherein said step S3 obtains valid pairs of feature points that match each other by: When the number of the characteristic point pairs of the characteristic point pair set output by the trunk matching layer is not smaller than a preset threshold value, acquiring the effective characteristic point pairs based on the output of the trunk matching layer; When the number of the feature point pairs output by the trunk matching layer is smaller than a preset threshold, determining a nearest neighbor search result output by the secondary matching layer, and if the number of the key point pairs of the nearest neighbor search result is not smaller than the preset threshold, obtaining the effective feature point pairs based on the output of the secondary matching layer; And when the number of key point pairs output by the secondary matching layer is smaller than a preset threshold value, obtaining the effective characteristic point pairs based on the corner pairs of the similarity calculation result output by the spam matching layer.
9. An electronic device, comprising: A processor; A computer readable storage medium having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of any of claims 1 to 8.

Description

Submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault tolerance matching Technical Field The invention relates to the technical field of underwater target detection, in particular to a submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault tolerance matching. Background In the daily inspection and maintenance of submarine cables, marine resource surveys and underwater pipelines, the problems of color cast (such as bluing and greening), greatly reduced contrast, sparse texture and the like of the underwater inspection images are caused by serious illumination attenuation of the underwater environment, different absorption rates of water bodies to light rays with different wavelengths and interference of suspended particles. Conventional image stitching and enhancement techniques face the following core technical challenges when processing such underwater submarine cable images: first, feature extraction contradicts mutual exclusion of color fidelity. In order to improve the success rate of image registration, contrast enhancement is usually required to be carried out on the image, but the image can damage the real physical color of the target surface (such as attachments and rusting), so that serious deviation occurs in the downstream defect grading; Second, frequency information is underutilized and inter-domain complementarity is absent. The influence difference of the underwater environment on the frequency domain amplitude component and the phase component is obvious, the existing method lacks an explicit modeling and effective cross-domain fusion mechanism for the double-domain key information, and color fidelity and detail authenticity are difficult to be considered; Third, the engineering robustness of the matching algorithm is insufficient. When facing to a pure-color sand bed or a weak texture region, traditional algorithms such as SIFT, ORB and the like are extremely easy to fail, and when edge equipment is deployed, a deep learning model (such as LoFTR) based on a transducer often fails reasoning due to limited calculation force or sudden blurring, so that a matched pipeline is collapsed. Disclosure of Invention In view of the above, the application provides a submarine cable image splicing method based on dual-domain decoupling enhancement and multi-level fault tolerance matching, which at least solves the problems that the frequency domain information is not fully utilized, the structural stability and the detail recovery are difficult to be compatible, and the splicing is easy to fail under extreme working conditions. Specifically, the application is realized by the following technical scheme: According to a first aspect of embodiments of the present disclosure, a submarine cable image stitching method based on dual-domain decoupling enhancement and multi-level fault tolerance matching is provided, including the following steps: Step S1, sea cable image pairs of adjacent frames are obtained, original color channel information of each frame image in the sea cable image pairs is extracted, and channel enhancement processing is carried out on each frame image to obtain channel enhancement images; Step S2, processing the channel enhanced image based on a preset double-domain decoupling enhanced network to obtain a final enhanced image, wherein the double-domain decoupling enhanced network comprises a feature extraction branch, a double-domain feature decoupling branch and a feature fusion branch, and the steps are as follows: the feature extraction branch is configured to perform multi-scale feature extraction on each frame of channel enhanced image to obtain a high-resolution spatial feature map and a low-resolution spatial feature map; The double-domain feature decoupling branch is configured to extract a spatial domain enhancement feature and a frequency domain related feature of the low-resolution spatial feature map, fuse the spatial domain enhancement feature and the frequency domain related feature to obtain a joint representation feature, correspondingly fuse a phase component and an amplitude component of the joint representation feature and a phase component and an amplitude component of the high-resolution spatial feature map, and recombine and transform the fused phase component and the fused amplitude component to obtain a frequency domain enhancement feature; The feature fusion branch is configured to cross-domain fuse the to-frequency domain enhancement feature with the spatial domain enhancement feature based on an attention strategy, and to perform multi-scale reconstruction based on the joint representation feature and the cross-domain fusion feature to obtain a final enhanced image; step S3, carrying out feature point matching on the final enhanced image pair based on a multi-level fault-tolerant matching network to obtain effective feature point pairs matched with each other; And