Search

CN-121982661-A - Directional ship detection method and system based on joint attention

CN121982661ACN 121982661 ACN121982661 ACN 121982661ACN-121982661-A

Abstract

The invention provides a directional ship detection method and a directional ship detection system based on joint attention, which comprise the steps of acquiring image acquisition data, carrying out feature extraction by utilizing a backbone network to obtain a multi-scale feature map, carrying out feature recalibration on the multi-scale feature map respectively, outputting an enhanced target feature map, carrying out convolution operation on the enhanced target feature map by adopting self-adaptive geometric convolution to obtain ship direction characterization information, generating candidate frames based on the ship direction characterization information, combining ship real frames, calculating intersection ratio statistical features, carrying out training sample self-adaptive distribution according to the intersection ratio statistical features, and carrying out ship directional detection by utilizing the training samples subjected to self-adaptive distribution based on an Anchor-free detection frame.

Inventors

  • Xia Caifeng
  • GAO HONGWEI
  • YANG WEI
  • LIU BO

Assignees

  • 沈阳理工大学

Dates

Publication Date
20260505
Application Date
20260123

Claims (10)

  1. 1. A method for directional vessel detection based on joint attention, comprising: Acquiring image acquisition data in an offshore monitoring scene; extracting features of the image acquisition data by using a preset backbone network to obtain a multi-scale feature map, and respectively carrying out feature recalibration of space dimension and channel dimension on the multi-scale feature map to output an enhanced target feature map; Performing convolution operation on the enhanced target feature map by adopting self-adaptive geometric convolution to obtain ship direction representation information in the target feature map; generating candidate frames based on the ship direction characterization information, combining ship real frames marked in advance in training data, calculating the cross-over ratio statistical characteristics of the candidate frames and the ship real frames, and carrying out training sample self-adaptive distribution according to the cross-over ratio statistical characteristics; based on a preset Anchor-free detection frame, carrying out ship orientation detection on the image acquisition data by using a training sample subjected to self-adaptive distribution to obtain a ship orientation detection result in the offshore monitoring scene; The self-adaptive geometric convolution is constructed by learning the deformable offset of the convolution kernel sampling points, introducing bilinear interpolation sampling and space constraint loss.
  2. 2. The method of claim 1, wherein the image acquisition data comprises one or more of visible light image data, synthetic aperture radar image data, and marine feature correlation data acquired by an artificial intelligence internet of things device; the ship feature associated data includes ship attitude associated image data and continuous video frame data including ship targets.
  3. 3. The method of claim 2, wherein the feature extraction of the image acquisition data using a preset backbone network to obtain a multi-scale feature map comprises: inputting the image acquisition data into a preset backbone network; Carrying out convolution, activation and pooling on ship feature associated data in the image acquisition data in sequence through the backbone network to obtain an initial feature map, wherein the initial feature map comprises feature information with different resolutions and different semantic levels; Inputting the initial feature map into a feature pyramid network, and up-sampling feature information with high resolution and low semantic level through the feature pyramid network to obtain up-sampled features; performing cross-scale feature fusion on the up-sampling features and the down-sampling features to generate a multi-scale feature map; Wherein the backbone network is ResNet convolutional neural network.
  4. 4. The method of claim 1, wherein the convolving the enhanced target feature map with an adaptive geometric convolution to obtain ship direction characterization information in the target feature map comprises: Inputting the enhanced target feature map into a self-adaptive geometric convolution layer; The variable offset corresponding to each spatial sampling point of a convolution kernel is learned through the offset prediction convolution branch in the adaptive geometric convolution layer, and meanwhile, the scalar modulation factor corresponding to each spatial sampling point is learned through the modulation factor prediction convolution branch in the adaptive geometric convolution layer; based on the learned deformable offset, performing spatial position adjustment on each spatial sampling point of the convolution kernel to obtain adjusted coordinates of each spatial sampling point; extracting a first characteristic value corresponding to the space sampling point in the enhanced target characteristic diagram aiming at the space sampling point with the adjusted coordinates being an integer; aiming at the space sampling point with the adjusted coordinates being a non-integer, calculating a second characteristic value of the space sampling point by adopting a bilinear interpolation method; according to the first characteristic value and the second characteristic value, obtaining effective sampling characteristics of each space sampling point; The contribution amplitude of each effective sampling feature is subjected to weighted modulation through the scalar modulation factor, and all modulated effective sampling features are subjected to summation operation to obtain initial ship direction characterization features; And carrying out data calibration on the initial ship direction characterization features through a preset loss function to obtain ship direction characterization information in the target feature map.
  5. 5. The method of claim 4, wherein the loss function is expressed as follows: ; In the formula, ; Wherein, the Representing a loss function; Representing a classification loss function; representing an initial stage positioning loss function; Representing the weight coefficient of the initial stage; representing a positioning loss function in a refinement stage; representing weight coefficients of a refinement stage; Representing the total number of positive sample point sets; An identification representing a single positive sample point; a set identifier representing a positive sample point set; Representing a single positive sample point The number of the corresponding output characteristic points; Representing a single positive sample point Representing a corresponding set of output feature points; Representing output feature points Is a predictive category probability vector of (1); Representing a true category label vector.
  6. 6. The method of claim 1, wherein the generating a candidate frame based on the vessel direction characterization information, in combination with a vessel real frame pre-labeled in training data, calculates a statistical feature of an intersection ratio of the candidate frame and the vessel real frame, comprises: extracting key parameter information from the ship direction characterization information; constructing an initial candidate frame set according to the key parameter information, wherein each initial candidate frame corresponds to a potential ship target, and the geometric form of each initial candidate frame is consistent with the ship geometric form and the direction characteristic in the ship direction representation information; acquiring ship real frames marked in advance in training data, and calculating the sum of angular point distances between each initial candidate frame in the initial candidate frame set and the ship real frames; screening a preset number of initial candidate frames with minimum sum of angular point distances for each ship real frame to form a target candidate frame subset corresponding to the ship real frames; For each ship real frame and a corresponding target candidate frame subset thereof, respectively calculating the intersection ratio of each candidate frame in the target candidate frame subset and the ship real frame to obtain an intersection ratio set corresponding to the ship real frame; carrying out statistical analysis on each cross-over ratio set to obtain cross-over ratio statistical characteristics; The key parameter information comprises one or more of predicted center coordinates, predicted width, predicted height and predicted direction angles of ship targets; The ship real frame comprises one or more of the following real center coordinates, real width, real height, real direction angle and real angular point coordinate information of a ship target; And all target candidate frame subsets corresponding to the ship real frames jointly form a target candidate frame set.
  7. 7. The method of claim 6, wherein said adaptively assigning training samples based on said cross-ratio statistics comprises: Calculating a positive and negative sample screening threshold corresponding to each ship real frame based on the intersection ratio statistical characteristics corresponding to each ship real frame; judging whether positive sample judgment conditions are met one by one according to candidate frames in a target candidate frame subset corresponding to each ship real frame, wherein if the intersection ratio of the candidate frames to the ship real frames is larger than a corresponding screening threshold value, the center point of the candidate frames is positioned in the ship real frames, the candidate frames are judged to be positive samples, and the positive samples are included in a positive sample set; If a single candidate frame simultaneously meets positive sample judgment conditions of a plurality of ship real frames, calculating the intersection ratio of the candidate frame and each ship real frame, selecting the ship real frame corresponding to the maximum intersection ratio as a matching object, and reserving the candidate frame as a positive sample corresponding to the matching object; Uniformly judging all candidate frames which are not judged to be positive samples in the initial candidate frame set as negative samples, and incorporating the negative samples into a negative sample set; And obtaining a training sample set comprising the positive sample set and the negative sample set according to the positive sample set and the negative sample set.
  8. 8. The method of claim 7, wherein the performing ship orientation detection on the image acquisition data based on the preset Anchor-free detection frame using the adaptively assigned training samples to obtain the ship orientation detection result in the offshore surveillance scene comprises: inputting the positive sample set and the negative sample set into a preset Anchor-free detection frame for training to obtain a trained Anchor-free detection frame; Performing size adjustment and normalization preprocessing on the image acquisition data to generate standardized image data; Inputting the standardized image data into a feature processing link in the trained Anchor-free detection framework, and outputting an enhanced feature map, wherein the feature processing link comprises a backbone network, a feature pyramid network and a self-adaptive geometric convolution layer; respectively inputting the enhanced feature map into a classification branch and a directional boundary box regression branch in the trained Anchor-free detection frame to obtain a ship directional detection result in the offshore monitoring scene; the ship orientation detection result comprises position information of a ship target, category confidence and orientation boundary frame parameters; the orientation bounding box parameters include one or more of center coordinates, width, height, and direction angle information.
  9. 9. A joint attention-based directional vessel inspection system, comprising: the data acquisition module is used for acquiring image acquisition data in an offshore monitoring scene; The characteristic calibration module is used for carrying out characteristic extraction on the image acquisition data by utilizing a preset backbone network to obtain a multi-scale characteristic image, carrying out characteristic recalibration on the multi-scale characteristic image in space dimension and channel dimension respectively, and outputting an enhanced target characteristic image; The feature processing module is used for carrying out convolution operation on the enhanced target feature image by adopting self-adaptive geometric convolution to obtain ship direction representation information in the target feature image; the self-adaptive distribution module is used for generating candidate frames based on the ship direction characterization information, calculating the intersection ratio statistical characteristics of the candidate frames and the ship real frames by combining the ship real frames marked in advance in the training data, and carrying out self-adaptive distribution on training samples according to the intersection ratio statistical characteristics; The orientation detection module is used for carrying out ship orientation detection on the image acquisition data by using a training sample subjected to self-adaptive allocation based on a preset Anchor-free detection frame to obtain a ship orientation detection result in the offshore monitoring scene; The self-adaptive geometric convolution is constructed by learning the deformable offset of the convolution kernel sampling points, introducing bilinear interpolation sampling and space constraint loss.
  10. 10. The system of claim 9, wherein the image acquisition data comprises one or more of visible light image data, synthetic aperture radar image data, and marine feature correlation data acquired by an artificial intelligence internet of things device; the ship feature associated data includes ship attitude associated image data and continuous video frame data including ship targets.

Description

Directional ship detection method and system based on joint attention Technical Field The invention relates to the technical field of computer vision and artificial intelligence, in particular to a directional ship detection method and system based on joint attention. Background With the rapid development of the technology of the artificial intelligence internet of things (AIoT), a AIoT-powered offshore monitoring system (AIoT-MSS) has become a core support for guaranteeing the ordered operation of offshore traffic, and strict requirements are put on real-time accurate detection of ship targets. The marine vessel generally presents a strip-shaped structure and has a large length-width ratio, and is influenced by factors such as navigation attitude, acquisition view angle and the like, and the vessel presents the characteristic of random directional distribution in the monitoring image, so that the accurate representation of the vessel direction becomes one of the core challenges of the marine detection task. The current mainstream ship orientation detection method mainly comprises an anchor point-based orientation detection method (such as RoI-Transformer, R Det), an anchor point-free orientation detection method (such as FCOS-O, oriented RepPoints) and the like, wherein the methods depend on traditional angle regression or fixed receptive field convolution operation in the aspect of orientation characterization, the former determines the posture of a boundary frame through directly regressing a ship direction angle, but the limited periodicity of the ship direction angle and the direction definition difference of a rotating boundary frame cause difficulty in accurately describing large-scale direction changes of a ship geometric structure in complex sea conditions, the latter adopts standard convolution extraction characteristics of the fixed receptive field, the receptive field size and shape of each output pixel in a convolution layer are always fixed, cannot be dynamically adjusted according to the direction changes and geometric deformation of the ship, and further cannot effectively capture the fine edge characteristics of the ship in any direction, and the existing method lacks an adaptive optimization mechanism for the ship direction characteristics, and further reduces the robustness of the orientation boundary frame in the face of complex backgrounds such as sea surface reflection and wind wave interference, so that the orientation boundary frame regression accuracy is insufficient, the problem of seriously influencing the sea body fitting degree of the ship body deviation, the orientation prediction deviation and the like occurs, and the reliability of a detection system is seriously influenced. Disclosure of Invention In order to solve the problem that in the prior art, the ship direction characterization is inaccurate, the precise geometric edge characteristics of the ship in any direction are difficult to capture by a traditional angle regression method, so that the detection reliability of an offshore monitoring system is low, the invention provides a combined attention-based directional ship detection method, which comprises the following steps: Acquiring image acquisition data in an offshore monitoring scene; Extracting features of the image acquisition data by using a preset backbone network to obtain a multi-scale feature map, and respectively carrying out feature recalibration on the multi-scale feature map in space dimension and channel dimension to output an enhanced target feature map; Performing convolution operation on the enhanced target feature map by adopting self-adaptive geometric convolution to obtain ship direction representation information in the target feature map; generating candidate frames based on the ship direction characterization information, combining ship real frames marked in advance in training data, calculating the cross-over ratio statistical characteristics of the candidate frames and the ship real frames, and carrying out training sample self-adaptive distribution according to the cross-over ratio statistical characteristics; based on a preset Anchor-free detection frame, carrying out ship orientation detection on the image acquisition data by using a training sample subjected to self-adaptive distribution to obtain a ship orientation detection result in the offshore monitoring scene; The self-adaptive geometric convolution is constructed by learning the deformable offset of the convolution kernel sampling points, introducing bilinear interpolation sampling and space constraint loss. Optionally, the image acquisition data comprises one or more of visible light image data, synthetic aperture radar image data and ship characteristic association data acquired by artificial intelligence Internet of things equipment; the ship feature associated data includes ship attitude associated image data and continuous video frame data including ship targets. Optionally, th