Search

CN-121982720-A - Remote sensing instance division point self-adaptive optimization method of discrete grid strategy

CN121982720ACN 121982720 ACN121982720 ACN 121982720ACN-121982720-A

Abstract

The invention discloses a remote sensing instance division point self-adaptive optimization method of a discrete grid strategy. Including rocker images for targets With reference images Using visual feature extraction networks to accomplish both Stage feature extraction to The method comprises the steps of constructing a fixed-size discrete example grid for an object, mapping candidate positive and negative point sets to grid cells respectively, sampling visual characteristics of the grid cells containing candidate points, constructing state representation, constructing an example-level reinforcement learning environment, executing point type marks on the grid cells to generate an example-level point prompt optimization strategy, outputting the optimized positive and negative point sets of each target example according to the strategy, mapping the optimized positive and negative point sets to pixel coordinate point prompts, inputting a prompt type segmentation model, and obtaining an example segmentation mask After threshold denoising, a stable instance mask is output . The output point prompt of the invention is more in line with the characteristics of multi-scale and compact distribution of complex background and examples of the remote sensing image, thereby improving the segmentation precision and stability.

Inventors

  • YU XIAOBING
  • FAN ZHIHUI
  • ZHU YUNHUI
  • YAO YAO
  • ZHAN TIANMING

Assignees

  • 南京审计大学

Dates

Publication Date
20260505
Application Date
20260126

Claims (10)

  1. 1. The remote sensing instance division point self-adaptive optimization method of the discrete grid strategy is characterized by comprising the following steps of: Given a target rocker image And at least one reference image Target rocker image through visual feature extraction network Reference image A kind of electronic device Stage feature extraction, setting a matching threshold value, and obtaining a target rocker image Performing feature matching on the target rocker image, wherein the feature matching is successful In (a) Marking the corresponding pixel area as candidate positive points, building a candidate positive point set, wherein the matching degree is lower than a threshold value or no matching object exists Marking the corresponding region as candidate negative points, and constructing a candidate negative point set; for target rocker images Constructing a discrete example grid with a fixed size for each target example, and respectively mapping a candidate positive point set or a candidate negative point set to grid cells, wherein the grid cells adopt a characteristic extraction network to extract a target rocker image Performing feature extraction to generate visual features corresponding to each grid unit, and constructing a state representation comprising grid features, point type indication information and center mark information, wherein the grid units adopt 9×9 grids, performing visual feature sampling on the grid units comprising candidate points, and constructing the state representation; Setting up an example-level reinforcement learning environment, presetting a reward function, executing point type marking operation on the grid unit, and outputting a point prompt set; Training a strategy network by using a near-end strategy optimization algorithm to obtain an instance-level point prompt optimization strategy, outputting an optimization positive point set and an optimization negative point set of each target instance by using the instance-level point prompt optimization strategy, mapping the optimization positive points and the optimization negative points into pixel coordinate point prompts, taking the pixel coordinate point prompts as the input of a prompt type segmentation model, and outputting an instance segmentation mask Stable instance masks are obtained by thresholding the instance segmentation masks 。
  2. 2. The remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 1, wherein the specific steps of marking candidate positive points or candidate negative points are as follows: For a given target rocker image Proceeding with Dividing to obtain Personal (S) Then extracting features to obtain a rocker image feature matrix The calculation formula is as follows: ; wherein: A visual feature extraction network is represented, Representing a target rocker image The number of the pieces of the plastic material, Representing a feature dimension; For reference images Proceeding with Dividing to obtain Reference pictures Then extracting features to obtain a reference image feature matrix The calculation formula is as follows: ; wherein: A visual feature extraction network is represented, Representing reference images The feature matrix is used to determine the feature matrix, Representing reference images The number of the pieces of the plastic material, Representing a feature dimension; extracting reference image feature matrix through visual feature extraction network Rocker image feature matrix Each of (3) Is constructed by semantic feature vectors of (1) A library of features is provided which is a library of features, Given a reference mask Screening target rocker image feature matrix Setting up a query subset from the foreground and background in the database The calculation formula is as follows: ; wherein: representing annotated queries Quantity of (a) wherein ≤ ; With each reference image Is a feature distance minimum target rocker image As an index, to Each reference image in the feature library Nearest neighbor matching is performed, and the calculation formula is as follows: ; ; wherein: represent the first Reference pictures First, the Individual target rocker images Is a characteristic distance of (2); representing a binary norm; Representing and referencing images Is the first of (2) Individual target rocker images Is a reference to (a).
  3. 3. The remote sensing instance segmentation point adaptive optimization method of a discrete grid strategy according to claim 2, wherein the step of building the candidate positive point set is as follows: Mask the given reference Mapping to Target rocker image in feature library Reference foreground is obtained from the foreground of (a) Reference foreground is matched according to a set matching threshold value Matching to obtain candidate positive points, and building a candidate positive point set, wherein the calculation formula is as follows: ; wherein: A set of candidate positive points is represented, Representing reference foreground Is a set of indexes of (a); Mask the given reference Mapping to Target rocker image in feature library Background of (2) reference background According to the set matching threshold value, the reference background is matched Matching to obtain candidate negative points, and building a candidate negative point set, wherein the calculation formula is as follows: ; wherein: Representing a set of candidate negative points, Representing a reference background Is a set of indexes of (a); when the feature matching fails or the number of candidate points is insufficient, the center pixel coordinate of the example boundary frame is adopted as a rollback center point, and the center is positioned in the reference prospect And its neighborhood (Neighborhood) For the centre of the target rocker image ) As candidate positive points, the far-from-center region is candidate negative points, wherein the calculation formula of the center pixel coordinates of the boundary frame is as follows: ; wherein: 、 Representing example bounding box pixel coordinates, Representing example bounding box center pixel coordinates; mapping pixel points in a target rocker image to The grid coordinates and the one-dimensional index are recovered by the grid unit when the prompt is output, the grid coordinates of the grid unit are calculated through the formula (1), and the grid unit pixel center coordinates are calculated through the formulas (2) and (3), wherein the calculation formula is as follows: (1); (2); (3); wherein: the coordinates of the pixels are represented and, Representation of The coordinates of the grid are used to determine, Represents a one-dimensional index of the line order, Representing the grid line index, Representation of The size of the grid coordinates is determined, Representing grid cell pixel center coordinates.
  4. 4. A remote sensing instance segmentation point adaptive optimization method according to claim 3, wherein the specific steps of constructing a discrete instance grid of fixed size are as follows: the center pixel coordinates of the center position of the bounding box are acquired, Setting up a local grid with a fixed size in a neighborhood by taking the center pixel coordinate of an example boundary frame as a reference, and using a preset sampling position on a target rocker image corresponding to each grid unit for subsequent feature sampling and point type labeling, wherein the calculation formula of the number of the grid units is as follows: ; wherein: representing the side length of the local grid; representing the number of grid cells; Each grid cell location in a feature library of a target pattern The corresponding feature vectors are sampled to form example grid feature tensors, and the calculation formula is as follows: ; ; wherein: The feature sampling operator is represented as a function of the feature sampling operator, An example mesh feature tensor is represented, After the calculation of the example grid characteristic tensor is completed, mapping candidate positive points and candidate negative points into example grid units to form point type tensors, marking the center point units as center channels, and building a discrete example grid with fixed size, wherein the calculation formula is as follows: ; ; wherein: representing a point type tensor; the characteristics of the grid are represented and, Representing the reinforcement learning environment state.
  5. 5. The remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 1, wherein the specific steps of constructing the action space and feasibility constraint of the instance-level reinforcement learning environment are as follows: Acquiring discrete instance grids, wherein each grid cell corresponds to a sampling position in a target rocker image, constructing a grid cell set, selecting one grid cell at each moment, marking the network cell as a positive point, a negative point or a non-point, discretizing continuous position selection into a fixed number of discrete decisions, and constructing a reinforcement learning model; The reinforcement learning model selects one grid cell from the grid cell set in each step, and selects the point type in the grid cell, wherein the selected calculation formula is as follows: ; wherein: Respectively correspond to Therefore, the motion space size is: ; wherein: Representing the local grid side length of an instance, Representing the grid line index, A discrete label of the type of point is represented, Representing a set of action spaces; Two-dimensional grid position and point type joint coding into a discrete action index Direct modeling through joint coding of two-dimensional grid positions and point types, and discrete action index thereof The calculation formula of (2) is as follows: ; wherein: representing a discrete action index; Representing the grid line index, ; Representing the side length of the grid; A point type code is represented and, And then applying upper and lower limit constraint on the number of positive points and the number of negative points, and passing through the point type tensor Positive point in (a) calculating negative points to obtain the number of positive points And negative point count The calculation formula is as follows: ; ; In the formula, Representing an indication function; The calculation formulas of the upper limit and the lower limit super parameters of the points are as follows: ; ; wherein: Indicating the number of positive points present, Representing the number of negative points present, Indicating that the lower limit of the number of points exceeds the parameter, After the calculation of the upper limit and the lower limit super parameters of the points is completed, the central unit of the example boundary box is set as a protected unit, the central unit is forbidden to be marked as a non-point, and punishment is applied to at least actions which violate the point range, the central point protection or the type conflict constraint by calculating illegal actions according to the following calculation formula: ; wherein: Representing policy network indexing of discrete actions Is the original of the neural network output of (a) ; Representing punished , Indicating an illegal action instruction amount; 1 represents illegality, 0 represents legal; Represents a penalty factor for suppressing the illegal action probability, 。
  6. 6. The remote sensing instance division point self-adaptive optimization method of discrete grid strategy according to claim 5, wherein the method is characterized in that initial points are introduced and target point intervals are preset Current point number Defining point rewards, and calculating the following formula: ; wherein: representing the current positive or negative points; representing a target interval lower bound; Representing the upper boundary of the target interval; representing the point reward factor(s), Represents the penalty coefficient of the point number, And counting the nearest neighbor distance between candidate positive points, wherein when the nearest neighbor distance is uniformly distributed, the rewards are high, at least distance items, radius constraint, point items, uniformity items and out-of-range punishment are weighted and summed, and then the nearest neighbor distance, uniformity index and total rewards are cut into a fixed interval to improve the training stability of a near-end strategy optimization algorithm, wherein the calculation formula of the nearest neighbor distance, uniformity index and total rewards weighting is as follows: ; ; ; wherein: represent the first Positive pixel coordinates; Representing nearest neighbor distance; representing the mean; Representing standard deviation; representing a very small constant; Representing a uniformity score; represent the first Sub-rewards; represent the first A weight; Representing a total prize; Representation clipping to 。
  7. 7. A remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 3, wherein the method is characterized in that a reward function is preset, a point type marking operation is performed on grid cells, and a specific step of outputting a point prompt set is as follows: the spatial relationship between the candidate point and the center point is described by the physical distance, and the calculation formula is as follows: ; wherein: Representing a central unit index; representing candidate unit indexes; Representing pixel coordinates corresponding to the candidate unit; representing pixel coordinates corresponding to the center point; Representing the feature vector corresponding to the center point; representing the feature vector corresponding to the candidate unit; Representing a physical distance; Representing the feature distance; representing two norms according to a physical distance radius threshold Penalty or constraint is imposed on positive point selection exceeding the threshold, selected such that the candidate positive point set is within a predetermined range of the instance interior or boundary, where the radius threshold The calculation formula of (2) is as follows: ; wherein: Representing a current negative point set; Representing a negative point cell index; representing the number of nearest negative points of a preset radius; wherein Taking the minimum distance; Representing an aggregation operator; Representing the negative-point radius threshold.
  8. 8. The remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 1, wherein a strategy network is trained by adopting a near-end strategy optimization algorithm to obtain an instance-level point prompt optimization strategy, and the specific steps are as follows: Policy network basis Instance grid feature tensor in moment reinforcement learning environment state Sum point type tensor Corresponding to the tensor state Output action distribution And value estimation of output state The multi-step interaction track is collected in a parallel environment, the variance is reduced and the sample efficiency is improved through the dominance function, and the calculation formula is as follows: ; wherein: : Indicating time of day Is used for the control of the state of (a), Indicating time of day Is used for the action of (a), Representing the parameters as Is provided with a policy distribution of (a), The value function is represented by a function of the value, Representing the value estimation, reintroducing generalized dominance estimation, TD time-series differential error The calculation advantage is recursively calculated again Updating a near-end strategy optimization algorithm by taking advantages as learning signals of good and bad directions, wherein TD time sequence differential errors The calculation formula of (2) is as follows: ; ; wherein: Representing a reward; representing a termination mark; Representing a discount factor; Representation of Coefficients; Representation of Error; Representing a dominance function, limiting the training collapse by limiting the new strategy and the old strategy through superposition of value loss and entropy positive, wherein the calculation formula is as follows: ; ; wherein: representing current policy parameters; Representing old policy parameters; represents an importance ratio; Representing a truncated coefficient; representing the truncated target of the near-end policy optimization algorithm.
  9. 9. The remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 8, wherein the optimization positive point set and the optimization negative point set output based on the instance-level point prompt optimization strategy are mapped into pixel point prompts, and the specific steps are as follows: Deriving all grid cell index sets marked as positive points as an optimized positive point set Deriving all grid cell index sets marked as negative points as an optimized negative point set The calculation formula is as follows: ; ; wherein: representing a point type tensor; representing grid line index in grid cells Whether it is a positive point; Representing a grid cell; Inputting the derived grid point set as a prompt type segmentation model, converting grid coordinates into pixel coordinates, and corresponding to grid units As point hint coordinates, the calculation formula is as follows: ; ; wherein: representing a grid cell row and column index; Representation of Size of the material; Representing the pixel coordinates of the pixel center point.
  10. 10. The remote sensing instance segmentation point self-adaptive optimization method of a discrete grid strategy according to claim 1, wherein the specific steps of inputting the optimization point prompt into the segmentation model and outputting the instance mask are as follows: Taking the optimized positive points and the optimized negative points as point prompt input of a prompt type segmentation model, and predicting an instance mask on a target image according to prompt by the prompt type segmentation model Optimizing positive points for indicating target interior regions, negative points for suppressing background or adjacent instance interference, wherein instance masks are predicted The calculation formula of (2) is as follows: ; wherein: representing a target remote sensing image; Representing an optimized positive point pixel coordinate set; representing an optimized negative point pixel coordinate set; representing a prompt type segmentation model reasoning function; And (3) representing the output instance segmentation mask, and then carrying out threshold denoising on the output instance segmentation mask to obtain a stable instance mask, wherein the calculation formula is as follows: ; wherein: Representing a connected domain operator; Representing the area of the connected domain; Representing an area threshold; representing the post-processed instance mask.

Description

Remote sensing instance division point self-adaptive optimization method of discrete grid strategy Technical Field The invention particularly relates to a remote sensing instance division point self-adaptive optimization method of a discrete grid strategy. Background Remote sensing image instance segmentation aims at separating and identifying target instances (such as buildings, ships, vehicles and the like) with independent semantics and boundaries from high-resolution remote sensing images, and the task is widely applied to scenes such as homeland mapping, city updating, disaster assessment, military reconnaissance and the like. Because the remote sensing image generally has the characteristics of large coverage, obvious target scale change, complex background texture, compact distance between targets and the like, the example segmentation still faces higher difficulty in engineering practice. In recent years, the segmentation model based on prompt can quickly generate a segmentation result through point prompt, box prompt or mask prompt, and has strong interaction flexibility. However, in the remote sensing image, the model often depends on manual click or manual interaction correction to obtain a high-quality result, so that the use cost is high, the batch processing efficiency is low, and the requirement of large-scale remote sensing data automatic processing is difficult to meet. To reduce human interaction, there have been attempts to automatically generate hint points by saliency detection, heuristic rules, or simple feature matching. However, the method generally lacks global optimization capability for 'point set quality', and the problems of insufficient coverage of prompt points, weak boundary discrimination capability, redundancy of positive and negative points, or mutual conflict and the like easily occur, so that the segmentation performance and stability are limited. Disclosure of Invention The invention aims to provide a remote sensing instance division point self-adaptive optimization method of a discrete grid strategy, which solves the problems existing in the prior art. The technical scheme is that the remote sensing instance division point self-adaptive optimization method of the discrete grid strategy comprises the following steps: Given a target rocker image And at least one reference imageTarget rocker image through visual feature extraction networkReference imageA kind of electronic deviceStage feature extraction, setting a matching threshold value, and obtaining a target rocker imagePerforming feature matching on the target rocker image, wherein the feature matching is successfulIn (a)Marking the corresponding pixel area as candidate positive points, building a candidate positive point set, wherein the matching degree is lower than a threshold value or no matching object existsMarking the corresponding region as candidate negative points, and constructing a candidate negative point set; for target rocker images Constructing a discrete example grid with a fixed size for each target example, and respectively mapping a candidate positive point set or a candidate negative point set to grid cells, wherein the grid cells adopt a characteristic extraction network to extract a target rocker imagePerforming feature extraction to generate visual features corresponding to each grid unit, and constructing a state representation comprising grid features, point type indication information and center mark information, wherein the grid units adopt 9×9 grids, performing visual feature sampling on the grid units comprising candidate points, and constructing the state representation; Setting up an example-level reinforcement learning environment, presetting a reward function, executing point type marking operation on the grid unit, and outputting a point prompt set; Training a strategy network by using a near-end strategy optimization algorithm to obtain an instance-level point prompt optimization strategy, outputting an optimization positive point set and an optimization negative point set of each target instance by using the instance-level point prompt optimization strategy, mapping the optimization positive points and the optimization negative points into pixel coordinate point prompts, taking the pixel coordinate point prompts as the input of a prompt type segmentation model, and outputting an instance segmentation mask Stable instance masks are obtained by thresholding the instance segmentation masks。 Preferably, the specific steps of marking the candidate positive or negative points are as follows: For a given target rocker image Proceeding withDividing to obtainPersonal (S)Then extracting features to obtain a rocker image feature matrixThe calculation formula is as follows: ; wherein: A visual feature extraction network is represented, Representing a target rocker imageThe number of the pieces of the plastic material,Representing a feature dimension; For reference images Proceeding withDivid