CN-121981885-A - Fan blade image stitching method based on shape features and texture features

CN121981885ACN 121981885 ACN121981885 ACN 121981885ACN-121981885-A

Abstract

The invention relates to the technical field of image processing and discloses a fan blade image splicing method based on shape characteristics and texture characteristics, which comprises the steps of obtaining a fan blade image, performing overlapping sliding window cutting to obtain a plurality of subgraphs, dividing the subgraphs to obtain a plurality of first mask images only comprising the fan blade, and performing pretreatment to obtain a plurality of second mask images; based on each two adjacent second mask graphs, taking one as a reference graph and the other as a graph to be registered, respectively carrying out affine parameter calculation based on texture features and affine parameter calculation based on shape features to obtain a first affine parameter and a second affine parameter, carrying out comprehensive optimization on the first affine parameter and the second affine parameter to obtain a final affine parameter, carrying out affine transformation on each graph to be registered and each subgraph based on the corresponding final affine parameter, and splicing to obtain a mask splicing result and a subgraph splicing result. The invention can maintain the consistency of global geometric shapes and realize the high-precision alignment of local textures.

Inventors

Jiang Enchao
ZHANG CE
SHI ZHIYUAN
SHENG ZHONGXI
YI TAIXUN
LIAO BANGMIN
ZHOU YUANBING
LIAN YI
DAI RUI

Assignees

东方电气集团数字科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260116

Claims (10)

1. A fan blade image splicing method based on shape characteristics and texture characteristics is characterized in that: The fan blade image splicing method comprises the following steps: s1, acquiring fan blade images, performing overlapping sliding window cutting to obtain a plurality of subgraphs, dividing the subgraphs to obtain a plurality of first mask images only comprising fan blades, and preprocessing to obtain a plurality of second mask images; S2, based on each two adjacent second mask patterns, taking one as a reference pattern and the other as a pattern to be registered, and respectively carrying out affine parameter calculation based on texture features and affine parameter calculation based on shape features to obtain a first affine parameter and a second affine parameter; S3, comprehensively optimizing the first affine parameter and the second affine parameter to obtain a final affine parameter; and S4, carrying out affine transformation on each graph and subgraph to be registered based on the corresponding final affine parameters, and splicing to obtain a mask splicing result and a subgraph splicing result.
2. The fan blade image stitching method based on shape features and texture features of claim 1, wherein: the specific steps of the pretreatment include: and denoising and smoothing each first mask image, and aligning the coordinates of each first mask image and the corresponding subgraph.
3. The fan blade image stitching method based on shape features and texture features of claim 1, wherein: the specific steps for carrying out affine parameter calculation based on texture features comprise the following steps: Performing mask cutting on the corresponding subgraph based on the second mask to obtain a blade subgraph only comprising fan blades, and performing graying and normalization processing on the blade subgraph; Extracting key points of two adjacent blade subgraphs by SuperPoint, matching the key points of the two adjacent blade subgraphs by using SuperGlue, generating confidence coefficient, constructing a point pair set, presetting a confidence coefficient threshold, removing point pairs with the confidence coefficient smaller than the confidence coefficient threshold to obtain an optimized point pair set, if the point pair of the optimized point pair set is zero, outputting a null value as a first affine parameter, if the point pair of the optimized point pair set is larger than zero, carrying out coordinate alignment on the blade subgraphs and a corresponding second mask, calculating the reprojection error of each point pair in the optimized point pair set, presetting a distance threshold, acquiring point pairs with the reprojection error smaller than the distance threshold, estimating an affine matrix, and constructing an inner point set; Calculating symmetric transmission errors of each point pair based on the affine matrix and the interior point set, presetting an error threshold, removing point pairs with the symmetric transmission errors larger than the error threshold to obtain an optimized interior point set, and estimating the affine matrix based on the point pairs in the optimized interior point set to obtain an optimized affine matrix; And analyzing a rotation angle, a scale factor, a transverse translation amount and a longitudinal translation amount from the optimized affine matrix, and constructing a first affine parameter.
4. A method of stitching fan blade images based on shape and texture features as recited in claim 3 wherein: the calculation formula for calculating the symmetrical transmission errors of each point is as follows: ; Wherein, the In order to make the transmission error symmetrical, Is the first The key point of one of the blade subgraphs in a point pair, Is the first The corresponding key point of the other blade subgraph in the pair of points, To optimize the affine matrix.
5. The fan blade image stitching method based on shape features and texture features of claim 1, wherein: The affine parameter calculation based on the shape characteristics comprises the following specific steps: Respectively carrying out edge detection on the reference image and the image to be registered, extracting straight-line segments, obtaining end points, lengths and angles of the straight-line segments, calculating center coordinates of the straight-line segments based on the end points, and dividing the straight-line segments into a first group and a second group based on the center coordinates; Normalizing the angles of the straight line segments to obtain normalized angles of the straight line segments, clustering the straight line segments of the first group and the second group based on the normalized angles to obtain a plurality of straight line clusters, calculating the average normalized angles of the straight line segments in the straight line clusters, respectively obtaining the straight line segments with the largest length in the straight line clusters in the first group and the second group, and constructing a first set and a second set; selecting any one straight line segment from the first set of the reference image and the image to be registered, calculating the angle difference of the two straight line segments, and constructing a first optimal straight line group by using the two straight line segments with the smallest angle difference; Based on the first optimal linear group and the second optimal linear group, obtaining two optimal edge lines of each of the reference image and the image to be registered, respectively constructing corresponding central lines based on the optimal edge lines, obtaining angles of the two central lines, and calculating a rotation angle; respectively calculating the intersection abscissa of the bottom edges of the reference image and the image to be registered and the corresponding central line, and calculating the difference value to obtain the transverse translation quantity; Constructing a width function based on the height and the width of the fan blade in the reference graph, calculating the bottom width of the graph to be registered, inputting the bottom width of the graph to be registered into the width function, solving to obtain a theoretical height, and calculating to obtain an initial longitudinal translation amount based on the bottom edge height and the theoretical height of the reference graph; adjusting an initial scale factor based on the initial longitudinal translation amount until the theoretical height is smaller than or equal to the bottom edge height of the reference graph, and taking the current initial longitudinal translation amount and the initial scale factor as the longitudinal translation amount and the scale factor; And constructing a second affine parameter based on the rotation angle, the transverse translation amount, the longitudinal translation amount and the scale factor.
6. The method for stitching fan blade images based on shape features and texture features of claim 5, wherein: the specific steps of calculating the intersection abscissa of the bottom edge of the reference picture and the corresponding central line are as follows: Selecting a plurality of horizontal lines within a certain range at the bottom of the reference graph, acquiring intersection points of each horizontal line and the corresponding central line, calculating the abscissa of each intersection point, and taking the median value of the abscissa of each intersection point as the intersection abscissa of the bottom edge of the reference graph and the corresponding central line.
7. The fan blade image stitching method based on shape features and texture features of claim 1, wherein: The specific step of comprehensively optimizing the first affine parameter and the second affine parameter in S3 includes: Presetting an optimization rule, selecting a first affine parameter or a second affine parameter as an initial parameter based on the optimization rule, carrying out affine transformation on the to-be-registered graph based on the initial parameter to obtain a registration graph, mapping the reference graph and the registration graph into the same coordinate system, and calculating the coincidence loss value of the registration graph and the reference graph; If the initial parameter is a first affine parameter, judging whether the coincidence loss value is smaller than or equal to a preset threshold value, if so, taking the first affine parameter as a final affine parameter, if not, optimizing the first affine parameter to obtain a third affine parameter, taking the third affine parameter as the initial parameter to calculate the coincidence loss value, judging whether the coincidence loss value is smaller than or equal to the preset threshold value, if so, taking the third affine parameter as the final affine parameter, if not, continuing optimizing the first affine parameter to update the third affine parameter until reaching a preset updating round, taking the third affine parameter with the smallest coincidence loss value as a first candidate parameter, taking the second affine parameter as the initial parameter to calculate the coincidence loss value, judging whether the coincidence loss value is smaller than or equal to the preset threshold value, if so, taking the second affine parameter as the final parameter; If the initial parameter is a second affine parameter, judging whether the coincidence loss value is smaller than or equal to a preset threshold value, if so, taking the second affine parameter as a final affine parameter, if not, optimizing the second affine parameter to obtain a fourth affine parameter, calculating the coincidence loss value by taking the fourth affine parameter as the initial parameter, judging whether the coincidence loss value is smaller than or equal to the preset threshold value, if so, taking the fourth affine parameter as the final affine parameter, if not, continuing optimizing the second affine parameter to update the fourth affine parameter until a preset updating round is reached, taking the fourth affine parameter with the smallest coincidence loss value as a second candidate parameter, comparing the coincidence loss values corresponding to the second candidate parameter and the second affine parameter, and taking the coincidence loss value with the smallest coincidence loss value as the final affine parameter.
8. The method for stitching fan blade images based on shape and texture features of claim 7, wherein: The specific steps of calculating the coincidence loss value of the registration graph and the reference graph are as follows: And acquiring an overlapping region of the reference image and the registration image, acquiring the pixel number of the reference image, the pixel number of the registration image and the overlapping pixel number of the reference image and the registration image based on the overlapping region, respectively calculating the duty ratio of the overlapping pixel number in the pixel number of the reference image and the duty ratio in the pixel number of the registration image, obtaining a first duty ratio and a second duty ratio, and calculating a loss value based on the first duty ratio and the second duty ratio.
9. The method for stitching fan blade images based on shape features and texture features of claim 8, wherein: the calculation formula for calculating the loss value based on the first duty ratio and the second duty ratio is as follows: ; Wherein, the In order to achieve a loss value, the value of the loss, At the first duty cycle of the first power supply, Is a second duty cycle.
10. The method for stitching fan blade images based on shape and texture features of claim 7, wherein: the specific steps of optimizing the first affine parameter are as follows: And setting a neighborhood based on the first affine parameter, sampling in the neighborhood, and generating a third affine parameter.

Description

Fan blade image stitching method based on shape features and texture features Technical Field The invention relates to the technical field of image processing, in particular to a fan blade image stitching method based on shape features and texture features. Background The fan blade is used as a key component of the wind generating set, the length of the fan blade can reach tens of meters, a single image often cannot completely cover the whole view of the blade, and in order to realize panoramic detection, a plurality of local high-resolution images are usually required to be acquired through an unmanned plane or a ground camera and spliced to form a continuous and complete panoramic view of the blade. The traditional method relies on manual marking or a general splicing algorithm based on a natural scene, but the blade has the characteristics of long strips, single texture and prominent boundary characteristics, so that the registration precision among images is often insufficient, the splicing result is misplaced, gaps or deformed, and the overall quality is influenced. Especially under the condition of illumination change, shooting angle difference and background interference, the splicing effect is more easily degraded, and the requirements of wind power plant operation and maintenance on high precision and high reliability are difficult to meet. In order to solve the technical problems in the traditional method, the industry tries to introduce the deep learning into the blade detection, but the research on the blade splicing is relatively insufficient, the blade geometric shape characteristics and the surface texture characteristics are not fused, so that affine transformation parameters cannot be accurately estimated in the splicing process, and the global geometric shape consistency and the local texture alignment are difficult to be considered. Disclosure of Invention The invention aims to provide a fan blade image splicing method based on shape characteristics and texture characteristics, which can maintain global geometric shape consistency and realize high-precision alignment of local textures. The technical scheme adopted by the invention is as follows: A fan blade image stitching method based on shape features and texture features comprises the following steps: s1, acquiring fan blade images, performing overlapping sliding window cutting to obtain a plurality of subgraphs, dividing the subgraphs to obtain a plurality of first mask images only comprising fan blades, and preprocessing to obtain a plurality of second mask images; S2, based on each two adjacent second mask patterns, taking one as a reference pattern and the other as a pattern to be registered, and respectively carrying out affine parameter calculation based on texture features and affine parameter calculation based on shape features to obtain a first affine parameter and a second affine parameter; S3, comprehensively optimizing the first affine parameter and the second affine parameter to obtain a final affine parameter; and S4, carrying out affine transformation on each graph and subgraph to be registered based on the corresponding final affine parameters, and splicing to obtain a mask splicing result and a subgraph splicing result. According to the technical measures, the fan blade area is limited by dividing the sub-graph after the fan blade image is cut, irrelevant background is removed, affine parameters are calculated in parallel on two paths of texture features and shape features, final affine parameters are obtained through comprehensive optimization, after affine transformation is carried out on each graph to be registered and the sub-graph based on the final affine parameters and the graph to be registered and the sub-graph are spliced, high-precision alignment of local textures can be achieved while global geometric shape consistency is maintained by a mask splicing result and a sub-graph splicing result, and splicing precision and quality are effectively improved. Further, the specific steps of the pretreatment include: and denoising and smoothing each first mask image, and aligning the coordinates of each first mask image and the corresponding subgraph. According to the technical measures, through denoising and smoothing, the continuity of the fan blade boundary is effectively ensured, the quality of the first mask images is improved, and through aligning the coordinates of each first mask image with the corresponding subgraph, the coordinate offset can be reduced, and the follow-up operation error is reduced. Further, the specific steps of carrying out affine parameter calculation based on texture features comprise: Performing mask cutting on the corresponding subgraph based on the second mask to obtain a blade subgraph only comprising fan blades, and performing graying and normalization processing on the blade subgraph; Extracting key points of two adjacent blade subgraphs by SuperPoint, matching the key points of the