CN-121982333-A - Single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis

CN121982333ACN 121982333 ACN121982333 ACN 121982333ACN-121982333-A

Abstract

The invention provides a single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis, which relates to the technical field of machine vision and image processing, the invention obtains the preliminary skeleton of all leaves on the surface of tea trees according to point cloud data in a depth map, then selects sampling points according to the positions marked by the skeleton, obtains a plurality of feature maps according to the colors of the sampling points, and respectively acquiring the frameworks in each feature map by the same method, and comparing the frameworks in all the feature maps with the frameworks in the depth map to acquire the real tea bud frameworks.

Inventors

LI YATAO
Tan Liuhuan
HE LEIYING
CHEN JIANNENG
WU CHUANYU
LIU YICHEN
Qian Jinghao

Assignees

浙江理工大学

Dates

Publication Date: 20260505
Application Date: 20260403

Claims (9)

1. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis is characterized by comprising the following specific steps of: Step 1, synchronously acquiring a depth image of tea trees and a gray level image generated by an RGB image, aligning pixel blocks of the two images, acquiring a gradient value of each pixel block in the depth image, and dividing the gradient values into groups according to the similarity and the closeness of the gradient values of different pixel blocks to form a plurality of pixel groups; Step 2, performing binarization processing on each pixel group, detecting two boundary lines of each pixel group, and starting from the two boundary lines, using parallel propagation to initiate propagating waves to the inside at the same time, wherein a line where the two waves meet is marked as a skeleton; Step 3, a sampling point is taken from each skeleton in the depth map, the pixel value of the sampling point in the gray map is recorded through the coordinate index of the sampling point, the pixel range is constructed by the pixel value, the pixel blocks in the gray map in the pixel range are reserved to form a feature map, and each sampling point generates a feature map; Step 4, calibrating each discrete pixel block in each feature map as a pixel group, acquiring a plurality of pixel groups in the feature map, acquiring the skeleton of each pixel group in the feature map through the step 2 again, comparing the skeleton in each feature map with the skeleton in the depth map respectively, and calibrating the skeleton to be confirmed in the feature map and the corresponding skeleton in the depth map according to the number of the same pixel blocks occupied by the skeletons in the two maps; And 5, generating similar scores according to the number of the frameworks to be confirmed in each characteristic graph and the similarity degree of the frameworks to be confirmed and the corresponding frameworks, and judging that the characteristic graph with the largest similarity score is a tea bud graph, wherein the frameworks to be confirmed in the tea bud graph are tea bud frameworks.
2. The method for rapidly extracting the single-view tea bud 3D skeleton based on the point cloud analysis according to claim 1, wherein a tea tree image is acquired through an RGB-D camera, the tea tree image comprises an RGB image and a depth image, the RGB image is converted into a gray scale image, each pixel block in the gray scale image records the pixel value of the position, each pixel block in the depth image records the depth value of the position, and the pixel blocks of the gray scale image and the depth image are aligned; And respectively processing the gray level image and the depth image by a median filtering method.
3. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis, which is disclosed in claim 2, is characterized in that a horizontal neighborhood gradient and a vertical neighborhood gradient of each pixel block in the depth map are respectively obtained through a Sobel convolution kernel, and gradient values are formed, wherein the size of the Sobel convolution kernel is 3x3; Obtaining a gradient array of the depth image, wherein the number of rows of the gradient array is equal to the number of rows of pixel blocks in the depth image, the number of columns of the gradient array is equal to the number of columns of pixel blocks in the depth image, the two-dimensional index of each element in the gradient array corresponds to the two-dimensional index of each pixel block in the depth image one by one, and each element in the gradient array is a gradient value of the corresponding pixel block.
4. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis of claim 3, wherein the pixel blocks in the depth image are grouped through a gradient array, and the grouping logic is as follows: Setting a gradient difference threshold value, in eight adjacent square grids of the pixel block, if the gradient value is smaller than the gradient value of the pixel block and the difference value is larger than the gradient difference threshold value, identifying the pixel block as an edge pixel block, otherwise, identifying the pixel block as not an edge pixel block, judging all continuously connected edge pixel blocks through a contour searching method, identifying and obtaining an edge pixel group forming a closed space, merging the pixel blocks in the closed space into a pixel group, and discarding the edge pixel blocks not forming the closed space; judging the number of pixel blocks in each pixel group, setting an element number threshold, removing the pixel groups with the pixel blocks lower than the element number threshold, and forming a plurality of pixel groups by using only the reserved pixel groups.
5. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis, which is characterized in that the skeleton is extracted for each pixel group respectively, and the logic is as follows: The pixel value of each pixel block in the pixel group is marked as 255, the pixel value of each pixel block outside the pixel group is marked as 0, the boundary line of the pixel group is obtained through a contour tracing algorithm, meanwhile, the end points of the pixel group are obtained through simple end point detection, the number of the end points is at least two, when the number of the end points of the pixel group exceeds two, the two end points are randomly stored, the rest of the end points are taken as points in the boundary line, two independent boundary lines are formed according to the two end points, the two boundary lines are taken as starting points, waves spreading into the pixel group are simultaneously initiated from the boundary line according to a parallel traveling wave propagation algorithm, and the line where the two waves meet is the skeleton.
6. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis of claim 5, wherein all skeletons in the depth image are obtained, the upper direction of the depth image is taken as the upper direction, the point of each skeleton at the Nmm position of the upper end point is taken as a sampling point, the coordinate index of each sampling point in a depth image pixel block is recorded, the coordinate index is compared with a gray level image, and the pixel value of the sampling point corresponding to each skeleton in the gray level image is obtained.
7. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis of claim 6, wherein a pixel value floating range is set, the pixel value obtained by each skeleton is taken as a center, and in the pixel value floating range, similar extraction is respectively carried out on the gray level map, and the logic of the similar extraction is as follows: After a skeleton forms a pixel value of a corresponding sampling point, a pixel range is formed by taking the pixel value as a center, the upper limit of the range is the pixel value of the sampling point plus a pixel value floating range, the lower limit of the range is the pixel value of the sampling point minus the pixel value floating range, a pixel block with the pixel value in the pixel range in a gray level diagram is reserved, the pixel value is calibrated to be 255, and the pixel values of other pixel blocks are normalized to be 0, so that a characteristic diagram is formed; and obtaining feature maps with the same number as the skeletons.
8. The method for rapidly extracting the 3D skeleton of the single-view tea bud based on the point cloud analysis of claim 7, wherein the skeletons in each feature map are respectively compared with the skeletons in the depth map to obtain skeletons to be confirmed which are possibly tea buds, and the comparison logic is as follows: Numbering each skeleton in the feature map, simultaneously recording pixel blocks passed by each skeleton and coordinate indexes of each pixel block, recording the pixel blocks passed by each skeleton and the coordinate indexes of each pixel block in the depth map, comparing the coordinate indexes of each skeleton with the coordinate indexes of each skeleton in the depth map according to the numbering sequence in the same feature map, checking whether repetition exists and the number of repeated pixel blocks, setting a repetition threshold, wherein the repetition threshold represents the number of the same pixel blocks occupied by the two skeletons, calibrating the skeletons in the feature map exceeding the repetition threshold as skeletons to be confirmed, and each skeleton to be confirmed has a corresponding skeleton in the depth map.
9. The method for rapidly extracting the 3D skeleton of the single-view tea buds based on the point cloud analysis of claim 8, wherein the tea bud feature diagrams are selected according to the number and the similarity degree of the skeletons to be confirmed in each feature diagram, and the selection logic is as follows: the method comprises the steps of obtaining the ratio of the number of the frameworks to be confirmed in the feature map to the number of all frameworks in the feature map, simultaneously obtaining the number of the repeated pixel blocks of the frameworks to be confirmed and the corresponding frameworks in the feature map, and constructing similar scores, wherein the logic of the similar scores is as follows: The number of the occupied ratio and the repeated pixel blocks is an independent variable, the similarity score is a dependent variable, the number of the occupied ratio and the repeated pixel blocks is weighted and summed to form the similarity score, the weight of the occupied ratio is larger than that of the repeated pixel blocks, the more the occupied ratio or the more similar score of the repeated pixel blocks is larger, and the influence of the occupied ratio on the similarity score is larger than that of the repeated pixel blocks; And obtaining a characteristic diagram with the maximum similarity score, marking the characteristic diagram as a tea bud diagram, and obtaining a framework to be confirmed in the tea bud diagram as a tea bud framework.

Description

Single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis Technical Field The invention relates to the technical field of machine vision and image processing, in particular to a single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis. Background Along with the development of tea market, the demand for high-quality tea is higher and higher, tea buds are the most important materials in tea, traditional tea bud picking is usually manual picking by tea farmers, and the process requires great personnel and labor force and consumes time, so that automatic mechanical picking becomes a necessary requirement, and tea bud identification is the first link of automatic picking, so that the identification of tea buds by a machine vision method has important significance for automatic picking. In the process of identification and picking, in order to avoid the waste of computing resources and improve the picking efficiency, only the framework of tea buds needs to be identified to provide data support for the subsequent picking action, so that a visual picking method based on image processing needs to be formed. In the prior art, publication number CN117078926A discloses a tea leaf tender tip positioning method based on a 3D point cloud topological structure, which comprises the steps of firstly obtaining RGB images and point clouds of tea leaf tender tips through a depth camera, utilizing an improved target detection network to realize identification and coarse positioning of the tea leaf tender tips, extracting target point clouds based on an original point cloud set in combination with a target area, carrying out preprocessing such as point cloud denoising and downsampling, carrying out clustering segmentation on the preprocessed three-dimensional point clouds, carrying out skeleton point extraction, connecting skeleton points to obtain tender tip skeletons, and finally carrying out neighbor searching on skeleton points of the tea leaf tender tip skeleton structures to obtain the grade of the tea leaf tender tips and positioning points of picking positions. The disclosed technical document realizes skeleton recognition of tea buds, but obtains skeleton information by recognizing the whole appearance of the tea buds, and has dense blades and more shielding phenomenon in tea trees, and the skeleton robustness is lower only by means of point cloud data, so that the recognition success rate is affected. The above information disclosed in the background section is only for enhancement of understanding of the background of the disclosure and therefore it may include information that does not form the prior art that is already known to a person of ordinary skill in the art. Disclosure of Invention The invention aims to provide a single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis, so as to solve the problems in the background technology. In order to achieve the above purpose, the present invention provides the following technical solutions: A single-view tea bud 3D skeleton rapid extraction method based on point cloud analysis comprises the following specific steps: Step 1, synchronously acquiring a depth image of tea trees and a gray level image generated by an RGB image, aligning pixel blocks of the two images, acquiring a gradient value of each pixel block in the depth image, and dividing the gradient values into groups according to the similarity and the closeness of the gradient values of different pixel blocks to form a plurality of pixel groups; Step 2, performing binarization processing on each pixel group, detecting two boundary lines of each pixel group, and starting from the two boundary lines, using parallel propagation to initiate propagating waves to the inside at the same time, wherein a line where the two waves meet is marked as a skeleton; Step 3, a sampling point is taken from each skeleton in the depth map, the pixel value of the sampling point in the gray map is recorded through the coordinate index of the sampling point, the pixel range is constructed by the pixel value, the pixel blocks in the gray map in the pixel range are reserved to form a feature map, and each sampling point generates a feature map; Step 4, calibrating each discrete pixel block in each feature map as a pixel group, acquiring a plurality of pixel groups in the feature map, acquiring the skeleton of each pixel group in the feature map through the step 2 again, comparing the skeleton in each feature map with the skeleton in the depth map respectively, and calibrating the skeleton to be confirmed in the feature map and the corresponding skeleton in the depth map according to the number of the same pixel blocks occupied by the skeletons in the two maps; And 5, generating similar scores according to the number of the frameworks to be confirmed in each characteristic graph and the similarity degree of the frameworks to be confirmed and the correspon