CN-122023865-A - Weed flexibility identification method based on deep learning

CN122023865ACN 122023865 ACN122023865 ACN 122023865ACN-122023865-A

Abstract

The invention relates to a weed flexibility recognition method based on deep learning, which comprises the steps of collecting pictures of an obstacle under different illumination at different time, calculating six-dimensional parameters and seven Hu invariant moments of the coverage area, convex hull area, perimeter, length-width ratio, actual height and gravity center height of the obstacle by combining the deep learning and image processing, designing an optimal artificial neural network structure and super parameters by utilizing characteristic engineering processing parameters, realizing prediction of the folding resistance of the obstacle, comparing the folding resistance with the stable bearing force of an experimental vehicle, determining the flexibility attribute of the obstacle, and finally determining whether the obstacle needs to bypass. According to the invention, the flexibility recognition is introduced, the flexibility characteristics of the targets in the images are analyzed, the intelligent discrimination of weeds and other obstacles in the field or photovoltaic power station environment is realized, the robot behaviors can be dynamically adjusted according to different operation environments, such as agricultural fields or photovoltaic power stations, and safer, more efficient and intelligent operation control is realized.

Inventors

WANG YONG
GU HAIYANG
GUO KAI

Assignees

飒沓机器人科技(苏州)有限公司
苏州大学

Dates

Publication Date: 20260512
Application Date: 20251209

Claims (5)

1. The weed flexibility identification method based on deep learning is characterized by comprising the following steps of: Acquiring obstacle pictures under different illumination at different time, and calculating six-dimensional parameters and seven Hu invariant moments of the coverage area, convex hull area, perimeter, length-width ratio, actual height and gravity center height of the obstacle by combining deep learning and image processing; Designing an optimal artificial neural network structure and super parameters by using characteristic engineering processing parameters, and realizing the prediction of the breaking force of the obstacle; And comparing the bending resistance with the stable bearing force of the experimental vehicle to determine the flexibility attribute of the obstacle, and finally determining whether the obstacle needs to detour.
2. The weed compliance identification method based on deep learning as claimed in claim 1, wherein the semantic segmentation is performed by using U-NET, and specifically comprising the steps of: s1.1, manufacturing an image classification sample set by adopting an interval frame taking method; s1.2, labeling samples, including weeds and/or backgrounds; Step S1.3, data enhancement, namely performing random clipping, color enhancement, contrast enhancement and horizontal overturn on an image to enhance the robustness of the data; and S1.4, carrying out semantic segmentation training by adopting the U-NET to obtain the optimal model weight.
3. The weed compliance identification method based on deep learning as claimed in claim 2, wherein the data set construction is performed, specifically comprising the steps of: s2.1, extracting a mask contour based on the images segmented by the U-NET, wherein the mask contour is used for extracting the appearance of weeds; Step S2.2, calculating a coverage area S1 and an aspect ratio P, wherein the number of the statistical masks is used as the coverage area through the mask contour information obtained in the step S2.1, and the aspect ratio P is obtained according to the length-width pixel proportion of the boundary box; S2.3, calculating the perimeter C of the obstacle, eliminating internal points of the obtained mask, only leaving surrounding pixel points and traversing all mask pixel points, wherein when the 8 neighborhood of the pixel points is in the mask, the internal points are the boundary points, otherwise, finally counting all the pixel points and subtracting the internal points to obtain the perimeter of the weeds; step S2.4, calculating the gravity center W of the image, using first-order origin moment calculation of the image, and representing gray values at (x, y) by I (x, y) in the two-dimensional image, wherein the calculation formula is as follows: Wherein: Wherein M 00 is the pixel area sum of the binary images, M 10 is the sum of all the binary images X, M 01 is the sum of all the binary images y, and the calculation formula of the gravity center (X 0 ,Y 0 ) is as follows: X 0 ＝M 10 /M 00 , Y 0 ＝M 01 /M 00 ; Step S2.5, the convex hull area S2 is used as an enhancement index when the segmentation is poor, the function is findContours which is realized in OpenCV, a binary image is transmitted to obtain a convex hull, and the number of pixels in the convex hull area is calculated as the convex hull area; step S2.6, actual height H, according to similar triangle: Wherein Y represents the actual height of the object, Y represents the pixel height of the object in the imaging plane, Z represents the object distance, and Z represents the distance between the imaging plane and the optical center, i.e. the focal length, and the actual height H of the object is deduced according to the above formula; step S2.7, obtaining image moments, namely adding 7 parameters, wherein the image moments are used for extracting characteristic parameters, and the calculation formulas of 7 Hu invariant moments are as follows: Wherein: γ=p+q+1, and step S2.8, fusing the characteristic parameters of the steps S2.2 to S2.7.
4. The weed compliance identification method based on deep learning of claim 3, wherein the artificial neural network building step specifically comprises the following steps: Step S3.1, refining based on the obtained data set of the 13-dimensional data: Step S3.11, firstly, analyzing the characteristic correlation, and calculating the relation between the characteristics by using the pearson correlation coefficient, wherein the formula is as follows: cov=E(XY)-E(X)E(Y), Step S3.12, normalization processing, namely, the sample distribution difference of the data is large, the dimension of the pixel area is in the thousand levels, and the aspect ratio is in the number level, so that the normalization processing is carried out on the data in each dimension, noise is introduced in the data set, and a Min-Max method is adopted, wherein the formula is as follows: Step S3.13, detecting and eliminating abnormal values, and eliminating abnormal values by using a 3 sigma criterion, wherein the formula is as follows: step S3.2, designing a neural network structure and super parameters: s3.21, constructing an artificial neural network, wherein the number of input neuron nodes is 13, the number of output neurons is 1 corresponding to each dimension value of each sample, and the number of output neurons is the sample bending resistance; Step S3.22, designing the hidden layer number and the neuron number of the hidden layer based on an empirical formula: Wherein m is the number of hidden layer neurons, l is the number of output neurons, n is the number of input neurons, 2 hidden layers are selected, and the MSE, the model time consumption T and the decision coefficient R 2 are used for considering the performance; wherein, the MES and R 2 formulas of the result evaluation indexes are as follows: Where N represents the total number of samples used for evaluation, W i represents the actual magnitude of the bending resistance, and E i represents the estimated magnitude of the bending resistance of the neural network; Where N represents the total number of samples evaluated, y i represents the test set samples, y predict represents the predicted results of the model on the test set, and y mean represents the average value of the test set samples; And S3.23, designing an activation function and an optimization algorithm, using a Tanh function as the activation function, and selecting Adagrad as the optimization algorithm.
5. The weed compliance identification method based on deep learning as claimed in claim 4, wherein the specific steps of the method for judging the compliance of the obstacle are as follows: s4.1, acquiring a real-time frame image; s4.2, inputting the artificial neural network; Step S4.3, predicting the obstacle folding resistance F according to each frame of image, wherein the function obtained by fitting the input data is as follows: And S4.4, judging the flexibility of the obstacle, setting a threshold K according to the anti-collision performance of the vehicle body, judging the vehicle body as a rigid obstacle when F is larger than K, and judging the vehicle body as a flexible obstacle otherwise.

Description

Weed flexibility identification method based on deep learning Technical Field The invention relates to the technical field of intelligent robots, in particular to a weed flexibility identification method based on deep learning. Background With the rapid development of intelligent robot technology, the application of inspection robots and mowing robots in outdoor scenes such as photovoltaic power stations and farmlands is increasingly popular. In the prior art, a robot mainly relies on vision or a laser sensor to detect ground targets or obstacles, but most systems can only identify the positions and morphological characteristics of the targets, further distinction of physical properties of the targets is lacking, the targets are directly judged as the obstacles, a patrol robot can avoid the obstacles, and a mowing robot can mow the grass directly. In practical applications, the difference of the flexibility characteristics of different types of targets is remarkable. For the inspection robot, targets with lower flexibility and harder structure usually cause potential collision risks to equipment, an avoidance strategy is needed, and targets with higher flexibility and deformation under the action of external force can directly pass through without affecting operation safety. In an actual operation scene, weeds or other obstacles have flexibility difference, namely vegetation with higher flexibility can be directly mowed or rolled without avoiding, vegetation or obstacles with lower flexibility and harder structure can damage equipment during direct operation, and at the moment, a robot should adopt strategies such as detouring, decelerating or stopping alarming to prevent the robot from influencing the operation environment and mowing quality under the photovoltaic panel. However, most of the existing mowing robots only rely on geometric forms or color information of images to identify targets, and lack accurate judging capability on flexibility characteristics, so that operation strategies are single, misjudging rate is high, mowing efficiency is affected, and service life of equipment is prolonged. As shown in FIG. 1, the scene is the condition of a desert photovoltaic power station, a small haloxylon ammodendron tree is arranged between two rows of photovoltaics, part of the haloxylon ammodendron tree is lignified, the whole haloxylon ammodendron tree is relatively hard and can possibly avoid the haloxylon ammodendron tree during inspection, and as shown in FIG. 2, the scene is the growth condition of salix psammophila in the desert photovoltaic power station, and the salix psammophila is large and needs to be mowed by a mowing robot. As shown in fig. 3-5, the common profile and corresponding dimensions of the haloxylon ammodendron tree in a desert power station. Therefore, a weed flexibility identification method based on deep learning is needed, intelligent judgment on target flexibility is achieved through image sensing and feature analysis, a more accurate decision basis is provided for inspection and mowing operation, and the operation intelligence level of the robot in a complex environment is improved. Disclosure of Invention The invention aims to overcome the problems in the prior art, and provides a weed compliance identification method based on deep learning, which is used for analyzing the compliance characteristics of targets in images and realizing intelligent discrimination of weeds and other obstacles in a field or photovoltaic power station environment, thereby providing accurate decision basis for robot operation. In order to achieve the technical purpose and the technical effect, the invention is realized by the following technical scheme: a weed compliance identification method based on deep learning, the method comprising: Acquiring obstacle pictures under different illumination at different time, and calculating six-dimensional parameters and seven Hu invariant moments of the coverage area, convex hull area, perimeter, length-width ratio, actual height and gravity center height of the obstacle by combining deep learning and image processing; Designing an optimal artificial neural network structure and super parameters by using characteristic engineering processing parameters, and realizing the prediction of the breaking force of the obstacle; And comparing the bending resistance with the stable bearing force of the experimental vehicle to determine the flexibility attribute of the obstacle, and finally determining whether the obstacle needs to detour. Furthermore, the U-NET is adopted for semantic segmentation, and the method specifically comprises the following steps: s1.1, manufacturing an image classification sample set by adopting an interval frame taking method; s1.2, labeling samples, including weeds and/or backgrounds; Step S1.3, data enhancement, namely performing random clipping, color enhancement, contrast enhancement and horizontal overturn on an image to enhance the robus