CN-122023933-A - Tomato dynamic growth monitoring method based on robot inspection and digital twin

CN122023933ACN 122023933 ACN122023933 ACN 122023933ACN-122023933-A

Abstract

The invention discloses a tomato dynamic growth monitoring method based on robot inspection and digital twinning. Firstly, selecting and identifying test tomato plants according to planting management requirements; then, constructing a patrol map by utilizing the ROS and SLAM algorithm, wherein the patrol robot patrol according to a set track and accurately identify a sample plant through the identification information and park; then, detecting plants by two depth cameras mounted on the liftable rotating support, obtaining a three-dimensional point cloud model of tomatoes, and extracting key phenotype data such as plant height, stem thickness, fruit cluster number, flowering number and the like; and finally, combining a digital twin technology, constructing a periodically acquired three-dimensional point cloud model and phenotype data into a tomato dynamic growth model, and combining environmental factors and water and fertilizer records to update and optimize a tomato growth management strategy in real time.

Inventors

ZHANG JUNXIONG
Duan Yichu
LI WEI
GUO JIAYANG
ZHAO CHENGWEI
LIU JIAWEI
CHEN YING

Assignees

中国农业大学

Dates

Publication Date: 20260512
Application Date: 20260214

Claims (5)

1. A tomato dynamic growth monitoring method based on robot inspection and digital twin is characterized by comprising the following steps: s1, selecting corresponding test plants according to test requirements of tomato planting managers, and carrying out unique identification on each sample plant; S2, constructing a patrol map by utilizing a robot operating system ROS and a simultaneous positioning and map construction SLAM algorithm, wherein the patrol robot is used for patrol according to a given track and accurately identifying a sample plant through identification information and parking; S3, expanding the coverage range of a field of view by using two depth cameras through a lifting rotating bracket mounted on a greenhouse inspection robot, and synchronously collecting key phenotype information such as plant height, stem thickness, fruit cluster number, flowering number and the like of a target plant; and S4, integrating the periodically acquired tomato three-dimensional point cloud model and phenotype data such as plant height, stem thickness, fruit cluster number, flowering number and the like into a digital twin platform to construct an evolvable dynamic growth model, wherein the model synchronously combines real-time environment variables and water and fertilizer records, and supports growth trend prediction and management strategy optimization.
2. The method for monitoring the dynamic growth of tomatoes based on robot inspection and digital twinning according to claim 1, wherein the step S1 specifically comprises the following steps: S11, selecting tomato plants meeting test conditions from a greenhouse as samples; S12, sticking a label 1 with a two-dimensional code on the top of each plant sample, and respectively sticking a label 2, a label 3 and a label 4 on the middle part, the middle lower part and the bottom of the plant in the height direction, wherein the labels 1, the label 2, the label 3 and the label 4 adopt the same color, and only the label 1 comprises the two-dimensional code which is used for uniquely identifying the identity information of the plant; And S13, associating the two-dimensional code information in the tag 1 with the plants and storing the two-dimensional code information into a database so as to realize the follow-up tracking and monitoring of the growth states of the plants of each sample.
3. The method for monitoring the dynamic growth of the tomatoes based on the robot inspection and digital twinning according to claim 1, wherein the step S2 specifically comprises the following steps: S21, scanning a greenhouse planting environment by using a SLAM mapping method to generate a yaml-format environment map file of greenhouse tomatoes, wherein the map contains space structures and barrier information of all areas in a greenhouse and is a navigation basis of a greenhouse inspection robot; s22, calibrating key positions and coordinate points in a greenhouse to generate an automatic inspection path, wherein the greenhouse inspection robot inspects according to a set route; S23, after the inspection robot enters between ridges, starting an upper camera to synchronously acquire RGB image flow and depth image flow for detecting two-dimensional code information in the tag 1, and when the real-time acquired depth value of the tag 1 is in a preset threshold range [ alpha min, alpha max ], executing preliminary parking by the inspection robot, wherein the position sitting mark is as follows 。
4. The method for monitoring the dynamic growth of the tomatoes based on the robot inspection and digital twinning according to claim 1, wherein the step S3 specifically comprises the following steps: S31, performing vertical upward visual field splicing by two depth cameras on a greenhouse inspection robot, wherein the visual field of a first depth camera covers the upper middle region of a plant, the visual field of a second depth camera covers the lower middle region and the stem base region of the plant, and the visual fields of the two cameras are complementary in the vertical direction and the same in the horizontal direction so as to ensure that key organs of the whole tomato plant are in an effective observation range; s32, based on the position of the inspection robot in S23 Combining the color identification information of the labels 1 to 4, identifying the positions of the labels in the image, and detecting the abscissa of the center of the frame by the label 1 Abscissa of center of detection frame with tag 4 As a left-right boundary reference of plants; S33, acquiring left and right boundary abscissa of plant based on identification And Determining a geometric constraint reference of a scanning track, controlling a rotating bracket to drive a depth camera to start from an initial observation point, constructing a scanning circular arc track which takes a plant side outside space point as a virtual circle center through the horizontal movement of a patrol robot and the real-time linkage compensation of a bracket rotation angle, enabling the depth camera to move from the plant side rear to the plant side front along the circular arc track, synchronously acquiring plant side phenotype structure data and three-dimensional point cloud, and realizing the completion of dynamic growth data of a plant front view shielding area; s34, fitting a central axis curve of a main stem of a tomato plant based on space coordinates of the tag 1, the tag 2, the tag 3 and the tag 4 in a three-dimensional point cloud model, and calculating the length of the curve from a base to a top to obtain the actual growth height of the plant as a plant height phenotype parameter; S35, extracting a target point cloud segment of a main stem of a tomato plant based on the obtained complete plant point cloud model, sampling a horizontal section slice in the segment along the central axis direction of the main stem, performing ellipse fitting on the obtained section point cloud by using a least square method, calculating the mean value of the major axis and the minor axis of the fitted ellipse, and defining the mean value of the major axis and the minor axis as a stem thick phenotype parameter of the sampling position; and S36, detecting the number of fruit clusters and the number of flowers based on a target detection model, pre-training by collecting a data set of tomato flowers and a full-growth period of tomatoes in advance, calling a trained target detection algorithm in the process of inspection, and simultaneously combining a target tracking algorithm to eliminate the repeated counting problem caused by repeated target identification, and finally, realizing accurate statistics of the total number of the fruit clusters and the number of flowers of the individual tomatoes.
5. The method for monitoring the dynamic growth of the tomatoes based on the robot inspection and digital twinning according to claim 1, wherein the step S4 specifically comprises the following steps: S41, controlling a greenhouse inspection robot to periodically inspect tomato plants of the same batch of samples according to a preset date interval, and repeatedly executing the steps S1 to S3 to acquire three-dimensional point cloud data and corresponding phenotype parameters of each time point; s42, carrying out identity matching and time alignment on the data acquired by the same plant at different time points through the two-dimensional code in the label 1 to form a time sequence data set taking the plant as a unit; s43, carrying out time sequence modeling on key phenotype parameters such as plant height, stem thickness, fruit cluster number, flowering number and the like based on the time sequence data set, and fitting the growth variation trend of the key phenotype parameters; S44, fusing the fitting result with the three-dimensional point cloud model, and constructing a dynamic growth model of the plant in the digital twin platform to realize visual backtracking and future growth state prediction; S45, synchronously collecting greenhouse environmental factor data and water and fertilizer integrated management records, associating the environmental factor and water and fertilizer integrated records with dynamic growth models of corresponding plants according to time stamps, and constructing an enhanced digital twin body fusing growth-environment-water and fertilizer information; And S46, according to the simulation scene input by the manager, adjusting virtual water and fertilizer parameters or environment setting, deducing the future growth state of the tomato plant by the model, and outputting optimization suggestions.

Description

Tomato dynamic growth monitoring method based on robot inspection and digital twin Technical Field The invention relates to the field of facility agriculture robots, in particular to a tomato dynamic growth monitoring method based on robot inspection and digital twinning. Background Tomato is used as an important vegetable crop widely planted worldwide, and has high nutritive value and remarkable economic value. However, the existing tomato planting management in a large-scale multi-span greenhouse still highly depends on manual operation to perform tomato growth detection and management regulation, and has the main three outstanding problems that firstly, the manual measurement mode is time-consuming and labor-consuming and low in efficiency, and is difficult to meet the requirement of large-scale cultivation, secondly, the manual recording and arrangement of a large amount of discrete phenotype data is complex in process and easy to make mistakes, thirdly, the acquired discrete data is difficult to fully utilize and excavate, and the construction and industrial intelligent upgrading of an intelligent decision system are restricted. In recent years, with the rapid development of agricultural robots and machine vision technologies, the existing researches focus on the identification and detection of specific targets, such as tomato string positioning, single fruit segmentation, maturity discrimination or disease identification, etc., such as the "inspection robot real-time position identification and tomato counting yield evaluation method" proposed in the patent of publication number CN116681964a, these researches generally have stronger single task guidance, lack systematic integration of overall three-dimensional configuration and agronomic traits of plants, and are difficult to support comprehensive and dynamic evaluation of tomato growth states. In the field of building plant growth models, for example, a method for building a plant three-dimensional model based on a generator adopted in a patent with publication number CN118967971A is characterized in that the core of the method depends on template scaling and splicing in a preset database, and input is only an image and an environment variable, so that the three-dimensional structural change of a real plant cannot be reflected. Based on the method, the invention provides a tomato dynamic growth monitoring method based on robot inspection and digital twin. According to the method, data acquisition is carried out on tomato plants through a patrol robot carrying two depth cameras, and a digital twin model of individual experimental plants is built by combining a three-dimensional reconstruction technology, so that dynamic growth monitoring of tomatoes is realized. On the basis, the system can integrate environmental parameters and water and fertilizer records, support deduction growth scene simulation as required, provide scientific and accurate intelligent decision basis for planting managers, and effectively promote intelligent tomato production. Disclosure of Invention In order to solve the technical problems, the invention provides a tomato dynamic growth monitoring method based on robot inspection and digital twinning. The method for monitoring the dynamic growth of the tomatoes based on the robot inspection and digital twinning comprises the following steps: s1, selecting corresponding test plants according to test requirements of tomato planting managers, and carrying out unique identification on each sample plant; S2, constructing a patrol map by utilizing a robot operating system ROS and a simultaneous positioning and map construction SLAM algorithm, wherein the patrol robot is used for patrol according to a given track and accurately identifying a sample plant through identification information and parking; S3, expanding the coverage range of a field of view by using two depth cameras through a lifting rotating bracket mounted on a greenhouse inspection robot, and synchronously collecting key phenotype information such as plant height, stem thickness, fruit cluster number, flowering number and the like of a target plant; And S4, integrating the periodically acquired tomato three-dimensional point cloud model and plant height, stem thickness, fruit cluster number, flowering and other phenotype data into a digital twin platform to construct an evolvable dynamic growth model, wherein the model synchronously combines real-time environment variables and water and fertilizer records, and supports growth trend prediction and management strategy optimization. Preferably, the step S1 specifically includes the following steps: S11, selecting tomato plants meeting test conditions from a greenhouse as samples; S12, sticking a label 1 with a two-dimensional code on the top of each plant sample, and respectively sticking a label 2, a label 3 and a label 4 on the middle part, the middle lower part and the bottom of the plant in the height direction, wherein