CN-121982530-A - Intelligent recognition system for leaf age of rice seedlings based on depth camera

CN121982530ACN 121982530 ACN121982530 ACN 121982530ACN-121982530-A

Abstract

The invention discloses a depth camera-based intelligent recognition system for leaf ages of rice seedlings, which belongs to the technical field of agricultural intelligent equipment and machine vision, and comprises a monitoring point gridding path planning module, a mechanical arm co-positioning data acquisition module, an RGB-D data preprocessing module, a dynamic foreground seedling accurate extraction module, a leaf age recognition confidence evaluation module, a leaf age data visual presentation module and an integrated control strategy feedback module; the RGB color features and the connected domain analysis are combined to optimize the mask, the single seedlings in the front row are accurately segmented, background interference is eliminated, the confidence coefficient is dynamically calibrated through the deep learning model and the image quality features, the confidence coefficient after calibration is generated by combining the texture definition and the edge continuity features, the confidence coefficient evaluation is directly related to the agronomic decision, the leaf age result is ensured to be generated only based on reliable data, the personnel review requirement is reduced, and the decision efficiency is improved.

Inventors

ZHANG BO
CUI HAOYU
LI WENXIAO
LIU YUFEI
NIU TAO
ZHUANG KEJIN

Assignees

黑龙江八一农垦大学

Dates

Publication Date: 20260505
Application Date: 20260123

Claims (10)

1. The intelligent recognition system for the leaf age of the rice seedlings based on the depth camera is characterized by comprising a monitoring point gridding path planning module, a mechanical arm co-positioning data acquisition module, an RGB-D data preprocessing module, a dynamic foreground seedling accurate extraction module, a leaf age recognition confidence evaluation module, a leaf age data visual presentation module and an integrated control strategy feedback module; The monitoring point gridding path planning module performs coordinate planning of the monitoring points and generation of a moving path through space gridding according to the size of the seedling raising greenhouse and the working range of the mechanical arm; The mechanical arm co-positioning data acquisition module performs fixed point movement and soft poking on seedling clusters by controlling the mechanical arm to move and the end effector according to the path coordinates, and simultaneously triggers the depth camera to acquire RGB-D images; The RGB-D data preprocessing module performs data preprocessing through registration, filtering and calibration according to the collected RGB-D data to obtain a depth map with aligned pixel levels and accurate distance information; The dynamic foreground seedling accurate extraction module is used for accurately extracting the foreground seedlings through dynamic distance layering and a binarization mask according to the processed depth map and is applied to RGB images to divide single plant targets; The leaf age identification confidence evaluation module is used for carrying out leaf age grade identification and confidence evaluation through a pre-trained deep learning model according to the extracted clean seedling image; the leaf age data visual presentation module stores leaf age data according to the identification result and the information of the monitoring points, performs statistics and graphical display according to the points through a database and visualization; and the integrated control strategy feedback module carries out collaborative command and strategy feedback instruction generation of each module through the system general control and scheduling logic according to the data analysis result and the confidence evaluation.
2. The intelligent recognition system of the leaf age of rice seedlings based on the depth camera is characterized in that the monitoring point gridding path planning module obtains the three-dimensional size of a seedling raising greenhouse, the boundary coordinates of a seedling bed, the origin of a basic coordinate system of a mechanical arm, the maximum working radius, each joint movement range and imaging constraint conditions of the depth camera, models the internal space of the greenhouse and the reachable range of the mechanical arm in a virtual three-dimensional coordinate system, determines an acquisition plane parallel to the plane of the seedling bed according to the optimal working distance of the camera right above the seedling bed, initially generates a series of candidate monitoring points in an equidistant grid lattice mode according to the repeated positioning precision of the mechanical arm and the visual field diameter of the camera on the acquisition plane, screens out candidate monitoring points which cannot reach the imaging constraint by the mechanical arm through inverse kinematics solving based on the constructed working space model to form a final three-dimensional coordinate set of the candidate monitoring points 。
3. The intelligent recognition system of the leaf age of the rice seedlings based on the depth camera according to claim 2, wherein the monitoring point gridding path planning module traverses all the monitoring points Triggering from the initial position of the mechanical arm, sequentially returning paths with shortest distance through all monitoring points, solving by path searching to generate an optimal monitoring point path sequence s, performing virtual simulation and collision detection on the generated path sequence, and finally determining the three-dimensional coordinates of all the monitoring points which are accessible to the mechanical arm 。
4. The intelligent recognition system for the leaf age of rice seedlings based on the depth camera is characterized in that the mechanical arm is cooperatively positioned with the data acquisition module, the mechanical arm sequentially and stably moves to the coordinate position of each monitoring point from an initial position according to a path sequence s, the relative position of an end effector and the depth camera is dynamically calibrated through a real-time pose feedback system, the end effector is controlled to execute a preset soft poking action at the position of each monitoring point, after the poking action is completed, the system enters a stable state, the depth camera is automatically triggered to synchronously acquire RGB images and depth images, the vibration amplitude and the image definition of the mechanical arm are monitored in real time in the acquisition process, if pose fluctuation or image blurring is detected, the system automatically pauses and finely adjusts the pose of the mechanical arm until a preset image quality threshold is met, and after poking and image acquisition of all monitoring points are sequentially completed, the system generates a standardized acquisition result comprising a time stamp, a monitoring point identifier and an RGB-D image data stream.
5. The intelligent recognition system of the rice seedling leaf age based on the depth camera is characterized in that the RGB-D data preprocessing module is used for respectively carrying out distortion correction on an original RGB image and a depth image based on internal reference calibration data of the depth camera, carrying out geometric correction on the image through distortion coefficients in camera calibration parameters, carrying out multi-scale noise suppression processing on the depth image through self-adaptive spatial filtering, carrying out distortion correction and scale normalization processing on the depth image based on camera calibration parameters and an ambient light compensation model, detecting the spatial consistency and the depth value stability of the RGB-D data in real time in the preprocessing process, and automatically triggering a resampling mechanism if registration deviation is found until output meets the pixel level alignment and distance precision standard preset by the system, so as to generate a depth image with accurate pixel level alignment and distance information.
6. The intelligent recognition system of the leaf age of the rice seedlings based on the depth camera, which is characterized in that the dynamic foreground seedling accurate extraction module acquires a depth map, analyzes the height distribution characteristics of a seedling canopy in the depth map, and adaptively calculates a dynamic distance threshold range, wherein the dynamic distance threshold range is realized as follows: , In the formula (i) the formula (ii), Representing a dynamic distance threshold value, Represents the average value of the height of the seedling canopy in the depth map, Represents the standard deviation of the height of the seedling canopy in the depth map, The normalized coefficient is represented by a value of the normalized coefficient, Representing density influence coefficient, controlling the influence degree of seedling density on threshold value, The seedling density of the current monitoring point is represented; according to the dynamically calculated distance threshold value, layering the depth map according to the depth value, classifying the pixel points into a foreground layer and a background layer, and generating a preliminary foreground binary mask, wherein the implementation is as follows: , In the formula (i) the formula (ii), The foreground binary mask is represented by a binary mask, The coordinates in the depth map are represented, Representing pixel coordinates.
7. The intelligent recognition system of the leaf age of the rice seedlings based on the depth camera, which is characterized in that the dynamic foreground seedling accurate extraction module performs morphological optimization on a binary mask generated by depth layering by combining color features of RGB images and connected domain analysis, performs pixel level fusion on the optimized binary mask and the RGB images, and extracts RGB pixel data of a corresponding foreground area, and is realized as follows: , In the formula (i) the formula (ii), Pixel values representing foreground regions in the RGB image, RGB color values of pixels representing coordinates in an original RGB image are screened out areas conforming to the physical characteristics of single seedlings based on the area, shape ratio, contour regularity and other characteristics of the connected areas.
8. The intelligent recognition system of the leaf age of the rice seedlings based on the depth camera, which is characterized in that the leaf age recognition confidence evaluation module inputs the preprocessed seedling images into a pre-trained deep convolutional neural network model, generates probability distribution vectors of leaf age grades through forward propagation of the deep convolutional neural network model, acquires a prediction probability value of each leaf age category, and is realized as follows: , In the formula (i) the formula (ii), Representing the predicted probability of the ith leaf-age category, Logits outputs representing the deep convolutional neural network model, The temperature coefficient is represented, and N represents the total number of leaf age categories; extracting the category with the highest probability value from the predicted probability as the leaf age grade of the current seedling, and recording the highest probability value as the basic confidence coefficient 。
9. The intelligent recognition system of the leaf age of the rice seedlings based on the depth camera, which is characterized in that the leaf age recognition confidence assessment module calculates the texture definition and the edge continuity of the rice seedling image based on the basic confidence, performs weighted fusion on the characteristic value and the basic confidence score, and generates the calibrated confidence, and is realized as follows: , In the formula (i) the formula (ii), Representing the confidence after calibration, T representing texture clarity, E representing edge continuity, Representing the calibration coefficient of the device, Representing a small constant.
10. The intelligent recognition system for the leaf age of the rice seedlings based on the depth camera, which is disclosed in claim 1, is characterized in that the visual presentation module of the leaf age data stores the leaf age data into a structured database according to time sequence and space coordinates, establishes a composite index of a time stamp and a monitoring point ID, automatically marks a low confidence result, and automatically generates the following dynamic chart according to the statistical result.

Description

Intelligent recognition system for leaf age of rice seedlings based on depth camera Technical Field The invention belongs to the technical field of agricultural intelligent equipment and machine vision, and particularly relates to an intelligent recognition system for leaf age of rice seedlings based on a depth camera. Background In the seedling raising stage, the leaf age is most determined by technicians entering a shed to pull out seedlings for visual inspection, so that the seedling belt is damaged, the labor intensity is high, and the subjectivity is high. In the close planting seedling raising tray, the seedlings are crossed front and back, the background is complex, the traditional identification method only based on RGB images can only use two-dimensional colors and textures, the front seedlings, rear plants and ground sundries are difficult to distinguish, misjudgment and missed judgment are easy to generate, and the identification precision and stability are insufficient. However, the existing intelligent recognition system for leaf ages of rice seedlings has certain defects that in a close planting seedling raising scene, the traditional system only depends on RGB images or fixed depth threshold values, the front seedlings and the rear seedlings cannot be distinguished, the recognition misjudgment rate is high, accurate single leaf age information cannot be obtained, a fixed distance threshold value is adopted to process a depth map, dynamic adjustment along with the change of the density of the seedlings is impossible, the high-density area is easy to miss detection, the low-density area is easy to misdetection, the diversity requirement of seedling raising environment cannot be met, double calibration of image quality and model confidence is not carried out on recognition results, the misjudgment rate is obviously increased under low-quality images, the agronomic decision basis is unreliable, the system is unidirectional data flow, an automatic feedback mechanism is not available, and the mechanical arm seedling pulling force, the acquisition path or the resampling strategy cannot be dynamically adjusted according to the recognition results. Disclosure of Invention The invention aims to provide an intelligent recognition system for the leaf age of rice seedlings based on a depth camera, so as to solve the problems in the background technology. In order to achieve the purpose, the intelligent rice seedling leaf age identification system based on the depth camera comprises a monitoring point gridding path planning module, a mechanical arm co-positioning data acquisition module, an RGB-D data preprocessing module, a dynamic foreground seedling accurate extraction module, a leaf age identification confidence assessment module, a leaf age data visual presentation module and an integrated control strategy feedback module; The monitoring point gridding path planning module performs coordinate planning of the monitoring points and generation of a moving path through space gridding according to the size of the seedling raising greenhouse and the working range of the mechanical arm; The mechanical arm co-positioning data acquisition module performs fixed point movement and soft poking on seedling clusters by controlling the mechanical arm to move and the end effector according to the path coordinates, and simultaneously triggers the depth camera to acquire RGB-D images; The RGB-D data preprocessing module performs data preprocessing through registration, filtering and calibration according to the collected RGB-D data to obtain a depth map with aligned pixel levels and accurate distance information; The dynamic foreground seedling accurate extraction module is used for accurately extracting the foreground seedlings through dynamic distance layering and a binarization mask according to the processed depth map and is applied to RGB images to divide single plant targets; The leaf age identification confidence evaluation module is used for carrying out leaf age grade identification and confidence evaluation through a pre-trained deep learning model according to the extracted clean seedling image; the leaf age data visual presentation module stores leaf age data according to the identification result and the information of the monitoring points, performs statistics and graphical display according to the points through a database and visualization; and the integrated control strategy feedback module carries out collaborative command and strategy feedback instruction generation of each module through the system general control and scheduling logic according to the data analysis result and the confidence evaluation. Preferably, the monitoring point gridding path planning module is in wireless connection with the mechanical arm co-location data acquisition module and the leaf age data visualization presentation module, the mechanical arm co-location data acquisition module is in wireless connection with the RGB-D