CN-121982475-A - Plant disease AI intelligent agent diagnostic system

Abstract

The invention discloses a plant disease AI intelligent agent diagnosis system, which comprises data preprocessing and model fine tuning, collecting and maintaining large-scale plant disease picture label pairs and plant disease question-answer data, training and fine tuning a large language model by using the data, a decision making module, a diagnosis tool executing module, a result aggregation module and a decision fusion method, wherein the prompting engineering is used for guiding the fine-tuned large model to output a thinking process of the fine-tuned large model and embedding a specific label into the fine-tuned large model to call related operation tools, the diagnosis tool executing module can intelligently select and configure a special diagnosis tool to realize multidimensional analysis and cover disease identification, disease severity assessment prediction and accurate control suggestion, and the result aggregation module integrates diagnosis results of different sources, adopts a decision fusion method to improve diagnosis reliability and provides an interpretable diagnosis report. The invention realizes the high-efficiency and accurate plant disease diagnosis function by integrating the above parts.

Inventors

• WANG QI
• Qin Lufu
• WU XINGCAI
• WU XUE
• ZHANG JIALIN
• WANG YAZHOU
• XIAO YUANYUAN
• YU PEIJIA

Assignees

• 贵州大学

Dates

Publication Date

20260505

Application Date

20260113

Claims (8)

1. 1. The plant disease AI intelligent agent diagnosis system is characterized by comprising the following steps: Step 1, preprocessing plant disease images and multi-mode instruction data, constructing training data, and fine-tuning a large language model based on a multi-mode alignment architecture, wherein parameters of a visual encoder are frozen, and a fine-tuning language decoder and an image-text projection bridging layer are subjected to fine-tuning; Step 2, receiving plant disease images and queries input by a user, analyzing the intention of the user by utilizing a finely-tuned multi-mode large language model, judging whether an external tool needs to be called through thinking chain reasoning, and generating a tool calling decision comprising a tool name and parameters; Step 3, executing the tool calling decision, calling a diagnosis tool in an operation memory, and obtaining a structural diagnosis result, wherein the diagnosis result comprises a tool name identifier and diagnosis content; And 4, recombining the diagnosis result and the original input of the user into a context, inputting the trimmed multi-mode large language model for result aggregation, and generating a diagnosis report.
2. 2. The plant disease AI intelligent agent diagnostic system of claim 1, wherein step 1 comprises using the plant disease image dataset and the multimodal instruction following dataset to uniformly resize the image and normalize the image to construct instruction-following data comprising an image reference identifier.
3. 3. The plant disease AI intelligent agent diagnostic system of claim 1, wherein the mental chain reasoning in step 2 comprises observing visual features of an image, parsing text to identify a disease, obtaining a control recommendation or verifying a diagnosis, and forming therefrom a mission plan to identify a plant species, locate a disease patch, determine a disease type, evaluate a severity, or generate a control strategy.
4. 4. The plant disease AI intelligent agent diagnostic system of claim 1, wherein the tool call decision of step 2 comprises a tool application program interface name and related parameter information.
5. 5. The plant disease AI agent diagnostic system of claim 1, wherein the diagnostic tool of step 3 comprises at least one of a disease classification tool, a target detection tool, and a semantic segmentation tool.
6. 6. The plant disease AI intelligent agent diagnostic system of claim 5, wherein the semantic segmentation tool performs pixel level segmentation of lesions and leaves by extracting multi-scale features, calculates a proportion of a lesion area to a total area of leaves, and maps to a predetermined severity level.
7. 7. The plant disease AI agent diagnostic system of claim 6 wherein the severity level is divided into five levels of health, mildness, middleness, severity and extreme severity based on a lesion area ratio.
8. 8. The plant disease AI intelligent agent diagnostic system of claim 1, wherein step 4 directs the multi-modal large language model to generate a diagnostic report comprising diagnostic results and control recommendations via prompt engineering techniques.

Description

Plant disease AI intelligent agent diagnostic system Technical Field The invention belongs to the technical field of agricultural information, and particularly relates to a plant disease AI intelligent agent diagnosis system based on a multi-mode large language model. Background With global climate change, agricultural planting structure adjustment and evolution of pathogenic microorganisms, the types and the hazard degree of crop diseases are increasingly aggravated, and huge losses are brought to agricultural production. Meanwhile, the traditional disease diagnosis depends on the experience judgment and laboratory detection of agricultural specialists, and the method is long in time consumption and high in cost, and is difficult to meet the rapid diagnosis requirement of large-scale planting of modern agriculture. To address these challenges, artificial intelligence technology is increasingly being introduced into agricultural production, particularly in crop disease control, presenting great potential. The computer vision is applied to the field of plant disease identification, but the existing model generally can only execute single tasks such as disease classification, lesion area segmentation or disease detection, and comprehensive and accurate disease diagnosis is difficult to realize. In addition, the existing plant disease recognition system is easily affected by factors such as illumination change, leaf shielding and the like when facing a complex field environment, and the diagnosis accuracy is reduced if the system depends on a separate detection model. The multi-modal large language model has made great progress in the general field, but still faces challenges in handling multi-task operations, particularly in the field of plant disease diagnosis. Current large-scale plant disease models, while attempting to address this problem, are limited to a narrow set of tasks and do not provide expert responses to specific diagnostic needs. Disclosure of Invention The present invention aims to solve the above problems and provide a plant disease AI intelligent agent diagnostic system capable of diagnosing plant diseases efficiently and accurately. The invention relates to a plant disease AI intelligent agent diagnosis system, which comprises the following steps: step 1, data preprocessing and model fine tuning Step 1.1 data collection and processing: Data collection, namely using a multi-mode instruction following data set containing more than 40 ten thousand pictures related to agricultural pests and plant diseases and a plant disease image data set covering 26 types of plant disease leaves in total; the data preprocessing, namely uniformly adjusting the image size to be 336 x 336 pixels, and carrying out normalization processing by adopting a standard ImageNet normalization mode required by a CLIP visual encoder; The command data format is that for the plant disease image data set, additionally constructing and generating command-following data, wherein the data comprises a data item identifier, an associated plant disease image file name and a group of dialogue contents comprising user questions and system responses, and the question contents comprise reference identifiers for images; Step 1.2 Fine tuning of the model On the basis of a multi-mode alignment architecture, parameters of a visual encoder are frozen, a language decoder and an image-text projection bridging layer are finely adjusted, the learning rate is selected from a range of 1e-5 to 5e-4, preferably 2e-4, and an optimizer is AdamW, adopts cosine annealing and is matched with warming. The batch size and the training wheel number are automatically adjusted according to the video memory. Training on a server equipped with 8 NVIDIA L20 GPUs for about 20-30 hours can achieve convergence, or LoRA or QLoRA can be used to reduce the memory footprint. The above parameters are preferred embodiments and are not intended to be limiting. Step 2 decision making Module Using prompt engineering to design corresponding prompt words to guide the trimmed large model to output the thinking process of the large model, and embedding tags of an API format and a data return structure into the large model to call related operation tools; Step 2.1 intent analysis: After receiving plant disease images input by a user and corresponding queries, deeply analyzing the intention of the user by utilizing a multi-mode large language model, and adding a self-thinking mechanism at the same time, so that the model can perform multi-angle interpretation, check whether the prior knowledge is enough or whether a professional tool is needed in the answer process, and before generating responses, the model starts a thinking chain reasoning mechanism to gradually decompose the problems, namely firstly observing visual characteristics of the images, then analyzing text intention, judging whether the prior knowledge is sufficient or needs to call an external tool, and forming task