CN-122021845-A - Multi-modal agricultural knowledge graph construction method based on large language model

CN122021845ACN 122021845 ACN122021845 ACN 122021845ACN-122021845-A

Abstract

The invention provides a multi-mode agricultural knowledge graph construction method based on a large language model, which comprises the steps of firstly carrying out standardized pretreatment on multi-mode data such as texts, images and sensors in the agricultural field, then adopting a mixed mode of combining top-down and bottom-up to construct an initial agricultural knowledge body, identifying new concepts based on the large language model to realize dynamic update of the body, then fusing text entities, image feature vectors and sensor time sequence features into multi-mode triples to be stored in a graph database, finally analyzing user query intention by using the large language model, and carrying out multi-hop path reasoning by combining a graph neural network to generate a structured result. The invention solves the problems of difficult multi-mode fusion, lag in knowledge updating and weak semantic reasoning capability in the prior art, improves the integrity, timeliness and intelligent decision support capability of the knowledge graph, can combine light deployment and federal learning mechanisms, adapts to an edge computing scene and meets the data privacy protection requirement.

Inventors

HU ZELIN
CHEN REN
DING XINRU
LIU MINJUE
ZHANG WENQIANG
YANG RUIYU
WANG WENFU

Assignees

赣南师范大学
江西鲜行者数智科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251203

Claims (7)

1. A multi-modal agricultural knowledge graph construction method based on a large language model is characterized by comprising the following steps: S1, acquiring text data, image data and sensor data in the agricultural field, and carrying out standardized preprocessing on the multi-mode data; s2, constructing an initial agricultural knowledge body by adopting a mixed mode combining top-down and bottom-up, and identifying a new concept based on a large language model to realize dynamic update of the body; S3, fusing the text entity, the image feature vector and the sensor time sequence feature into a multi-mode triplet, and storing the multi-mode triplet into a graph database; and S4, analyzing the user query intention by using the large language model, traversing the knowledge graph path by combining the graph neural network, and generating a structured reasoning result.
2. The multi-modal agricultural knowledge graph construction method based on the large language model is characterized in that in the step S1, text data comprise unstructured texts in agricultural journals, agricultural technical questioning and answering platforms and electronic commerce comments, image data comprise crop disease spot images and agricultural operation videos, target detection models are adopted to extract disease spot area features, image feature vectors are generated through cross-modal models, sensor data comprise time sequence data acquired by a weather station and a soil sensor, and the time sequence models are used to extract features and map the features to a knowledge graph relation chain.
3. The method for building the multi-modal agricultural knowledge graph based on the large language model according to claim 1, wherein in the step S2, the top-down mode is defined by domain experts to form a core ontology structure, the bottom-up mode is used for analyzing agricultural texts through the large language model, sub concepts are automatically extracted and ontology levels are optimized, and the dynamic updating comprises the steps of periodically crawling agricultural documents, identifying new concepts, judging relations between the new concepts and existing entities through entity linking technology, and triggering ontology updating.
4. The method for constructing a multi-modal agricultural knowledge graph based on a large language model according to claim 1, wherein in the step S3, the multi-modal triples comprise entities, relations and corresponding multi-modal attributes, the entity relations are stored by using a graph database, and the sensor raw data are stored in a distributed file system and are associated with graph nodes through indexes.
5. The multi-modal agricultural knowledge graph construction method based on the large language model according to claim 1 is characterized in that in the step S4, the large language model is used for analyzing natural language query of a user and identifying key entities and query intentions; the output results are structured information containing variety name, resistance data, and planting advice.
6. The multi-mode agricultural knowledge graph construction method based on the large language model of claim 1 is characterized by further comprising a light deployment scheme, wherein the large language model is compressed by adopting knowledge distillation and quantization technology, the model volume and the computing resource requirement are reduced, and the method is suitable for agricultural Internet of things equipment and mobile terminal application scenes.
7. The multi-modal agricultural knowledge graph construction method based on the large language model of claim 1, further comprising a federal learning mechanism, wherein the multi-modal agricultural knowledge graph construction method based on the large language model is characterized by comprising the steps of jointly training models among a plurality of farms to construct regional knowledge graphs, realizing knowledge fusion on the premise of not sharing original data, and meeting the data privacy protection requirement.

Description

Multi-modal agricultural knowledge graph construction method based on large language model Technical Field The invention relates to the technical fields of agricultural information technology and artificial intelligence, in particular to a multi-modal agricultural knowledge graph construction method based on a large language model. Background With the rise of the agricultural informatization level, agricultural information systems have accumulated massive amounts of multimodal data, including text, images, sensor data, and the like. The knowledge graph is used as a structured semantic network knowledge base, and strong knowledge organization and semantic association capability is shown in the agricultural field. However, existing agricultural knowledge graph construction methods face the following technical bottlenecks: The traditional method mainly relies on single text data construction, and is difficult to effectively fuse multi-mode information such as images (such as fruit tree lesion images), sensor data (such as soil moisture content and temperature and humidity) and the like. For example, when dealing with "wheat stripe rust", existing methods merely extract symptoms by text description, and cannot correlate disease image features with environmental sensor data (e.g., high incidence at temperatures ∈20 ℃ and humidity > 85%), resulting in map semantic fragmentation and insufficient integrity. The dynamic knowledge updating mechanism is lack, the knowledge in the agricultural field has strong timeliness (such as new crop varieties, pest resistance variation, climate change influence and the like), but the existing static map updating is highly dependent on manual intervention, and cannot respond to data change in real time. For example, when new pesticides are marketed or pest resistance monitoring data are updated, it is difficult for the conventional method to update control schemes synchronously in time, resulting in a lag of map knowledge from actual production requirements. The cross-domain semantic association is weak, namely the agricultural industry chain covers multiple links such as planting, processing, storage, sales and the like, but the existing map is limited to a single sub-domain (such as crop planting or pest control), and the cross-domain semantic association capability (such as a chain relation of 'crop variety-growing environment-processing technology-market demand') is lacking. For example, agricultural product traceability maps do not integrate planting environment data and processing quality standards, and it is difficult to support full-link risk management and decision optimization from field to dining table. The complex semantic reasoning capability is limited, the traditional reasoning depends on a simple rule engine (such as 'temperature >30 ℃ and irrigation demand is increased'), but under a complex scene (such as multi-factor interaction influences crop yield and pest resistance evolution mechanism), the multi-hop reasoning and implicit relation mining capability is lacked. For example, it is difficult to realize linkage analysis of precise fertilization and disease prevention through a multi-layer semantic chain of "soil ph→suitable fertilizer type→crop root development→disease resistance". Disclosure of Invention The invention aims to overcome the defects of the prior art and provides a multi-modal agricultural knowledge graph construction method based on a large language model, which realizes effective fusion of multi-modal data, dynamic knowledge updating and intelligent semantic reasoning. In order to achieve the above purpose, the invention adopts the following technical scheme: A multi-modal agricultural knowledge graph construction method based on a large language model comprises the following steps: S1, acquiring text data, image data and sensor data in the agricultural field, and carrying out standardized preprocessing on the multi-mode data; s2, constructing an initial agricultural knowledge body by adopting a mixed mode combining top-down and bottom-up, and identifying a new concept based on a large language model to realize dynamic update of the body; S3, fusing the text entity, the image feature vector and the sensor time sequence feature into a multi-mode triplet, and storing the multi-mode triplet into a graph database; and S4, analyzing the user query intention by using the large language model, traversing the knowledge graph path by combining the graph neural network, and generating a structured reasoning result. Further, in the step S1, the text data comprises unstructured texts in agricultural journals, agricultural technical questioning and answering platforms and electronic commerce comments, the image data comprises crop disease spot images and agricultural machinery operation videos, a target detection model is adopted to extract disease spot area features, image feature vectors are generated through a cross-modal model, the sensor data comprises