Search

CN-122021651-A - Semantic-driven distributed reasoning task arrangement method

CN122021651ACN 122021651 ACN122021651 ACN 122021651ACN-122021651-A

Abstract

The invention discloses a distributed reasoning task arrangement method based on semantic driving, which comprises the steps of constructing a domain ontology model and establishing semantic mapping, analyzing task demands, executing multi-level semantic fusion data source retrieval, generating an execution path based on multi-objective collaborative optimization, distributing tasks and adopting model differential updating to reduce transmission expenditure, and executing self-adaptive migration and network partition fault tolerance based on check points and a two-stage submission protocol. The invention solves the technical problems of state loss caused by heterogeneous data semantic gap, static scheduling and dynamic environment conflict and coarse-granularity fault tolerance, realizes high-efficiency distributed reasoning of data immobility and calculation flow, and is suitable for the fields of multi-source heterogeneous data such as ocean science, meteorological monitoring, industrial Internet of things and the like.

Inventors

  • ZHENG GUOZHOU
  • WANG WENYI
  • YI HAIBO

Assignees

  • 苏州深渊矩阵科技有限公司

Dates

Publication Date
20260512
Application Date
20260407

Claims (10)

  1. 1. A distributed reasoning task arrangement method based on semantic driving is characterized by comprising the following steps: Constructing a domain ontology model based on a knowledge structure of a target domain, and establishing a semantic mapping relation between semantic descriptions of heterogeneous data sources and the domain ontology model; analyzing an reasoning task request submitted by a user, extracting semantic features of task requirements, and generating a structured task requirement description; based on the semantic mapping relation and the structured task demand description, performing multi-level semantic fusion data source retrieval in the distributed nodes, and determining available data sources; generating a target task execution path based on a multi-target collaborative optimization strategy according to the real-time state information of the node where the available data source is located; Distributing the reasoning task to the node where the data is located according to the target task execution path to execute local calculation, and converging calculation results; and performing adaptive task migration and resource reallocation based on the dynamic monitoring information in the task execution process.
  2. 2. The method for orchestrating distributed inference tasks based on semantic driving according to claim 1, wherein the establishing semantic mapping relationships comprises: And establishing a mapping relation between the Schema description of the heterogeneous data source and the domain ontology model by calculating and aligning the semantic similarity based on the domain word vector and the structure.
  3. 3. The semantic-driven distributed inference task orchestration method according to claim 1, wherein the multi-level semantic-fused data source retrieval comprises: An accurate matching layer, which is used for searching a data source of which the semantic tag is completely consistent with the structural task demand description; the semantic generalization layer is used for searching data sources with upper and lower or synonymous relations with the demand description concept based on semantic similar relations; a space-time approximation layer, which is used for searching data sources adjacent to the demand description in space-time dimension based on space-time fuzzy reasoning; And determining available data sources according to priorities of the accurate matching layer over the semantic generalization layer and the semantic generalization layer over the space-time approximation layer, or fusing retrieval results of all layers according to preset weight proportions to determine the available data sources.
  4. 4. The semantic-driven distributed inference task orchestration method according to claim 1, wherein the multi-objective collaborative optimization strategy comprises: Generating a candidate task execution path set by taking transmission delay, resource consumption and task success rate as optimization targets; determining a non-dominant solution set of the candidate task execution path set by adopting Pareto front analysis; And scoring each path in the non-dominant solution set based on a weighted scoring function, and selecting the path with the highest score as a target task execution path.
  5. 5. A method of semantic-driven distributed inference task orchestration according to claim 1, wherein the weighted scoring function takes the form of a linear weighted combination: Score = w 1 ×BW_norm + w 2 ×MEM_norm + w 3 ×(1/QUEUE_norm) + w 4 ×SUCCESS_rate; Wherein BW_norm E [0,1] is normalized network bandwidth, MEM_norm E [0,1] is normalized Graphics Processor (GPU) video memory capacity, QUEUE_norm E (0, 1) is normalized QUEUE waiting time, SUCCESS_rate E [0,1] is historical task SUCCESS rate, w 1 、w 2 、w 3 、w 4 is corresponding weight coefficient and w 1 +w 2 +w 3 +w 4 =1.
  6. 6. The semantic-driven distributed inference task orchestration method according to claim 1, wherein the adaptive task migration and resource reassignment comprises: monitoring network state and node performance based on a dynamic adaptive threshold; when an exception is detected, the intermediate state of task execution is saved by a checkpointing mechanism, and the task migration and resource reassignment are performed using a Two-phase commit protocol (Two-Phase Commit Protocol, 2 PC).
  7. 7. The semantic-driven distributed inference task orchestration method according to claim 1, wherein the two-phase commit protocol comprises: the preparation stage comprises synchronizing the check point data to the target node and verifying consistency; completing traffic route switching and source node resource release; if the preparation stage detects that the target node does not respond or the network partition is abnormal, the method rolls back to the source node to continue execution, and marks the target node as an unavailable state.
  8. 8. The semantic-driven distributed inference task orchestration method according to claim 1, wherein distributing the inference tasks according to the target task execution path comprises: determining a model distribution sequence and a result transmission relation between nodes according to the target task execution path; the model differential updating mechanism is adopted to transmit the changed model parameters, and only the parameter increment relative to the reference model is transmitted; And executing encryption transmission on the converged calculation result.
  9. 9. The method of any one of claims 1-8, wherein the domain ontology model comprises: the core concept layer comprises an observation entity, an observation parameter, a space-time characteristic and a data quality attribute; a semantic relationship layer defining inheritance, association and constraint relationships among concepts; And the standard mapping layer establishes mapping with the domain standard vocabulary and supports standard term importing and domain expanding.
  10. 10. A computer device comprising a processor and a memory, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-9 when executing the computer program.

Description

Semantic-driven distributed reasoning task arrangement method Technical Field The invention relates to the technical field of distributed computing and artificial intelligence reasoning, in particular to a semantic-driven distributed reasoning task arrangement method oriented to a heterogeneous data environment, which can be applied to the fields of multi-source heterogeneous data such as ocean science, meteorological monitoring, geological exploration, industrial Internet of things and the like. Background With the development of the Internet of things, edge computing and scientific observation networks, mass heterogeneous data are stored in nodes in geographic distribution in a scattered manner, and how to realize efficient artificial intelligence reasoning analysis on the premise of not intensively migrating original data becomes a key technical challenge in the field of distributed computing. The prior art mainly has three types of technical defects, and deep technical conflicts exist among the defects: First, resource discovery fails due to semantic gaps. Traditional distributed scheduling systems (e.g., hadoop YARN, kubernetes) rely primarily on grammar-level resource descriptions (e.g., JSON/YAML configurations), lacking an understanding of the semantic content of the data. In particular, with respect to unstructured/semi-structured scientific data (such as NetCDF files of ocean observations, CSV logs of meteorological sites and JSON data streams of industrial sensors), the existing scheduler can only identify physical attributes such as file paths or storage capacity, and the like, and cannot understand reasoning task semantics including field concepts (offshore, surface and warm salt) such as "acquire offshore surface warm salt data". When faced with data sources that employ different field naming (e.g., SST/sea_surface_temp, TS/temp_saliency) but are semantically equivalent, existing systems fail to automatically establish associations, resulting in resource retrieval accuracy below 80%. And secondly, the conflict between the static scheduling strategy and the dynamic environment. Existing distributed AI inference frameworks (e.g., tensorFlow Federated, pyTorch Distributed) typically employ preset fixed scheduling strategies, with data location constraints severely disjointed from dynamic computational force matching. For example, the Kubernetes scheduler, while supporting "node affinity" to direct tasks to the node where the data resides, cannot be adaptively adjusted based on network state fluctuations (e.g., satellite link delay spikes) or node load changes (e.g., GPU memory tension). When a fixed path is executed, network congestion or resource contention will lead to an increase in the rate of task failure, lacking a mechanism to dynamically trade-off between "data nearby computation" and "resource sufficiency assurance". Third, the coarse-grained fault tolerance mechanism creates a state loss. In the existing task migration mechanism, a process level restart or full data feedback is adopted, when a task is inferred by deep learning through node migration, the task needs to be re-run from the beginning due to the loss of calculation states such as model parameters, optimizer states, intermediate feature graphs and the like, and continuity and data consistency of the inference task running for a long time cannot be guaranteed. The three defects are not isolated, but are interweaved, namely, the semantic understanding deficiency leads to failure in accurately positioning the optimal data source, the static scheduling cannot adapt to dynamic environment change, and coarse-granularity fault tolerance cannot guarantee long-time task continuity. The prior art fails to provide a systematic solution for deep fusion of semantic understanding-resource awareness-state retention. Therefore, a dynamic task scheduling method integrating semantic understanding and resource perception is needed to realize efficient distributed reasoning of data immobility and computational flow. Disclosure of Invention The invention aims to solve the technical problems of semantic understanding deficiency, scheduling policy rigidness, coarse granularity of fault-tolerant mechanism and the like in distributed reasoning task arrangement in a heterogeneous data environment, and provides a method for fusing domain semantic modeling, intelligent resource discovery, multi-objective path optimization and fine granularity state migration depth. In order to achieve the above purpose, the invention adopts the following technical scheme: A distributed reasoning task arrangement method based on semantic driving comprises the following steps: Constructing a domain ontology model based on a knowledge structure of a target domain, and establishing a semantic mapping relation between semantic descriptions of heterogeneous data sources and the domain ontology model; analyzing an reasoning task request submitted by a user, extracting semantic