CN-121683727-B - Data labeling method and system based on prompt word driving

CN121683727BCN 121683727 BCN121683727 BCN 121683727BCN-121683727-B

Abstract

The application belongs to the technical field of data processing, and provides a data labeling method and a system based on prompt word driving, wherein a joint guide vector is generated by fusing field characteristics of labeling tasks and operation behavior vectors of labeling personnel, so that labeling errors caused by guide deviation are greatly reduced, and labeling efficiency and primary labeling quality are improved; the method comprises the steps of analyzing an initial labeling result, identifying the field distribution difference of a labeling defect type and an associated prompting word, providing clear targeting for the follow-up optimization of the knowledge graph parameters, constructing a reward function by taking the labeling defect type and the field distribution difference of the prompting word as a state space and combining labeling accuracy and field adaptability, continuously outputting high-quality prompting words by strengthening learning iteration to correct parameters of the prompting word knowledge graph, and after the prompting word knowledge graph iteration is stable, updating fusion coefficients of a joint guiding vector based on quality evaluation data feedback to ensure the collaborative adaptation of the joint guiding vector and the optimized knowledge graph.

Inventors

WU ANQIN
XU XIAOJUN
LIANG MENGYIN
XUE MINGDE

Assignees

杭州索益网络科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260210

Claims (9)

1. The data labeling method based on the prompt word driving is characterized by comprising the following steps of: Receiving a labeling task request, analyzing the labeling task to extract the domain characteristics of the labeling task, extracting the operation behavior vectors in the operation behaviors of labeling personnel, and fusing the domain characteristics and the operation behavior vectors to generate a joint guide vector; retrieving the prompt words from the prompt word knowledge graph based on the combined guide vector, processing the data to be marked and the prompt words to output an initial marking result and a confidence level, and analyzing the initial marking result to identify the difference between the marking defect type and the field distribution of the associated prompt words; the identifying the field distribution difference between the labeling defect type and the associated prompt word comprises the following steps: Screening the divergence parts with the confidence coefficient smaller than the confidence threshold based on the confidence coefficient of the initial annotation result and the divergence parts of the two types of annotation results, and classifying the divergence parts according to divergence sources to identify annotation defect types; binding the marking defect type with the field prompt word and the operation prompt word for generating the initial marking result, and counting the frequency of triggering the corresponding marking defect type by the prompt words with different proportions of fusion coefficients; Calculating defect induction rates of the same prompting word under different subdivision dimensions based on the frequency according to the subdivision dimensions of the field to which the current labeling task belongs, and counting the distribution duty ratio of labeling defect types among different subdivision dimensions to obtain field distribution differences of the associated prompting words corresponding to the labeling defect types; screening an initial labeling result with the confidence coefficient smaller than a confidence threshold value, determining a reward function based on labeling accuracy and field adaptability, taking the field distribution difference of a labeling defect type and an associated prompting word as a state space, and correcting parameters of a prompting word knowledge graph through reinforcement learning iteration; When the continuous iteration tends to be stable, the iteration optimization is terminated, a labeling data set is output, and the fusion coefficient of the joint guide vector is fed back and updated based on the quality evaluation data of the iteration process.
2. The method for labeling data based on prompt word driving as recited in claim 1, wherein the extracting the domain features of the labeling task comprises: Receiving a labeling task request, and executing basic semantics, task targets and hierarchical semantics splitting of data constraint on text description of the labeling task request to obtain initial characteristics; The method comprises the steps of calling historical field features of a historical labeling task, matching the distance between an initial feature and the historical field features through cosine similarity, and screening the similar historical labeling task with feature similarity not smaller than a similarity threshold; Identifying cross-domain overlapping features in the initial features, weighting and calculating the labeling accuracy and feature similarity of the similar historical labeling tasks to obtain weights, and reserving the high-weight cross-domain overlapping features as core features; distributing basic weights for the core features, dynamically adjusting the basic weights based on the contribution degree of the core features in the similar historical labeling tasks to the labeling accuracy, and forming the field features of the labeling tasks after weighting and fusion; The method comprises the steps of marking the initial characteristics of a current marking task, wherein the cross-domain overlapping characteristics represent overlapping parts of the characteristics of the history field of the similar marking task with the history in the initial characteristics of the current marking task, distributing corresponding weight duty ratio for marking accuracy and characteristic similarity, and obtaining the weight of the cross-domain overlapping characteristics after weighting calculation of the marking accuracy and the characteristic similarity by the corresponding weight duty ratio so as to judge the importance degree of the cross-domain overlapping characteristics.
3. The method for labeling data based on a hint word driver of claim 2, wherein extracting the operational behavior vector comprises: Acquiring operation behaviors of marking personnel corresponding to basic operations, interactive operations and decision operations at different times in the similar historical marking tasks; based on the history labeling accuracy of similar history labeling tasks, dynamically distributing the behavior weights of different operation behaviors according to operation types; Constructing a behavior feature matrix based on the behavior weight and the operation behavior, clustering the operation modes through a density clustering algorithm, removing abnormal operation behaviors, and extracting behavior features of different operation modes; And adjusting the coding dimension weight by combining the field features of the labeling task, performing mixed coding and normalization processing on the behavior features to generate an operation behavior vector consistent with the field feature dimension, wherein the coding dimension weight is adjusted according to the association relationship between the field features of the labeling task and the behavior features.
4. The method for generating a joint guidance vector according to claim 3, wherein the generating the joint guidance vector comprises: A preset domain ontology knowledge base is called, quantized values of feature importance of the domain to which the current labeling task belongs are extracted, and domain adaptation weights are distributed for domain features according to the duty ratio of the quantized values; Selecting small sample to-be-marked data of a current marking task, finishing small sample marking based on an operation behavior vector, and determining the behavior effective weight of the operation behavior vector based on marking conformity of marking results and standards; And taking the domain adaptation weight and the behavior effective weight as fusion coefficients, and carrying out weighted fusion on the domain characteristics with consistent dimensions and the operation behavior vector to generate a joint guide vector.
5. The method for labeling data based on prompt word driving as recited in claim 4, wherein the search prompt word comprises: Preprocessing the prompting word knowledge graph, classifying and quantifying prompting words of the prompting word knowledge graph into domain class vectors according to the classes of the domain, and associating operation behavior labels of labeling personnel in the corresponding domain; Extracting corresponding field components in the combined guide vector, performing cosine similarity rough matching with field category vectors of the prompt word knowledge graph, and screening candidate prompt words in the field to which the current labeling task belongs; Extracting corresponding operation components in the joint guide vector, calculating the matching degree of operation behavior labels and the operation components in candidate prompt words, and reserving the prompt words with the matching degree larger than a matching threshold value to obtain available candidate sets; and carrying out weighted descending sorting on the prompt words of the available candidate set based on the fusion coefficient of the combined guide vector, presetting the number of the prompt words based on the labeling complexity of the current labeling task, and outputting the prompt words according to the sorting result, wherein the fusion coefficient of the combined guide vector comprises the field adaptation weight and the behavior effective weight.
6. The method for labeling data based on a hint word driver of claim 5, wherein outputting the initial labeling result and the confidence level comprises: classifying the output prompting words according to the proportion of the fusion coefficients, and respectively setting prompting words with the proportion of the corresponding fusion coefficients being larger than the preset proportion as field prompting words and operation prompting words; Executing field semantic annotation on the data to be annotated based on the field prompt words and the field features, executing operation adaptation annotation based on the operation prompt words and the operation behavior vectors, and generating a field annotation result and an operation annotation result; calculating the degree of fit between the two types of labeling results, fusing the two types of labeling results into initial labeling results if the degree of fit is larger than a fit threshold, otherwise, calling a preset labeling standard fragment to correct the divergence part, and generating the initial labeling result; And generating result consistency based on the fit degree, the matching degree of the prompt word and the fit degree of the operation labeling result, and the confidence degree of the suitability of the prompt word and the operation association, and outputting the confidence degree of the initial labeling result after weighted summation.
7. The method for labeling data based on prompt word driving as recited in claim 6, wherein the iteratively correcting parameters of the knowledge graph of the prompt word comprises: screening initial labeling results with the confidence coefficient smaller than the confidence threshold, determining a reward function based on labeling accuracy and field adaptability, and dynamically adjusting parameters of the reward function according to the field distribution difference; Carrying out quantization characterization on the field distribution difference of the marked defect type and the associated prompt word to serve as a state space, and carrying out hierarchical correction on the parameters of the prompt word knowledge graph through reinforcement learning processing according to the frequency of the marked defect type corresponding to the prompt word and the different proportions of the fusion coefficient; after each round of correction, the prompt words are retrieved again and marked based on the corrected knowledge graph of the prompt words, the accumulated score of the reward function is calculated, and if the accumulated score does not reach the convergence threshold, the state space is updated based on the new domain distribution difference until iteration tends to be stable.
8. The method for labeling data based on hint word driving of claim 7, wherein the feedback updating the fusion coefficient of the joint guidance vector comprises: Extracting annotation accuracy and defect induction rate based on quality evaluation data of an iterative process, and respectively associating fusion coefficients of the combined guide vectors to form association relations; Determining an adjustment rule according to the association relation and the deviation direction of the quality evaluation data, updating a fusion coefficient based on the adjustment rule, regenerating a combined guide vector through the data to be marked of the small sample, and completing marking by using a search prompt word and monitoring marking effect; And comparing the labeling effect with a quality standard when iteration tends to be stable so as to judge whether the fusion coefficient of the combined guide vector after solidification and update is obtained.
9. A data labeling system based on prompt word driving, which is used for realizing the data labeling method based on the prompt word driving according to any one of claims 1-8, and is characterized by comprising a task analysis module, a data labeling module, an intelligent iteration module and a quality feedback module; The task analysis module is used for receiving the labeling task request, analyzing the labeling task to extract the domain characteristics of the labeling task, extracting the operation behavior vector in the operation behaviors of labeling personnel, and fusing the domain characteristics and the operation behavior vector to generate a joint guide vector; The data labeling module retrieves the prompt words from the prompt word knowledge graph based on the combined guide vector, processes the data to be labeled and the prompt words to output an initial labeling result and confidence coefficient, and analyzes the initial labeling result to identify the labeling defect type and the field distribution difference of the associated prompt words; The intelligent iteration module is used for screening initial labeling results with the confidence coefficient smaller than the confidence threshold value, determining a reward function based on labeling accuracy and field adaptability, taking the field distribution difference of the labeling defect type and the associated prompting word as a state space, and correcting parameters of the prompting word knowledge graph through reinforcement learning iteration; And the quality feedback module is used for terminating the iterative optimization and outputting a labeling data set when the continuous iteration tends to be stable, and feeding back and updating the fusion parameters of the joint guide vector based on the quality evaluation data of the iterative process.

Description

Data labeling method and system based on prompt word driving Technical Field The application relates to the technical field of data processing, in particular to a data labeling method and system based on prompt word driving. Background Along with the popularization of artificial intelligence and big data technology, data annotation is used as a basic link of model training and algorithm optimization, and a technical system of the data annotation is continuously evolved. Early data annotation depends on manual sentence-by-sentence and frame-by-frame annotation, the efficiency is low, the influence of subjective experience is large, and the consistency of annotation quality is difficult to guarantee. In order to solve the problem, the industry gradually develops semi-automatic methods such as rule driving labeling and templated labeling, manual intervention is reduced through preset field rules or labeling templates, and labeling efficiency is improved. In recent years, a labeling technology driven by a prompt word becomes a mainstream development direction, and the core logic of the labeling technology is to guide labeling personnel or automation tools to complete labeling tasks through the prompt word, so that a high-efficiency flow of prompt guidance and quick labeling is realized. The technology breaks through the dependence of the traditional rule driving method on a fixed scene, can cover more labeling requirements by expanding the prompting words, is widely applied to multi-type data labeling scenes such as texts, images and audios, and promotes the data labeling to develop towards the intelligent and flexible directions. Although the prompt word driving labeling technology has advanced to a certain extent, a plurality of limitations still exist in practical application, the labeling requirements of high precision and high adaptability are difficult to meet, the conventional prompt word driving method is mostly dependent on single-dimension guidance, the deep coordination of prompt profession and labeling operation behaviors is not realized, the searched prompt words deviate from the specification of labeling tasks or the labeling efficiency and flow normalization are reduced, defects generated in the labeling process are mostly counted integrally in the prior art, the precise association of defect types and the prompt words is not established, the guiding deviation of the prompt words cannot be positioned, and the follow-up optimization lacks definite targeting. Based on the defects in the prior art, the technical problem to be solved by the application is how to realize the accuracy, the efficiency and the quality stability of data annotation through the collaborative guidance of the prompt words and the operation. Disclosure of Invention The application aims to overcome the defects of the prior art and provides a data labeling method and system based on prompt word driving. In order to achieve the above purpose, the application adopts the following technical scheme: The first aspect provides a data labeling method based on prompt word driving, which comprises the steps of receiving a labeling task request, analyzing the labeling task to extract the domain characteristics of the labeling task, extracting the operation behavior vector in the operation behaviors of labeling personnel, and fusing the domain characteristics and the operation behavior vector to generate a joint guide vector; retrieving the prompt words from the prompt word knowledge graph based on the combined guide vector, processing the data to be marked and the prompt words to output an initial marking result and a confidence level, and analyzing the initial marking result to identify the difference between the marking defect type and the field distribution of the associated prompt words; screening an initial labeling result with the confidence coefficient smaller than a confidence threshold value, determining a reward function based on labeling accuracy and field adaptability, taking the field distribution difference of a labeling defect type and an associated prompting word as a state space, and correcting parameters of a prompting word knowledge graph through reinforcement learning iteration; When the continuous iteration tends to be stable, the iteration optimization is terminated, a labeling data set is output, and the fusion coefficient of the joint guide vector is fed back and updated based on the quality evaluation data of the iteration process. Optionally, the extracting the domain features of the labeling task includes: Receiving a labeling task request, and executing basic semantics, task targets and hierarchical semantics splitting of data constraint on text description of the labeling task request to obtain initial characteristics; The method comprises the steps of calling historical field features of a historical labeling task, matching the distance between an initial feature and the historical field features through cosi