CN-122020506-A - Information pollution evaluation method and system in large model

CN122020506ACN 122020506 ACN122020506 ACN 122020506ACN-122020506-A

Abstract

The application provides an information pollution evaluation method and system in a large model, wherein the method comprises the steps of obtaining a pollution corpus used for large model training, and constructing an input pollution vector for each corpus in the pollution corpus; the method comprises the steps of generating an output set by utilizing a large model, constructing an output pollution vector for each piece of output content in the output set, matching corpus and output content of which the actual semantics meet similarity requirements according to the actual semantics of a plurality of corpus and a plurality of output content, calculating inherited pollution quantity between each group of output content and corpus, calculating pollution influence coefficients according to the output pollution vector and inherited pollution quantity, constructing nodes in an application system of the large model, recording node risk values of the output content, and calculating global pollution risk indexes according to the input pollution vector, the output pollution vector and the node risk values. The problem that negative results or wrong results are generated after the large model is influenced by polluted content is solved.

Inventors

SHI YIJIE
ZHAO LUJIN
QIN SUJUAN

Assignees

北京邮电大学

Dates

Publication Date: 20260512
Application Date: 20251219

Claims (10)

1. A method for information pollution assessment in a large model, the method comprising: Acquiring a pollution corpus used for the large model training, and constructing an input pollution vector for each corpus in the pollution corpus; inputting the pollution corpus into the large model, generating an output set by using the large model, and constructing an output pollution vector for each piece of output content in the output set; according to the actual semantics of the corpus and the output contents, matching the corpus and the output contents, wherein the actual semantics meet the similarity requirement, and calculating the inherited pollution amount between each group of matched output contents and the corpus; calculating pollution influence coefficients of the large model according to the output pollution vector and the inheritance pollution quantity; Constructing nodes in the application system of the large model, and recording node risk values of the output content in the application system by using the nodes; And calculating a global pollution risk index according to the input pollution vectors, the output pollution vectors and the node risk values, and representing the information pollution degree of the large model by the pollution influence coefficient and the global pollution risk index.
2. The information pollution evaluation method in a large model according to claim 1, wherein said matching said corpus and said output content, in which said actual semantics meet a similarity requirement, based on actual semantics of a plurality of said corpora and a plurality of said output contents, comprises: mapping a plurality of corpora and a plurality of output contents into the same semantic space by using the same semantic code; Calculating the similarity between a plurality of corpus and a plurality of output contents; And matching the corpus with the highest similarity with each piece of output content.
3. The method for evaluating information pollution in a large model according to claim 2, wherein said calculating an inherited pollution amount between each set of the matched output contents and the corpus comprises: calculating the weight of the corpus matched with each output content according to the following formula: ; Wherein alpha j,i is the weight of the corpus i matched with the output content j, lambda represents the concentration degree of weight distribution, h i is the corpus after semantic coding, For the output content after semantic encoding, d i ∈N（y j ) is the corpus meeting the similarity requirement; Calculating the inherited pollution amount according to the following formula: ; Wherein, the For the said amount of contamination to be inherited, -Providing said input pollution vector; Calculating pollution influence coefficients of the large model according to the output pollution vector and the inheritance pollution amount, wherein the pollution influence coefficients comprise: The pollution influence coefficient is calculated according to the following formula: ; Wherein A j is the pollution influence coefficient, Epsilon is a constant that prevents the denominator from being zero for the output pollution vector.
4. The information pollution evaluation method in a large model according to claim 1, wherein constructing a node in an application system of the large model, recording a node risk value of the output content in the application system using the node, comprises: Recording dynamic behavior information of the output content in the system by using the node, wherein the dynamic behavior information comprises first writing time, latest access time and calling times; Calculating a node risk value according to the following formula: ; ; In particular, the method comprises the steps of, For node risk values in the dimension being calculated, Is a node, m is a node ordinal number, For the time of the last access to be described, For the first write time, k is the dimension, As a pollution vector in the dimension, For the node's survival period, For the number of calls.
5. The method of information pollution assessment in a large model according to claim 1, wherein said calculating a global pollution risk index from a plurality of said input pollution vectors, a plurality of said output pollution vectors, and a plurality of said node risk values comprises: Calculating input pollution mean values of a plurality of input pollution vectors of a plurality of corpus, calculating output pollution mean values of a plurality of output pollution vectors of a plurality of output contents, and calculating system node mean values of a plurality of node risk values of the plurality of output contents; The overall risk index is calculated according to the following formula: ; wherein R k is the integrated risk index, For the input pollution mean value, For the output pollution mean value, Beta 1 、β 2 、β 3 is a scene coefficient, and k is a dimension; calculating the global pollution risk index according to the following formula: ; Wherein R global is the global pollution risk index, n is the total dimension number, and w k is the weight of dimension k.
6. The method for estimating information pollution in a large model according to claim 1, wherein said constructing an input pollution vector for each corpus in the pollution corpus comprises: Constructing the input pollution vectors of multiple dimensions for each corpus in the pollution corpus; Said constructing an output pollution vector for each piece of output content in said output set, comprising: the output pollution vector of multiple dimensions is constructed for each piece of output content of the output set.
7. The method of information pollution assessment in a large model of claim 6, wherein the input pollution vector of multiple dimensions and the output pollution vector of multiple dimensions each include a factual distortion risk dimension, a logical structure risk dimension, a semantic quality risk dimension, and a mood bias wind dimension; Calculating the input pollution vector or the output pollution vector in the de facto distortion risk dimension according to the following formula: ; Wherein, the For the input pollution vector or the output pollution vector of the fact-distortion risk dimension, d i is the corpus, Is a triplet set of corpus, h, r and t are respectively a head entity, a relation and a tail entity in the triplet, A triplet consistency score; Splitting the corpus into sentence sequences, and calculating the input pollution vector or the output pollution vector in the logical structure risk dimension according to the following formula: ; ; Wherein, the For the input pollution vector or the output pollution vector of the logical result risk dimension, For the sequence of sentences to be described, The contradictory probabilities of two adjacent sentences; Calculating the input pollution vector or the output pollution vector in the semantic quality risk dimension according to the following formula: ; Wherein, the For the input pollution vector or the output pollution vector of the semantic quality risk dimension, q syn is a syntactic structure score of the corpus, q coref is a reference consistency score of the corpus; calculating the input pollution vector or the output pollution vector in the emotion-biased wind dimension according to the following formula: ; Wherein, the For the input pollution vector or the output pollution vector of the emotion bias wind dimension, e (d i ) is emotion intensity, and b (d i ) is bias expression intensity.
8. The method of information pollution assessment in a large model according to claim 1, characterized by, after characterizing the information pollution level of the large model by the pollution influence coefficient and the global pollution risk index, comprising: setting a pollution risk index threshold, and intercepting the output content when the global pollution risk index is larger than the pollution risk index threshold; analyzing the intercepted knowledge reference link of the output content, inquiring the input content corresponding to the output content, and correcting the input content.
9. The method of information pollution assessment in a large model according to claim 1, characterized by, after characterizing the information pollution level of the large model by the pollution influence coefficient and the global pollution risk index, comprising: And correcting reasoning logic of the large model in response to the numerical value of the pollution influence coefficient representing that the large model has amplifying behavior on the pollution content of the corpus.
10. An information pollution assessment system in a large model, the system comprising: The input risk module is used for acquiring a pollution corpus used for the large model training and constructing an input pollution vector for each corpus in the pollution corpus; the output risk module is used for inputting the pollution corpus into the large model, generating an output set by utilizing the large model, and constructing an output pollution vector for each piece of output content in the output set; The first matching module is used for matching the corpus and the output content, the actual semantics of which meet the similarity requirement, according to the actual semantics of the corpus and the output content, and calculating the inherited pollution amount between the output content and the corpus after each group of matching; The first calculation module is used for calculating the pollution influence coefficient of the large model according to the output pollution vector and the inheritance pollution quantity; A system risk module, which is used for constructing nodes in the application system of the large model and recording node risk values of the output content in the application system by using the nodes; And the evaluation module is used for calculating a global pollution risk index according to the plurality of input pollution vectors, the plurality of output pollution vectors and the plurality of node risk values, and representing the information pollution degree of the large model by the pollution influence coefficient and the global pollution risk index.

Description

Information pollution evaluation method and system in large model Technical Field The application relates to the technical field of computers, in particular to an information pollution evaluation method and system in a large model. Background In recent years, with large-scale application of large models in scenes such as searching, question answering, content generation, code assistance, decision support and the like, the degree of dependence of models on training data and running environments continues to deepen. Large models typically rely on the automated learning of language laws, knowledge structures and reasoning patterns from large-scale internet corpora, the authenticity, logic and security of their generated content being largely dependent on the quality and distribution structure of the training corpora. However, open internet data sources are complex, unverified false reports, exaggerated narratives, emotional text, outdated information, letterpress expression, copyrighted content, potentially illicit data, etc. are mixed in the corpus. Once the low-quality or biased corpus is absorbed by the model, a bias representation or error association relation is formed in a parameter space, and further the low-quality or biased corpus is displayed in various forms such as fact errors, logic confusion, semantic blurring, emotion misguidance, value bias and the like in the reasoning and generation stage, so that the reliability and compliance of model output are weakened. On the other hand, the output content of the large language model is not a 'disposable consumer product', but is multiplexed and amplified for many times in an actual platform, and can be recycled and spread in a closed loop link of 'training corpus → model generation → system storage and indexing → retraining/re-exposure', so that pollution is accumulated in the internal circulation of the system and is difficult to eliminate. Therefore, there is an urgent need for a method of assessing pollution risk in large models that reduces the negative impact of pollution information on the inference logic and the generated content of large models. Disclosure of Invention Therefore, the application aims to provide an information pollution evaluation method and system in a large model, which solve the problem that a negative result or an erroneous result is generated after the large model is influenced by polluted content. In order to achieve one of the above disclosed objects, the present application provides an information pollution evaluation method in a large model, the method comprising: Acquiring a pollution corpus used for the large model training, and constructing an input pollution vector for each corpus in the pollution corpus; inputting the pollution corpus into the large model, generating an output set by using the large model, and constructing an output pollution vector for each piece of output content in the output set; according to the actual semantics of the corpus and the output contents, matching the corpus and the output contents, wherein the actual semantics meet the similarity requirement, and calculating the inherited pollution amount between each group of matched output contents and the corpus; calculating pollution influence coefficients of the large model according to the output pollution vector and the inheritance pollution quantity; Constructing nodes in the application system of the large model, and recording node risk values of the output content in the application system by using the nodes; And calculating a global pollution risk index according to the input pollution vectors, the output pollution vectors and the node risk values, and representing the information pollution degree of the large model by the pollution influence coefficient and the global pollution risk index. As a further improvement of an embodiment of the present application, the matching the corpus and the output content, in which the actual semantics meet the similarity requirement, according to the actual semantics of a plurality of the corpora and a plurality of the output content includes: mapping a plurality of corpora and a plurality of output contents into the same semantic space by using the same semantic code; Calculating the similarity between a plurality of corpus and a plurality of output contents; And matching the corpus with the highest similarity with each piece of output content. As a further improvement of an embodiment of the present application, the calculating the inherited pollution amount between the output content and the corpus after each set of matching includes: calculating the weight of the corpus matched with each output content according to the following formula: ; Wherein alpha j,i is the weight of the corpus i matched with the output content j, lambda represents the concentration degree of weight distribution, h i is the corpus after semantic coding, For the output content after semantic encoding, d i∈N（yj) is the corpus m