CN-121412453-B - Feature interaction method and device based on dynamic diagram constraint

CN121412453BCN 121412453 BCN121412453 BCN 121412453BCN-121412453-B

Abstract

The invention discloses a feature interaction method and device based on dynamic graph constraint, wherein an input feature field is mapped into a low-dimensional dense embedded vector to form an embedded matrix, similarity between any two fields is obtained through calculation to form a dense similarity matrix, K most relevant neighbor fields of each field are obtained through screening by a microtop-K algorithm and are given with weights to generate a field adjacent matrix, in a similar-transform attention module, the field adjacent matrix is used as an attention mask, each field is interacted with the K most relevant neighbor fields connected in the field adjacent matrix to obtain a graph constraint high-order interaction result, in a similar-FM module, a pair second-order feature cross result is obtained through calculation and is multiplied by the corresponding weights to obtain a graph constraint second-order interaction result, the two are fused, prediction processing is carried out, and prediction probability is output. The method and the device can solve the problems of more noise, low calculation efficiency, poor interpretation and the like in the prior art.

Inventors

RONG JIAN
BO LE
GE LONGLONG

Assignees

北京天源迪科网络科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251027

Claims (6)

1. The feature interaction method based on the dynamic diagram constraint is characterized by comprising the following steps of: The method comprises the steps of mapping input characteristic fields into low-dimensional dense embedded vectors, forming an embedded matrix by all the embedded vectors, wherein the input characteristic fields comprise user gender, age, click history, commodity class and price interval; The method comprises the steps of obtaining similarity between any two fields through calculation based on the embedded matrix, forming a dense similarity matrix based on the similarity, screening the dense similarity matrix through a micro Top-K algorithm to obtain K most relevant neighbor fields of each field, giving weight, and generating a field adjacent matrix; In a class-FM module, a paired second-order feature crossing result is obtained through calculation, and the paired second-order feature crossing result is multiplied by corresponding weights in the field adjacency matrix to obtain a graph constraint second-order interaction result; the graph constraint high-order interaction result and the graph constraint second-order interaction result are fused to obtain a fusion result, and the fusion result is input into a prediction network to be subjected to prediction processing, so that the prediction probability is output; In the similar transducer attention module, the field adjacency matrix is used as an attention mask, and the steps of interacting each field with K most relevant neighbor fields connected in the field adjacency matrix are as follows: based on the embedded matrix, respectively calculating to obtain a query matrix, a key matrix and a value matrix through a leachable projection matrix; Constructing a mask matrix based on the field adjacency matrix; Obtaining an attention score by calculation based on the query matrix, the key matrix, and the mask matrix; Obtaining an attention weight matrix by applying a softmax function to the attention score; based on the attention weight matrix and the value matrix, aggregation is carried out on neighbor fields, and a graph constraint high-order interaction result is obtained; In the similar FM module, a paired second-order characteristic crossing result is obtained through calculation, and the paired second-order characteristic crossing result is multiplied by the corresponding weight in the field adjacency matrix to obtain a graph constraint second-order interaction result, wherein the steps of obtaining the graph constraint second-order interaction result are as follows: Extracting a first embedded vector and a second embedded vector of a set characteristic field pair from the embedded matrix; Obtaining the original second-order cross characteristic of the set characteristic field pair by performing element-by-element product calculation on the first embedded vector and the second embedded vector; Acquiring the corresponding weight of the set characteristic field pair from the field adjacency matrix; multiplying the original second-order cross feature with the corresponding weight to obtain a graph constraint second-order cross feature; and accumulating the graph constraint second-order cross features of all field pairs to obtain the graph constraint second-order interaction result.
2. The feature interaction method based on dynamic diagram constraint of claim 1, wherein in the process of obtaining the similarity between any two fields through calculation based on the embedding matrix, a calculation formula of the similarity is: wherein S is a dense similarity matrix; 、 the method comprises the steps of obtaining a projection matrix which can be learned, d is the dimension of an embedded vector, and E is the embedded matrix.
3. The feature interaction method based on dynamic graph constraint of claim 2, wherein the dense similarity matrix is screened through a microtop-K algorithm to obtain K most relevant neighbor fields of each field and give weight, in the process of generating a field adjacency matrix, gumbel noise is generated through independent uniform random variables distributed with the same degree, elements of the dense similarity matrix and the corresponding Gumbel noise are calculated to obtain normalized scores, the normalized scores are processed through a softmax function to obtain soft weight distribution, indexes corresponding to a previous K large value in the soft weight distribution are screened to form a mask, and normalization processing is performed after element-by-element multiplication is performed on the soft weight distribution and the mask to generate the field adjacency matrix; The expression for generating the field adjacency matrix is: In the formula, Vector of field adjacency matrix ith row; An ith row vector which is soft weight distribution; The i-th row vector of the mask is the product of elements; To avoid a very small constant for zero division, F is the total number of feature fields.
4. A feature interaction device based on dynamic graph constraint, which adopts the feature interaction method based on dynamic graph constraint as claimed in any one of claims 1-3, and is characterized by comprising: The system comprises an embedding matrix acquisition unit, an embedding matrix generation unit, a processing unit and a processing unit, wherein the embedding matrix acquisition unit is used for mapping an input characteristic field into a low-dimensional dense embedding vector; The field adjacency matrix generation unit is used for obtaining the similarity between any two fields through calculation based on the embedded matrix, forming a dense similarity matrix based on the similarity, screening the dense similarity matrix through a microtop-K algorithm to obtain K most relevant neighbor fields of each field, giving weight, and generating a field adjacency matrix; The graph constraint high-order interaction and graph constraint second-order interaction unit is used for interacting each field with K most relevant neighbor fields connected in the field adjacency matrix by taking the field adjacency matrix as an attention mask in a similar transducer attention module to obtain a graph constraint high-order interaction result; The prediction probability acquisition unit is used for fusing the graph constraint high-order interaction result and the graph constraint second-order interaction result to obtain a fusion result; the graph constraint high-order interaction and graph constraint second-order interaction unit comprises: based on the embedded matrix, respectively calculating to obtain a query matrix, a key matrix and a value matrix through a leachable projection matrix; Constructing a mask matrix based on the field adjacency matrix; Obtaining an attention score by calculation based on the query matrix, the key matrix, and the mask matrix; Obtaining an attention weight matrix by applying a softmax function to the attention score; based on the attention weight matrix and the value matrix, aggregation is carried out on neighbor fields, and a graph constraint high-order interaction result is obtained; the graph constraint high-order interaction and graph constraint second-order interaction unit comprises: Extracting a first embedded vector and a second embedded vector of a set characteristic field pair from the embedded matrix; Obtaining the original second-order cross characteristic of the set characteristic field pair by performing element-by-element product calculation on the first embedded vector and the second embedded vector; Acquiring the corresponding weight of the set characteristic field pair from the field adjacency matrix; multiplying the original second-order cross feature with the corresponding weight to obtain a graph constraint second-order cross feature; and accumulating the graph constraint second-order cross features of all field pairs to obtain the graph constraint second-order interaction result.
5. The feature interaction device based on dynamic diagram constraint of claim 4, wherein in the field adjacency matrix generation unit, in a process of obtaining the similarity between any two fields through calculation based on the embedded matrix, a calculation formula of the similarity is as follows: wherein S is a dense similarity matrix; 、 the method comprises the steps of obtaining a projection matrix which can be learned, d is the dimension of an embedded vector, and E is the embedded matrix.
6. The feature interaction device based on dynamic graph constraint according to claim 5, wherein in the field adjacency matrix generation unit, the dense similarity matrix is screened through a microtop-K algorithm to obtain K most relevant neighbor fields of each field and give weight, in the field adjacency matrix generation unit, gummel noise is generated through independent uniform random variables distributed uniformly, elements of the dense similarity matrix and the gummel noise are calculated to obtain normalized scores, the normalized scores are processed through a softmax function to obtain soft weight distribution, indexes corresponding to a front K large value in the soft weight distribution are screened to form a mask, and normalization processing is performed after element-by-element multiplication of the soft weight distribution and the mask to generate the field adjacency matrix; The expression for generating the field adjacency matrix is: In the formula, Vector of field adjacency matrix ith row; An ith row vector which is soft weight distribution; The i-th row vector of the mask is the product of elements; To avoid a very small constant for zero division, F is the total number of feature fields.

Description

Feature interaction method and device based on dynamic diagram constraint Technical Field The invention relates to the technical field of recommendation systems and click rate prediction, in particular to a feature interaction method and device based on dynamic diagram constraint. Background In the field of recommendation systems and click rate (CTR) prediction, feature interaction is a core link for mining potential association of user interests and object attributes and improving model prediction accuracy, and the existing mainstream technology is widely applied to scenes such as electronic commerce recommendation, information flow distribution and the like. The method comprises the steps of enabling DeepFM to become one of basic models in industry through a parallel structure of FM explicit second-order interaction and DNN implicit high-order interaction, enabling xDeepFM to achieve explicit modeling of high-order interaction by means of a CIN module, enabling the method to be applied to scenes needing strong interpretation, enabling AutoInt to capture flexible high-order interaction modes by means of a self-attention mechanism and adapt to business scenes with medium field numbers, enabling part of methods to attempt to introduce graphs or fixed structure priors for optimizing interaction pertinence, and enabling the technologies to jointly form a main technical system of current feature interaction. However, the prior art still has significant drawbacks in practical applications. On the one hand, most methods lack a unified interaction range constraint mechanism, namely DeepFM carries out second-order interaction on all field pairs without differences, a large amount of noise is easy to introduce, autoInt 'attention spreading' is easy to occur in global attention calculation, weight is distributed on irrelevant fields, the complexity is as high as O (F2), the efficiency of field increasing is suddenly reduced, xDeepFM is relatively strong in interpretation, but when the number of orders and the number of fields are increased, the calculation and memory cost are exponentially increased, and the requirement of large-scale industrial deployment is difficult to meet. On the other hand, depending on a fixed prior or offline composition method, the field association relation cannot be dynamically adjusted along with data distribution, and the second-order interaction process and the high-order interaction process cannot be simultaneously constrained, so that the model has poor adaptability and difficult end-to-end optimization, and the interaction effectiveness, the calculation efficiency and the interpretability are difficult to balance. Therefore, it is needed to invent a feature interaction method based on dynamic diagram constraint, which solves the problems of more noise, low calculation efficiency, poor interpretation and the like in the prior art. Disclosure of Invention Therefore, the invention provides a feature interaction method and device based on dynamic diagram constraint, which solve the problems of more noise, low calculation efficiency, poor interpretation, high calculation and memory cost, difficulty in simultaneously considering second-order and high-order interaction optimization and the like caused by lack of uniform interaction range constraint in the prior art. In order to achieve the above purpose, the invention provides a feature interaction method based on dynamic diagram constraint, which comprises the following steps: mapping the input characteristic field into an embedded vector with low dimension density; The method comprises the steps of obtaining similarity between any two fields through calculation based on the embedded matrix, forming a dense similarity matrix based on the similarity, screening the dense similarity matrix through a micro Top-K algorithm to obtain K most relevant neighbor fields of each field, giving weight, and generating a field adjacent matrix; In a class-FM module, a paired second-order feature crossing result is obtained through calculation, and the paired second-order feature crossing result is multiplied by corresponding weights in the field adjacency matrix to obtain a graph constraint second-order interaction result; Fusing the graph constraint high-order interaction result with the graph constraint second-order interaction result to obtain a fusion result; and inputting the fusion result into a prediction network for prediction processing, and outputting a prediction probability. As a preferred scheme of the feature interaction method based on the constraint of the dynamic graph, in the process of obtaining the similarity between any two fields through calculation based on the embedded matrix, a calculation formula of the similarity is as follows: wherein S is a dense similarity matrix; 、 the method comprises the steps of obtaining a projection matrix which can be learned, d is the dimension of an embedded vector, and E is the embedded matrix. As a