CN-114090848-B - Data recommendation and classification method, feature fusion model and electronic equipment

CN114090848BCN 114090848 BCN114090848 BCN 114090848BCN-114090848-B

Abstract

The embodiment of the application provides a data recommendation and classification method, a feature fusion model and electronic equipment. The data recommendation method comprises the steps of determining target data, determining at least one associated data related to the target data in a plurality of data according to associated information among the data, fusing characteristic information of the target data and characteristic information of the at least one associated data to obtain fused characteristics of the target data, and recommending at least one recommended data for a user based on the fused characteristics of the target data. According to the technical scheme provided by the embodiment of the application, the characteristic information of the target data is fused with the characteristic information of at least one associated data to obtain the fusion characteristic of the target data, and then the data is recommended to the user based on the fusion characteristic fused with the characteristic information of other associated data, so that the diversity of recommendation is improved, and the fusion of posterior information is brought.

Inventors

Yang Susen
Lei Chenyi
WANG GUOXIN
TANG HAIHONG

Assignees

阿里巴巴（中国）有限公司
阿里巴巴（中国）有限公司

Dates

Publication Date: 20260421
Application Date: 20211025
Priority Date: 20211025

Claims (12)

1. A data recommendation method, comprising: Determining target data, wherein the target data comprises at least one of text data, image data, video data and audio data; determining at least one associated data related to the target data in a plurality of data according to the associated information between the data, wherein the associated information between the data comprises the same label and the same label number of the two data; configuring corresponding embedded features for the target data and the at least one associated data respectively; Determining characteristic information of the target data according to the multi-mode characteristics of the target data and the embedded characteristics corresponding to the target data; Determining feature information of the at least one associated data according to the multi-modal feature of the at least one associated data and the embedded feature corresponding to the at least one associated data, wherein the multi-modal feature of the associated data or the target data comprises at least one of text feature, audio feature and image feature; Inputting the characteristic information of the target data and the characteristic information of the at least one associated data into a characteristic fusion model, and executing the characteristic fusion model to obtain the fusion characteristic of the target data, wherein the characteristic fusion model is obtained through a pre-training process; And recommending at least one recommendation data for the user based on the fusion characteristics of the target data.
2. The data recommendation method according to claim 1, wherein determining at least one associated data related to the target data among a plurality of data based on the association information between the data, comprises: Constructing a relation graph according to the association information between the data, wherein the relation graph comprises a plurality of nodes and side information reflecting the relation between the nodes; and aiming at the target node, sampling the nodes in the relation graph to sample at least one associated node related to the target node, wherein the data corresponding to the associated node is associated data related to the target data.
3. The data recommendation method according to claim 2, wherein sampling nodes in the relationship graph for the target node to sample at least one associated node related to the target node comprises: acquiring target times and sampling quantity; In a primary sampling process, taking the target node as a sampling origin, and sampling at least one neighbor node adjacent to the sampling origin in the relation graph; Judging whether the sampling times reach the target times or not; In the next sampling iteration, taking any one neighbor node in the at least one neighbor node as a sampling origin, and sampling at least one neighbor node adjacent to the sampling origin in the relation graph; And when the sampling times are greater than the target times, determining the sampling number of neighbor nodes as associated nodes from the neighbor nodes sampled in the sampling iterations of the target times.
4. The data recommendation method according to claim 3, wherein when a plurality of associated data related to the target data is sampled, the method further comprises: determining importance of a plurality of associated data; according to the importance degree, configuring corresponding embedded features for the plurality of associated data respectively; configuring embedded features for the target data; acquiring the characteristics of at least one mode of the target data and the multi-mode characteristics of any one of the plurality of associated data; determining characteristic information of the target data according to the characteristics of at least one mode of the target data and the embedded characteristics corresponding to the target data; and determining characteristic information of the associated data according to the characteristic of at least one mode of any associated data in the plurality of associated data and the embedded characteristic corresponding to the associated data.
5. The data recommendation method of claim 4 wherein the embedded features comprise a location feature and a role feature, the target data has a multi-modal feature comprising features of multiple modalities, and Determining feature information of the target data according to the multi-modal feature of the target data and the embedded feature corresponding to the target data, including: determining a corresponding weight for each of the multi-modal features of the target data; Determining content characteristics of the target data according to the multi-modal characteristics of the target data and the weight of the characteristics of each modal in the multi-modal characteristics; And aggregating the content characteristics of the target data, the position characteristics corresponding to the target data and the role characteristics to obtain the characteristic information of the target data.
6. The data recommendation method according to any one of claims 1 to 5, wherein fusing the feature information of the target data and the feature information of the at least one associated data to obtain a fused feature of the target data, comprises: Determining feature similarity of feature information of the target data and feature information of the at least one associated data; Determining the attention weight corresponding to the at least one associated data based on the feature similarity; and carrying out fusion coding on the characteristic information of the target data and the characteristic information of the at least one associated data according to the attention weight corresponding to the at least one associated data to obtain the fusion characteristic of the target data.
7. The data recommendation method according to any one of claims 1 to 5, further comprising: Obtaining a sample graph, wherein the sample graph comprises sample nodes and side information reflecting the relation among the sample nodes; Sampling sample nodes in the sample graph for a first sample node in the sample graph to sample at least one second sample node related to the first sample node; Determining characteristic information of the first sample node and characteristic information of at least one second sample node according to the characteristic of at least one mode of the first sample node and the characteristic of at least one mode of at least one second sample node; Inputting the characteristic information of the first sample node and the characteristic information of the at least one second sample node into the characteristic fusion model to obtain fusion characteristics of the first sample node and fusion characteristics of the at least one second sample node; Based on the fusion characteristics of the first sample node and the fusion characteristics of the at least one second sample node, executing a graph reconstruction task and a mask sample node characteristic reconstruction task to obtain execution results corresponding to the tasks; And optimizing parameters in the feature fusion model according to the execution results corresponding to the tasks.
8. A data recommendation method, comprising: responding to the operation of a user on the interactive interface, and outputting first multimedia data; determining at least one second multimedia data related to the first multimedia data; configuring corresponding embedded features for the first multimedia data and the at least one second multimedia data respectively; Determining feature information of the first multimedia data according to the multi-mode features of the first multimedia data and the embedded features corresponding to the first multimedia data; Determining feature information of the at least one second multimedia data according to the multi-modal feature of the at least one second multimedia data and the embedded feature corresponding to the at least one second multimedia data; Inputting the characteristic information of the first multimedia data and the characteristic information of the at least one second multimedia data into a characteristic fusion model, and executing the characteristic fusion model to obtain the fusion characteristic of the first multimedia data, wherein the characteristic fusion model is obtained through a pre-training process; determining recommendation data based on the fusion characteristics of the first multimedia data; and outputting the recommended data for the user when the output condition is met.
9. The method of claim 8, wherein determining at least one second multimedia data associated with the first multimedia data comprises: Determining at least one second multimedia data based on historical data associated with the user; Acquiring the multi-modal characteristics of the first multimedia data, the multi-modal characteristics of the at least one second multimedia data and the behavior data of the user; determining association information between multimedia data according to the multi-modal characteristics of the first multimedia data, the multi-modal characteristics of the at least one second multimedia data and the behavior data of the user; the relation graph is constructed according to the association information among the multimedia data, wherein the relation graph comprises a plurality of nodes and side information reflecting the relation among the nodes, and the relation graph comprises a first node corresponding to the first multimedia data and at least one second node corresponding to the at least one second multimedia data; and aiming at the first node, sampling the nodes in the relation graph to sample at least one associated node related to the first node, wherein data corresponding to the associated node is second multimedia data related to the first multimedia data.
10. A method of classifying data, comprising: Determining at least one second multimedia data related to the first multimedia data; configuring corresponding embedded features for the first multimedia data and the at least one second multimedia data respectively; Determining feature information of the first multimedia data according to the multi-mode features of the first multimedia data and the embedded features corresponding to the first multimedia data; Determining feature information of the at least one second multimedia data according to the multi-modal feature of the at least one second multimedia data and the embedded feature corresponding to the at least one second multimedia data; Inputting the characteristic information of the first multimedia data and the characteristic information of the at least one second multimedia data into a characteristic fusion model, and executing the characteristic fusion model to obtain the fusion characteristic of the first multimedia data, wherein the characteristic fusion model is obtained through a pre-training process; and determining the category to which the first multimedia data belongs according to the fusion characteristics of the first multimedia data.
11. A graph-based feature fusion system, comprising: The node sampling layer is used for sampling the nodes in the relation graph aiming at a first node in the relation graph to obtain at least one second node related to the first node; The node characteristic information determining layer is used for respectively configuring corresponding embedded characteristics for the first node and the at least one second node according to the multi-mode information of the first node, the multi-mode information of the at least one second node and the side information between the first node and the at least one second node, determining the characteristic information of the first node according to the multi-mode information of the first node and the embedded characteristics corresponding to the first node, and determining the characteristic information of the at least one second node according to the multi-mode information of the at least one second node and the embedded characteristics corresponding to the at least one second node, wherein the multi-mode information of the first node or the second node comprises at least one of text characteristics, audio characteristics and image characteristics; the feature fusion layer is used for fusing the feature information of the first node and the feature information of the at least one second node to obtain the fusion feature of the first node; and the optimization layer is used for executing the graph reconstruction task and the masking node feature reconstruction task according to the fusion features of the first node to obtain execution results corresponding to the tasks, and optimizing parameters in the feature fusion model according to the execution results corresponding to the tasks.
12. An electronic device comprising a memory and a processor, wherein the memory is configured to store a program, wherein the processor is coupled to the memory and configured to execute the program stored in the memory to implement the steps of the data recommendation method according to any one of claims 1 to 7, or to implement the steps of the data recommendation method according to any one of claims 8 to 9, or to implement the steps of the data classification method according to claim 10.

Description

Data recommendation and classification method, feature fusion model and electronic equipment Technical Field The present application relates to the field of computer technologies, and in particular, to a data recommendation and classification method, a feature fusion model, and an electronic device. Background Data recommendations, such as text recommendations, video recommendations, music recommendations, etc., are now widely used. Currently, data recommendations are typically based on analyzing user preferences based on user history data and then recommending to the user based on the user preferences. Alternatively, text, video or music related to the currently browsed text, video or music being watched or music being played is recommended to the user according to the currently browsed text, video being watched or music being played or the like. In the prior art, the data recommendation scheme only recommends similar data to the user, and the recommendation diversity is insufficient. Disclosure of Invention The application provides a data recommendation and classification method, a feature fusion model and electronic equipment which solve the problems or at least partially solve the problems. In one embodiment of the application, a data recommendation method is provided. The method comprises the following steps: Determining target data; Determining at least one associated data related to the target data in a plurality of data according to the associated information among the data; Fusing the characteristic information of the target data and the characteristic information of the at least one associated data to obtain the fused characteristic of the target data; And recommending at least one recommendation data for the user based on the fusion characteristics of the target data. In another embodiment of the present application, a data recommendation method is also provided. The method comprises the following steps: responding to the operation of a user on the interactive interface, and outputting first multimedia data; determining at least one second multimedia data related to the first multimedia data; Fusing the characteristic information of the first multimedia data and the characteristic information of the at least one second multimedia data to obtain the fused characteristic of the first multimedia data; determining recommendation data based on the fusion characteristics of the first multimedia data; and outputting the recommended data for the user when the output condition is met. In yet another embodiment of the present application, a data classification method is also provided. The method comprises the following steps: Determining at least one second multimedia data related to the first multimedia data; Fusing the characteristic information of the first multimedia data and the characteristic information of the at least one second multimedia data to obtain the fused characteristic of the first multimedia data; and determining the category to which the first multimedia data belongs according to the fusion characteristics of the first multimedia data. In yet another embodiment of the present application, a feature fusion model based on a relationship graph is provided. The feature fusion model based on the relation graph comprises the following steps: The node sampling module is used for sampling the nodes in the relation graph aiming at the first node in the relation graph to obtain at least one second node related to the first node; The node characteristic information determining module is used for respectively configuring corresponding embedded characteristics for the first node and the at least one second node according to the multi-mode information of the first node, the multi-mode information of the at least one second node and the side information between the first node and the at least one second node; determining characteristic information of the first node according to the multi-mode information of the first node and the embedded characteristics corresponding to the first node; determining characteristic information of the at least one second node according to the multi-mode information of the at least one second node and the embedded characteristics corresponding to the at least one second node; the feature fusion module is used for inputting the feature information of the first node and the feature information of the at least one second node into a feature fusion model, and executing the feature fusion model to obtain fusion features of the first node; The optimization module is used for executing a graph reconstruction task and a masking node feature reconstruction task according to the fusion features of the first sample node to obtain execution results corresponding to the tasks, and optimizing parameters in the feature fusion model according to the execution results corresponding to the tasks. In yet another embodiment of the present application, an electronic device is also provided. T