CN-122019869-A - Large model cross-domain sequence recommendation method based on low-rank module weaving fusion

CN122019869ACN 122019869 ACN122019869 ACN 122019869ACN-122019869-A

Abstract

The invention relates to the technical field of large model article recommendation, and discloses a large model cross-domain sequence recommendation method based on low-rank module weaving fusion, which comprises the steps of merging each source domain data set with a target domain data set respectively to obtain a plurality of corresponding mixed training data sets; the method comprises the steps of respectively carrying out fine adjustment on a large model on a target domain data set and a mixed training data set to obtain parameters of a target domain expert module and a corresponding number of mixed expert modules, carrying out parameter fusion on the expert modules, loading the fused low-rank expert into the large model in an reasoning stage, and predicting the next possibly interacted object of the target domain user history interaction sequence. The invention can improve the migration capability of the large model to the source domain recommendation knowledge by means of the low-rank module and the model fusion method, and can improve the recommendation effect of the large model on the target domain in Shan Yuanyu, multi-source domain and cross-platform scenes, thereby bringing more accurate cross-domain sequence recommendation to users.

Inventors

HOU MIN
LIU XIN
WU LE
HE CHENYI
LIU HAO
LI CHENG
LI XIN
WEI SI

Assignees

合肥工业大学

Dates

Publication Date: 20260512
Application Date: 20260106

Claims (8)

1. A large model cross-domain sequence recommendation method based on low-rank module braiding fusion is characterized by comprising the following steps: Acquiring a target domain data set and a plurality of source domain data sets, respectively constructing corresponding text training data sets, merging the text training data sets of each source domain with the text training data sets of each source domain to form a mixed training data set corresponding to each source domain, wherein each interaction record in the target domain data set and the source domain data sets comprises a user, an article interacted by the user and an interaction time stamp; based on a target domain text training data set, performing fine adjustment on a target domain low-rank expert module loaded into a large model to obtain an optimized target domain low-rank expert module; Based on each mixed training data set, respectively carrying out fine adjustment on the mixed low-rank expert modules loaded into the large model to obtain optimized mixed low-rank expert modules corresponding to each mixed training data set; weighting and fusing the optimized target domain low-rank expert module and the optimal parameters of all the optimized mixed low-rank expert modules to obtain a fused low-rank expert module; And loading the fused low-rank expert module into the large model for carrying out reasoning and predicting on the sequence recommendation task of the target domain user.
2. The large model cross-domain sequence recommendation method based on low-rank module weave fusion according to claim 1, wherein the obtaining a target domain data set and a plurality of source domain data sets comprises: Acquiring a target domain dataset And an nth source domain dataset , , Is the total number of source domain datasets, wherein, Representation of Is the first of (2) The interaction record of the strip source domain, Represent the first A source domain user in a strip source domain interaction record, Represent the first Source domain items in the stripe source domain interaction record, Represent the first Source domain time stamps in the strip source domain interaction records; Represent the first A total number of source domain interaction records for the individual source domain datasets; representing the first of the target domain data sets The interaction record of the strip target domain, Represent the first Target domain users in the bar target domain interaction record, Represent the first Target domain items in the bar target domain interaction record, Representation of A target domain timestamp in the strip target domain interaction record; Representation of The total number of target domain interaction records in the database.
3. The large model cross-domain sequence recommendation method based on low rank module weave fusion according to claim 2, wherein the respectively constructing corresponding text training data sets, merging the text training data set of each source domain with the text training data set of the target domain to form a mixed training data set corresponding to each source domain, specifically comprises: for each source domain dataset , Will be The items interacted with the same user are arranged from small to large according to the time stamp, each user has a sequence of interacted items, the items are represented by the title character strings thereof, The set of interaction sequences for all users in (1) is named ; For a target domain dataset And then is processed and obtained The same processing procedure is adopted to obtain a set of target domain user interaction sequences ; For the following And Any interaction sequence in (a) Adopting a leave-one method to make the first in the interactive sequence Personal article Separation, used as test set label, the remainder Text instructions for building training sets by inputting large models Including natural language descriptions of recommended tasks and user history And a candidate set consisting of And 29 negative samples, tag Is the correct ordering of items, with true items Arranged at the forefront, can be respectively constructed in such a way Corresponding source field text training data set And Corresponding target field text training data set , wherein, Representation of Is the first of (2) The text data of the strip is displayed, Representation of Is the first of (2) The entry of pieces of text data into the text instructions of the large model, Representation of Is used for the identification of the tag of (c), Representation of The total number of text data in the database; Representation of Is the first of (2) The text data of the strip is displayed, Representation of Is the first of (2) A text instruction of the bar is given, Representation of Is used for the identification of the tag of (c), Representation of The total number of text data in (a); Training data set of text of each source domain Respectively with the text training data set of the target domain And merging to obtain the mixed training data set corresponding to each source domain.
4. The large model cross-domain sequence recommendation method based on low-rank module woven fusion according to claim 1, wherein the large model comprises an embedded layer and Layer transducer submodule definition Is the first Pre-trained weight matrix of layer transducer sub-modules, , Representation of Is used for the length of the steel wire, Representation of Is of the width of (a); Wherein the nth mixed training data set is corresponding to Hybrid low rank expert module of layers The two low rank matrices of a layer are denoted as And , , For mixing the total number of training data sets, let Target domain low rank expert module of layer The two low rank matrices of a layer are denoted as And ; Representing the rank of the low rank matrix.
5. The large-model cross-domain sequence recommendation method based on low-rank module braiding fusion according to claim 4, wherein the target domain low-rank expert module loaded into the large model is subjected to fine tuning based on a target domain text training data set, and the optimized target domain low-rank expert module is obtained, and specifically comprises the following steps: text instruction for text data of target field text training data set Inputting into a large model, and obtaining an initial target domain embedded representation after the embedded layer processing ; Will be Respectively inputting the L-layer transformation submodule and the L-layer target domain low-rank expert module to perform fusion processing to obtain a first layer Target domain embedded representation of a layer Further obtain the target domain embedded representation of the L layer : ; Represent the first Embedding a representation of a target domain of a layer; Fine tuning optimization target for constructing embedded layer, L layer transformation submodule and L layer target domain low-rank expert module Fine tuning the target domain low rank expert module of the L layer to obtain the optimal parameters of the fine-tuned target domain low rank expert module , wherein, And Representation of And The corresponding optimal low rank matrix: ; The parameters representing the large model are represented by, = Is a parameter of the target domain low rank expert module of the L layer, Is that Is used for the identification of the tag of (c), Representing input Predicted as Probability of (2); Representation of The total number of text data in the database.
6. The large-model cross-domain sequence recommendation method based on low-rank module weave fusion according to claim 4, wherein the method is characterized in that based on each mixed training data set, the mixed low-rank expert modules loaded into the large model are respectively subjected to fine tuning to obtain optimized mixed low-rank expert modules corresponding to each mixed training data set, and specifically comprises the following steps: for each mixed dataset Text instruction Inputting into a large model, and obtaining an initial target domain embedded representation after the embedded layer processing Will (i) be Respectively inputting the L-layer transformation submodule and the L-layer target domain low-rank expert module to perform fusion processing to obtain a first layer Target domain embedded representation of a layer Further obtain the target domain embedded representation of the L layer : ; Represent the first Embedding a representation of a target domain of a layer; fine tuning optimization target for constructing embedded layer, L layer transformation submodule and L layer mixed low-rank expert module And fine tuning the mixed low-rank expert module of the L layers to obtain the optimal parameters of the fine-tuned mixed low-rank expert module , wherein, And Representation of And The corresponding optimal low rank matrix: ; The parameters representing the large model are represented by, = Is the first Parameters of the L layer of the hybrid low rank expert module, Is that Is used for the identification of the tag of (c), Representing input Predicted as Probability of (2); Representation of The total number of text data in (a); through the above operation, obtain Optimized parameters of optimized mixed low rank expert module corresponding to each mixed training data set , , wherein, And Represent the first The first mixed low rank expert module Two optimal low rank matrices for a layer.
7. The large model cross-domain sequence recommendation method based on low rank module braiding fusion according to claim 1, wherein the weighting fusion is performed on the optimized target domain low rank expert module and the optimal parameters of all the optimized mixed low rank expert modules to obtain a fused low rank expert module, and the method specifically comprises the following steps: Obtaining the first step based on the optimized optimal parameters of the target domain low-rank expert module and the optimal parameters of the mixed low-rank expert modules Two fused low rank matrices of a layer's fused low rank expert modules And Thereby obtaining Fusion parameters of layer fusion low rank expert module : ; ; When i=0, the number of the cells, Low rank expert module for optimized target domain Two optimal low-rank matrixes of the layer form a target domain low-rank expert module (LMM) Optimal parameters of the layer when 1 In the time-course of which the first and second contact surfaces, Expert module number of mixed low rank after optimization Two optimal low-rank matrixes of layers form a mixed low-rank expert module Optimal parameters of the layer; represents the fusion coefficient, satisfies 。
8. The large-model cross-domain sequence recommendation method based on low-rank module woven fusion according to claim 1, wherein the loading of the fused low-rank expert module into the large model is used for carrying out reasoning prediction on sequence recommendation tasks of target domain users, and specifically comprises the following steps: calculating parameters of large model loaded with fused low-rank expert module : ; As an original parameter of the large model, Fusion parameters for fusing the low-rank expert modules; And predicting the test text instruction in the test set by using the large model loaded with the fused low-rank expert module to obtain the object sequence which is most likely to be interacted by the user next.

Description

Large model cross-domain sequence recommendation method based on low-rank module weaving fusion Technical Field The invention relates to the technical field of large model article recommendation, in particular to a large model cross-domain sequence recommendation method based on low-rank module braiding fusion. Background The large model cross-domain sequence recommendation can understand multi-source domain user interaction logic by means of text instructions, breaks through dependence of traditional recommendation on cross-domain user/object overlapping, supplements object semantic information by utilizing general world knowledge of a large model, provides recommendation results with individuation and interpretability for users, supports efficient fine adjustment of parameters of the large model (such as LoRA), can adapt to multi-domain data on the premise of not updating full parameters, balances training cost and knowledge migration efficiency, and can quickly improve target domain recommendation performance through source domain knowledge in the face of sparse target domain data or new domain expansion scenes, and various advantages make the recommendation result an important direction for solving the problem of cross-domain recommendation pain points. In order for a large model to migrate multi-domain recommendation knowledge more efficiently, existing correlation methods can be generalized to one paradigm, the data merge paradigm. Such paradigms typically integrate multi-domain and multi-task recommendation data, build a unified instruction trim dataset, train a single large model to fit a full scene cross-domain recommendation task. The core of the paradigm is to design a universal instruction template to align multi-domain interaction data, so that a large model can master multi-domain recommendation knowledge through one-time training, and a universal cross-domain recommendation system with one model covering multiple scenes is developed. However, we consider that this paradigm has two limitations, namely inflexibility, a model retraining from scratch when one domain is added or subtracted according to scene needs, and data collision, forced multi-domain data co-training, which is prone to cross-domain preference collision and gradient collision in the training process, resulting in reduced recommended performance of the model on the target domain. The above limitations motivate us to find new paradigms for building better cross-domain sequence recommendation models. Fortunately, model fusion provides us with a viable solution. Model fusion is a distinct idea from the data merge paradigm in that by fusing parameters of multiple single-task models, a single model can perform multiple tasks better without retraining. This is a promising technique, which would naturally solve both of the above limitations if successfully applied in the field of cross-domain sequence recommendations. A naive model fusion paradigm aims at realizing knowledge migration by fusing model parameters independently trained in each domain, and mainly comprises the steps of independently training a large model or a large model low-rank module adapting to each domain, fusing parameters of a plurality of single-domain models by adopting simple strategies such as weight average, interpolation and the like or a front edge model fusion method, and avoiding high cost of total data retraining. Specifically, considering huge parameter quantity of a large model, the model is mainly trained by adopting a LoRA lightweight low-rank adapter for domain model, and a single-domain recommendation task can be adapted only by updating a small quantity of low-rank matrix parameters, and meanwhile, parameter fusion is used as a core means for realizing cross-domain knowledge migration. However, in a naive model fusion model, the source domain model is not adapted to the distribution of the target domain, and direct fusion is easy to degrade the model performance, so that effective source domain knowledge migration is difficult to realize under a sparse or cross-platform scene of the target domain data, and even the original recommended performance of the model in the target domain is damaged. Disclosure of Invention In order to overcome the limitation of a paradigm of the prior large-model cross-domain sequence recommendation, the invention provides a large-model cross-domain sequence recommendation method based on low-rank module weaving fusion, so that the large model can promote the understanding capability of cross-domain recommendation knowledge, and more accurate recommendation performance is brought to users in a target domain. In order to solve the technical problems, the invention adopts the following technical scheme: a large model cross-domain sequence recommendation method based on low-rank module braiding fusion comprises the following steps: Acquiring a target domain data set and a plurality of source domain data sets, respectively co