CN-122019883-A - User interest quality evaluation method and device, storage medium and electronic equipment

CN122019883ACN 122019883 ACN122019883 ACN 122019883ACN-122019883-A

Abstract

The application relates to a method and a device for evaluating user interest quality, a storage medium and electronic equipment. The method comprises the steps of collecting historical behavior data of a user in a first time window, generating an old interest point sequence and a new interest point sequence according to the historical behavior data through an old interest model and a new interest model respectively, converting the old interest point sequence and the new interest point sequence into a first recommended content sequence and a second recommended content sequence for predicting interest preference of the user through a preset recall mechanism respectively, collecting real behavior data of the user in a second time window, quantifying the real behavior data according to watching time length to generate a reference content sequence reflecting the real interest preference of the user, and calculating normalized damage accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence according to the reference content sequence respectively and performing comparison analysis. The method solves the technical problems that the interest points are difficult to be directly related to the user behaviors and the evaluation cost is high.

Inventors

DU HONGGUANG
HE KAI
XU XIAOLE
ZHANG QUNFANG
ZHANG SHIJUN

Assignees

北京奇艺世纪科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260205

Claims (10)

1. A method for evaluating quality of interest of a user, comprising: Collecting historical behavior data of a user in a first time window, and generating an old interest point sequence and a new interest point sequence according to the historical behavior data through an old interest model and a new interest model respectively; Converting the old interest point sequence and the new interest point sequence into a first recommended content sequence and a second recommended content sequence for predicting the interest preference of the user through a preset recall mechanism; Acquiring real behavior data of the user in a second time window, quantifying the real behavior data according to the watching duration, and generating a reference content sequence reflecting the real interest preference of the user, wherein the second time window is positioned behind the first time window in time; And according to the reference content sequence, respectively calculating normalized damage accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence, and performing comparison analysis to determine the quality of the new interest model relative to the old interest model.
2. The method of claim 1, wherein generating an old point of interest sequence and a new point of interest sequence from the historical behavioral data by an old interest model and a new interest model, respectively, comprises: generating an old interest point set and a new interest point set according to the historical behavior data through the old interest model and the new interest model respectively; And respectively sorting the old interest point set and the new interest point set according to a preset interest point sorting rule, and respectively selecting a plurality of interest points with the same number from the old interest point set and the new interest point set as the old interest point sequence and the new interest point sequence.
3. The method of claim 1, wherein converting the old and new point of interest sequences into first and second recommended content sequences, respectively, that predict the user interest preferences by a preset recall mechanism comprises: for each interest point in the old interest point sequence and the new interest point sequence, invoking the preset recall mechanism to respectively acquire a first candidate content set and a second candidate content set associated with the interest point; And respectively sorting the first candidate content set and the second candidate content set according to a preset content sorting rule, and respectively selecting the first plurality of content with the same quantity from the first candidate content set and the second candidate content set as the first recommended content sequence and the second recommended content sequence.
4. The method of claim 1, wherein quantifying the real behavior data according to a viewing duration, generating a reference content sequence reflecting real interest preferences of the user, comprises: obtaining the corresponding watching time length of each content in the real behavior data; Based on the watching time length, ordering all contents in the real behavior data in a descending order to obtain a first content sequence; mapping the viewing time length of each content in the first content sequence into a correlation weight value to form a second content sequence with the correlation weight value; respectively selecting a plurality of first contents in each recall result from a plurality of recall results obtained by the preset recall mechanism triggered by all interest points based on the preset recall mechanism, and combining the first contents to form a third candidate content set; And performing intersection operation on the second content sequence and the third candidate content set, and performing reverse order sorting according to the relevance weight value to generate the reference content sequence.
5. The method of claim 4, wherein selecting the first plurality of content in each recall result from a plurality of recall results obtained by the preset recall mechanism triggered based on all points of interest of the preset recall mechanism, and combining to form a third set of candidate content comprises: taking each interest point in the preset recall mechanism as a recall key word, and respectively triggering the preset recall mechanism to obtain a fourth candidate content set associated with each interest point; For each fourth candidate content set, sorting according to the relevance of the interest points of the fourth candidate content set, and selecting a plurality of first contents to form a corresponding fifth candidate content set; and merging and de-duplicating the fifth candidate content set corresponding to each interest point to form the third candidate content set.
6. The method of claim 4, wherein calculating normalized break cumulative gains for the first recommended content sequence and the second recommended content sequence, respectively, based on the reference content sequence comprises: calculating to obtain ideal breakage accumulated gain based on the correlation weight value and the sequencing position of each content in the reference content sequence; matching the content in the first recommended content sequence with the reference content sequence respectively, and calculating to obtain the damage accumulated gain of the first recommended content sequence according to the position of the matched content in the first recommended content sequence and the corresponding correlation weight value; matching the content in the second recommended content sequence with the reference content sequence respectively, and calculating to obtain the damage accumulated gain of the second recommended content sequence according to the position of the matched content in the second recommended content sequence and the corresponding correlation weight value; And respectively carrying out normalization processing on the damage accumulation gains of the first recommended content sequence and the second recommended content sequence and the ideal damage accumulation gain to obtain corresponding normalization damage accumulation gains.
7. The method of any one of claims 1 to 6, wherein performing a comparison analysis to determine the merits of the new interest model relative to the old interest model comprises: determining that the new interest model is better than the old interest model when the normalized impairment accumulation gain of the first recommended content sequence is less than the normalized impairment accumulation gain of the second recommended content sequence; determining that the old interest model is better than the new interest model when the normalized damage accumulated gain of the first recommended content sequence is greater than the normalized damage accumulated gain of the second recommended content sequence; And under the condition that the normalized damage accumulated gain of the first recommended content sequence is equal to the normalized damage accumulated gain of the second recommended content sequence, determining that the old interest model and the new interest model have the same quality.
8. An apparatus for evaluating quality of interest of a user, comprising: The first generation module is used for collecting historical behavior data of a user in a first time window and generating an old interest point sequence and a new interest point sequence according to the historical behavior data through an old interest model and a new interest model respectively; the conversion module is used for respectively converting the old interest point sequence and the new interest point sequence into a first recommended content sequence and a second recommended content sequence for predicting the interest preference of the user through a preset recall mechanism; The second generation module is used for collecting real behavior data of the user in a second time window and quantifying the real behavior data according to the watching duration to generate a reference content sequence reflecting the real interest preference of the user, wherein the second time window is positioned behind the first time window in time; And the comparison module is used for respectively calculating normalized damage accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence according to the reference content sequence, and carrying out comparison analysis so as to determine the advantages and disadvantages of the new interest model relative to the old interest model.
9. A computer-readable storage medium, having stored thereon a computer program, characterized in that the computer program, when executed by a processor, performs the method of any of claims 1 to 7.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of the claims 1 to 7 by means of the computer program.

Description

User interest quality evaluation method and device, storage medium and electronic equipment Technical Field The present application relates to the field of multimedia intelligent processing technologies, and in particular, to a method and apparatus for evaluating quality of interest of a user, a storage medium, and an electronic device. Background With the wide application of personalized recommendation systems, user interest points play a central role in recommendation result generation. For example, in a movie platform, a user's interest preferences in stars, types, or topics are typically represented by a user portrait. However, the prior art still has significant shortcomings in point of interest quality assessment. The existing method mainly relies on an on-line A/B experiment or manual sampling evaluation to verify the accuracy of the interest points, and can reflect the recommendation effect to a certain extent, but has the problems of high evaluation cost, long period, complex cross-team cooperation and the like. In addition, the offline evaluation method is generally only based on interest data or historical behavior characteristics generated by the model, lacks direct association with actual behaviors of the user, and cannot verify the updated effect of the interest points in time. Because the interest data belongs to the intermediate representation, the interest data is difficult to map to the user viewing behavior directly, obvious faults exist between the interest points and the recommended results, and quick and low-cost evaluation of the quality of the interest points is difficult to realize. Meanwhile, after the interest model is updated, a long time is often required to wait for actual feedback to judge the effect, which further limits the efficiency of recommending algorithm iteration and user portrait updating. Disclosure of Invention The application provides a method and a device for evaluating user interest quality, a storage medium and electronic equipment, and aims to solve the technical problems that interest points are difficult to be directly related to user behaviors and evaluation cost is high. The application provides a method for evaluating user interest quality, which comprises the steps of collecting historical behavior data of a user in a first time window, generating an old interest point sequence and a new interest point sequence according to the historical behavior data through an old interest model and a new interest model respectively, converting the old interest point sequence and the new interest point sequence into a first recommended content sequence and a second recommended content sequence for predicting user interest preference through a preset recall mechanism respectively, collecting real behavior data of the user in the second time window, quantifying the real behavior data according to watching duration, generating a reference content sequence reflecting the user real interest preference, wherein the second time window is positioned behind the first time window in time, calculating normalized breakage accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence according to the reference content sequence respectively, and comparing and analyzing to determine the superiority and inferiority of the new interest model relative to the old interest model. The application provides a device for evaluating the user interest quality, which comprises a first generation module, a conversion module and a second generation module, wherein the first generation module is used for collecting historical behavior data of a user in a first time window and generating an old interest point sequence and a new interest point sequence according to the historical behavior data through an old interest model and a new interest model respectively, the conversion module is used for converting the old interest point sequence and the new interest point sequence into a first recommended content sequence and a second recommended content sequence for predicting the user interest preference through a preset recall mechanism respectively, the second generation module is used for collecting real behavior data of the user in a second time window and quantifying the real behavior data according to the watching duration to generate a reference content sequence reflecting the real interest preference of the user, the second time window is positioned behind the first time window in time, and the comparison module is used for calculating normalized loss accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence respectively according to the reference content sequence and comparing and analyzing the normalized loss accumulation gains corresponding to the first recommended content sequence and the second recommended content sequence to determine the new interest preference of the new mod