US-12626145-B2 - Apparatus and method for recommending collaborative filtering based on learnable-time ordinary differential equation

US12626145B2US 12626145 B2US12626145 B2US 12626145B2US-12626145-B2

Abstract

A collaborative filtering recommending apparatus includes an initial-embedding generation module configured to generate first and second initial embeddings; a dual co-evolving ordinary differential equation module comprising first and second ordinary differential equation processing members which repeat, for a specific time, a process of receiving first and second previous embeddings, respectively, and outputting first and second subsequent embeddings, respectively, through mutual influence as a learning time elapses, the first and second previous embeddings initially corresponding to the first and second initial embeddings, respectively; and a final-embedding generation module configured to generate a first final embedding by cumulative summing the first previous and subsequent embeddings, and to generate a second final embedding by cumulative summing the second previous and subsequent embeddings.

Inventors

No Seong Park
Jeong Whan CHOI
Jin Sung JEON

Assignees

UIF (UNIVERSITY INDUSTRY FOUNDATION), YONSEI UNIVERSITY

Dates

Publication Date: 20260512
Application Date: 20211228
Priority Date: 20211213

Claims (14)

1 . A collaborative filtering recommending apparatus based on a learnable-time ordinary differential equation, the apparatus comprising: an initial-embedding generation module configured to generate first and second initial embeddings; a dual co-evolving ordinary differential equation module comprising first and second ordinary differential equation processing members which repeat, for a specific time, a process of receiving first and second previous embeddings, respectively, and outputting first and second subsequent embeddings, respectively, through mutual influence as a learning time elapses, the first and second previous embeddings initially corresponding to the first and second initial embeddings, respectively; and a final-embedding generation module configured to generate a first final embedding by cumulative summing the first previous and subsequent embeddings, and to generate a second final embedding by cumulative summing the second previous and subsequent embeddings, wherein the process repeated by the dual co-evolving ordinary differential equation module for the specific time extracts a plurality of user embedding matrixes at a first set of time points and a plurality of product embedding matrixes at a second set of time points within a continuous time domain, wherein each time point is implemented as a learnable parameter stored in computer memory that is iteratively updated during training by: (i) computing a gradient of a loss function with respect to the time point using an adjoint sensitivity method that performs reverse-mode integration through the neural ordinary differential equations; and (ii) adjusting a numerical value of the time point stored in the memory based on the computed gradient, and wherein the initial-embedding generation module, the dual co-evolving ordinary differential equation module, and the final-embedding generation module are each implemented via at least one processor.
2 . The collaborative filtering recommending apparatus of claim 1 , wherein the dual co-evolving ordinary differential equation module outputs the user embedding matrix (u(t)) at a specific point of the learnable-time as the first subsequent embedding.
3 . The collaborative filtering recommending apparatus of claim 2 , wherein the dual co-evolving ordinary differential equation module outputs the product embedding matrix (p(t)) at a specific point of the learnable-time as the second subsequent embedding.
4 . The collaborative filtering recommending apparatus of claim 3 , wherein the dual co-evolving ordinary differential equation module constructs a set of co-evolving ordinary differential equations with the user embedding matrix and the product embedding matrix.
5 . The collaborative filtering recommending apparatus of claim 3 , wherein the dual co-evolving ordinary differential equation module outputs a plurality of the user embedding matrixes (u(t 1 ), u(t 2 ), . . . , u(t n )) and a plurality of the product embedding matrixes (p(t 1 ), p(t 2 ), . . . , p(t n )) at a plurality of discrete times which is settable by a user.
6 . The collaborative filtering recommending apparatus of claim 1 , wherein the dual co-evolving ordinary differential equation module constructs each of first and second ordinary differential equations as a non-parameterized and non-time-dependent ordinary differential equation.
7 . The collaborative filtering recommending apparatus of claim 1 , wherein the final-embedding generation module generates the first final embedding by weighted summing a plurality of user matrixes generated at a plurality of discrete times, and generates the second final embedding by weighted summing a plurality of product matrixes generated at the plurality of discrete times.
8 . A collaborative filtering recommending method based on a learnable-time ordinary differential equation, the method comprising: an initial embedding generation step of generating first and second initial embeddings; a dual co-evolving ordinary differential equation step of repeating, for a specific time, a process of receiving first and second previous embeddings, respectively, and outputting first and second subsequent embeddings, respectively, through mutual influence as a learning time elapses, through first and second ordinary differential equation processing members, the first and second previous embeddings initially corresponding to the first and second initial embeddings, respectively; and a final-embedding generation step of generating a first final embedding by cumulative summing the first previous and subsequent embeddings, and generating a second final embedding by cumulative summing the second previous and subsequent embeddings, wherein the process repeated by the dual co-evolving ordinary differential equation step for the specific time extracts a plurality of user embedding matrixes at a first set of time points and a plurality of product embedding matrixes at a second set of time points within a continuous time domain, wherein each time point is implemented as a learnable parameter stored in computer memory that is iteratively updated during training by: (i) computing a gradient of a loss function with respect to the time point using an adjoint sensitivity method that performs reverse-mode integration through the neural ordinary differential equations; and (ii) adjusting a numerical value of the time point stored in the memory based on the computed gradient.
9 . The collaborative filtering recommending method of claim 8 , wherein the dual co-evolving ordinary differential equation step comprises outputting the user embedding matrix (u(t)) at a specific point of the learnable-time as the first subsequent embedding.
10 . The collaborative filtering recommending method of claim 9 , wherein the dual co-evolving ordinary differential equation step comprises outputting the product embedding matrix (p(t)) at a specific point of the learnable-time as the second subsequent embedding.
11 . The collaborative filtering recommending method of claim 10 , wherein the dual co-evolving ordinary differential equation step comprises constructing a set of co-evolving ordinary differential equations with the user embedding matrix and the product embedding matrix.
12 . The collaborative filtering recommending method of claim 10 , wherein the dual co-evolving ordinary differential equation step comprises outputting a plurality of the user embedding matrixes (u(t 1 ), u(t 2 ), . . . , u(t n )) and a plurality of the product embedding matrixes (p(t 1 ), p(t 2 ), . . . , p(t n )) at a plurality of discrete times which is settable by a user.
13 . The collaborative filtering recommending method of claim 8 , wherein the dual co-evolving ordinary differential equation step comprises constructing each of first and second ordinary differential equations as a non-parameterized and non-time-dependent ordinary differential equation.
14 . The collaborative filtering recommending method of claim 8 , wherein the final embedding generation step comprises generating the first final embedding by weighted summing a plurality of user matrixes generated at a plurality of discrete times, and generating the second final embedding by weighted summing a plurality of product matrixes generated at the plurality of discrete times.

Description

ACKNOWLEDGEMENT National R&D Project Supporting the Present Invention Assignment number: 1711126082Project number: 2020-0-01361-002Department name: Ministry of Science and Technology Information and CommunicationResearch and management institution: Information and Communication Planning and Evaluation InstituteResearch project name: Information and Communication Broadcasting Innovation Talent Training(R&D)Research project name: Artificial Intelligence Graduate School Support(Yonsei University)Contribution rate: 1/1Organized by: Yonsei University Industry-Academic Cooperation FoundationResearch period: 20200101 to 20211231 CROSS-REFERENCE TO RELATED APPLICATION This application claims priority to Korean Patent Application No. 10-2021-0177928 (filed on Dec. 13, 2021), which is hereby incorporated by reference in its entirety. BACKGROUND The present disclosure relates to a learning-based recommendation technology and, more particularly, to an apparatus and method for recommending learnable-time ordinary differential equation based collaborative filtering, which redesign a linear GCN based on a neural ODE, thus improving a collaborative filtering performance. Collaborative filtering (CF), which is to predict users' preferences from patterns, is a long-standing research subject in the field of recommender systems. Further, it is common to learn user and product embedding vectors and calculate their dot-products for recommendation. Matrix factorization is one approach, which is well-known in the field of the recommender system. There have also been proposed various other application examples. Recently, attention is being focused on graph convolutional networks (GCNs) for the purpose of collaborative filtering. GCNs have been proposed to process not only CF-related graphs but also other general graphs. GCNs may be mainly categorized into two types: spectral GCNs and spatial GCNs. GCNs for CF may fall into the first category due to its appropriateness for CF. However, there is still needed research on what is the optimal GCN architecture for CF. Neural ordinary differential equations (NODEs) may be to learn implicit differential equations from data. NODEs may calculate h⁡(t1)=h⁡(t0)+∫t0 t1f⁡(h⁡(t),t;θf)⁢dt, where ƒ is a neural network parameterized by θƒ that approximates dh⁡(t)dt, to derive h(t1) from h(t0), when t1>t0. It is to be noted that θƒ is trained from data, in other words, dh⁡(t)dt is trained from data. A variable t may be called as a time variable, which represents the layer concept of neural networks. It is to be noted that t may be a non-negative integer in conventional neural networks, whereas it may be any non-negative real number in NODEs. In this regard, NODEs may be considered as continuous generalizations of neural networks. Various ODE solvers may solve the integral problem, and may generalize various neural network architectures. For instance, the general form of residual connection may be expressed as h(t+1)=h(t)+ƒ(h(t); θ), which may be identical to the explicit Euler method to solve ODE problems. CITED DOCUMENT Patent Document Korean Patent Laid-Open Publication No. 10-2021-0031197 (Mar. 19, 2021) SUMMARY In view of the above, the present disclosure provides an apparatus and method for recommending learnable-time ordinary differential equation based collaborative filtering, which redesign a linear GCN based on a neural ODE, thus improving a collaborative filtering performance. The present disclosure provides a collaborative filtering recommending apparatus based on a learnable-time ordinary differential equation, the apparatus including an initial-embedding generation module configured to generate first and second initial embeddings; a dual co-evolving ordinary differential equation module including first and second ordinary differential equation processing members which repeat, for a specific time, a process of receiving first and second previous embeddings, respectively, and outputting first and second subsequent embeddings, respectively, through mutual influence as a learning time elapses, the first and second previous embeddings initially corresponding to the first and second initial embeddings, respectively; and a final-embedding generation module configured to generate a first final embedding by cumulative summing the first previous and subsequent embeddings, and to generate a second final embedding by cumulative summing the second previous and subsequent embeddings. The dual co-evolving ordinary differential equation module may output a user embedding matrix (u(t)) at a specific point of the learnable-time as the first subsequent embedding. The dual co-evolving ordinary differential equation module may output a product embedding matrix (p(t)) at a specific point of the learnable-time as the second subsequent embedding. The dual co-evolving ordinary differential equation module may construct a set of co-evolving ordinary differential equations with the user embedding matrix and the p