US-12626107-B2 - Systems and methods for time series forecasting

Abstract

Systems and methods for providing a neural network system for time series forecasting are described. A time series dataset that includes datapoints at a plurality of timestamps in an observed space is received. The neural network system is trained using the time series dataset. The training the neural network includes: generating, using an encoder of the neural network system, one or more estimated latent variables of a latent space for the time series dataset; generating, using an auxiliary predictor of the neural network system, a first latent-space prediction result based on the one or more estimated latent variables; transforming, using a decoder of the neural network system, the first latent-space prediction result to a first observed-space prediction result; and updating parameters of the neural network system based on a loss based on the first observed-space prediction result.

Inventors

• Chenghao LIU
• Chu Hong Hoi
• Kun Zhang

Assignees

• SALESFORCE, INC.

Dates

Publication Date

20260512

Application Date

20220916

Claims (18)

1. 1 . A method of providing a neural network system for time series forecasting, comprising: receiving, via an input interface, a time series dataset that includes datapoints at a plurality of timestamps in an observed space; training the neural network system using the time series dataset, wherein the training the neural network system includes: providing a non-parametric state space model for a dynamic system associated with the time series dataset, wherein the non-parametric state space model includes a nonparametric latent transition model and an emission model, wherein the emission model includes a post-nonlinear transformation associated with distortion in instrument measurements; generating, using a first encoder of the neural network system based on the non-parametric state space model, one or more estimated latent variables of a latent space for the time series dataset; generating, using an auxiliary predictor of the neural network system, a first latent-space prediction result based on the one or more estimated latent variables; transforming, using a first decoder of the neural network system, the first latent-space prediction result to a first observed-space prediction result; and updating parameters of the neural network system using a first objective based on the first latent-space prediction result and the first observed-space prediction result.
2. 2 . The method of claim 1 , wherein each of the latent variables is identifiable from a corresponding observed data.
3. 3 . The method of claim 1 , wherein the one or more estimated latent variables are determined based on one or more estimated time-varying change factors.
4. 4 . The method of claim 3 , wherein the training the neural network system includes: providing, using the auxiliary predictor, a first estimated noise based on the one or more estimated latent variables and the one or more estimated time-varying change factors; and generating, using the auxiliary predictor, the first latent-space prediction result based on the one or more estimated latent variables and the first estimated noise.
5. 5 . The method of claim 1 , wherein the neural network system includes a variational autoencoder (VAE) including the first encoder and the first decoder, and wherein the first objective includes an augmented evidence lower bound (ELBO) objective.
6. 6 . The method of claim 1 , wherein the first objective is based on an auxiliary predictor distribution associated with the first latent-space prediction result.
7. 7 . A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform a method comprising: receiving, via an input interface, a time series dataset that includes datapoints at a plurality of timestamps in an observed space; training a neural network system using the time series dataset, wherein the training the neural network system includes: providing a non-parametric state space model for a dynamic system associated with the time series dataset, wherein the non-parametric state space model includes a nonparametric latent transition model and an emission model, wherein the emission model includes a post-nonlinear transformation associated with distortion in instrument measurements; generating, using a first encoder of the neural network system based on the non-parametric state space model, one or more estimated latent variables of a latent space for the time series dataset; generating, using an auxiliary predictor of the neural network system, a first latent-space prediction result based on the one or more estimated latent variables; transforming, using a first decoder of the neural network system, the first latent-space prediction result to a first observed-space prediction result; and updating parameters of the neural network system using a first objective based on the first latent-space prediction result and the first observed-space prediction result.
8. 8 . The non-transitory machine-readable medium of claim 7 , wherein each of the latent variables is identifiable from a corresponding observed data.
9. 9 . The non-transitory machine-readable medium of claim 7 , wherein the one or more estimated latent variables are determined based on one or more estimated time-varying change factors.
10. 10 . The non-transitory machine-readable medium of claim 9 , wherein the training the neural network system includes: providing, using the auxiliary predictor, a first estimated noise based on the one or more estimated latent variables and the one or more estimated time-varying change factors; and generating, using the auxiliary predictor, the first latent-space prediction result based on the one or more estimated latent variables and the first estimated noise.
11. 11 . The non-transitory machine-readable medium of claim 7 , wherein the neural network system includes a variational autoencoder (VAE) including the first encoder and the first decoder, and wherein the first objective includes an augmented evidence lower bound (ELBO) objective.
12. 12 . The non-transitory machine-readable medium of claim 7 , wherein the first objective is based on an auxiliary predictor distribution associated with the first latent-space prediction result.
13. 13 . A system, comprising: a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform a method comprising: receiving, via an input interface, a time series dataset that includes datapoints at a plurality of timestamps in an observed space; training a neural network system using the time series dataset, wherein the training the neural network system includes: providing a non-parametric state space model for a dynamic system associated with the time series dataset, wherein the non-parametric state space model includes a nonparametric latent transition model and an emission model, wherein the emission model includes a post-nonlinear transformation associated with distortion in instrument measurements; generating, using a first encoder of the neural network system based on the non-parametric state space model, one or more estimated latent variables of a latent space for the time series dataset; generating, using an auxiliary predictor of the neural network system, a first latent-space prediction result based on the one or more estimated latent variables; transforming, using a first decoder of the neural network system, the first latent-space prediction result to a first observed-space prediction result; and updating parameters of the neural network system using a first objective based on the first latent-space prediction result and the first observed-space prediction result.
14. 14 . The system of claim 13 , wherein each of the latent variables is identifiable from a corresponding observed data.
15. 15 . The system of claim 13 , wherein the one or more estimated latent variables are determined based on one or more estimated time-varying change factors.
16. 16 . The system of claim 15 , wherein the training the neural network system includes: providing, using the auxiliary predictor, a first estimated noise based on the one or more estimated latent variables and the one or more estimated time-varying change factors; and generating, using the auxiliary predictor, the first latent-space prediction result based on the one or more estimated latent variables and the first estimated noise.
17. 17 . The system of claim 13 , wherein the neural network system includes a variational autoencoder (VAE) including the first encoder and the first decoder, and wherein the first objective includes an augmented evidence lower bound (ELBO) objective.
18. 18 . The system of claim 13 , wherein the first objective is based on an auxiliary predictor distribution associated with the first latent-space prediction result.

Description

RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63/344,495 filed May 20, 2022 which is incorporated by reference herein in its entirety. TECHNICAL FIELD The embodiments relate generally to time series forecasting, and more specifically to systems and methods for learning latent causal dynamics in time series forecasting. BACKGROUND Time series forecasting has been widely used in research and industries such as economic planning, epidemiology study, or energy consumption. State space models, together with deep learning, have been used for time series analysis and prediction. However, these methods usually rely on stringent assumptions regarding the nature of causal relationships that may not hold in practice. For example, if the forms of transition and emission processes with stringent assumptions in these models cannot approximate the actual data generation process, the results could be sub-optimal. Therefore, there is a need for improved time series forecasting. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a simplified block diagram of a computing device for implementing the time series forecasting framework based on a non-parametric state space model described in embodiments herein. FIG. 2 is a simplified block diagram of a networked system suitable for implementing the time series forecasting framework based on a non-parametric state space model described in embodiments herein. FIG. 3 is an example logic flow diagram illustrating a method of providing time series forecasting based on a non-parametric state space model for a dynamical system underlying the time series data, according to some embodiments described herein. FIG. 4 is an example logic flow diagram illustrating a method of providing time series forecasting based on the non-parametric state space model, according to some embodiments described herein. FIG. 5 is an example block diagram illustrating an example architecture for a time series forecasting framework based on the non-parametric state space model in model training, according to some embodiments described herein. FIG. 6 is an example block diagram illustrating an example transition prior modeling used in the time series forecasting framework of FIG. 5, according to some embodiments described herein. FIGS. 7A and 7B illustrate comparison between example modeling dependencies in observation space and latent space used in the time series forecasting framework, according to some embodiments described herein FIG. 8 illustrates a graph representation of a generation process of a dynamical system with time-varying change factors as used in the time series forecasting framework, according to some embodiments described herein. FIGS. 9-18 provide example data tables and experimental results illustrating example data performance of the time series forecasting framework based on a non-parametric state space model described in relation to FIGS. 1-7B, according to some embodiments described herein. In the figures, elements having the same designations have the same or similar functions. DETAILED DESCRIPTION As used herein, the term “network” may comprise any hardware or software-based framework that includes any artificial intelligence network or system, neural network, or system and/or any training or learning models implemented thereon or therewith. As used herein, the term “module” may comprise hardware or software-based framework that performs one or more functions. In some embodiments, the module may be implemented on one or more neural networks. Time series forecasting using deep learning models is often challenging because the deep learning models may not have a priori knowledge of the time series. For example, in modelling a pandemic transmission and recovery model, the assumptions of protocols such as containment strategies at different periods may not be apparent in the time-series data itself. For example, protocols such as self-quarantine and social distancing mandates, and the like, which were implemented at certain periods during a lookback time window, can often influence the outcome of time series forecasts. However, such knowledge of historical protocols may not be captured by the deep learning model by merely learning the time series data such as the number of new cases, 7-day average, and/or the like within a past time period. Furthermore, deep learning models are often directly trained to minimize forecasting loss or reconstruction loss. However, deep learning models that are trained to minimize forecasting loss or reconstruction loss may not recover the correct latent processes. The lack of a correct latent process may result in models that capture spurious or over-completed dependencies with noise, which eventually impair the training performance. In view of the need for a more accurate and computationally efficient time series forecasting mechanism, embodiments described herein provide a state space-model based framework that adop