Towards Flexibility, Efficiency, and Multi-Dimensional Dependency of Gaussian Process State-Space Models

The state-space model (SSM) is a mathematical framework that encapsulates the dynamics of a system through its states, transition functions governing state evolution, and observed system outputs. This versatile framework finds applications across various sectors, from military to civilian domains, including missile interception, autonomous navigation, epidemic tracking, and more. However, classic SSMs rely on predefined or known transition functions to estimate latent states, which can be challenging or even infeasible to specify accurately in complex scenarios. Such a mismatch in underlying system dynamics can lead to significant performance degradation in state estimation and model prediction. To address this challenge, machine learning and data-driven models have emerged as promising solutions. This thesis focuses on a specific class of data-driven (Bayesian nonparametric) SSMs, namely the Gaussian process state-space model (GPSSM for nonlinear dynamical system modeling, learning, and inference. The main objective of this thesis is to investigate two fundamental aspects within GPSSMs: 1) the model’s representational capacity, and 2) learning and inference efficiency. Regarding model representational capacity, we first note that GP-based transition functions within GPSSMs face challenges in modeling complex real-world system dynamics, particularly those that are non-smooth and nonlinear. To overcome this limitation, we propose a novel approach using normalizing flows to transform standard GPs into more expressive stochastic processes, thereby constructing a flexible and interpretable SSM called the transformed GPSSM (TGPSSM). We provide a rigorous derivation and analysis of TGPSSM, demonstrating its feasibility and superiority in handling complex systems. Additionally, existing GPSSMs often assume that the outputs of multi-dimensional transition functions are independent. While this assumption simplifies the modeling process, it fails to accurately capture the dependencies present in practical multi-dimensional data. To address this, we introduce a multi-output GP based on the linear model of coregionalization (LMC) in GPSSMs to enhance the model’s representational power. Leveraging inductive biases across dimensions, the resulting output-dependent GPSSM (ODGPSSM) shows significant advantages in handling dynamical systems with partially observable dimensions. To improve learning and inference efficiency, we first streamline existing variational inference methods. Specifically, to address challenges such as large-scale optimization of variational parameters or extensive Monte Carlo sampling that can lead to inefficient learning and inference, we propose a novel ensemble Kalman filter (EnKF)-integrated variational inference algorithm. This algorithm harnesses the inherent differentiability of EnKF to streamline the inference process for latent states within a non-mean-field variational inference framework. Furthermore, we extend and propose an online version of the algorithm to further improve its scalability and efficiency. To tackle the issue of escalating parameter proliferation and computational complexity associated with higher-dimensional latent state spaces, we then propose a new method by integrating efficiently transformed Gaussian processes (ETGP) into GPSSM. Specifically, we employ multiple normalizing flows on a shared Gaussian process to transform it into multiple flexible and correlated random processes. These random processes act as priors for transition functions within high-dimensional latent state spaces. The resulting model, named the efficient GPSSM (EGPSSM), substantially reduces computational complexity while demonstrating competitive performance in inference tasks compared to existing methods.