Disentanglement for improved data-driven modeling of dynamical systems

Modeling dynamical systems is a fundamental task in various scientific and engineering domains, requiring accurate predictions, robustness to varying conditions, and interpretability of the underlying mechanisms. Traditional data-driven approaches often struggle with long-term prediction accuracy, generalization to out-of-distribution (OOD) scenarios, and providing insights into the system’s behavior. This thesis explores the integration of supervised disentanglement into deep learning models as a means to address these challenges. We begin by advancing the state-of-the-art in modeling wave propagation governed by the Saint-Venant equations. Utilizing U-Net architectures and purposefully designed training strategies, we develop deep learning models that significantly improve prediction accuracy. Through OOD analysis, we highlight the limitations of standard deep learning models in capturing complex spatiotemporal dynamics, demonstrating how integrating domain knowledge through architectural design and training practices can enhance model performance. We further extend our supervised disentanglement approach to high-dimensional observations by adapting it to convolutional neural network architectures. Specifically, we integrate supervised disentanglement into the Recurrent State-Space Model (RSSM) and U-Net architectures, evaluating their performance on modeling pendulum motion from images and wave propagation dynamics. Our results suggest that supervised disentanglement can be applicable in complex, high-dimensional settings and may offer benefits in improving model robustness. We conduct an analysis of the learned latent space representations using various disentanglement metrics. The findings indicate that supervised disentanglement can lead to more structured and interpretable latent spaces, potentially facilitating better parameter inference and model interpretability. Despite the improvements, we recognize potential trade-offs, such as slight decreases in short-term prediction accuracy in some high-dimensional experiments. These observations highlight the need for careful consideration when applying supervised disentanglement and the effect of the system dynamics and observation space. Further research is needed to fully assess its potential and limitations. By bridging data-driven approaches with physical understanding, supervised disentanglement offers a novel tool for achieving accurate long-term predictions, robustness to OOD scenarios, and enhanced interpretability. This methodology has potential applications across diverse fields, including climate modeling, engineering, and healthcare, where understanding and predicting complex system behaviors are crucial. In conclusion, this thesis contributes to the advancement of modeling dynamical systems by demonstrating the efficacy of supervised disentanglement. It provides a foundation for future research into integrating domain knowledge into machine learning models, exploring new methods for disentangled representation learning, and applying these techniques to a wider range of dynamical systems. By continuing to merge data-driven models with physical understanding, we move toward more reliable, interpretable, and effective tools for modeling the dynamic processes that shape our world.