A Factor Graph Approach to Variational Sparse Gaussian Processes

Abstract

A Variational Sparse Gaussian Process (VSGP) is a sophisticated nonparametric probabilistic model that has gained significant popularity since its inception. The VSGP model is often employed as a component of larger models or in a modified form across numerous applications. However, re-deriving the update equations for inference in these variations is technically challenging, which hinders broader adoption. In a separate line of research, message passing-based inference in factor graphs has emerged as an efficient framework for automated Bayesian inference. Despite its advantages, message passing techniques have not yet been applied to VSGP-based models due to the lack of a suitable representation for VSGP models in factor graphs. To address this limitation, we introduce a Sparse Gaussian Process (SGP) node within a Forney-style factor graph (FFG). We derive variational message passing update rules for the SGP node, enabling automated and efficient inference for VSGP-based models. We validate the update rules and illustrate the benefits of the SGP node through experiments in various Gaussian Process applications.