Reactive Probabilistic Programming for Scalable Bayesian Inference
Abstract
The Bayesian framework is a crucial technology at the core of modern AI with applications such as speech and image recognition and generation, biomedical analysis, robot navigation, and more. The framework describes how a rational agent should update its beliefs when new information is revealed by the agent’s environment. In order to update internal beliefs, the agent must execute the so-called Bayes rule, which follows straight from the fundamental rules of probability theory. Unfortunately, the exact execution of Bayes rule is often computationally challenging or even intractable, due to the need to evaluate high-dimensional integrals that may have no closed-form solution. In addition, real-world scenarios comprise non-stationary environments that may lead to a need for online and real-time execution of Bayes rule in time-varying conditions. In short, while Bayes rule is arguably the correct way to support computations in advanced AI applications, in practice the application of a Bayesian approach to AI systems has been hindered by serious computational issues. In this dissertation, we attempt to approach this computational issue from a fresh perspective. This dissertation focuses on the realization of efficient Bayesian inference in large-scale probabilistic models, targeting real-time signal processing and control applications under real-world conditions. We present a practical architecture based on reactive message passing-based inference in a factor graph representation of the probabilistic model under study. Factor graphs not only offer an insightful visual representation but also support an efficient inference process that takes advantage of statistical independence assumptions in the probabilistic model. For a given factor graph, we first associate the Bayesian inference problem with the minimization of a Constrained Bethe Free Energy (CBFE) functional, which can be interpreted as an approximate, but computationally lighter, approach to Bayesian inference. We then develop an automatated message passing approach to CBFE minimization. As a unique feature, in this dissertation, we introduce a reactive programming style implementation of the message passing process. –
