Towards Universal Probabilistic Programming with Message Passing on Factor Graphs

Abstract

This thesis presents efficient and automated probabilistic inference algorithms for intelligent system design. An intelligent system is a decision-making agent that can take reasonable and reliable actions in uncertain environments. Probability theory and Bayesian inference constitute the theoretical framework for intelligent system design. Hence, probabilistic modeling and inference are of great importance in building intelligent agents and applications. Modeling a real-world phenomenon with the language of probability theory is often intuitive, and hence it can be carried out by experts in the application field. In contrast, the inference part often necessitates expertise in Bayesian statistics and precludes experts and scientists from diverse application fields from utilizing probabilistic modeling. This thesis focuses on automating Bayesian inference procedures to make probabilistic modeling more accessible for non-experts of Bayesian statistics. The works presented in this thesis can be subsumed under the umbrella of probabilistic programming. Probabilistic programming is a programming paradigm that automates inference procedures in probabilistic models.