Autoregressive (AR) models are one of the most popular ways to describe different time-varying processes in nature, economics, etc. However, their parameters are often estimated in a batch manner which makes them inefficient for handling large-scale real-time data. In our work, we investigate the feasibility of online parameter estimation for these types of models. We translate the AR model to a probabilistic factor graph which takes advantage of the factorization of the model by implementing inference as a message passing algorithm. Due to the intractability of exact parameter inference for these types of models, sum-product message passing becomes impractical. This suggests to use alternative message passing algorithms based on approximate inference, e.g., variational message passing (VMP) which tries to find variational distributions that serve as good proxies for the exact solution. With VMP, the computations for online state and parameter estimation can be automated.