State-space modeling of non-stationary natural signals is a notoriously difficult task. As a result of context switches, the memory depth of the model should ideally be adapted online. Stabilized linear forgetting (SLF) has been proposed as an elegant method for state-space tracking in context-switching environments. In practice, SLF leads to state and parameter estimation tasks for which no analytical solutions exist. In the literature, a few approximate solutions have been derived, making use of specific model simplifications. This paper proposes an alternative approach, in which SLF is described as an inference task on a generative probabilistic model. SLF is then executed by a variational message passing algorithm on a factor graph representation of the generative model. This approach enjoys a number of advantages relative to previous work. First, variational message passing (VMP) is an automatable procedure that adapts appropriately under changing model assumptions. This eases the search process for the best model. Secondly, VMP easily extends to estimate model parameters. Thirdly, the modular make-up of the factor graph framework allows SLF to be used as a click-on feature in a large variety of complex models. The functionality of the proposed method is verified by simulating an SLF state-space model in a context-switching data environment.