Realizing Synthetic Active Inference Agents, Part II: Variational Message Updates

Thijs van de Laar; Magnus Koudahl; Bert de Vries

doi:10.1162/neco_a_01713

Realizing Synthetic Active Inference Agents, Part II: Variational Message Updates

Neural Comput. 2024 Sep 23:1-38. doi: 10.1162/neco_a_01713. Online ahead of print.

Authors

Thijs van de Laar¹, Magnus Koudahl^{2

3}, Bert de Vries^{2

4}

Affiliations

¹ Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands t.w.v.d.laar@tue.nl.
² Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands.
³ VERSES AI Research Lab, Los Angeles, CA, 90016, U.S.A. magnus.koudahl@verses.ai.
⁴ GN Hearing Benelux BV, 5612 AB, Eindhoven, The Netherlands bert.de.vries@tue.nl.

PMID: 39312496
DOI: 10.1162/neco_a_01713

Abstract

The free energy principle (FEP) describes (biological) agents as minimizing a variational free energy (FE) with respect to a generative model of their environment. Active inference (AIF) is a corollary of the FEP that describes how agents explore and exploit their environment by minimizing an expected FE objective. In two related papers, we describe a scalable, epistemic approach to synthetic AIF by message passing on free-form Forney-style factor graphs (FFGs). A companion paper (part I of this article; Koudahl et al., 2023) introduces a constrained FFG (CFFG) notation that visually represents (generalized) FE objectives for AIF. This article (part II) derives message-passing algorithms that minimize (generalized) FE objectives on a CFFG by variational calculus. A comparison between simulated Bethe and generalized FE agents illustrates how the message-passing approach to synthetic AIF induces epistemic behavior on a T-maze navigation task. Extension of the T-maze simulation to learning goal statistics and a multiagent bargaining setting illustrate how this approach encourages reuse of nodes and updates in alternative settings. With a full message-passing account of synthetic AIF agents, it becomes possible to derive and reuse message updates across models and move closer to industrial applications of synthetic AIF.