expected free energy

For the

• $\pi$ - policy - sequence of actions
• $Q$ - current mental model
• $\tilde{x}$ - set of future hidden states
• $\tilde{y}$ - set of future outcomes/ perceptions
• $C$ - set of preference parameters

$$\begin{array}{rl}G(\pi )& =-{\mathbb{E}}_{Q(\tilde{x},\tilde{y}|\pi )}[D_{KL}[Q(\tilde{x}|\tilde{y},\pi )||Q(\tilde{x}|\pi )]]-{\mathbb{E}}_{Q(\tilde{y}|\pi )}[\ln P(\tilde{y}|C)]\\ & ={\mathbb{E}}_{Q(\tilde{x}|\pi )}[H[P(\tilde{y}|\tilde{x})]]+D_{KL}[Q(\tilde{y}|\pi )||P(\tilde{y}|C)]\\ & ={\mathbb{E}}_{Q(\tilde{x},\tilde{y}|\pi )}[\ln P(\tilde{y},\tilde{x}|C)]-H[Q(\tilde{x}|\pi )]\end{array}$$

where $Q(\tilde{x},\tilde{y}|\pi )\triangleq Q(\tilde{x}|\pi )P(\tilde{y}|\tilde{x})$

---

References

@parr2022b