Markov chain matrix

Markov chain can be compactly/ efficiently described using matrices + vectors.

For a vector or probabilities (can be a prior distribution) of the possible states ${\vec{s}}_t$:

${\vec{s}}_{t+1}=A{\vec{s}}_t$

$A$ is the transition matrix where $a_{ij}=P(S_{t+1}=q_i|S_t=q_j)$

Note that the columns of the transition matrix must add up to 1. The probabilities of all the possible states given a current state must add up to 1 by probability axioms.

---

References

• @lancaster2004
• @sucar2020