Graphs and Markov chains
J. V. Greenman · The Mathematical Gazette · 1977
In its simplest form the theory of Markov chains states that, if the ( i,j ) element of matrix M is the probability p ij of a one-step transition from state i to state j , then the ( ij ) element of the matrix M k gives the probability of transition from state i to state j in k steps. Since the probabilities of transition to the different states add to 1, the sum of the elements in any row of M is necessarily equal to 1. (Readers are warned that in some elementary treatments the alternative convention is adopted of writing p ij in the ( j,i ) position, so that the column sums are 1, in order that the probability vectors should be column matrices, which are more familiar in school mathematics.)