Need a bit of help with how this Markov chain calculation works out.
On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.
Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.
Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),
P(X3=5)=k=1∑5p1kpk5(2) however for P(X4=5),
P(X4=5)=k=1∑5p1k(2)pk5(2)And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.