You are Here: Home >< Maths

Markov Chains

Announcements Posted on
Four hours left to win £100 of Amazon vouchers!! Don't miss out! Take our short survey to enter 24-10-2016
1. Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
2. (Original post by Blue7195)
Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
I'm only just studying Markov chains myself, but it may be because 3 is an odd amount of steps so you can only use the one step matrix then the two step, whereas with 4 you can go in two steps and then another two steps as it's even.
3. (Original post by Blue7195)
Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix of a multi-step transition is simply given by a power:

where is the 1-step transition matrix. This immediately means that

wherever for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

for any positive integral a and b that sum to n.
4. (Original post by Gregorius)
SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix of a multi-step transition is simply given by a power:

where is the 1-step transition matrix. This immediately means that

wherever for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

for any positive integral a and b that sum to n.
Thank you so much!

Register

Thanks for posting! You just need to create an account in order to submit the post
1. this can't be left blank
2. this can't be left blank
3. this can't be left blank

6 characters or longer with both numbers and letters is safer

4. this can't be left empty
1. Oops, you need to agree to our Ts&Cs to register

Updated: April 30, 2016
TSR Support Team

We have a brilliant team of more than 60 Support Team members looking after discussions on The Student Room, helping to make it a fun, safe and useful place to hang out.

This forum is supported by:
Today on TSR

Who is getting a uni offer this half term?

Find out which unis are hot off the mark here

Poll
Useful resources

Maths Forum posting guidelines

Not sure where to post? Read here first

How to use LaTex

Writing equations the easy way

Study habits of A* students

Top tips from students who have already aced their exams

Chat with other maths applicants