You are Here: Home >< Maths

# Markov Chains watch

1. Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
2. (Original post by Blue7195)
Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
I'm only just studying Markov chains myself, but it may be because 3 is an odd amount of steps so you can only use the one step matrix then the two step, whereas with 4 you can go in two steps and then another two steps as it's even.
3. (Original post by Blue7195)
Need a bit of help with how this Markov chain calculation works out.

On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

however for P(X4=5),

And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix of a multi-step transition is simply given by a power:

where is the 1-step transition matrix. This immediately means that

wherever for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

for any positive integral a and b that sum to n.
4. (Original post by Gregorius)
SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix of a multi-step transition is simply given by a power:

where is the 1-step transition matrix. This immediately means that

wherever for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

for any positive integral a and b that sum to n.
Thank you so much!

TSR Support Team

We have a brilliant team of more than 60 Support Team members looking after discussions on The Student Room, helping to make it a fun, safe and useful place to hang out.

This forum is supported by:
Updated: April 30, 2016
Today on TSR

### Three reasons you may feel demotivated right now

...and how to stay positive

### Can I get A*s if I start revising now?

Discussions on TSR

• Latest
Poll
Useful resources

### Maths Forum posting guidelines

Not sure where to post? Read the updated guidelines here

### How to use LaTex

Writing equations the easy way

### Study habits of A* students

Top tips from students who have already aced their exams

## Groups associated with this forum:

View associated groups
Discussions on TSR

• Latest

The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE