Hey there! Sign in to join this conversationNew here? Join for free
    • Thread Starter
    Offline

    7
    ReputationRep:
    Need a bit of help with how this Markov chain calculation works out.

    On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

    Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

    Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

    P(X_3=5) = \displaystyle\sum_{k=1}^{5} p_{1k} p_{k5}^{(2)}

    however for P(X4=5),

    P(X_4=5) = \displaystyle\sum_{k=1}^{5} p_{1k}^{(2)} p_{k5}^{(2)}

    And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
    • Very Important Poster
    Online

    21
    ReputationRep:
    Very Important Poster
    (Original post by Blue7195)
    Need a bit of help with how this Markov chain calculation works out.

    On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

    Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

    Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

    P(X_3=5) = \displaystyle\sum_{k=1}^{5} p_{1k} p_{k5}^{(2)}

    however for P(X4=5),

    P(X_4=5) = \displaystyle\sum_{k=1}^{5} p_{1k}^{(2)} p_{k5}^{(2)}

    And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
    I'm only just studying Markov chains myself, but it may be because 3 is an odd amount of steps so you can only use the one step matrix then the two step, whereas with 4 you can go in two steps and then another two steps as it's even.
    Offline

    13
    ReputationRep:
    (Original post by Blue7195)
    Need a bit of help with how this Markov chain calculation works out.

    On state space S = {1,2,3,4,5}, with transition probability matrix P and 2-step transition probability matrix P^2.

    Initial probability distribution is P(X0=1) = 1 and P(X0=i) = 0 for i=2,3,4,5.

    Question is asking for find P(X3=5) and P(X4=5), and in the answers is states that, for P(X3=5),

    P(X_3=5) = \displaystyle\sum_{k=1}^{5} p_{1k} p_{k5}^{(2)}

    however for P(X4=5),

    P(X_4=5) = \displaystyle\sum_{k=1}^{5} p_{1k}^{(2)} p_{k5}^{(2)}

    And I am very confused as to how they managed to get these values, how would you know whether its the 2-step transition matrix you are using or the first? How can you tell? Thank you for any help.
    SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix P^{(n)} of a multi-step transition  X_i \rightarrow X_{i+n} is simply given by a power:

     \displaystyle P^{(n)} = P^n

    where P is the 1-step transition matrix. This immediately means that


     \displaystyle P^{(n)} = P^a P^b

    wherever  a + b = n for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

     \displaystyle \sum_{k=1}^{M} P_{i, k}^{a} P_{k, j}^b = \sum_{k=1}^{M} P_{i, k}^{(a)} P_{k, j}^{(b)}

    for any positive integral a and b that sum to n.
    • Thread Starter
    Offline

    7
    ReputationRep:
    (Original post by Gregorius)
    SeanFM is basically spot on here, but let me just add a bit. The Markov property of a Markov chain means that the transition matrix P^{(n)} of a multi-step transition  X_i \rightarrow X_{i+n} is simply given by a power:

     \displaystyle P^{(n)} = P^n

    where P is the 1-step transition matrix. This immediately means that


     \displaystyle P^{(n)} = P^a P^b

    wherever  a + b = n for positive integers a & b. So, unraveling this, it means that if you want to get from state i to state j after n steps in an M state Markov chain you can do it via

     \displaystyle \sum_{k=1}^{M} P_{i, k}^{a} P_{k, j}^b = \sum_{k=1}^{M} P_{i, k}^{(a)} P_{k, j}^{(b)}

    for any positive integral a and b that sum to n.
    Thank you so much!
 
 
 
Poll
Who is your favourite TV detective?
Useful resources

Make your revision easier

Maths

Maths Forum posting guidelines

Not sure where to post? Read the updated guidelines here

Equations

How to use LaTex

Writing equations the easy way

Student revising

Study habits of A* students

Top tips from students who have already aced their exams

Study Planner

Create your own Study Planner

Never miss a deadline again

Polling station sign

Thinking about a maths degree?

Chat with other maths applicants

Can you help? Study help unanswered threads

Groups associated with this forum:

View associated groups

The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

Write a reply...
Reply
Hide
Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.