Just one sec...
Hey! Sign in to get help with your study questionsNew here? Join for free to post

Markov chain help

Announcements Posted on
Take our short survey, £100 of Amazon vouchers to be won! 23-09-2016
    • Thread Starter
    Offline

    3
    ReputationRep:
    Just a quick couple of questions:
    1
    What is meant by the 'equilibrium distribution vector' concerning a Markov chain of probabilities?

    Take this example:

    1---------2----------3

    "An object moves along the graph shown above, so that at each step it moves with equal probability to a neighbouring point."

    a) Write down the transition matrix P for this Markov process (i think i've got this right, so i'm not bothered about this one)
    b) Calculate the equilibrium distribution vector for this Markov process.

    2
    How do you calculate the limit as n -> infinity of A^n for a square matrix A?
    Offline

    2
    ReputationRep:
    From my rather hazy memory, an equilibrium (or invariant) distribution vector is a stochastic row vector \pi with \pi P = \pi.
    • Thread Starter
    Offline

    3
    ReputationRep:
    (Original post by generalebriety)
    From my rather hazy memory, an equilibrium (or invariant) distribution vector is a stochastic row vector \pi with \pi P = \pi.
    Thanks, that makes sense now.
    Online

    3
    ReputationRep:
    In general, finding \lim_{n\to\infty} A^n is tricky (even assuming the limit exists).

    But barring a few pathological cases(*), if M is a stochastic matrix, and \pi is an invariant vector for M, then M^n x \to \pi \for all x. Which means M^n converges to the matrix with all columns equal to \pi.

    (*) I think it always works as longs as M has no elements of size 1, but that's intuition and hope speaking rather than detailed knowledge.
 
 
 
Write a reply…

Reply

Submit reply

Register

Thanks for posting! You just need to create an account in order to submit the post
  1. this can't be left blank
    that username has been taken, please choose another Forgotten your password?
  2. this can't be left blank
    this email is already registered. Forgotten your password?
  3. this can't be left blank

    6 characters or longer with both numbers and letters is safer

  4. this can't be left empty
    your full birthday is required
  1. Oops, you need to agree to our Ts&Cs to register
  2. Slide to join now Processing…

Updated: February 12, 2009
TSR Support Team

We have a brilliant team of more than 60 Support Team members looking after discussions on The Student Room, helping to make it a fun, safe and useful place to hang out.

Poll
Who will be the next permanent England boss?
Useful resources

Make your revision easier

Maths

Maths Forum posting guidelines

Not sure where to post? Read here first

Equations

How to use LaTex

Writing equations the easy way

Student revising

Study habits of A* students

Top tips from students who have already aced their exams

Study Planner

Create your own Study Planner

Never miss a deadline again

Polling station sign

Thinking about a maths degree?

Chat with other maths applicants

Can you help? Study help unanswered threads

Groups associated with this forum:

View associated groups
Study resources

The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.