Hey there! Sign in to join this conversationNew here? Join for free

Quick markov question Watch

    • Thread Starter

    I am doing a question on markov chains but I am already stuck on how to set the transition matrix up:

    A Quiz machine is in a pub, the probability of winning depends only on the two previous results.

    Result at n-2: L L W W
    Result at n-1: L W L W
    Prob of win at n:a 0.5 0.5 1-b

    create a transition matrix of a 4 state markov chain. To do this take the states at time n to be the four possible pairs of results at n-1 and n?

    I'm afraid I can't remember how to write matrices in latex so will have to just give a quick reply.
    The four states that you are interested in are WW, WL, LW and LL

    If your previous state was WW then the probability of LW or LL after the next go is clearly 0, the prob of WW is 1-b (the prob that you win the next go given WW) and the prob is WL is b (the prob that you lose the next go given WW).

    Similar approach for the others will allow you to complete your matrix- you should have two 0 probabilities in each row as two of the outcomes will always be impossible.
    • Thread Starter

    Ah so like this for LL LW WL WW:

    \[ \left( \begin{array}{cccc}

1-a & 0 & 0.5 &0 \\

a & 0 & 0.5& 0\\


0 & 0.5 & 0 & 1-b\end{array} \right)\]


    Exactly, but it should be the other way round

    \[ \left( \begin{array}{cccc}

1-a & a & 0 &0 \\

0 & 0 & 0.5& 0.5\\


0 & 0 & b & 1-b\end{array} \right)\]

    The row represents your current state and the columns represent the state after the next turn.
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • Poll
    Will you be richer or poorer than your parents?
    Useful resources

    Make your revision easier


    Maths Forum posting guidelines

    Not sure where to post? Read the updated guidelines here


    How to use LaTex

    Writing equations the easy way

    Student revising

    Study habits of A* students

    Top tips from students who have already aced their exams

    Study Planner

    Create your own Study Planner

    Never miss a deadline again

    Polling station sign

    Thinking about a maths degree?

    Chat with other maths applicants

    Can you help? Study help unanswered threads

    Groups associated with this forum:

    View associated groups
  • See more of what you like on The Student Room

    You can personalise what you see on TSR. Tell us a little about yourself to get started.

  • The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

    Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

    Quick reply
    Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.