You are Here: Home >< Maths

# Quick markov question Watch

Announcements
1. I am doing a question on markov chains but I am already stuck on how to set the transition matrix up:

A Quiz machine is in a pub, the probability of winning depends only on the two previous results.

Result at n-2: L L W W
Result at n-1: L W L W
Prob of win at n:a 0.5 0.5 1-b

create a transition matrix of a 4 state markov chain. To do this take the states at time n to be the four possible pairs of results at n-1 and n?
2. I'm afraid I can't remember how to write matrices in latex so will have to just give a quick reply.
The four states that you are interested in are WW, WL, LW and LL

If your previous state was WW then the probability of LW or LL after the next go is clearly 0, the prob of WW is 1-b (the prob that you win the next go given WW) and the prob is WL is b (the prob that you lose the next go given WW).

Similar approach for the others will allow you to complete your matrix- you should have two 0 probabilities in each row as two of the outcomes will always be impossible.
3. Ah so like this for LL LW WL WW:

?
4. Exactly, but it should be the other way round

The row represents your current state and the columns represent the state after the next turn.

TSR Support Team

We have a brilliant team of more than 60 Support Team members looking after discussions on The Student Room, helping to make it a fun, safe and useful place to hang out.

This forum is supported by:
Updated: December 4, 2010
Today on TSR

### Things NOT to do at uni

Have you done any of these?

### How much revision have yr12s done?

Discussions on TSR

• Latest
• ## See more of what you like on The Student Room

You can personalise what you see on TSR. Tell us a little about yourself to get started.

• Poll
Useful resources

### Maths Forum posting guidelines

Not sure where to post? Read the updated guidelines here

### How to use LaTex

Writing equations the easy way

### Study habits of A* students

Top tips from students who have already aced their exams

## Groups associated with this forum:

View associated groups
Discussions on TSR

• Latest
• ## See more of what you like on The Student Room

You can personalise what you see on TSR. Tell us a little about yourself to get started.

• The Student Room, Get Revising and Marked by Teachers are trading names of The Student Room Group Ltd.

Register Number: 04666380 (England and Wales), VAT No. 806 8067 22 Registered Office: International House, Queens Road, Brighton, BN1 3XE

Reputation gems: You get these gems as you gain rep from other members for making good contributions and giving helpful advice.