You are Here: Home >< Maths

# Markov Chains: Urns and Molecules watch

1. M molecules are distributed in 2 urns, urn A and urn B. Consider the following discrete Markov chain with M +1 states where the states are enumerated by j = 0, 1, ..., M and j is the number of molecules in urn A. A transition fromone state into another occurs by choosing a molecule at random (from either urn)and putting it in the other urn.
(a) Write down the transition matrix for general M.
(b) Consider the case M = 2. Is the chain ergodic? Explain.

I am struggling to get started on this question. I think we the transition matrix is going to be MxM. Do I need to populate this matrix with values for question a?

I am also kind of confused about the transitional probabilities in this matrix. For example what does mean? I think it means that A starts with 0 molecules, and something has happened to make A have 1 molecule? But this molecule must have been removed from urn B and put into urn A. But how do we know urn B has molecules for this to happen.
Attached Images

2. (Original post by aspiring_doge)
M molecules are distributed in 2 urns, urn A and urn B. Consider the following discrete Markov chain with M +1 states where the states are enumerated by j = 0, 1, ..., M and j is the number of molecules in urn A. A transition fromone state into another occurs by choosing a molecule at random (from either urn)and putting it in the other urn.
(a) Write down the transition matrix for general M.
(b) Consider the case M = 2. Is the chain ergodic? Explain.

I am struggling to get started on this question. I think we the transition matrix is going to be MxM. Do I need to populate this matrix with values for question a?

I am also kind of confused about the transitional probabilities in this matrix. For example what does mean? I think it means that A starts with 0 molecules, and something has happened to make A have 1 molecule? But this molecule must have been removed from urn B and put into urn A. But how do we know urn B has molecules for this to happen.
Let's see if we can get you started. This markov chain has M+1 states, labelled by the number of molecules in urn A. Let us work out the state transition probabilities for state j going to state j+1 and for state j going to state j-1. If you think about it for a minute, you'll see that these are the only possible state transitions, so the collection of them fully specifies the state transition matrix.

If you are in state j, then there are j molecules in urn A and M-j in urn B. If you choose a molecule at random, then the probability that it is in urn A is (j/M) and the probability that it is in urn B is (j-M)/M. If the molecule is in urn A, then the state transition that occurs is j to j-1; if it is in urn B, the state transition that occurs is j to j+1. Can you see that this gives you the required state transition probabilities?
3. (Original post by Gregorius)
Let's see if we can get you started. This markov chain has M+1 states, labelled by the number of molecules in urn A. Let us work out the state transition probabilities for state j going to state j+1 and for state j going to state j-1. If you think about it for a minute, you'll see that these are the only possible state transitions, so the collection of them fully specifies the state transition matrix.

If you are in state j, then there are j molecules in urn A and M-j in urn B. If you choose a molecule at random, then the probability that it is in urn A is (j/M) and the probability that it is in urn B is (j-M)/M. If the molecule is in urn A, then the state transition that occurs is j to j-1; if it is in urn B, the state transition that occurs is j to j+1. Can you see that this gives you the required state transition probabilities?
I think I follow your explanation. So the transitional probability is the number of molecules in urn A from state j to j+1 or j-1?

Also why is there M+1 states? I thought there are M molecules so how is the possible?
4. (Original post by aspiring_doge)
I think I follow your explanation. So the transitional probability is the number of molecules in urn A from state j to j+1 or j-1?
The transition probabilities are not the number of molecules - but the proportion of them. If you go from state j to state j-1 then you need to have selected a molecule from urn A; if you go from j to j+1 then you need to have selected from urn B.

Also why is there M+1 states? I thought there are M molecules so how is the possible?
There are M molecules, so you can have from 0 to M molecules in urn A, therefore M+1 states.
5. (Original post by Gregorius)
.....
Thanks! I understand it now.

### Related university courses

TSR Support Team

We have a brilliant team of more than 60 Support Team members looking after discussions on The Student Room, helping to make it a fun, safe and useful place to hang out.

This forum is supported by:
Updated: March 14, 2016
Today on TSR

### My dream is to become a WAG

I wont settle for anything else!

Poll
Useful resources

### Maths Forum posting guidelines

Not sure where to post? Read the updated guidelines here

### How to use LaTex

Writing equations the easy way

### Study habits of A* students

Top tips from students who have already aced their exams