Markov Chain

$c_n$ is the cumulative number of rolls of a die up to time n, with state space {0,1,2,...}.
$c_0 = 0$.

I need to find out the probability that 9 or 10 get mentioned (in all the different paths).

So I started thinking about the no. of ways 9 can get mentioned for up to 9 rolls. I.e. number of ways in 9 rolls is 9, in 8 rolls is 8, I'm not sure how to carry this process on and find a succinct way of working it out.

As no one else has responded:

May not be able to help, but as it stands I can't even follow your question.

It reads to me as if you roll a die (presumably standard 6-sided), and the cumulative number of rolls up to time n, is simply n rolls. I doubt that's what you intend, but it's the only way I can interpret what you'lve written.

Then you talk about number of ways 9 can get mentioned. What does "9 gets mentioned" mean?

$c_n$ is the cumulative score, of rolls up to time n.

So if we have paths of cumulative scores $c_n(\omega_i)$, for instance I've considered, the path
$c_1(\omega_1) =1, c_2(\omega_1) =2, c_3(\omega_1) =3, \cdots c_9(\omega_1)=9, \cdots$ that is, getting a score of 1, for the first 9 rolls, then after that it doesn't matter. I'm trying to work out the probability of 9 or 10 appears in any path $\omega_i$ (the different ways the scores can appear)
I know that at least two rolls are required. that is $c_1(\omega_i) = 6$ followed by $c_1(\omega_i) = 3$ or the other way around, for some path $\omega_i$

Can't see a particularly easy way to do it. Matlab would help.

If you're going to do it via transition matrices, then:

Since you're not interested in anything greater than 10, your states could be {0,1,...,9,10,>10}

Then the probability would be the sum of the third & second to last two elements of the state vector, from:

$(T^9+T^8+... T)(1,0,...0)^T$ or the other way around - vaguely recall there was something different about Markov transition matrices

But you'll need to adjust for getting 9 at stage n and 10 at stage n+1, as these would be double counted.

Don't know if that's any use to use,...

My knowledge of Markov chains is very much "practical" (as in ways to solve maths puzzles and similar rather than theoretial), but I think you can do a little better here.

Make the states 0, 1, 2, 3, ..., 8 (as before), and then "got 9 or 10" and then "skipped 9 or 10" (which you might as well still call > 10). So "9 or 10" and ">10" are both 'stuck states' (is the formal term absorbing?) (i.e. p(9 or 10 -> 9 or 10) = p(>10 -> >10) = 1). Building the transition matrix M is going to be a little tedious, but your final result will just be the "9 or 10" element of M^10 (1 0 0 0 ... 0)^T.

$c_n$ is the cumulative number of rolls of a die up to time n, with state space {0,1,2,...}.
$c_0 = 0$.

I need to find out the probability that 9 or 10 get mentioned (in all the different paths).

So I started thinking about the no. of ways 9 can get mentioned for up to 9 rolls. I.e. number of ways in 9 rolls is 9, in 8 rolls is 8, I'm not sure how to carry this process on and find a succinct way of working it out.

Tricky little question this, if you're to avoid awkward algebra. One of the wrinkles here is that if you start counting trajectories that include your targets 9 or 10, then it's not entirely clear what the denominator should be to get a probability. So, I'd calculate probabilities directly using a decomposition of the problem. I'll outline a simpler version and leave the rest to you!

The probability of the chain ever hitting 9 is just the sum of the probabilities of hitting 9 after exactly n throws, for n = 1 though to infinity. But most of these terms are zero, and we're left with the sum from 2 to 9. So, calculate (or just look up)

(i) The probability of landing on 9 in exactly 2 throws.
(ii) The probability of landing on 9 in exactly 3 throws.
(iii) etc.

Then add them up.

I'm not sure how to carry this process on and find a succinct way of working it out.

I have to admit, I'm not seeing a pleasant way of calculating, say, the probability of landing on 9 in exactly 5 throws.

It's one of those standard tortures about generating functions for undergraduates in advanced elementary probability theory! See here for the details.

Using your initial transition matrix method, I get the same - 0.523 (3 dec.pl), or $\dfrac{52...22}{6^9}$

Not to flog a dead horse, but it seems that page pretty much wimps out on doing any calculations with more than 4 dice (the most likely roll for n dice is fairly obvious by symmetry anyhow).

The formula wanted is the one for "c" (probability of scoring p, throwing n s-sided dice), just a few lines into the working. It ends up being a simple calculation.