The Student Room Group

Likelihood functions

I have vaguely been learning about likelihood functions and was wondering if there is a method for converting them to probability density functions, specifically for the binomial distribution (for example to calculate confidence intervals). I cannot seem to find any suggestion that this is possible or impossible. Any help?
(edited 4 years ago)
Original post by Knortfoxx
I have vaguely been learning about likelihood functions and was wondering if there is a method for converting them to probability density functions, specifically for the binomial distribution (for example to calculate confidence intervals). I cannot seem to find any suggestion that this is possible or impossible. Any help?


Since likelihood functions are derived from probability distributions, you can simply reverse the process and recover the probability density from a likelihood; but this would be rather pointless and is not, I think, what you mean. In general, the answer to your question is no; there are plenty of likelihood functions that integrate to infinity, and therefore cannot be normalized to be a probability density. (Take the uniform distribution on (0,θ)\left(0, \theta\right), for example).

The properties of likelihood functions lead you off in another direction entirely. Although likelihood functions are not themselves probability distributions, the logarithm of a likelihood ratio is. So if you take the likelihood evaluated at one value of the parameter, divide this by the value of the likelihood at another parameter, and take the logarithm, you do get a random variable. You can then use this fact to either set up a statistical test, or to derive (likelihood based) confidence intervals. This turns out to be a very good thing to do, as by the Neyman-Pearson lemma this statistical test is the most powerful test you can construct (i.e. it is the test that makes the best use of the data you have available).

In fact the log likelihood ratio is asymptotically distributed as a χ2\chi^2 random variable, making these tests very easy to carry out in practice. The likelihood ratio test is ubiquitous in higher statistics!
Original post by Gregorius
Since likelihood functions are derived from probability distributions, you can simply reverse the process and recover the probability density from a likelihood; but this would be rather pointless and is not, I think, what you mean. In general, the answer to your question is no; there are plenty of likelihood functions that integrate to infinity, and therefore cannot be normalized to be a probability density. (Take the uniform distribution on (0,θ)\left(0, \theta\right), for example).

The properties of likelihood functions lead you off in another direction entirely. Although likelihood functions are not themselves probability distributions, the logarithm of a likelihood ratio is. So if you take the likelihood evaluated at one value of the parameter, divide this by the value of the likelihood at another parameter, and take the logarithm, you do get a random variable. You can then use this fact to either set up a statistical test, or to derive (likelihood based) confidence intervals. This turns out to be a very good thing to do, as by the Neyman-Pearson lemma this statistical test is the most powerful test you can construct (i.e. it is the test that makes the best use of the data you have available).

In fact the log likelihood ratio is asymptotically distributed as a χ2\chi^2 random variable, making these tests very easy to carry out in practice. The likelihood ratio test is ubiquitous in higher statistics!

While that makes a lot of sense, is it technically possible to simply scale a likelihood function (that doesn't integrate to infinity) and have a probability distribution?
Original post by Theloniouss
While that makes a lot of sense, is it technically possible to simply scale a likelihood function (that doesn't integrate to infinity) and have a probability distribution?


If a function integrates to one (or if it has a finite integral that can be normalized to one) then you can consider it to be a probability distribution. But the interesting question is whether this probability distribution relates in some probabilistic sense to what you started with. There are two answers to this, depending on which school of probability interpretation you follow:

(1) If you follow the frequentist school (where probabilities are considered to be limits of repeated experiments) then the answer is a flat "no". The reason for this is that in frequentist probability, when you write down a probability distribution p(xθ)p(x | \theta) that depends on a parameter θ \theta , that parameter is taken to be a (possibly unknown) fixed parameter. It's not a random variable. So when you write L(θx)=p(xθ)L(\theta | x) = p(x | \theta) you're not making a probability statement, as you can only make probability statements about random variables.

(2) On the other hand, if you're a Bayesian, then everything in sight is considered to be a random variable, including that parameter θ\theta. So here you can make probability statements about it. How would we do this? You have Bayes' theorem:

p(θx)p(x)=p(xθ)p(θ) \displaystyle p(\theta | x) p(x) = p(x | \theta) p(\theta)

If you divide both sides of the equation by p(x)p(x), then you have a probability distribution built out of the likelihood function. But note: to make it into a probability distribution in a consistent way, we have to multiply it by that term p(θ)p(\theta), which is called the prior probability of θ\theta.

Quick Reply

Latest