Maximum Likelihood Estimation

Setting of the problem

1. Samples are i.i.d from the density
2. is of the form , in which the parameters are unknown, deterministic vectors. We need to estimate from the given samples.

Solution

We have the likelihood function,

Let’s simplify this by taking the ,

We can define the optimal as,

Examples

For Gaussian distribution, the ML estimation for mean and variance is

But remember that the value of here is biased. The unbiased variance estimator is,