Bayesian Estimation

Setting

1. Samples are i.i.d from density
2. is of the form where are random vectors with prior density of
   • Remember that it is different from Maximum Likelihood Estimation where the are unknown deterministic vectors.

Solution

We need to do minimal risk estimation. Loss Function of an Estimation

Expected Risk

Risk conditional on

Empirical Risk on all training data

Bayesian Estimation

Example

With the loss function , the optimal value of is,

But we still don’t know what is or .

1. We set the prior
2. Write down the joint density (conditional density):

3. Calculate the posterior

We do Bayesian Estimation iteratively. When we obtain a new sample,