Bayesian Decision Theory Framework with Costs

The setup is as follow:

1. Random feature vectors .
2. State space , possible classes.
3. Decision space , where we have possible decisions.
4. Loss function , the cost of deciding when the true state is .

The goal is to minimize the expected loss. We define the conditional risk for one sample as:

Now, we need to calculate the overall risk for all the samples.

Note that here is a decision rule that maps each to a decision. Next, to find the optimal decision rule ,

This might be hard to calculate, so we minimize the integral by minimizing the integrand at each pooint. This is called Minimal Risk Decision.

To calculate , same as the above equation actually.