Naive Bayes Classifier

Remember that for classification, we want to find:

$$P({\omega }_i|\mathbf{x})=\frac{P(\mathbf{x}|{\omega }_i)P({\omega }_i)}{P(\mathbf{x})}$$

To classify, we just pick the class with the maximum posterior probability:

$$\hat{\omega }=\arg \underset{i}{\max }P({\omega }_i|\mathbf{x})=\arg \underset{i}{\max }P(\mathbf{x}|{\omega }_i)P({\omega }_i)$$

The naive assumption assumes all features are conditionally independent given the class.

$$P(\mathbf{x}|{\omega }_i)=P(x_1,x_2,\dots ,x_d|{\omega }_i)=\prod _{j=1}^{d}P(x_j|{\omega }_i)$$

To make it less naive, we have a smarter Bayesian Decision Theory Framework with Costs.