The main idea is to create many weak classifiers and then do voting.
The simple idea is this:
- We try to use each features to see which one classifies best.
- For the misclassified examples using the best classifier, we put more weight into it.
- Then, we sample a new dataset and prioritizing the misclassified examples.
- We repeat to step 1 until a set amount of step.
- During inference, we do voting using the “best trees” for each round.