Kuhn Tucker theorem

At the saddle point, the following equalities hold:

Primal feasibility

Inside the room.

$$\begin{array}{rl}& \min \Phi (\mathbf{w})=\frac{1}{2}(\mathbf{w}\cdot \mathbf{w})\text{w.r.t }\mathbf{w},\\ & \text{ s.t. }{y}_i[({\mathbf{x}}_i\cdot \mathbf{w})+b]\ge 1,i=1,2,\dots ,l\end{array}$$

Dual room

Nonnegative wall forces ${\alpha }_i\ge 0$.

$$\begin{array}{rl}& \underset{\mathbf{\alpha }}{\max }W(\mathbf{\alpha })=\sum _{i=1}^l{\alpha }_i-\frac{1}{2}\sum _{i,j}^l{\alpha }_i{\alpha }_j{y}_i{y}_j({\mathbf{x}}_i\cdot {\mathbf{x}}_j)\\ & \text{s.t. }{\alpha }_i\ge 0,i=1,\dots ,l\\ & \text{and }\sum _{i=1}^l{\alpha }_i{y}_i=0\end{array}$$

Stationary

Forces balance.

$$\mathbf{w}=\sum _i{\alpha }_i{y}_i{x}_i,\sum _i{\alpha }_i{y}_i=0$$

Complementary slackness

Only touching walls push.

$${\alpha }_i^0\left\{\begin{array}{c}[{\mathbf{x}}_i\cdot {\mathbf{w}}_0+b_0]y_i-1\end{array}\right\}=0,i=1,\dots ,l$$

Since $y_i[({\mathbf{x}}_i\cdot \mathbf{w})]\ge 1$, we have ${\alpha }_i\ne 0$ only for cases where $y_i[({\mathbf{x}}_i\cdot \mathbf{w})+b]=1$, which are the support vectors.