Concept

Maximal Margin Separator Problem

To restate our problem: Given a linear separable dataset {xˉi,y(i)}in\{\bar{x}_i, y^{(i)}\}_i^n, and linear decision boundary defined by normal, offset θˉ,b\bar{\theta}, b we would like to solve the following optimization problem with constraints:

maxθˉ,bθˉxˉi+bθˉ\max_{\bar{\theta}, b} \frac{\bar{\theta} \cdot \bar{x}_i + b}{||\bar{\theta}||} subject to (y(i)(θˉxˉi+b))>0  i\text{subject to } (y^{(i)} (\bar{\theta} \cdot \bar{x}_i + b)) > 0 \ \ \forall i

This problem simplifies to minθˉ,b12θˉ2\min_{\bar{\theta}, b} \frac{1}{2}||\bar{\theta}||^2 subject to (y(i)(θˉxˉi+b))1  i\text{subject to } (y^{(i)} (\bar{\theta} \cdot \bar{x}_i + b)) \geq 1 \ \ \forall i

The nitty-gritty of this proof can be seen in this nodes source, though it might be worth writing out in full here. (#TODO Connect the linear algebra relation node to this node for explanation).

0

2

Updated 2020-02-25

Tags

Data Science