Concept icon
Concept

Maximal Margin Separator Problem

To restate our problem: Given a linear separable dataset {xˉi,y(i)}in\{\bar{x}_i, y^{(i)}\}_i^n, and linear decision boundary defined by normal, offset θˉ,b\bar{\theta}, b we would like to solve the following optimization problem with constraints:

max⁡θˉ,bθˉ⋅xˉi+b∣∣θˉ∣∣\max_{\bar{\theta}, b} \frac{\bar{\theta} \cdot \bar{x}_i + b}{||\bar{\theta}||} subject to (y(i)(θˉ⋅xˉi+b))>0∀i\text{subject to } (y^{(i)} (\bar{\theta} \cdot \bar{x}_i + b)) > 0 \forall i

This problem simplifies to min⁡θˉ,b12∣∣θˉ∣∣2\min_{\bar{\theta}, b} \frac{1}{2}||\bar{\theta}||^2 subject to (y(i)(θˉ⋅xˉi+b))≥1∀i\text{subject to } (y^{(i)} (\bar{\theta} \cdot \bar{x}_i + b)) \geq 1 \forall i

The nitty-gritty of this proof can be seen in this nodes source, though it might be worth writing out in full here. (#TODO Connect the linear algebra relation node to this node for explanation).

0

2

Concept icon
Updated 2020-02-25

Tags

Data Science