A support vector machine written with numpy: hinge loss, mini-batch subgradient descent, no solver. Three features, since three dimensions is the most you can still draw. The point is to see the margin, not only to score it.
The data is 500 synthetic points with 10% of the labels flipped on purpose, which puts the ceiling on accuracy
somewhere near 90% and makes the soft margin do real work. On the held-out 20%, the hand-rolled solver gets
85.0%. scikit-learn's SVC with a linear kernel, solving the same objective as a quadratic program, gets 88.0%
on the same split, and the two planes sit 19.6 degrees apart while agreeing on 91% of test predictions.
The gap is the epoch budget rather than the method. Twenty times the epochs closes it to 12.7 degrees and 87.0% while the loss moves by one percent: the objective is nearly flat near the bottom, so the last stretch is slow.
Sweeping C from 0.1 to 50 pulls the margin from 33 units wide down to 1.9. At the wide end the regularizer has
flattened the weights to near zero and the model has stopped committing to anything.
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
jupyter lab svm.ipynbEverything is seeded, so a clean run reproduces the numbers above and rewrites the figures in images/.