Linear Classifier

Hypothesis class of machine learning models.

A simple linear classifier in $d$ dimensions is defined by the parameters $\theta \in {\mathbb{R}}^d$ and ${\theta }_0\in \mathbb{R}$:

$$h(x;\theta ,{\theta }_0)=\text{sign}({\theta }^{T}x+{\theta }_0)=\{\begin{array}{l}+1\text{ if }{\theta }^{T}x+{\theta }_0>0\\ -1\text{ otherwise}\end{array}$$

Example

Let $\theta =\left[\begin{array}{c}-1\\ 1.5\end{array}\right]$ and ${\theta }_0=3$. Classify the points $x^{(1)}=\left[\begin{array}{c}3\\ 2\end{array}\right]$ and $x^{(2)}=\left[\begin{array}{c}4\\ -1\end{array}\right]$.

Algorithms:

• Perceptron
• Averaged Perceptron