Gram-Schmidt Process

The Gram-Schmidt process is an algorithm that constructs an orthogonal basis from a non-orthogonal basis.

For example, consider the basis $\{a_1,a_2\}$ for the subspace $V=\text{span}\{a_1,a_2\}$, where

$$a_1=\left[\begin{array}{c}1\\ -1\end{array}\right],a_2=\left[\begin{array}{c}4\\ 6\end{array}\right]$$

The Gram-Schmidt process involves 2 steps:

1. Set $v_1=a_1$
2. Find a vector $v_2$ in $V$ that’s orthogonal to $v_1$. This can be achieved by computing

$$\begin{array}{rl}{v}_2& =a_2-{\text{proj}}_{v_1}{a}_2\\ & =a_2-\frac{a_2\cdot v_1}{v_1\cdot v_1}{v}_1\end{array}$$

Notice that $v_2$ is written as a linear combination of $a_1(=v_1)$ and $a_2$. This means that $v_2\in V$ and $v_2\perp v_1$. Moreover, $\text{span}\{a_1,a_2\}=V=\text{span}\{v_1,v_2\}$. The set $B=\{v_1,v_2\}$ is then an orthogonal basis for $V$.