Matrix Basics

Let’s say we have vectors $\vec{u},\vec{v},\vec{w}$ such that

$$\vec{u}=\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right],\vec{v}=\left[\begin{array}{c}0\\ 1\\ -1\end{array}\right],\vec{w}=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]$$

We can create a matrix $A$ with the above vectors in its columns:

$$A=\left[\begin{array}{ccc}1& 0& 0\\ -1& 1& 0\\ 0& -1& 1\end{array}\right]$$

A linear combination $x_1\vec{u}+x_2\vec{v}+x_3\vec{w}=b$ can be expressed as:

$$A\vec{x}=\left[\begin{array}{ccc}1& 0& 0\\ -1& 1& 0\\ 0& -1& 1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]=\left[\begin{array}{c}{x}_1\\ -x_1+x_2\\ -x_2+x_3\end{array}\right]$$

The product of a matrix and a vector is a combination of the columns of the matrix. When we say $x_1\vec{u}+x_2\vec{v}+x_3\vec{w}=b$, we’re thinking about multiplying numbers be vectors. When we say $A\vec{x}=\vec{b}$, we’re thinking about multiplying a matrix (whose columns are $\vec{u},\vec{v},\vec{w}$) by the numbers.

A more important question: given $A$ and $\vec{b}$, for what vectors $\vec{x}$ does $A\vec{x}=\vec{b}$? In the above example, this would mean solving 3 equations as such:

$$A\vec{x}=\left[\begin{array}{ccc}1& 0& 0\\ -1& 1& 0\\ 0& -1& 1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]=\left[\begin{array}{c}{x}_1\\ x_2-x_1\\ x_3-x_2\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\end{array}\right]$$

This is equivalent to solving

$$\begin{array}{r}{x}_1=b_1\\ x_2-x_1=b_2\\ x_3-x_2=b_3\end{array}$$

Since we have $x_1=b_1$, we have $x_2=b_1+b_2$, so our solution is

$$\vec{x}=\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]=\left[\begin{array}{c}{b}_1\\ b_1+b_2\\ b_1+b_2+b_3\end{array}\right]$$

But this is in fact just

$$x=\left[\begin{array}{ccc}1& 0& 0\\ 1& 1& 0\\ 1& 1& 1\end{array}\right]\left[\begin{array}{c}{b}_1\\ b_2\\ b_3\end{array}\right]$$

or $\vec{x}=A^{-1}\vec{b}$. If the matrix $A$ is invertible, we can multiply both sides by $A^{-1}$ to find a unique $\vec{x}$ to solve $A\vec{x}=\vec{b}$.

We can say that $A$ represents a transform $\vec{x}\to \vec{b}$ that has an inverse transform $\vec{b}\to \vec{x}$, such that if $\vec{b}=\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]$ then we also have $\vec{x}=\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]$.

A Second Example

Let’s keep the same columns $\vec{u}$ and $\vec{v}$ but use a different $\vec{w}$ such that we have:

$$C=\left[\begin{array}{ccc}1& 0& -1\\ -1& 1& 0\\ 0& -1& 1\end{array}\right]$$

Then,

$$C\vec{x}=\left[\begin{array}{ccc}1& 0& -1\\ -1& 1& 0\\ 0& -1& 1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]=\left[\begin{array}{c}{x}_1-x_3\\ x_2-x_1\\ x_3-x_2\end{array}\right]$$

Here, the system of equations is circular. Where before $Ax=0$ implied $x=0$, there are non-zero factors $x$ for which $Cx=0$, as this also happens for or any $\vec{x}$ where $x_1=x_2=x_3$. This is a significant difference, as we can’t just multiply both sides of $C\vec{x}=0$ by an inverse to find a non-zero solution $\vec{x}$.

The systems of equations encoded in $C\vec{x}=\vec{b}$ is:

$$\begin{array}{r}{x}_1-x_3=b_1\\ x_2-x_1=b_2\\ x_3-x_2=b_3\end{array}$$

If we add these together we get:

$$0=b_1+b_2+b_3$$

This tells us that $C\vec{x}=b$ has a solution $\vec{x}$ only when the components of $\vec{b}$ sum to 0. In a physical system, this might imply that the system is stable as long as forces are balanced.