Covariance Matrix

A covariance matrix is a square matrix that represents that covariance between each pair of elements in a given multivariate random variable.

For a 2D random variable, the covariance matrix is

$$\Sigma =\left[\begin{array}{cc}{\sigma }_{xx}& {\sigma }_{xy}\\ {\sigma }_{yx}& {\sigma }_{yy}\end{array}\right]=\left[\begin{array}{cc}{\sigma }_x^2& {\sigma }_{xy}\\ {\sigma }_{yx}& {\sigma }_y^2\end{array}\right]=\left[\begin{array}{cc}\text{VAR}(x)& \text{COV}(x,y)\\ \text{COV}(y,x)& \text{VAR}(y)\end{array}\right]$$

The off-diagonal entries are equal since $\text{COV}(x,y)=\text{COV}(y,x)$. If $x$ and $y$ are uncorrelated, the off-diagonal entries of the covariance matrix are zero.

For a $n$-dimensional random variable, the covariance matrix is given by

$$\Sigma =\left[\begin{array}{cccc}{\sigma }_1^2& {\sigma }_{12}& \dots & {\sigma }_{1n}\\ {\sigma }_{21}& {\sigma }_2^2& \dots & {\sigma }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\sigma }_{n1}& {\sigma }_{n2}& \dots & {\sigma }_{nn}\end{array}\right]$$

Properties

1. The