Gaussian Maximum Likehood Estimation

Recall that a Gaussian is expressed as

$$\mathcal{N}(x|\mu ,{\sigma }^2)=\frac{1}{(2\pi {\sigma }^2{)}^{1/2}}\exp \left\{-\frac{1}{2{\sigma }^2}(x-\mu {)}^2\right\}$$

The log of the likelihood function can be written as

$$\begin{array}{rl}\ln p(x|\mu ,{\sigma }^2)& =\ln \left(\prod _{i=1}^N\mathcal{N}(x_n|\mu ,{\sigma }^2)\right)\\ & =-\frac{1}{2{\sigma }^2}\sum _{n=1}^N(x_n-\mu {)}^2-\frac{N}{2}\ln {\sigma }^2-\frac{N}{2}\ln (2\pi )\end{array}$$

This can then be maximized by setting partial derivatives with respect to $\mu$ and ${\sigma }^2$ to zero and solving.

For $\mu$:

$$\begin{array}{rl}\frac{\partial }{\partial \mu }\ln (p|x,\mu ,{\sigma }^2)& =\frac{\partial }{\partial \mu }\left(-\frac{1}{2{\sigma }^2}\sum _{n=1}^N(x_n-\mu {)}^2\right)\\ & =\frac{1}{{\sigma }^2}\sum _{n=1}^N(x_n-\mu )\end{array}$$

Setting to 0 and solving:

$$\begin{array}{rl}\sum _{n=1}^N{x}_n-N\mu & =0\\ \boxed{{\mu }_{\text{ML}}=\frac{1}{N}\sum _{n=1}^N{x}_n}& \end{array}$$

which is the sample mean – the mean of the observed values.

For ${\sigma }^2$:

$$\frac{\partial }{\partial {\sigma }^2}\ln p(x|\mu ,{\sigma }^2)=\frac{\partial }{\partial {\sigma }^2}\left(-\frac{1}{2{\sigma }^2}\sum _{n=1}^N(x_n-\mu {)}^2-\frac{N}{2}\ln {\sigma }^2\right)$$

This simplifies to:

$$-\frac{N}{2{\sigma }^2}+\frac{1}{2({\sigma }^2{)}^2}\sum _{n=1}^N(x_n-\mu {)}^2$$

Setting to zero and solving for ${\sigma }^2$:

$$\boxed{{\sigma }_{\text{ML}}^2=\frac{1}{N}\sum _{n=1}^N(x_n-{\mu }_{ML}{)}^2}$$

This is the sample variance measured with respect to the sample mean $\mu$.