Running Averages

Running averages is a computational strategy for estimating a possibly weighted average of a sequence of data.

Let our data sequence be $a_1,a_2,\dots$ then we define a sequence of running average values, $A_0,A_1,A_2,\dots$ using the equations

$$\begin{array}{rl}{A}_0& =0\\ A_t& ={\gamma }_t{A}_{t-1}+(1-{\gamma }_t)a_t\end{array}$$

where ${\gamma }_t\in (0,1)$. If ${\gamma }_t$ is constant, then this is a moving average, in which:

$$\begin{array}{rl}{A}_T& =\gamma A_{T-1}+(1-\gamma )a_T\\ & =\gamma (\gamma A_{T-2}+(1-\gamma )a_{T-1})+(1-\gamma )a_T\\ & =\sum _{t=0}^T{\gamma }^{T-t}(1-\gamma )a_t\end{array}$$

So, we are essentially tracking of all the $a$ values that we have seen. But the ones that are further away have less influence; you can see that inputs at closer to the end of the sequence have more effect on $A_t$ than early inputs. If, instead, we set ${\gamma }_t=(t-1)/t$, then we get the actual average.