Predictive Coding

Can a real brain do backprop? We are constrained by physics and chemistry:

• Synaptic updates can only be based on local info
• Connection weights cannot be copied to other connections

In backprop, the error gradients are somehow propagated down through the network.

There are some architectures that implement something like backprop, but in a biologically plausible way.

In predictive coding, predictions/commands are sent one way through the network, and errors/deviations are sent the other way. A good way to think about this is a military chain of command:

Comparing feedforward networks and predictive coding networks:

In a PC network, each hidden node is split into two parts: an error node and a state/value node.

Let’s represent a whole layer as a single circle:

${\mu }^i$ is the prediction being sent up to layer $i$:

$${\mu }^i=\sigma (x^{i-1})M^{i-1}+{\beta }^i$$

• For now, assume $(W^i{)}^T=M^i$.

The error node ${ϵ}^i$ is the difference between $x^i$ and ${\mu }^i$. It has dynamics:

$$\tau \frac{d{ϵ}^i}{dt}=x^i-{\mu }^i-{\nu }^i{ϵ}^i$$

At equilibrium, we get:

$$\begin{array}{r}{ϵ}^i=\frac{x^i-{\mu }^i}{{\nu }^i}\end{array}$$

The goal for training the PC network is as follows. Given dataset $(X,Y),\theta =\{M^i,W^i{\}}_{i=1,\dots ,n}$:

$$\underset{\theta }{\max }p(Y(X),\theta )$$

where

$$\begin{array}{rl}p(Y(X),\theta )& =p(Y(X)|\theta )p(\theta )\\ & =p(Y|{\mu }^n)p(x^{n-1}|{\mu }^{n-1})\dots p(x^2|{\mu }^2)p(\theta )\end{array}$$

Consider $p(x^i|{\mu }^i)$. Assume $x^i\sim \mathcal{N}({\mu }^i,{\nu }^i)$ is normally distributed:

$$\begin{array}{rl}p(x^i|{\mu }^i)& =\frac{1}{\boxed{}}{e}^{\frac{-||x^i-{\mu }^i|{|}^2}{2{\nu }^i}}\\ -\ln p(x^i|{\mu }^i)& =c+\frac{1}{2{\nu }^i}||x^i-{\mu }^i|{|}^2\\ \therefore -\ln (p(Y(X)),\theta )& \equiv \sum _{i=1}^n\frac{||x^i-{\mu }^i|{|}^2}{2{\nu }^i}\end{array}$$

Hopfield function:

$$F=\frac{1}{2}\sum _{i=1}^n{\nu }^i||{ϵ}^i|{|}^2$$

• Recall that ${ϵ}^i=\frac{x^i-{\mu }^i}{{\nu }^i}$

now we show that the network activity acts to decrease the Hopfield energy.

Consider ${\nabla }_{x^{\ell }}F$, noting that $x^{\ell }$ appears in ${ϵ}^{\ell }$ and ${ϵ}^{l+1}$. Then:

$$\begin{array}{rl}{ϵ}^{\ell }& =\frac{1}{v^{\ell }}(x^{\ell }-{\mu }^{\ell })\\ & =\frac{1}{{\nu }^{l+1}}(x^{l+1}-{\mu }^{l+1})\\ & =\frac{1}{{\nu }^{l+1}}(x^{l+1}-\sigma (x^{\ell })M^{\ell })\\ \therefore {\nabla }_{x^{\ell }}F& ={ϵ}^{\ell }-{\sigma }^{\prime }(x^{\ell })\odot [{ϵ}^{l+1}(M^{\ell }{)}^T]\end{array}$$

Thus, gradient descent gives us:

$$\tau \frac{d{x}^{\ell }}{dt}={\sigma }^{\prime }(x^{\ell })\odot ({ϵ}^{l+1}{W}^{\ell })-{ϵ}^{\ell }$$

Training

To train the network, we clamp the input on both ends:

We then hold those inputs and running the network to equilibrium.

At equilibrium:

$$\frac{d{x}^{\ell }}{dt}=\frac{d{ϵ}^{\ell }}{dt}=0$$

Running to equilibrium allows all the parts of the network to interact. At equilibrium, we have:

$$\tau \frac{d{x}^i}{dt}={\sigma }^{\prime }(x^i)\odot ({ϵ}^{i+1}{W}^i)-{ϵ}^i=0$$

Then:

$${ϵ}^i={\sigma }^{\prime }(x^i)\odot ({ϵ}^{i+1}{W}^i)$$

or

$${ϵ}^i={\sigma }^{\prime }(x^i)\odot ({ϵ}^{i+1}(M^i{)}^T)$$

Link to Backprop

Starting with the top gradient:

$${\nabla }_{{\mu }^n}F=\frac{{\nu }^n}{2}{\nabla }_{{\mu }^n}||{ϵ}^n|{|}^2=-{ϵ}^n$$

But we also derived that, at equilibrium

$${ϵ}^i={\sigma }^{\prime }(x^i)\odot ({ϵ}^{i+1}(M^i{)}^T)$$

Comparing to the backprop formulas:

$${\nabla }_{z^{(\ell )}}E={\sigma }^{\prime }(z^{(\ell )})\odot ({\nabla }_{z^{(\ell +1)}}E\cdot (W^{(\ell )}{)}^T)$$

Thus, ${ϵ}^i$ is the gradient of the output error with respect to the prediction ${\mu }^i$.

Updating weights

Consider ${\nabla }_{{\mu }^{\ell }}F$. We have

$${\nabla }_{{\mu }^{\ell }}F=-\sigma (x^{\ell })\otimes {ϵ}^{\ell +1}$$

Likewise:

$${\nabla }_{W^{\ell }}=-{ϵ}^{\ell +1}\otimes \sigma (x^{\ell })$$

Therefore:

$$\begin{array}{rl}\delta \frac{d{M}^{\ell }}{dt}& =\sigma (x^{\ell })\otimes {ϵ}^{\ell +1}\\ \delta \frac{d{W}^{\ell }}{dt}& ={ϵ}^{\ell +1}\otimes \sigma (x^{\ell })\end{array}$$

• These learning rules only use info from nodes adjacent to the connection.

These weight updates formulas are the same type of “delta” used in backprop:

$$\frac{\partial E}{\partial W^{)\ell }}=\underset{\sigma (x)}{\underbrace{\left[\begin{array}{c}|\\ h^{(l)}\\ |\end{array}\right]}}\underset{ϵ}{\underbrace{\left[\begin{array}{ccc}\text{textemdash}& {\nabla }_{z^{(\ell +1)}}& \text{textemdash}\end{array}\right]}}$$

The time constant for the weights $\delta$ is larger than the time constant for the nodes $\tau$. This allows the value nodes and error nodes to converge to equilibrium faster, setting up the pieces needed for the weight updates. The full system of differential equations is:

$$\begin{array}{rl}\tau \frac{d{x}^{\ell }}{dt}& ={\sigma }^{\prime }(x^{\ell })\odot ({ϵ}^{l+1}{W}^{\ell })-{ϵ}^{\ell }\\ \tau \frac{d{ϵ}^i}{dt}& =x^i-{\mu }^i-{\nu }^i{ϵ}^i\\ \delta \frac{d{M}^{\ell }}{dt}& =\sigma (x^{\ell })\otimes {ϵ}^{\ell +1}\\ \delta \frac{d{W}^{\ell }}{dt}& ={ϵ}^{\ell +1}\otimes \sigma (x^{\ell })\end{array}$$

Testing

To run it, we just clamp the input $x$ and run the network to equilibrium. Once at equilibrium, $x^n$ is the network’s output.