Diffusion Decoder

When we learn a diffusion model, we learn the reverse process of the encoder. In other words, we learn a series of probabilistic mappings back from latent variable $z_T$ to $z_{T-1}$, from $z_{T-1}$ to $z_{T-2}$, and so on, until we reach the data $x$.

The true reverse distributions $q(z_{t-1}|z_t)$ of the diffusion process are complex multi-modal distributions that depend on the data distribution $Pr(x)$. We approximate these as normal distributions:

$$\begin{array}{rl}Pr(z_T)& ={\text{Norm}}_{z_T}[0,I]\\ Pr(z_{t-1}|z_t,{\varphi }_t)& ={\text{Norm}}_{z_{t-1}}[f_t[z_t,{\varphi }_t],{\sigma }_t^{2}I]\\ Pr(x|z_1,{\varphi }_1)& ={\text{Norm}}_x[f_1[z_1,{\varphi }_1],{\sigma }_1^{2}I]\end{array}$$

where:

• $f_t[z_t,{\varphi }_t]$ is a neural network that computes the mean of the normal distribution in the estimated mapping from $z_t$ to the preceding latent variable $z_{t-1}$.
• The terms $\{{\sigma }_t^2\}$ are predetermined.
• If the noise schedule hyperparameters ${\beta }_t$ in the diffusion process are close to zero (and the number of time steps $T$) is large, then this normal approximation will be reasonable.

We generate new examples from $Pr(x)$ using ancestral sampling:

• First, we draw $z_T$ from $Pr(z_T)$.
• Then, we sample $z_{T-1}$ from $Pr(z_{T-1}|z_{T,}{\varphi }_T)$, sample $z_{T-2}$ from $Pr(z_{T-2}|z_{T-1},{\varphi }_{T-1})$ and so on until we finally generate $x$ from $Pr(x|z_1,{\varphi }_1)$.