UDL Chapter 9 Problems

Problem 9.1

Consider a model where the prior distribution over the parameter is a normal distribution with mean zero and variance ${\sigma }_{\varphi }^2$ so that

$$Pr(\varphi )=\prod _{j=1}^J{\text{Norm}}_{{\varphi }_j}[0,{\sigma }_{\varphi }^2]$$

where $j$ indexes the model parameters. We now maximize ${\sum }_{i=1}^{I}Pr({\mathbf{y}}_i|{\mathbf{x}}_i,\varphi )Pr(\varphi )$. Show that the associated loss function of this model is equivalent to L2 regularization.

Problem 9.2

How do the gradients of the loss function change when L2 regularization is added?

Problem 3

Problem 4

Problem 5

Problem 6

Problem 7

Problem 8

Problem 9

Problem 10