Deep Learning

Notes from:

• Understanding Deep Learning
• Alice’s Adventures in a Differentiable Wonderland

Some of these notes are also included in Machine Learning.

Foundations

• Tensor
• Einstein Notation

Shallow Neural Networks

• Supervised Learning
• Discriminative vs. Generative Models
• Shallow Neural Network
• Universal Approximation Theorem
• Multivariate Inputs and Outputs
• Shallow vs. Deep Networks

Deep Neural Networks

• Composing Shallow Networks
• Deep Neural Network
  • Linear Regions Per Parameter for Neural Network

Learning Theory

• Neural Tangent Kernel

Practical

• Einops
• PyTorch Tensor Creation
• Einsum

Exercises

• UDL Chapter 2 Problems
• UDL Chapter 3 Problems
• UDL Chapter 4 Problems