Deep Learning

Notes from:

• Understanding Deep Learning

Some of these notes are also included in Machine Learning.

Foundations

• Tensor
• Einstein Notation

Neural Networks

• Supervised Learning
  • Linear Regression
• Discriminative vs. Generative Models

Shallow Neural Networks

• Shallow Neural Network
• Universal Approximation Theorem
• Multivariate Inputs and Outputs

Deep Neural Networks

• Composing Shallow Networks
• Deep Neural Network
• Shallow vs. Deep Networks

Loss Functions

• Loss Function
• Conditional Probabilistic Perspective of Learning
• Maximum Likelihood Criterion
• Log-Likelihood Criterion
• Loss Function Recipe
  • Univariate Regression as MLE
  • Binary Classification
  • Multi-class Classification

Learning Theory

• Neural Tangent Kernel

Practical

• Einops
• PyTorch Tensor Creation
• Einsum

Exercises

• UDL Chapter 2 Problems
• UDL Chapter 3 Problems
• UDL Chapter 4 Problems