/notes/

Recent

  • Nonlinear State Space Models

    Oct 24, 2025

    • mte484
  • MTE 484 Lab 2

    Oct 23, 2025

    • mte484
  • MTE 484 A5Q3c

    Oct 22, 2025

    See 1486 more →

    Home

    ❯

    ML

    ❯

    Optimizing Neural Networks

    Optimizing Neural Networks

    Jan 28, 20241 min read

    • ml

    An optimizer takes in a gradient and decides how to update the parameters based on the gradients. The simplest form is Stochastic Gradient Descent.

    • Adaptive step-size
      • Running Averages
      • Momentum (ML)
    • Adadelta
    • Adaptive moment estimation

    Graph View

    Created with Quartz v4.4.0 © 2025

    • Main Website
    • GitHub
    • LinkedIn