Optimizing Neural Networks

• Adaptive step-size
  • Running Averages
  • Momentum (ML)
  • Adadelta
  • Adam