Introduction to Online Learning

I am publishing the lecture notes of my class “Introduction to Online Learning” at Boston University, Fall 2019. I strongly suggest to go over the lectures in order: I will introduce all the required math tools slowly over the lectures, rather than all at once in single lectures.


  1. Introduction to Online Learning
  2. Online Gradient Descent
  3. Subgradients and Online-to-Batch Conversion
  4. More Online-to-Batch Examples and Strong Convexity
  5. Adaptive Algorithms: L* bounds and AdaGrad
  6. Lower Bounds for Online Linear Optimization
  7. Online Mirror Descent I: Bregman version
  8. Online Mirror Descent II: Regret and Mirror Version
  9. Online Mirror Descent III: Examples and Learning with Expert Advice
  10. Follow-The-Regularized-Leader I: Regret Equality
  11. Follow-The-Regularized-Leader II: Applications