M. Pawan Kumar
 
 

HOME

RESEARCH

PUBLICATIONS

GROUP

TALKS

TEACHING

CV

 

 

 

 

 

 

LECTURES

Lecture 5, Part 1: Empirical risk minimization
PPT   PDF

Lecture 5, Part 2: Optimization for deep learning
PPT   PDF

LINKS FOR EMPIRICAL RISK MINIMIZATION

Max-Margin Markov Networks
B. Taskar, C. Guestrin and D. Koller
NIPS, 2003

Support Vector Machine Learning for Interdependent and Structured Output Spaces
I. Tsochantaridis, T. Hofmann, T. Joachims and Y. Altun
ICML, 2004

LINKS FOR CONVEX OPTIMIZATION OVERVIEW

Convex Optimization, Stephen Boyd and Lieven Vandenberghe.
WWW

Convex Optimization: Algorithms and Complexity
S. Bubeck
Foundations and Trends in Machine Learning, 2014

LINKS FOR MOMENTUM

A Method for Solving a Convex Programming Problem with Convergence Rate O(1/k2)
Y. Nesterov
Soviet Mathematics Doklady, 1983

On the Importance of Initialization and Momentum in Deep Learning
I. Sutskever, J. Martens, G. Dahl and G. Hinton
ICML, 2013

LINKS FOR SMOOTHING

Smooth Minimization of Non-Smooth Functions
Y. Nesterov
Mathematical Programming, 2005

Smoothing and First Order Methods: A Unified Framework
A. Beck and M. Teboulle
SIAM Journal of Optimization, 2012

Smooth Loss Functions for Deep Top-k Classification
L. Berrada, A. Zisserman and M. Pawan Kumar
ICLR, 2018

LINKS FOR ADAPTIVE GRADIENTS

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
J. Duchi, E. Hazan and Y. Singer
JMLR, 2012

AdaDelta: An Adaptive Learning Rate Method
M. Zeiler
Arxiv Technical Report, 2012

Adam: A Method for Stochastic Optimization
D. Kingma and J. Ba
ICLR, 2015

On the Convergence of Adam and Beyond
S. Reddi, S. Kale and S. Kumar
ICLR, 2018

The Marginal Value of Adaptive Gradient Methods in Machine Learning
A. Wilson, R. Roelofs, M. Stern, N. Srebro and B. Recht
NIPS, 2017

LINKS FOR DIFFERENCE-OF-CONVEX OPTIMIZATION

Variations and Extensions of the Concave-Convex Procedure
T. Lipp and S. Boyd
Optimization and Engineering, 2016

Trusting SVM for Piecewise Linear CNNs
L. Berrada, A. Zisserman and M. Pawan Kumar
ICLR, 2017