M. Pawan Kumar
 
 

HOME

RESEARCH

PUBLICATIONS

GROUP

TALKS

TEACHING

CV

 

 

 

 

 

 

EFFICIENT FRANK-WOLFE FOR DENSE CRFs AND PIECEWISE LINEAR CNNs

M. Pawan Kumar

Pre-Doc Summer School, ETH Zurich, 2017

The course consists of two parts. In the first part, we will consider the challenging problem of energy minimization in dense CRFs, which is typically addressed using the mean-field algorithm. The popularity of mean-field is largely due to the fact that each iteration of the algorithm can be performed efficiently using a filtering method based on the permutohedral lattice. We will consider several continuous relaxations for the problem, which provide strong theoretical guarantees on the quality of the solution, and show that they can also be solved efficiently by leveraging the power of the Frank-Wolfe algorithm. Specifically, each iteration of Frank-Wolfe can also be performed efficiently using the same filtering method.
The second part of the course will look at a novel approach for parameter estimation in piecewise linear neural networks, that is, networks where the non-linear activations are piecewise-linear functions of their inputs. We will show how the problem of estimating the parameters of one layer of the network while keeping others fixed can be viewed as a latent support vector machine (SVM). This observation allows us to exploit the powerful block-coordinate Frank-Wolfe algorithm for SVMs to obtain an accurate set of parameters.

Topic 1: Dense CRF  [PPT]   [PDF]

Topic 2: Piecewise Linear CNN  [PPT]   [PDF]

Practical: QP for Dense CRF  [WWW]   [Completed Code]

An octave version of the code is also available here, thanks to Mikhail Usvyatsov.