|M. Pawan Kumar|
SELF-PACED LEARNING FOR LATENT VARIABLE MODELS
M. Pawan Kumar, B. Packer and D. Koller
In Proceedings of Advances in Neural Information Processing Systems (NIPS), 2010
Latent variable models are a powerful tool for addressing several tasks in machine learning. However, the algorithms for learning the parameters of latent variable models are often prone to getting stuck in a bad local optimum. To allieviate this problem, we build on the intuition that the learning algorithm should be presented with the training data in a meaningful order. The order of the samples is determined by how easy they are. The main challenge is that often we are not provided with a readily computable measure of the easiness of samples. We address this issue by proposing a novel, iterative self-paced learning algorithm where each iteration simultaneously selects easy samples and learns a new parameter vector. The number of samples selected is governed by a weight that is annealed until the entire training data has been considered. We empirically demonstrate that the self-paced learning algorithm outperforms the state of the art method for learning a latent structural SVM on four applications: object localization, noun phrase coreference, motif finding and handwritten digit recognition.