M. Pawan Kumar
 
 

HOME

RESEARCH

PUBLICATIONS

TALKS

TEACHING

CV

 

 

 

 

 

 

EM Algorithm for Likelihood Maximization

maxw F(w) where F(w) = Σi log Pr(xi, yi; w) = Σi log Pr(xi, yi, hi ; w) - Σi log Pr(hi | xi, yi; w)

Inference step: Obtain the exceptation of F(w) under the distribution Pr(hi | xi, yi; wt), where wt is the estimate of the parameters at iteration t.

Update step: Update the parameters by maximizing the expectation of F(w). Specifically

wt+1 = argmaxw Σi Pr(hi | xi, yi; wt) log Pr(xi, yi, hi ; w).

See [1,2] for a more detailed description of the EM algorithm.

References

[1] A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 39(1): 1-38, 1977.
[2] A. Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman and Hall, 1995.