|M. Pawan Kumar|
EM Algorithm for Likelihood Maximization
maxw F(w) where F(w) = Σi log Pr(xi, yi; w) = Σi log Pr(xi, yi, hi ; w) - Σi log Pr(hi | xi, yi; w)
Inference step: Obtain the exceptation of F(w) under the distribution Pr(hi | xi, yi; wt), where wt is the estimate of the parameters at iteration t.
Update step: Update the parameters by maximizing the expectation of F(w). Specifically
wt+1 = argmaxw Σi Pr(hi | xi, yi; wt) log Pr(xi, yi, hi ; w).
See [1,2] for a more detailed description of the EM algorithm.
 A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 39(1): 1-38, 1977.