M. Pawan Kumar 
HOME

EM Algorithm for Likelihood Maximization max_{w} F(w) where F(w) = Σ_{i} log Pr(x_{i}, y_{i}; w) = Σ_{i} log Pr(x_{i}, y_{i}, h_{i} ; w)  Σ_{i} log Pr(h_{i}  x_{i}, y_{i}; w) Inference step: Obtain the exceptation of F(w) under the distribution Pr(h_{i}  x_{i}, y_{i}; w_{t}), where w_{t} is the estimate of the parameters at iteration t. Update step: Update the parameters by maximizing the expectation of F(w). Specifically w_{t+1} = argmax_{w} Σ_{i} Pr(h_{i}  x_{i}, y_{i}; w_{t}) log Pr(x_{i}, y_{i}, h_{i} ; w). See [1,2] for a more detailed description of the EM algorithm. References
[1] A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 39(1): 138, 1977.
