1. Graphical model software for machine learning Kevin Murphy University of British Columbia December, 2005
2.
3. Supervised learning as Bayesian inference Y 1 X 1 Y N X N Y * X * Y n X n Y * X * N Training Testing
4. Supervised learning as optimization Y 1 X 1 Y N X N Y * X * Y n X n Y * X * N Training Testing
5.
6.
7. 1D chain CRFs for sequence labeling Y n1 Y nm Y n2 X n A 1D conditional random field (CRF) is an extension of logistic regression to the case where the output labels are sequences, y n 2 {1,…,C} m Local evidence Edge potential i ij
8. 2D Lattice CRFs for pixel labeling A conditional random field (CRF) is a discriminative model of P(y|x). The edge potentials ij are image dependent.
9. 2D Lattice MRFs for pixel labeling A Markov Random Field (MRF) is an undirected graphical model. Here we model correlation between pixel labels using ij (y i ,y j ). We also have a per-pixel generative model of observations P(x i |y i ) Local evidence Potential function Partition function
20. Complexity of exact inference In general, the running time is (N K w ), where w is the treewidth of the graph; this is the size of the maximal clique of the triangulated graph (assuming an optimal elimination ordering). For chains and trees, w = 2. For n £ n lattices, w = O(n).
21. Approximate sum-product O(N K I) General Mean field O(N K I) General Swendsen-Wang O(N K I) General Gibbs O(N K 2c I) c = cluster size General Generalized BP O(N K I) Restricted BP+FFT (exact iff tree) O(N K 2 I) General BP (exact iff tree) Time N=num nodes, K = num states, I = num iterations Potential (pairwise) Algorithm
22. Approximate max-product O(N K I) General SLS (stochastic local search) O(N K I) General ICM (iterated conditional modes) O(N 2 K I) [?] Restricted Graph-cuts (exact iff K=2) O(N K 2c I) c = cluster size General Generalized BP O(N K I) Restricted BP+DT (exact iff tree) O(N K 2 I) General BP (exact iff tree) Time N=num nodes, K = num states, I = num iterations Potential (pairwise) Algorithm
![Outline ,[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)