Maximum likelihood-set - introduction

Language Modeling with the Maximum Likelihood Set (Karakos & Khudanpur, ISIT-2006) http://dx.doi.org/10.1109/ISIT.2006.261575 Yusuke Matsubara Tsujii lab. Meeting 2006-06-22

Necessity of smoothing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Maximum Likelihood Set [ Jedynak & Khudanpur 2005 ] ,[object Object],[object Object],[object Object],p 1 +p 2 +p 3 =1 MLEs from possible counts 1 1 1 0 Sample size=3 #(word set) =3

The Maximum Likelihood Set (formal definition) k 2 linear inequality constraints

The Maximum Likelihood Set n = 3 (#samples) k = 3 (#word set) n = 10 (#samples) k = 3 (#word set) Larger samples Nearer to MLE

Choosing a pmf from a MLS ,[object Object],[object Object],reference MLS

Conditional pmf estimation ,[object Object],[object Object],[object Object],[object Object],[object Object]

Experimental results ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Bigram Trigram Witten-Bell Kneser-Ney Witten-Bell Kneser-Ney Reference 8.47 8.36 8.21 8.08 MLS 8.44 8.38 8.24 8.12

Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

(0, 0, 1) (0, 1/3, 2/3) (0, 2/3, 1/3) (0, 1, 0) (0, 2/3, 1/3) (0, 1/3, 2/3) (0, 0, 1) (2/3, 0, 1/3) (1/3, 0, 2/3) (1/3, 1/3, 1/3)

Maximum likelihood-set - introduction

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (12)

En vedette

En vedette (12)

Similaire à Maximum likelihood-set - introduction

Similaire à Maximum likelihood-set - introduction (20)

Maximum likelihood-set - introduction