1. The document presents the Multi Sense Skip-gram (MSSG) model for learning multiple embeddings per word in vector space. 2. MSSG assigns a separate embedding to each sense of a word using a context vector. It extends the Skip-gram model by learning sense-specific embeddings. 3. The Non-Parametric MSSG (NP-MSSG) model extends MSSG by using a non-parametric approach to learn the context vectors instead of fixed vectors, allowing an unbounded number of senses per word.

1. EMNLP2014 @PFI
Efficient Non-parametric Estimation
of Multiple Embeddings per Word in
Vector Space
Arvind Neelakantan, Jeevan Shankar,
Alexandre Passos, Andrew McCallum
®Preferred Infrastructure
š~ôø (@unnonouno)

2. nìûÏ
š~ôø (@unnonouno)
! n՞ǽ‹6ODKPYARbF6ÿå
! ´ò½‹57'+6:2
! NLP¾q/fµt©†k2014-
! rc§‹¨Û¹IBM¢ýÊPFI

3. îÓ
NLP¾q/fYANS
! YANSIbXJBZ9o
! ¾qÊÒl40ÈãØ9/¾qÊÒln
Ã/ÊÒ˜4;$4/¸2:
! s6¤jCbJR@/»œ5üç
! ƒd0mê* $

4. ! YANSĀ3o
! žÇ½‹fŒecf’yg.ĀÑf 2
! à-;õ3f*

5. “þ
! word2vecÙÉ )1(/àÇ.í™/”×
WEPaÁ;
! àÇ/—å+ÇÂĄă•¥¿tx.¥
! ”×WEPa/™nŽ*4;w„5«£
);

6. ù*=;Skip-gram [Mikolov+13]
$(#%'
éā
0
Ă
*
;

7. ‰{€™*=;Skip-gram
! àÇ/WEPa v(wt) +GbODKP/WEP
a v(c) /|ÐIFAQ.;
Ă*;

8. Multi Sense Skip-gram (MSSG) model
1. ÞÚàÇ/GbODKPWEPa9›æG
bODKP±h
2. ”×âÀ/g9iËGbODKP/ßú $
”׈ó
3. #/”×.v¼;”×WEPaˆó
4. ˆó$”×WEPa.Ý')ÞÚGbOD
KPz );+.;Skip-gram+
t!

9. ù*=;MSSG
Skip-gram
$(#%'!

10. ‰{€™*=;MSSG
! ˆó $”×WEPa¯%
! àÇ/WEPa0”×ï.ª:GbODKP
/WEPa01(%ª;

11. @aH`LZ*=;MSSG
! ÞÚGbO
DKP
¯')”×
ˆó;
G
Qu
;%

12. Non-Parametric MSSG (NP-MSSG) model
! MSSG*0”×WEPa/™04¬%'$
! NP-MSSG*0nŽ*4;
! 4w0àö.ƒ2*+GbODKPWEP
a/›æƶ$9} ”×WEPa
͜;%

13. NP-MSSG.;”×/ˆó/Ôw
! ƒ2*+ÆGbODKPh— $9}9
”×Ëä²:‘)8
! k(wt)0ƒ2*wt.²:‘)$”×/™
! vcontext0wt/ݾGbODKPWEPa
! (wt, k)0kˉ/”×/GbODKPg¦

14. q„/2+4
! MSSG
! àÇ.v )kÅ/”×WEPa²:‘);S_
[
Mkè
!

15. !
! ˆó$”×WEPa¯')ñ‚.Skip-gram
! NP-MSSG
! kÅ/”×WEPa²:‘);/0t!
!

16. #
! ˆó$”×WEPa*Skip-gram;/0t!

17. ŠÌ¡°Apple
! ° +J^VK.p;

18. ŠÌ¡°Run)
! ”×/Ąă-àÇ.0º/”×WEPa³Á;

19. ”×/ßú…­—*;/ŠÌ
localSimiËú$”×t¹/÷ÖcŸÜ

20. ·Î
! Š01+?,t!ŠÌ6')$(^o^)
! E_K™nŽ*z;w„.áð )$
/*?-/*/ +ëÄ
! ÞÚàÇ*4;8:QD][bPŒ‡/P
TNE*4$1nÕ-–;
! S_F_UWEPa¯