Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Learning to understand phrases by embedding the dictionary

1 701 vues

Publié le

review of "Learning to Understand Phrases by Embedding the Dictionary" by Felix Hill, Kyunghyun Cho, Anna Korhonen, Yoshua Bengio

at KTH's Deep Learning reading group:
www.csc.kth.se/cvap/cvg/rg/

Publié dans : Sciences
  • Soyez le premier à commenter

Learning to understand phrases by embedding the dictionary

  1. 1. @graphific Roelof Pieters Learning  to  Understand   Phrases  by  Embedding  the   Dic9onary 11  April  2015  
 Deep  Learning  Reading  Group www.csc.kth.se/~roelof/ http://arxiv.org/abs/1504.00548 Review of Felix Hill, Kyunghyun Cho, Anna Korhonen, Yoshua Bengio
  2. 2. 2 Core Idea • Model that learns useful representations of phrases and sentences • bridging gap between lexical [word meaning] and phrasal [or compositional] semantics [phrase/ sentence meaning]
  3. 3. 3 Model • RNN which maps dictionary definitions (phrases) to (lexical) representations (of the words those definitions define) • 2 tasks: 1. reverse dictionary/concept finder 2. general-knowledge crossword question answerer
  4. 4. 4 RNN (Recurrent Neural Network) 1. Latent features are modelled as distributed dense vector of hidden layers 2. Can operate on sequential data of variable length 3. Vanishing/exploding gradient
  5. 5. 5 LSTM (Long Short-Term Memory) 1. Scales connections between memory cell and the input/ output layers 2. Gates to control input/memory/outputs 3. Lessens vanishing/exploding gradient problem
  6. 6. RNN (LSTM) I. Cross-Lingual Reverse Dictionaries Dictionary with definitions Target word embeddings
 (Word2Vec, CBOW, 8B w) Model (training)
  7. 7. I. Cross-Lingual Reverse Dictionaries RNN Embedding Space testing Input phrase
  8. 8. I. Cross-Lingual Reverse Dictionaries
  9. 9. I. Cross-Lingual Reverse Dictionaries
  10. 10. II. General Knowledge (crossword) 
 Question Answering

×