Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
MachineLearningonKnowledgeGraphs
AQuickTourofKnowledgeGraphEmbeddings
LucaCostabello,AccentureLabsDublin
@lukostaz
KnowledgeGraph G
G = {(s, p, o)} ⊆ E × R × E
E : setofentitiesofG R : setofrelationsofG 2
SomeExamples
KnowledgeGraph         Statements             Entities                  
120M 10M
610M 51M
1.3B 6M
3.5B 364M
3
Applications
KnowledgeDiscovery
Question‑answeringsystems
Identify&disambiguateentitiesintext
Domain‑specificdecisionsuppo...
Applications
KnowledgeDiscovery
Question‑answeringsystems
Identify&disambiguateentitiesintext
Domain‑specificdecisionsuppo...
KnowledgeDiscovery:LinkPrediction
6
KnowledgeDiscovery:LinkPrediction
Predictprobabilityofexistenceofcandidatestatements
7
KnowledgeGraphEmbeddingModels
Automatic,supervisedlearningofembeddings,i.e.projectionsof
entitiesandrelationsintoacontinuo...
[Cai17]
9
[Cai17]
10
ModelRequirements
Expressiveness
ModelKGpropertiesandregularitiesasmuchaspossible
(symmetry,antisymmetry,inversion,composi...
AnatomyofaKGEmbeddingModel
KnowledgeGraph(KG)G
Negativesgenerationstrategy
Scoringfunctionforatriplef(t)
LossfunctionL
Opt...
Negativesgenerationstrategies
Uniformsampling:GenerateanduniformlysamplecorruptionsC:
C((s, p, o)) = {( , p, o)∣ ∈ E} ∪ {(...
Example(Uniformsampling)
E = {Mike, Liverpool, AcmeInc, George, LiverpoolFC}
R = {bornIn, friendWith}
t ∈ G = (Mike bornIn...
Scoringfunctionf
f assignsascoretoatriple(s, p, o):
TransE:TranslatingEmbeddings[Bordes13]
f = −∣∣(e +r ) −e ∣∣TransE s p ...
[Anonymous19]
16
LossfunctionL
Pairwisemargin‑based:
L(Θ) = max(0, [γ + f(t ; Θ) − f(t ; Θ)])
Negativelog‑likelihood
...
t ∈G+
∑
t ∈C−
∑ − ...
OptimizationProcedure
EstimatetheoptimalparametersΘ with(samevariantof)SGD:
L(Θ)
On‑the‑flynegativesgenerationwiththenegat...
EvaluationMetrics
LearningtoRankmetrics
Howwellarepositivetriplesrankedagainsttheircorruptions?
Hits@N = 1 if rank ≤ N
MR ...
Example
s p o score rank
Mike bornIn AcmeInc 0.789 1
Mike bornIn Liverpool 0.753 2 *
Mike bornIn LiverpoolFC 0.695 3
Georg...
LatestResults
[Anonymous19]
21
OpenResearchDirections
Howtodesignmoreexpressivemodels?
Howtosupporttime?
Howtocomeupwithexplainableresults?
Howtoinjectba...
Questions?
LucaCostabello,AccentureLabsDublin
@lukostaz
23
References
[Anonymous19]Anonymous,"RotatE:KnowledgeGraphEmbeddingbyRelationalRotationin
ComplexSpace",SubmittedtoICLR2019(...
Prochain SlideShare
Chargement dans…5
×

Machine Learning on Knowledge Graphs: a Quick Tour of Knowledge Graph Embeddings

Machine Learning on Knowledge Graphs: a quick and incomplete 15-min tour of knowledge graph embeddings

  • Soyez le premier à commenter

Machine Learning on Knowledge Graphs: a Quick Tour of Knowledge Graph Embeddings

  1. 1. MachineLearningonKnowledgeGraphs AQuickTourofKnowledgeGraphEmbeddings LucaCostabello,AccentureLabsDublin @lukostaz
  2. 2. KnowledgeGraph G G = {(s, p, o)} ⊆ E × R × E E : setofentitiesofG R : setofrelationsofG 2
  3. 3. SomeExamples KnowledgeGraph         Statements             Entities                   120M 10M 610M 51M 1.3B 6M 3.5B 364M 3
  4. 4. Applications KnowledgeDiscovery Question‑answeringsystems Identify&disambiguateentitiesintext Domain‑specificdecisionsupportsystems 4
  5. 5. Applications KnowledgeDiscovery Question‑answeringsystems Identify&disambiguateentitiesintext Domain‑specificdecisionsupportsystems 5
  6. 6. KnowledgeDiscovery:LinkPrediction 6
  7. 7. KnowledgeDiscovery:LinkPrediction Predictprobabilityofexistenceofcandidatestatements 7
  8. 8. KnowledgeGraphEmbeddingModels Automatic,supervisedlearningofembeddings,i.e.projectionsof entitiesandrelationsintoacontinuouslow‑dimensionalspace. 8
  9. 9. [Cai17] 9
  10. 10. [Cai17] 10
  11. 11. ModelRequirements Expressiveness ModelKGpropertiesandregularitiesasmuchaspossible (symmetry,antisymmetry,inversion,composition,etc.) Space/timecomplexity O(N ),O(N ),O(N ) intime&memory Bothrequirementsareequallyimportant! KnowledgeGraphEmbeddingsstrikeagoodtrade‑off. E R G 11
  12. 12. AnatomyofaKGEmbeddingModel KnowledgeGraph(KG)G Negativesgenerationstrategy Scoringfunctionforatriplef(t) LossfunctionL Optimizationalgorithm 12
  13. 13. Negativesgenerationstrategies Uniformsampling:GenerateanduniformlysamplecorruptionsC: C((s, p, o)) = {( , p, o)∣ ∈ E} ∪ {(s, p, )∣ ∈ E} Basedongenerativeadversarialnetwork[Cai17] Self‑adversarialnegativesampling[Anonymous19] s^ s^ o^ o^ 13
  14. 14. Example(Uniformsampling) E = {Mike, Liverpool, AcmeInc, George, LiverpoolFC} R = {bornIn, friendWith} t ∈ G = (Mike bornIn Liverpool) C = Mike bornIn AcmeInc Mike bornIn LiverpoolFC George bornIn Liverpool AcmeInc bornIn Liverpool i ti 14
  15. 15. Scoringfunctionf f assignsascoretoatriple(s, p, o): TransE:TranslatingEmbeddings[Bordes13] f = −∣∣(e +r ) −e ∣∣TransE s p o n 15
  16. 16. [Anonymous19] 16
  17. 17. LossfunctionL Pairwisemargin‑based: L(Θ) = max(0, [γ + f(t ; Θ) − f(t ; Θ)]) Negativelog‑likelihood ... t ∈G+ ∑ t ∈C− ∑ − + 17
  18. 18. OptimizationProcedure EstimatetheoptimalparametersΘ with(samevariantof)SGD: L(Θ) On‑the‑flynegativesgenerationwiththenegativesstrategy min Θ 18
  19. 19. EvaluationMetrics LearningtoRankmetrics Howwellarepositivetriplesrankedagainsttheircorruptions? Hits@N = 1 if rank ≤ N MR = rank [MeanRank] MRR = [MeanReciprocalRank] ∣Q∣ 1 ∑i=1 ∣Q∣ (s,p,o)i ∣Q∣ 1 ∑i=1 ∣Q∣ (s,p,o)i ∣Q∣ 1 ∑i=1 ∣Q∣ rank(s,p,o)i 1 19
  20. 20. Example s p o score rank Mike bornIn AcmeInc 0.789 1 Mike bornIn Liverpool 0.753 2 * Mike bornIn LiverpoolFC 0.695 3 George bornIn Liverpool 0.695 3 AcmeInc bornIn Liverpool 0.234 5 s p o score rank George worksFor AcmeInc 0.901 1 * George worksFor Liverpool 0.345 2 George worksFor LiverpoolFC 0.293 3 George worksFor Mike 0.201 4 LiverpoolFC worksFor AcmeInc 0.156 5 Hits@3 = 1 if rank ≤ 3 = 1 Hits@1 = 1 if rank ≤ 1 = 0.5 2 1 ∑i=1 2 (s,p,o)i 2 1 ∑i=1 2 (s,p,o)i 20
  21. 21. LatestResults [Anonymous19] 21
  22. 22. OpenResearchDirections Howtodesignmoreexpressivemodels? Howtosupporttime? Howtocomeupwithexplainableresults? Howtoinjectbackgroundknowledge?(i.e.text) 22
  23. 23. Questions? LucaCostabello,AccentureLabsDublin @lukostaz 23
  24. 24. References [Anonymous19]Anonymous,"RotatE:KnowledgeGraphEmbeddingbyRelationalRotationin ComplexSpace",SubmittedtoICLR2019(underreview). [Bordes13]Bordesetal."Translatingembeddingsformodelingmulti‑relationaldata.",NIPS 2013. [Cai17]Cai,Hongyunetal."AComprehensiveSurveyofGraphEmbedding:Problems, TechniquesandApplications",IEEETrans.OnKnowledgeandDataEng,2017 [Dettmers18]Dettmers,etal."Convolutional2DKnowledgeGraphEmbeddings",AAAI2018 [Nickel15]Nickeletal."Areviewofrelationalmachinelearningforknowledgegraphs."Procsof theIEEE104.1,2016 [Nickel17]Nickeletal."HolographicEmbeddingsofKnowledgeGraphs."AAAI2016 [Troullion17]Trouillon,Theo,etal."KnowledgeGraphCompletionviaComplexTensor Factorization.",ICML2017. [Yang15]BishanYang,etal."Embeddingentitiesandrelationsforlearningandinferencein knowledgebases",ICLR2015 24

×