Slides van de presentatie gegeven bij de DH Clinics Universiteitsbibliotheken op 17 oktober 2017. https://dhclinics.github.io/dag/2017-dag3

1. Natural Language Processing
& Named Entity Recognition
marieke.van.erp@huygens.knaw.nl
merpeltje

6. Natural Language Processing
• Het verwerken van natuurlijke
(menselijke) taal door computers
• Doel: ‘begrijpen’ van tekst
• In de praktijk: verrijken van collecties,
verbeteren zoekmachines,
automatisch vertalen
• Begrijpen van tekst wordt voor
computer opgedeeld in verschillende
lagen/stapjes:
Tokenisation
Lexical Analysis
Syntactic Analysis
Semantic Analysis
Pragmatic Analysis
Input text
Speaker's intended meaning

7. Natural Language Processing Stappen
Tokenisation
Lexical Analysis
Syntactic Analysis
Semantic Analysis
Pragmatic Analysis
Input text
Speaker's intended meaning
• Tokenisation: detectie van zinnen en woorden
• Lexicale analyse: woordsoorten (voorzetsel,
adjectief) en vervoegingen
• Syntactische analyse: zinsstructuur
(onderwerp, persoonsvorm, lijdend voorwerp)
• Semantische analyse: naar wie wordt
verwezen in de tekst, waar gaat de tekst
eigenlijk over?
• Pragmatische analyse: wat bedoelt de spreker

8. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

9. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

10. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

11. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

12. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

13. Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln

14. Waarom zijn entiteiten zo belangrijk?
• Entiteiten kunnen worden geïndexeerd
• Sentiment kan aan entiteiten gekoppeld
worden
• Entiteiten vormen het startpunt voor veel
andere informatie-extractietaken
bron: https://trends.google.nl/trends/topcharts

15. Entity recognition voor (literatuur)collecties

16. Entity recognition voor (literatuur)collecties

17. Hoe gaat Named Entity Recognition in zijn werk?
• Handgeschreven regels (bv. Persoonsnaam: “Meneer” gevolgd door
woord dat begint met hoofdletter)
• Hoge precisie, maar veel werk
• Wat te doen met woorden die ambigu zijn? Apple, Handel, De
Leeuw
• Statistische (machine learning) methodes
• Veel geannoteerde training data nodig
• Semi-supervised machine learning wordt onderzocht om dit op
te lossen

18. State-of-the-Art
*ACL: Association for Computational Linguistics

19. Valkuilen
• Hoge resultaten zijn uitsluitend op teksten ‘in domain’
teksten
• Meeste systemen zijn geoptimaliseerd voor hedendaagse
kranten
• Taalvariatie (tijd, dialect, status, type publicatie)
beïnvloeden resultaten
• Advies: do not try this at home

20. Als je er toch mee aan de slag wilt
• https://gate.ac.uk/
• https://nlp.stanford.edu/software/CRF-NER.shtml
• https://opennlp.apache.org/
• http://ixa2.si.ehu.es/
ixa-pipes/
• https://languagemachines.github.io/frog/

21. Evalueer & vraag hulp
• Controleer of het NER systeem de gewenste output geeft
• Bijvoorbeeld door de output met de hand na te
lopen
• Schakel een taaltechnoloog in!
• Computational Linguistics in the Netherlands
• http://www.let.rug.nl/vannoord/Clin/

22. Wat is Coreference Resolution
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.

23. Maar het is eigenlijk nog ingewikkelder
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.

24. Maar het is eigenlijk nog ingewikkelder
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.

25. Waarom is coreference resolutie belangrijk?
• Coreference is een frequent taalfenomeen
• Coreference resolution is essentieel om een tekst goed te begrijpen
• Coreference resolution is moeilijk omdat het over de grenzen van syntax,
semantiek en discours gaat
• Huidige state-of-the-art: 60~70% accuraat

26. Intussen in Semantic Web-land

27. Grounding Entities in the LOD cloud
“Abraham Lincoln”
http://dbpedia.org/resource/Abraham_Lincoln

29. Waar het mis kan gaan
Paris

30. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg

31. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg
https://upload.wikimedia.org/
wikipedia/commons/5/58/
Paris_Hilton_2%2C_2012.jpg

32. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg
https://upload.wikimedia.org/
wikipedia/commons/5/58/
Paris_Hilton_2%2C_2012.jpg
https://commons.wikimedia.org/
wiki/File:Helene_Paris_David.jpg

33. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg
https://upload.wikimedia.org/
wikipedia/commons/5/58/
Paris_Hilton_2%2C_2012.jpg
https://commons.wikimedia.org/
wiki/File:Helene_Paris_David.jpg
https://upload.wikimedia.org/wikipedia/commons/4/49/
Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG

34. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg
https://upload.wikimedia.org/
wikipedia/commons/5/58/
Paris_Hilton_2%2C_2012.jpg
https://commons.wikimedia.org/
wiki/File:Helene_Paris_David.jpg
https://upload.wikimedia.org/wikipedia/commons/4/49/
Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG

35. Waar het mis kan gaan
ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris-
france-tower-161853.jpeg
https://upload.wikimedia.org/
wikipedia/commons/5/58/
Paris_Hilton_2%2C_2012.jpg
https://commons.wikimedia.org/
wiki/File:Helene_Paris_David.jpg
https://upload.wikimedia.org/wikipedia/commons/4/49/
Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG

36. Maar je moet wel weten welke ‘Lincoln’ je moet hebben

38. Hoe gaat Named Entity Disambiguation in zijn werk?
• Er is een database van entiteiten met context
• Entiteiten worden geïdentificeerd in tekst
• Het systeem zoekt per gevonden entiteit in de tekst een
passende entiteit in de database
• Matching op woorden
• Matching op context
• Ranking (vaak naar populariteit van entiteit in de
database)

39. State-of-the-Art
• Lehman et al. (2010): 80,6
• TAC KBP Entity Linking*: persoon, organisatie, locatie
• Nieuwsartikelen, blogs en fora (Usenet)
* entity linking en entity disambiguation worden vaak door elkaar gebruikt

40. Valkuilen
• Vooral op ‘populaire’ entiteiten (beroemd genoeg voor
Wikipedia)
• Evaluatiedatasets vaak niet gebalanceerd (Van Erp et al.
2016)
• Moeilijk in te richten voor eigen thesauri

41. Als je er zelf mee aan de slag wilt
• https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki
• http://nerd.eurecom.fr
• FOX http://aksw.org/Projects/FOX.html
• https://www.ibm.com/watson/services/natural-
language-understanding/
• http://babelfy.org/

42. Evalueer & vraag hulp
• Controleer of het NEL systeem de gewenste output geeft
• Staan de entiteiten die voor mij relevant zijn wel in de
knowledge base?
• Schakel een Semantic Web expert in!
• http://nl.dbpedia.org/web/ (Nederlandstalige DBpedia )
• https://iswc2017.semanticweb.org/
• Platform Linked Data Nederland: http://www.pilod.nl/
• https://2017.semantics.cc/

43. Recap
• Named entity recognition en disambiguatie zijn potentiële
verrijkingen van databronnen
• Vaak nog onderzoeksstadium
• Domeinadaptatie blijft een punt van aandacht
• Maar er is wel veel interesse in taaltechnologie en
semantic web gemeenschap om dit te verbeteren!

44. Verder lezen
• Lehmann, J., Monahan, S., Nezda, L., Jung, A., & Shi, Y. (2010). LCC
Approaches to Knowledge Base Population at TAC 2010. In TAC.
• van Erp, M., Mendes, P. N., Paulheim, H., Ilievski, F., Plu, J., Rizzo, G., &
Waitelonis, J. (2016). Evaluating Entity Linking: An Analysis of Current
Benchmark Datasets and a Roadmap for Doing a Better Job. In LREC (Vol.
5, p. 2016).
• Maynard, D., Bontcheva, K., & Augenstein, I. (2016) Natural Language
Processing for the Semantic Web. Morgan Claypool.
• Introductiefilmpje Semantic Web: https://www.youtube.com/watch?
v=TJfrNo3Z-DU (Metaweb is opgekocht door Google, wat misschien iets
te maken heeft met de gestructureerde zoekresultaten die je nu terugkrijgt
die Seth noemde)

45. Img source: http://mwallacepro.com/content/uploads/2016/01/question-1500082_1920.jpg