"Automatic Generation of Quizzes from DBpedia According to Educational Standards" Presentation at the EKM 2018 Workshop of The Web Conference 2018.

1. Automatic Generation of Quizzes from DBpedia According
to Educational Standards
Oscar RODRÍGUEZ ROCHA
Catherine FARON ZUCKER
University Côte d’Azur, CNRS, Inria, I3S, France
oscar.rodriguez-rocha@inria.fr, faron@unice.fr
2018

2. Roadmap
• Introduction
• Research questions
• Related work
• Automatic generation of quizzes from KGs
• Educational references of knowledge and skills
• Empirical validation
• Conclusions and future work

3. Introduction
1 A. Papasalouros et al. Automatic Generation Of Multiple Choice Questions From Domain Ontologies. IADIS e-Learning 2008
2 O. Rodriguez Rocha and C. Faron Zucker. Automatic Generation of Educational Quizzes From Domain Ontologies. EduLearn 2017
Educational quizzes
• test or evaluate the knowledge acquired by learners
• support lifelong learning on various topics or subjects
• manual creation is time/resource consuming
• automatic generation is possible by means of:
• domain ontologies1
• SW technologies and LOD2
Semantic Web
• Based on standard technologies (RDF, OWL, SPARQL)
• Large and increasing number of datasets available on the LOD on various domains

4. Research Questions
How to select resources from the LOD on a specific subject
and relevant to a specific school level?
Which referential of knowledge and skills is the most
appropriate for the selection of suitable resources from the
LOD?
Which approach to use for the generation of quizzes about a
subject, that takes into account the school level of a student?

5. Automatic generation of quizzes
rdf:type
p
rdfs:subClassOf
?
?
?
?
O. Rodríguez Rocha and C. Faron Zucker. Automatic Generation of Educational Quizzes From Domain Ontologies. EduLearn 201
RDF Q&A
Generation
NL Questions
Generation
Ranking
Question
SELECTION
Knowledge
Graph
QuizQ&A

6. EduProgression
E d u c a t i o n a l r e f e r e n c e s o f k n o w l e d g e a n d s k i l l s
http://ns.inria.fr/semed/eduprogression
Oscar Rodríguez Rocha et al. A Formalization of the French Elementary School Curricula. EKM 2016

7. EduProgression based KG

8. Les Incollables
Structured knowledge base created from the MCQ of the game “Les
Incollables”
• Formalized in OWL
• Contains around 160,000 questions of the game
• Extracted from game cards, scanned documents and images
• From different French school courses (CP, CE1, CE2, CM1, CM2 and 6e)
• … and domains
• Questions are structured using 3 vocabularies:
• Linquest1
• Les Incollables
• FrenchEdu2
• QA elements enriched with DBpedia related resources
1 http://ns.inria.fr/semed/linquest
2 http://ns.inria.fr/semed/frenchedu/

9. Les Incollables based KG

10. Automatic generation of quizzes
rdf:type
p
rdfs:subClassOf
?
?
?
?
RDF Q&A
Generation
NL Questions
Generation
Ranking
Question
SELECTION
Knowledge
Graph
QuizQ&A

11. Automatic generation of quizzes
Knowledge
Graph
DBpedia
subgraph
Related
resources
discovery
Larger DBpedia
subgraph
Entity
linking

12. Selection of related
resources from DBpedia
Oscar Rodríguez Rocha, Catherine Faron Zucker. ITS 2018,
Extraction of Relevant Resources and Questions from DBpedia to Automatically Generate Quizzes on Specific Domains

13. Evaluation protocol
E v a l u a t i o n o f t h e k n o w l e d g e g r a p h s
• Goal: comparatively evaluate the two reference standards of
knowledge and skills: EduProgression and Les Incollables
• A KG from each reference standard is generated:
• Geography
• CM2 School year
• Questions are generated with each KG
• Evaluator: Geography teacher of the CM2 school year

14. Evaluation protocol
• precision of the resources generated from each KG
• 188 random extracted resources (100 per KG, no duplicates)
• Each resource was evaluated in the scale of 1 (not at all relevant) to 5 (very
relevant), =>3 (relevant)
• relevance of the generated questions from each KG
• 100 randomly extracted questions per KG
• Each question was evaluated in the scale of 1 (not at all relevant) to 5 (very
relevant), =>3 (relevant)
Figure 2: Precision of the resources of each reference stan-
dard
among aset of resourcesgenerated from agiven referencestandard
(equation (1)).
P =
Number of Relevant Resources
Total Set of ResourcesGenerated from a ReferenceStandard
(1)
We have considered that a resource is relevant if its score is
greater than or equal to 3.
The precision obtained by the resources of each educational
referenceisreported in table 2and shown in gure2.
As we can see, after this evaluation, the resources generated
from thestandard of referenceEduProgression weremorerelevant
according to thedomain and according to theschool level.
(on ascaleof oneto
been given a list of
extracted randomly
Theresultsof th
From theseresul
ated from the DBp
reference "EduProg
scoreof relevancew
consistent with the
extracted from each
7 CONCLUSI
In this paper, we h
generation of quizz
count theschool lev
reference, our appr
(corresponding to a
which thequizzesa
educational referen
have evaluated the

15. Experimental results
0.78
3.25
0.73
2.87
0
0.5
1
1.5
2
2.5
3
3.5
Resource's precision Question's average relevance
EduProgression Les Incollables

16. Conclusions and Future work
• Automatic generation of quizzes from DBpedia according to educational standards
• Evaluation of the knowledge graphs generated from each educational reference
• For the CM2 school year and the subject of Geography:
• The knowledge graph generated with EduProgresison has resources with the
highest precision and the questions generated from it have the highest relevance
• EP follows the French educational program, it was conceived as a referential
• Consequently, higher precision and relevance were expected
• Expand our evaluation on additional subjects/domains and school years
• Consider additional structured knowledge bases