Slides for the paper presented at 16th ESWC, Portorose, Slovenia

1. Boosting DL Concept Learners
Nicola FANIZZI, Giuseppe RIZZO and Claudia d'AMATO
LACAM — Dipartimento di Informatica & CILA
Università degli studi di Bari Aldo Moro
ESWC 2019 — June 4th, 2019

2. Outline
Introduction
The Framework:
Training
Strong learner
Weak learner
Test
Empirical Evaluation
Conclusions and Future Work

3. Introduction

4. Context & Motivations
Considering knowledge bases in the Web of Data
Goal: Inducing a classification function (predictive model) for a
target concept to assess the membership of (unseen) individuals
Supervised approach: each (training) example labeled as
positive/negative/uncertain
Possible Applications: approximate classification, semi-automatic KB
completion (assertional part)

5. Context & Motivations
Several approaches:
statistical approaches ⟶ prediction
concept learners ⟶ prediction + intensional definition
Poor model generalization (limited predictiveness)⚠

6. Context & Motivations
Weak Model 1
Strong Model
Weak Model 2
⋮ ⋮
⋮ ⋮
Weak Model n
Strong model more predictive than weak models
Ensemble Classification (boosting, bagging,...)ⓘ

7. Context & Motivations
Contribution
A framework for confidence-rated boosting for DL concept
learners
pool of weak models as concepts with the related score
each model induced to cover hardest examples (leading to
misclassifications)

8. Example
Learn a classification function for the individuals w.r.t. the target
concept .
Training set :
(instances of with a known child),
(instances of )
Ensemble Model:
Father
K = {
{
Man ⊑ Person, Woman ⊑ Person, Artist ⊑ Person, Man ⊑ ¬Woman} ∪
Man(a), Man(b), Man(c), Woman(d), Woman(f), Artist(e), Robot(g),
hasChild(a, d), hasChild(b, e)}
T
Ps = {a, b} Man
Ns = {d, f} Woman
Us = {c, e, g}
{Person [0.4], ¬Woman [0.8], Person [0.5], ∃hasChild. ⊤ [0.9]}

9. The Boosting Framework

10. Proposed Solution
Given and a set of training examples
Weight assigned to example (init. )
Ensemble of weak models (concepts) induced by a weak
learner ( DLF):
Confidence computed per weak model
Weights updated according to the coverage of the hypothesis
concept
decreased for examples covered by
increased for examples uncovered by
K T
wi ai = 1/|T|w
(0)
i
T Ct
ct
Ct
Ct

11. Coverage: Definition
An example covered by when
i.e., is correctly classified
x Ct
⎧
⎩
⎨
K ⊨ (x) if x ∈ PsCt
K ⊨ ¬ (x) if x ∈ NsCt
K ⊭ (x) ∧ K ⊭ ¬ (x) if x ∈ UsCt Ct
x

12. wDLF: Training
Given , weights associated to the examples, and a partial
description ( )
specialized via ref. operator ⟶ a set of concepts
score: difference between sum of the weights of the covered
ex.s and uncovered ex.s
heuristic: maximum difference between the scores of and
(gain)
T w
(t)
Ct = ⊤Ct
Ct ρ ⊑C
′
t
Ct
−∑
i| coveredai
wi
− −−−−−−−−
√
∑
i| uncoveredai
wi
− −−−−−−−−−−
√
C
′
t
Ct

13. Weight Updates
After is returned, weights of the examples updated:
for each example the new weight will be
where
with depending on the (ratio between the)sum of the weights of
covered and the uncovered (optimal value to minimize the training error)
update
Multiplied by for uncovered examples
Divided by for covered examples
Ct
ai w
(t)
i
= exp(− )w
(t)
i
w
(t)
i
ct
= − ⋅ (−1ct α^t )
I( , ( ))li ht ei
α^
exp ct
exp ct

14. Prediction
Given the model and a query individual
Prediction of single weak model :
if
if
otherwise
Final prediction: label according to weighted majority vote
sums of confidence values for the three labels
{ [ ]}Ct ct
T
t=1
a
Ct
+1 K ⊨ (a)Ct
−1 K ⊨ ¬ (a)Ct
0
=l
∗
argmax
l∈{+1,−1,0}
∑
t∈T
p( (x)=l)h
t
ct

15. Experimental Evaluation

16. Evaluation: Settings
5 KBs for producing 15 artificial problems/target concepts (per KB)
and related training & test sets
Parameters:
Ensemble size:
refinements generated via wDLF:
Baseline: wDLF
Other systems: C and DL-F
Boostrap procedure: 30 replications per learning problems/target
Evaluation indices:
match: exact predictions
commission: opposite predictions
omission: non definite predictions [for -label exs]
induction: definite predictions [for -label exs]
10
30
±1
0

17. Evaluation: Outcomes

18. Evaluation: Examples of Induced Models

19. Conclusions & Future Work

20. Conclusions
A boosted ensemble classifier
how to derive a weak learner
good performance for complex target concepts

21. Future Work
other (weak) learners exploiting ref.ops
dynamic ensemble dimension (validation set)
approximate reasoners for efficiency
formatted by Markdeep 1.04
✒