Session2 01.emad mohamed

Arabic-SOS
Segmenter, Stemmer, and Orthography Standardizer for the Arabic
Cultural Heritage
Emad Mohamed & Zeeshan Sayyed
May 2019
Emad Mohamed & Zeeshan Sayyed SOS: Segmenter, Stemmer, and Orthography Standardizer for the Arabic Cultural HeritageMay 9, 2019 1 / 47

Background
Emad Mohamed
Senior Lecturer, Research Group in Computational Lingustics,
University of Wolverhampton
Morphological Analysis, Syntactic Analysis, Computational Corpus
Linguistics, Language Resources
Zeeshan Ali Sayyed
PhD Candidate in Computer Science, Indiana University
Machine Learning, NLP, Parsing Morphologically-rich Languages
Research associate in the Arabic Cultural Analytics project at Doha
Institute for Graduate Studies, Qatar.

Roadmap
Segmentation
What is it, and why do we need it?
Data & Methods
Experiments & Results
Substandard Orthography
The Problem
Data & Methods
Experiments & Results
Eﬀect of Substandard Orthography on Segmentation
Stemming as a by-product of segmentation

whatissegmentation?
andwhydoweneedit?

AlktAb
Al
DEF
ktAb
NOUN

wAlktAb
w
CONJ
Al
DEF
ktAb
NOUN

wllktAb
w
CONJ
l
PREP
Al
DEF
ktAb
NOUN

wlktAbhm
w
CONJ
l
PREP
ktAb
NOUN
hm
POSS

Why is segmentation important
Word2vec is an algorithm for ﬁnding related words.
word2vec(book): book, books, novels, novel, manuscript, author,
ﬁction, essay, poem, poems
word2vec(ktAb): ktAb, AlktAb, wAlktAb, ktb, Alktb, llktAb, ktAby,
bAlktAb, fAlktAb, ktAbnA

Why is segmentation important
Segmentation is a required step for, or signiﬁcantly improves:
POS tagging
Parsing
Named Entity Recognition
Machine Translation
Lexical Analysis
...

Previous Work
Segmentation is a hot topic in Arabic NLP.
Many systems exist: MADA, AMIRA, MADAMIRA, FARASA
These systems handle Modern Standard Arabic or Colloquial Arabic
These systems fail on the Arabic cultural heritage. This is really
worrying given that Arabic is a continuum.
We focus on pre-MSA Arabic in this talk, but we are working
on a universal model for the Arabic language.

Previous Work
System MSA Accuracy CA Accuracy
MADAMIRA 98.3 94.3
FARASA 98.5 86
Table: Performance on Modern Standard Arabic and on Classical Arabic

Our approach to segmentation
Data & Annotation
Experiments
Results
Problems and Solutions

Annotation
Randomly selected a corpus from the Qur’an, Hadith, Islamic Law,
Islamic Philosophy, and the Al-Manar Magazine (1898-1935)
Segment it using a model built on the ATB
Select the most frequent ngrams, pass these to an annotator.
Iteratively do this to build a test set, training set, and dev set.

Set Source #Words
train 1 Al Manar 85 312
train 2 Al Manar + Classical 141 766
dev Al Manar 23 786
test 1 Al Manar 24 005
test 2 Classical 5 299
Table: Statistics of the datasets used for the experiments

Gradient Boosting Machines
Sequential Ensemble Method.
Uses Regression Decision Trees.
Multiple Iterations
Each subsequent iteration focuses on those parts of the problem that
the previous iterations got wrong.

Gradient Boosting Machines
Sequential Ensemble Method.
Uses Regression Decision Trees.
Multiple Iterations
Each subsequent iteration focuses on those parts of the problem that
the previous iterations got wrong.
”There are only two Machine Learning algorithms: Gradient Boosting
& Neural Netweorks”

Features for Segmentation

Experiments: Algorithms
Compare SVM’s, CRF’s and GBM’s
Gradient Boosting Machines produce the best results
XGBOOST
CATBOOST (Yandex)
LightGBM (Microsoft)
Settled on CATBOOST: best results

Al Manar Classical
CRF-Baseline 92.7% 94.96
SOS (Manar) 97.18% 97.17
SOS (Manar + Classical) 97.45 98.47
Table: Baselines segmentation accuracy using CRF

System Accuracy
SOS-Manar 97.17
SOS-Manar + Classical 98.47
Mohamed (2018) 96.8
MADAMIRA 94.7
SAPA 86.47
Table: Comparison with other segmenters on classical test set

Feature Ranking
Feature Value Feature Value
focus 15.6501 prev word suffix 4.2443
next2letters 11.857 chr position 3.2133
prev2letters 8.8664 minus2 3.1651
focus word prefix 7.8821 minus3 2.7478
plus1 7.3599 plus4 2.5857
focus word suffix 6.9752 plus5 2.566
plus3 6.7646 following word prefix 2.5203
plus2 5.5329 minus4 2.1905
minus1 4.7142 minus5 1.1644
Table: Feature importances ranked by the model

Takeaways
We have a good segmenter that achieves almost the same as the
MSA segmenters with less than one quarter of the data that has some
noise.
The context does not seem to help much as the most important
features are local.
Error analysis shows that ambiguity is the main culprit: most of the
ill-segmented words are ambiguous.
When we tried our segmenter on data available online, the
results were obviously worse. The reason: Substandard
Orthography

Substandard Orthography

The diﬀerent forms of hamza

Part of the stem
Question word: Do, Does, Is, Has, etc ..

Part of the stem. Cannot be segmented.

Assimilated question word. Must be segmented.

Accusative marker. May be segmented.

Figure: Three forms of hamza forming minimal pairs

t vs. h

t vs. h
Part of the stem: Cannot be segmented
3rd Person singular pronoun. Must be segmented.
3rd Person singular possessive pronoun. Must be segmented.

t vs. h
Almost always a singular feminine marker. May be segmented

y vs. a

y vs. a
Part of the stem. Cannot be segmented
First person possessive pronoun. Must be segmented.
First person pronoun. Must be segmented.
Imperfective preﬁx. May be segmented
17 diﬀerent functions in the Arabic Treebank.

y vs. a

Standardizing the Orthography
Data
Methods
Evaluation
Eﬀect of Orthography Standardization on Segmentation

Standardizing the Orthography: Data
Most available data are sub-standard
Formal publications: serious newspapers, magazines and books are
usually standard
The IslamWeb Library published over 1000 books all of which
rigorously checked and proofread.
We substandrdize this data
We select a sub-corpus of 35,666,914 words

Representing substandard orthography

Handling Substandard Orthography
We use the same set of features as we do in segmentation

Orthography Standardization Results

Eﬀect of Standardization on Segmentation

Stemming
Stemming can be derived from segmentation through a rule-based
system
Remove all the aﬃxes. Whatever remains is the stem
Theoretically you need POS tagging to disambiguate some rare cases
of ambiguity
Practically, those cases are so rare the number never get aﬀected
Stemming is at least as accurate as segmentation

Long-standing Problems & Possible Solutions
Problem Solution
Most of the errors in seg-
mentation are ambiguous
words
Widen the context to include n previ-
ous/following words, Add more data
(hard), synthetic data
Data imbalance in
orthography standardiza-
tion
Try several methods of under-
sampling for the dominant class(es)
Substandard sometimes
beats standardization
Still examining the problem

Ongoing & Future Research
Treat both segmentation and standardization as sequence to sequence
problems
Joint standardization and segmentation
Create artiﬁcial data for segmentation (successful ﬁrst attempts)
Trasnfer learning with contextual embeddings

Thank You
Github Repo: https://github.com/zeeshansayyed/ArabicSOS Email:
Zeeshan: zasayyed@indiana.edu
Emad: e.mohamed2@wlv.ac.uk
thankyouallforcomingandhopeyoufoundthisuseful
pleasefeelfreetoasksuggestorcriticise
”there is no such thing as a stupid question, only
stupid answers”

Session2 01.emad mohamed

Recommandé

Recommandé

Contenu connexe

Plus de IMPACT Centre of Competence

Plus de IMPACT Centre of Competence (20)

Dernier

Dernier (20)

Session2 01.emad mohamed