The document presents a method for joint English spelling error correction and part-of-speech (POS) tagging for language learners' writing. It proposes analyzing the text by deleting word boundaries and building a lattice to find the lowest cost path that considers both spelling corrections and POS tags. An experiment on ESL learner corpora shows the joint method improves over baselines in both POS tagging accuracy by up to 5.9% and spelling correction recall by 19.5%, demonstrating the mutual benefit of jointly modeling spelling and syntax.

1. Keisuke SAKAGUCHI
Joint English Spelling Error Correction and
POS Tagging for Language Learners Writing
Keisuke SAKAGUCHI, Tomoya MIZUMOTO,
Mamoru KOMACHI, and Yuji MATSUMOTO
Nara Institute of Science and Technology
(NAIST), Japan
@COLING 2012, Mumbai, India.

2. Keisuke SAKAGUCHI
Keywords
• Morphology
• English as a Second Language (ESL) leaners' writing
• POS tagging, Word Segmentation, Spelling Correction
2

3. Keisuke SAKAGUCHI
1. Background
2. Proposed method
3. Experiment
4. Analysis
3

4. Keisuke SAKAGUCHI
1. Background
2. Proposed method
3. Experiment
4. Analysis
4

5. Keisuke SAKAGUCHI
Spelling correction as preprocessing
5
Spelling Correction
POS tagging

6. Keisuke SAKAGUCHI
Spelling Errors in ESL writing
• typographical: *begginning/beginning
• merge: *dresscode/dress code
• split: *now a days/nowadays
• homophones: *hear/here
• confusion: *quiet/quite
• inflection: *please/pleased
• derivation: *badly/bad
6

7. Keisuke SAKAGUCHI
POS tagging for misspelled texts
7
I am quiet good.
('I', 'PRP'), ('am', 'VBP'), ('quiet', 'JJ'), ('good', 'JJ'), ...
'quite', 'RB'
She felt really badly after ...
... ('really', 'RB'), ('badly', 'RB'), ('after', 'IN'),
'bad', 'JJ'
Spelling errors interfere with POS tagging

8. Keisuke SAKAGUCHI
POS tagging for misspelled texts
8
... research and some analysys.
... ('and', 'CC'), ('some', 'DT'), ('analysys', 'NNS')
CANDIDATES: ('analysis', 'NN'), ('analyses', 'NNS')
POS tags help spelling correction.
I am hear to help you.
('I', 'PRP'), ('am', 'VBP'), ('hear', 'VB'), ...
CANDIDATES: ('hear', 'VB'), ('here', 'RB')

9. Keisuke SAKAGUCHI
Problems are related to each other
9
Spelling Correction
(will be improved by POS tags)
POS tagging
(interfered with spelling correction)

10. Keisuke SAKAGUCHI
Problems are related to each other
10
Spelling Correction
(will be improved by POS tags)
POS tagging
(interfered with spelling correction)

11. Keisuke SAKAGUCHI
Problems are related to each other
11
Spelling Correction
(will be improved by POS tags)
POS tagging
(interfered with spelling correction)
Joint analysis

12. Keisuke SAKAGUCHI
For merge & split errors
12
We donʼ’t have a casual dresscode.
'dress', 'NN' 'code', 'NN'
Word boundary disambiguation
(Japanese, Chinese, etc.)

13. Keisuke SAKAGUCHI
1. Background
2. Proposed method
3. Experiment
4. Analysis
13

14. Keisuke SAKAGUCHI
Joint spelling correction and POS tagging
Algorithm
1. delete all word boundaries of input
repeat //build a lattice of the input
2. read next letter and look up lexicon
until the end of the sentence
3. find the path with the lowest cost
14

15. Keisuke SAKAGUCHI
Example
15
I am hear to help you.

16. Keisuke SAKAGUCHI
Example
16
Iamheartohelpyou.
1. delete all word boundaries of input

17. Keisuke SAKAGUCHI
Example
17
Iamheartohelpyou.
I
(PRP) Lexicon
SURF : POS, COST, SPELLING
I : PRP, cost_̲of_̲I, I
2. read next letter and look up lexicon

18. Keisuke SAKAGUCHI
Example
18
Iamheartohelpyou.
I
(PRP)
am
(VBP)

19. Keisuke SAKAGUCHI
Example
19
Iamheartohelpyou.
I
(PRP)
am
(VBP)
hear
(VB)
here
(RB)
Lexicon
SURF : POS, COST, SPELLING
hear : VB, cost_̲of_̲hear, hear
hear : RB, cost_̲of_̲here, here
“hear” is also regarded as “here”

20. Keisuke SAKAGUCHI
Example
20
Iamheartohelpyou.
I
(PRP)
am
(VBP)
hear
(VB)
here
(RB)
to
(TO)
heart
(NN)

21. Keisuke SAKAGUCHI
Example
21
Iamheartohelpyou.
I
(PRP)
am
(VBP)
hear
(VB)
here
(RB)
to
(TO)
help
(VB)
you
(PRP)
.
(.)

22. Keisuke SAKAGUCHI
Example
22
Iamheartohelpyou.
I
(PRP)
am
(VBP)
hear
(VB)
here
(RB)
to
(TO)
help
(VB)
you
(PRP)
.
(.)
3. find the path with the lowest cost

23. Keisuke SAKAGUCHI
Proposed method (in detail)
• Cost calculation: Conditional Random Fields (CRF)
• Decoding: Viterbi Algorithm
• Correct Lexicon:
– SURF : POS, COST, SPELLING (= SURF)
• Lexicon of Spelling Errors
– SURF : POS, COST, SPELLING (≠ SURF)
– Typographical errors:
• error pairs from ESL learners corpus + speller suggestions
– Other spelling errors:
• error pairs from ESL learners corpus
– COST are copied from Correct Lexicon
23

24. Keisuke SAKAGUCHI
1. Background
2. Proposed method
3. Experiment
4. Analysis
24

25. Keisuke SAKAGUCHI
Experiment
• Goal: POS tagging and spelling correction
accuracy on the proposed method
– POS tagging
– Spelling Correction
Spelling
Correction
POS tagging
POS tagging
(baseline) Spelling Correction
(with LM)
POS tagging
Spelling
Correction
(with LM) Spelling
Correction
POS tagging
BASELINE PIPELINE JOINT
JOINTBASELINE
VS
VS
VS
25

26. Keisuke SAKAGUCHI
Experiment
• Dataset (ESL learners corpora)
– Cambridge Learners Corpus First Certificate in English
(CLC FCE) dataset
• Reference POS tags are automatically assigned
– Konan-‐‑‒JIEM learner corpus (KJ corpus)
• Reference POS tags are annotated by human
• Tools
– MeCab 0.98 (cost calc. + decoding)
– NAIST English Dictionary 0.2 (NAIST edic)
– GNU Aspell (speller)
– Language Model (IRSTLM + Google 1T corpus)
26

27. Keisuke SAKAGUCHI
Results (POS tagging)
27
Precision Recall F-‐‑‒value
CLC-‐‑‒FCE BASELINE 0.950 0.971 0.960
PIPELINE 0.961
(+1.1%)
0.976
(+0.5%)
0.968
(+0.8%)
JOINT 0.982
(+3.2%)*†
0.979
(+0.8%)
0.981
(+2.1%)*†
KJ corpus BASELINE 0.794 0.857 0.824
PIPELINE 0.827*
(+3.3%)
0.857
(±0.0%)
0.842
(+1.7%)
JOINT 0.853
(+5.9%)*†
0.871
(+1.4%)*†
0.862
(+3.8%)*†
* : statistically significant improvements over the baseline (p < 0.05)
† : statistically significant improvements over the pipeline (p < 0.05)

28. Keisuke SAKAGUCHI
Results (Spelling correction)
28
Precision Recall F-‐‑‒value
CLC-‐‑‒FCE BASELINE 0.519 0.427 0.468
JOINT 0.445
(-‐‑‒7.3%)*
0.622
(+19.5%)*
0.519
(+5.0%)*
* : statistically significant differences over the baseline (p < 0.05)

29. Keisuke SAKAGUCHI
1. Background
2. Proposed method
3. Experiment
4. Analysis
29

30. Keisuke SAKAGUCHI
Analysis
• True positives (typographical)
30
INPUT: … it is a surprice birthday party.
ANSWER: surprise (NN)
BASELINE: sur (JJ) price (NN)
PIPELINE: surprise (NN)
JOINT: surprise (NN)

31. Keisuke SAKAGUCHI
Analysis
• True positives (homophone)
31
INPUT: I am hear to help you.
ANSWER: here (RB)
BASELINE: hear (VB)
PIPELINE: hear (VB)
JOINT: here (RB)

32. Keisuke SAKAGUCHI
Analysis
• True positives (derivation)
32
INPUT: She felt really badly after ...
ANSWER: bad (JJ)
BASELINE: badly (RB)
PIPELINE: badly (RB)
JOINT: here (RB)

33. Keisuke SAKAGUCHI
Analysis
• True positives (typo + merge)
33
INPUT: ... a big swimingpool .
ANSWER: swimming (NN) pool (NN)
BASELINE: swim (RB) ing (VBP) pool (NN)
PIPELINE: seemingly (RB)
JOINT: swimming (NN) pool (NN)

34. Keisuke SAKAGUCHI
Error Analysis (Spelling)
• False positives
34
INPUT: After the concer finished ...
ANSWER: concert
BASELINE: cancer
JOINT: cancer
INPUT: technology has made life for ...
ANSWER: made
BASELINE: made
JOINT: mad

35. Keisuke SAKAGUCHI
Error Analysis (Spelling)
• False negatives
35
INPUT: ... your shown one evening ...
ANSWER: show
BASELINE: shown
JOINT: shown
INPUT: I was terrible hungry ...
ANSWER: terribly
BASELINE: terrible
JOINT: terrible

36. Keisuke SAKAGUCHI
Summary
• Spelling Errors in ESL writing
– typographical, homophone, confusion, split, merge, inflection,
derivation
• Joint spelling correction and POS tagging
– Morphological analysis with word boundary disambiguation
• correct lexicon + lexicon of spelling errors
• Result
– Joint model outperforms both baseline and pipeline.
36

37. Keisuke SAKAGUCHI
Thank you for listenning!
listening
37

38. Keisuke SAKAGUCHI
38

39. Keisuke SAKAGUCHI
Statistics of ESL learners corpora
39

40. Keisuke SAKAGUCHI
Evaluation metrics
40

41. Keisuke SAKAGUCHI
Features of CRFs
41

42. Keisuke SAKAGUCHI
Statistics of CLC-‐‑‒FCE spelling errors
42