TECH Meetup For MM Engineers In JP#2 Beat the Application Development with our Notes

1. The Abstract that Machine Learns to do
Languages Translation
Sai Htaung Kham
2019/02/15
@Tech Meetup For MM Engineers in JP #2

2. What is Language Translation?

3. - I decided to visit Myanmar so that I could meet and understand the locals in
person.
- Although there are many technologies out there, I do believe that the AI will be
one of the future technologies.

4. - I decided to visit Myanmar so that I could meet and understand the locals in
person.
- ဒေသခံဒ ွေန ဲ့ပုဂ္ဂိုလ်ဒရေးအရ ဒ ွေွေ့ဆံုနဂုင်ဒအောင် မြန်ြောမပည်ကဂု သွေောေးဖဂု ဲ့ကျွန်ဒ ော် ဆံုေးမဖ ်
လဂုက် ယ်။
- Although there are many technologies out there, I do believe that the AI will be
one of the future technologies.
- အေီြော နည်ေးပညောဒ ွေအြ ောေးကကီေး ရဂဒပြယဲ့် AI ဟော အနောဂ ် နည်ေးပညော စ်ခု
မဖစ်လဂြဲ့်ြယ်လဂု ဲ့ကျွန်ဒ ော် ထင် ယ်။
Translated by Google Translate

5. What are the building blocks of Language Translation
in Computer Science?

6. NLP (Natural Language Processing)

7. What is NLP?
Natural Language Processing is a range of technologies that enable machines to
understand, analyze and respond appropriately to natural language.
By Cedric Wagrez (Gengo)
This allows interacting with computers with natural language instead of
computer code.

8. Common Tasks in NLP
- Name Entity Recognition
- Sentiment Analysis
- Information Extraction
- Text Generation
- Machine Translation and so on.

9. Name Entity Recognition

10. Sentiment Analysis

11. A bit about Machine Translation
- Rule-based machine translation.
- Example-based machine translation.
- Statistical machine translation.
- Neural machine translation.

12. Example of Rule-based Machine Translation
Input I am a student .
Intermediate Layer ကျွန်ုပ် မဖစ်သည် စ်ဒယောက် ဒက ောင်ေးသောေး ။ Direct Translation
Output
ကျွန်ုပ် ဒက ောင်ေးသောေး စ်ဒယောက် မဖစ်သည် ။
Filter out or Rephrasing with defined
rules

13. Example-based Machine Translation
English Japanese
How much is that red umbrella? Ano akai kasa wa ikura desu ka.
How much is that small camera? Ano chiisai kamera wa ikura desu ka.
https://en.wikipedia.org/wiki/Example-based_machine_translation

14. Statistical Machine Translation - 1
http://people.csail.mit.edu/koehn/publications/tutorial2003.pdf

15. Statistical Machine Translation - 2
http://people.csail.mit.edu/koehn/publications/tutorial2003.pdf

16. Statistical Machine Translation - 3
https://thefrenchturtle.wordpress.com/2013/01/16/the-effectiveness-of-online-translators/

17. Statistical Machine Translation - 4

18. Neural Machine Translation
- Machine Learning, Deep Learning Based Approach.
- End to End Translation.

19. Some old but gold techniques
- 2014 - Sequence to Sequence
- 2014, 2015 - Attention Mechanism
- 2017 June - Transformer Architecture - Google AI
- 2018 May - Universal Language Model Fine Tuning - fast.ai
- 2018 November - BERT - Google AI
- 2019 February - OpenAI GPT2
- 2019 July - ERNIE - Baidu Research
- 2019 July - RoBERTa - Facebook Research
- 2019 August - MegatronLM - NVidia
- 2020 January - Turing Natural Language Generation (T-NLG)

20. State of the art Models with parameter counts
https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/?OCID=msr_blog_turing_hero

21. Part of Speech (POS) Tagging
ြဒကွေေး ခရဂုင် သည် အဒနောက် ဘက် ဧရောဝ ီ မြစ်ကြ်ေး စ်ဒလ ောက် ွေင် နံုေး င် ဒမြနုရပ်ကွေက် မဖစ် သည် ။
Noun Noun
Post-positional
Marker Noun Noun Noun Noun Noun
Post-positional
Marker Noun Noun Verb
Post-positional
Marker Punctuation

22. POS Tagging - 1
'(?:(?<!္)([က-ဪဿ၊-၏]|[၀-၉]+|[^က-၏]+)(?![္္ ]?[္္်္ ဲ့]))'
https://github.com/ye-kyaw-thu/sylbreak
https://github.com/swanhtet1992/ReSegment
- ဝဏ္ဏ (Syllable) Splitting
၂၀၁၈ခုနစ်အောရအောေးကစောေးပပဂိုင်ပွေ ွေင် အောေးကစောေးနည်ေးအဒရအ ွေက် ဂုေးမြငဲ့်လောခဲ့
၂၀၁၈', 'ခု', 'နစ်', 'အော', 'ရ', 'အောေး', 'က', 'စောေး', 'ပပဂိုင်', 'ပွေ', ' ွေင်', ' ', 'အောေး', 'က', 'စောေး', 'နည်ေး', 'အ', 'ဒရ', 'အ', ' ွေက်', ' ', ' ဂုေး', 'မြငဲ့်', 'လော', 'ခဲ့'

23. POS Tagging - 2
https://github.com/ye-kyaw-thu/sylbreak
https://github.com/swanhtet1992/ReSegment
- Training the Model
Model
ြ ဒကွေေး
Noun Noun

24. POS Tagging - 3
https://arxiv.org/pdf/1409.0473.pdf
Bahdanau et al., 2015

25. POS Tagging - 4
https://arxiv.org/pdf/1409.0473.pdf
20% 30% 10% 40%
This is an apple
ေါကပန်ေးသီေး

26. POS Tagging - 5
https://arxiv.org/pdf/1409.0473.pdf
I decided to visit Myanmar so that I could meet and understand the
locals in person.
ဒေသခံဒ ွေန ဲ့ပုဂ္ဂိုလ်ဒရေးအရ ဒ ွေွေ့ဆံုနဂုင်ဒအောင် မြန်ြောမပည် ကဂု သွေောေးဖဂု ဲ့
ကျွန်ဒ ော် ဆံုေးမဖ ် လဂုက် ယ်။

27. POS Tagging - 6
https://arxiv.org/pdf/1409.0473.pdf
Bahdanau et al., 2015
This is an apple
ေါကပန်ေးသီေး
20% 30% 10% 40%
Encoder
Decoder
Attention Layer

28. POS Tagging - 7
https://arxiv.org/pdf/1409.0473.pdf
Bahdanau et al., 2015
20% 30% 10% 40%
Encoder
Decoder
Attention Layer
နံုေး င် ဒမြနုရပ်ကွေက် မဖစ် သည်
Noun Noun Verb
Post-positional
Marker

29. POS Tagging - 8

30. POS Tagging - 9