自然言語処理における構文解析器の構造と解釈

1. ⮬↛ゝㄒฎ⌮䛻䛚䛡䜛 ᵓᩥゎᯒჾ䛾ᵓ㐀䛸ゎ㔘 Structure and Implementation of the Parsers for Natural Language Processing JOIss2014 @zakuro9715

2. ⮬ᕫ⤂௓

3. YuZakuro @zakuro9715 ᫂▼㧗ᑓ ༟⌫(㏆␥3఩) HN twitter Ꮫᰯ ㊃࿡

4. ᡂᯝ≀

5. https://github.com/zakuro9715/shino

6. 䛸䛣䜝䛷

7. 䝇䝷䜲䝗䛾ᯛᩘ䛜

8. 101ᯛ

9. 䛒䜛䛾䛷ᛴ䛞┠䛷䜔䜚䜎䛩

10. What is Parser? ᵓᩥゎᯒჾ 䛳䛶䠛

11. ᵓᩥゎᯒჾ ● ୚䛘䜙䜜䛯䝖䞊䜽䞁ิ䜢ゎᯒ䛧䚸 ᩥ䛾ᵓ㐀䜢⾲䛩䝕䞊䝍ᵓ㐀(୰㛫⾲⌧)䛻ኚ᥮ 䛩䜛䚹

12. ᵓᩥゎᯒჾ ● ୚䛘䜙䜜䛯䝖䞊䜽䞁ิ䜢ゎᯒ䛧䚸 ᩥ䛾ᵓ㐀䜢⾲䛩䝕䞊䝍ᵓ㐀(୰㛫⾲⌧)䛻ኚ᥮ 䛩䜛䚹構文木・AST・S式など

13. せ䛩䜛䛻…

14. I am You

15. ᩥ ྡモྃ ே⛠௦ྡモ I ືモྃ ືモ am ྡモྃ ᅛ᭷ྡモ You

16. Tips 1. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ 2. 䛔䜝䛔䜝䛺୰㛫⾲⌧

18. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ ● ᵓᩥゎᯒ䛻䛛䛡䜛䛯䜑䛻䛿䚸ᩥ䜢䝖䞊䜽䞁䛻ศゎ䛩䜛ᚲせ䛜䛒䜛 ● 䝖䞊䜽䞁䜈䛾ศゎ䛿Ꮠྃゎᯒჾ䛜⾜䛖

19. You are PRO ᩥᏐิ

20. [You, are, PRO] 䝖䞊䜽䞁ิ

21. 䛒䛺䛯䛿䝥䝻䛷䛩 ᩥᏐิ

22. [䛒䛺䛯, 䛿, 䝥䝻, 䛷䛩] 䝖䞊䜽䞁ิ

23. 䝥䝻 ᩥᏐิ

24. [䝥䝻] 䝖䞊䜽䞁ิ

26. S-Expression (S (NP (PeopleN (‘I’))) (VP (V (‘am’)) (NP (ProperN ('You')))))

27. Syntax Tree S NP PeopleN I VP V am NP ProperN You

28. 䛭䛾௚䛔䜝䛔䜝 ● Abstract Syntax Tree ● Reverse Polish Notation ● Polish Notation

29. Context Free Language ᩥ⬦⮬⏤ᩥἲ

30. ᩥ⬦⮬⏤ᩥἲ ● ᩥἲ䜢ᐃ⩏䛩䜛䛯䜑䛻౑⏝䛥䜜䜛ᩥἲ ● ṇつ⾲⌧䛷䛿ᐃ⩏䛷䛝䛺䛔ᩥἲ䜢ᐃ⩏䛷䛝䜛

31. ౛ S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes the dog 䛾ᵝ䛺ᩥἲ䜢⏕ᡂ䛩䜛CFG

32. Implements of typical parsers ௦⾲ⓗ䛺ᵓᩥゎᯒჾ䛸䛭䛾ᐇ⿦

33. ௦⾲ⓗ䛺ᵓᩥゎᯒჾ ● Recursive Descent Parser ● Shift-Reduce Parser

35. ≉ᚩ ● ẚ㍑ⓗᐇ⿦䛜ᐜ᫆䛷䛒䜛 ● 䜶䝷䞊䝯䝑䝉䞊䝆䜢⾲♧䛩䜛䛾䛜ᐜ᫆

36. ཎ⌮ ● ධຊ䛻ᑐ䛧䛶ᩥἲ䜢๓䛛䜙㡰䛻㐺⏝䛥䛫䜛

37. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”

38. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog

50. ௨ୗ␎

51. ᙅⅬ ● ᕥ෌ᖐ ● Ỵᐃ୙⬟ᛶ

52. ᕥ෌ᖐ ● ྑ䛾䜘䛖䛺CFG䜢⪃䛘䜛䛣䛾CFG䛿௵ពಶ䛾”a”䜢୪䜉䛯ᩥᏐ ิ䜢⾲䛩䛜䚸䝟䞊䝇䛩䜛䛸↓㝈䝹䞊䝥䛻㝗䜛 S -> S A | A A -> “a”

53. a a a a a S A -> -> S A | A “a”

54. a a a a a S A -> -> S A | A “a”

55. a a a a a S A -> -> S A | A “a”

56. a a a a a S A -> -> S A | A “a”

57. a a a a a S A -> -> S A | A “a”

58. a a a a a S A -> -> S A | A “a”

59. a a a a a S A -> -> S A | A “a”

60. a a a a a S A -> -> S A | A “a”

61. ゎỴ᪉ἲ ● ᩥἲ䜢௨ୗ䛾䜘䛖䛻ኚᙧ䛩䜛 S S_ A -> -> -> A S_ “” | A S_ “a”

62. Ỵᐃ୙⬟ᛶ ● ௒ぢ䛶䛔䜛䝖䞊䜽䞁䛷ᵓᩥ䜢Ỵᐃ䛩䜛䛾䛷䚸ఝ䛯 ᵓᩥ䛜䛒䜛ሙྜ䛿Ỵᐃ䛷䛝䛺䛔䛣䛜䛒䜛 ● 䛯䛸䛘䜀ୗ䛾䜘䛖䛺ᵓᩥ䛿A䜢ぢ䛯᫬Ⅼ䛷䛿Ỵ ᐃ䛷䛝䛺䛔 S A B C -> -> -> -> A B | A C “a” “b” “c”

63. ゎỴ᪉ἲ ● A䜢ぢ䛯᫬Ⅼ䛷䛿Ỵᐃ䛷䛝䛺䛔䛾䛷䚸䜒䛖䜂䛸䛴 ඛ䜢ぢ䛶ุ᩿䛩䜛䚹 S A B C -> -> -> -> A B | A C “a” “b” “c”

64. ゎỴ᪉ἲ ● ඛㄞ䜏䜢ቑ䜔䛩䜋䛹ィ⟬㔞䛜ቑ䛘䛶䛔䛟䛯䜑⌧ ᐇⓗ䛷䛿䛺䛔

66. ≉ᚩ ● LLἲ䛜ⱞᡭ䛸䛩䜛ᩥἲ䜢ゎᯒ䛷䛝䜛 ● 䜶䝷䞊䝯䝑䝉䞊䝆䜢⾲♧䛩䜛䛾䛜㞴䛧䛔

67. ཎ⌮ ● 䝖䞊䜽䞁䜢䝇䝍䝑䜽䛻✚䜏䛴䛴䚸䝇䝍䝑䜽䛾ୖ䛛䜙䝹䞊䝹䜢㐺⏝䛥䛫䜛 ● 䝇䝍䝑䜽䛻✚䜐ືస䜢Shift, 䝹䞊䝹䜢㐺⏝䛩䜛 ືస䜢Reduce䛸࿧䜆

68. a cat likes a dog

69. cat likes a dog a Shift

70. cat likes a dog Reduce Det

71. likes a dog Det Shift cat

72. likes a dog Reduce N Det

73. likes a dog Reduce NP

74. a dog NP Shift likes

75. a dog Reduce V NP

76. dog a NP Shift V

77. dog Reduce Det V NP

78. dog Det NP Shift V

79. Reduce N Det V NP

80. Reduce NP V NP

81. Reduce S

82. Recursive Descent Parser 䛾ᙅⅬ ● ᕥ෌ᖐ ● Ỵᐃ୙⬟ᛶ

83. ᕥ෌ᖐ ● ⮬᫂䛻Ⓨ⏕䛧䛺䛔

84. ᕥ෌ᖐ ● ᕥ෌ᖐ䛷↓㝈䝹䞊䝥䛻㝗䜛ཎᅉ䛿䚸ᩥἲ䛻ᑐ 䛧䛶䝖䞊䜽䞁䜢ᙜ䛶䛶䜖䛟䛛䜙 ● 䝖䞊䜽䞁䛻ᑐ䛧䛶ᩥἲ䜢ᙜ䛶䜛᪉ἲ䛷䛿Ⓨ⏕䛧䛺䛔

85. Ỵᐃ୙⬟ᛶ ● Recursive Descent Parser䛸ྠᵝ䛾ၥ㢟䛜Ⓨ⏕ 䛧䛺䛔䛣䛸䛿᫂䜙䛛

86. Ỵᐃ୙⬟ᛶ ● Recursive Descent Parser 䛷䛿᭱ึ䛾䝖䞊䜽䞁䜢Ⓨぢ䛧䛯᫬Ⅼ䛷ᩥἲ䜢Ỵᐃ䛧䛯 ● 䝖䞊䜽䞁䜢ぢ䛶䚸㐺⏝䛷䛝䜛ᩥἲ䜢᥈䛩᪉ἲ䛷䛿 ྠᵝ䛾ၥ㢟䛿Ⓨ⏕䛧䛺䛔

87. LLἲ䛾ᙅⅬ䜢ඞ᭹

88. 䜔䛳䛯䛽

89. 䜔䛳䛯䛽䛷䜒…

90. Shift-Reduce Parser 䛾ᙅⅬ ● Shift-Reduce⾪✺ ● Reduce-Reduce⾪✺

92. Shift-Reduce ⾪✺ ● ୗ䛾䜘䛖䛺ᩥἲ䛷䚸”a”䛜䝇䝍䝑䜽䛻䛒䜛䛸䛝 Reduce䛩䜛䛛Shift䛩䜛䛛䛜Ỵᐃ䛷䛝䛺䛔 S A B -> -> -> A | A B “a” “b”

93. ゎỴ᪉ἲ ● ඛㄞ䜏

95. Reduce-Reduce ⾪✺ ● ୗ䛾䜘䛖䛺ᩥἲ䛷䚸”a”䛜䝇䝍䝑䜽䛻䛒䜛䛸䛝䚸A1 䜢㐺⏝䛩䜛䛛A2䜢㐺⏝䛩䜛䛛Ỵᐃ䛷䛝䛺䛔 S A1 A2 -> -> -> A1 ‘x’ | A2 ‘y’ “a” “a”

96. ゎỴ᪉ἲ ● ඛㄞ䜏

97. Summary 䜎䛸䜑

98. ● Recursive Descent Parser 䛿ᐇ⿦䛜⡆༢ ○ 䛯䛰䛧ᙅⅬ䜒ከ䛔 ● Shift-Reduce Parser 䛿ᗄ䛴䛛䛾ᙅⅬ䜢ゎᾘ ○ ᪂䛧䛔ᙅⅬ䛜…

99. ⤖ㄽ

100. ඛㄞ䜏

101. 䛤Ύ⫈䛒䜚䛜䛸䛖䛤䛦䛔䜎䛧䛯

自然言語処理における構文解析器の構造と解釈

Recommandé

Recommandé

Contenu connexe

Dernier

Dernier (20)

En vedette

En vedette (20)

自然言語処理における構文解析器の構造と解釈