2. Suffix Array
Lucene

3. ✓Suffix Array
✓Lucene × Suffix Array
✓Suffix Array
✓

4. ※ →
abcde → abcde bcde cde de e

8. O(n^2 log n)

11. ✓Suffix Array
✓Lucene × Suffix Array
✓Suffix Array
✓

12. Java
http://lucene.apache.org/java/docs/
Analyzer
CJKAnalyzer
(n-gram
JapaneseAnalyzer
(
※JapaneseAnalyzer sen

13. •
-
•
-
•
- 10,000

14. • Lucene
- Version: 2.4.1 (newest:3.0.0)
- Index n-gram Index (CJKAnalyzer)
- Search Algorithm Inverted Index
• Suffix Array
- Version: full scratch(Java)
- Index Suffix Array build by ʻtwo stage algorithmʼ
- Search Algorithm binary search

15. •
- Server: Fujitsu RX200
- Cpu Intel Xeon CPU E5540 2.53GHz(Core 2 Quad)
- Memory 4GB
- HDD: 2TB
•
-
Suffix Array
-

16. •
-
-

17. ※ ms

18. ※ ms
Lucene : Suffix Array = 6:1

19. ※ ms
Lucene : Suffix Array = 6:1
40:1

20. Suffix
Array
※
(´ ω `)

21. ※Suffix Array

22. ※Suffix Array

23. Suffix Array

24. Suffix Array ※
※
→

25. → (´ ω `)
Suffix Array
Hadoop Suffix Array

26. Suffix Array
[ ]PFI Sedue Wikipedia Suffix
Array 25
→

27. ✓Suffix Array
✓Lucene × Suffix Array
✓Suffix Array
✓

28. 1.
2. 1. Suffix Array
3. 2.
Suffix Array

30. _
_
_ _
_
_ _
※ ”_” null 0x00

31. _
_
_ _
_
_ _
Suffix Array

32. 1. Suffix Array
_
_
_
_
_
_
_
_
_
_

33. 1. Suffix Array

34. 1. Suffix Array

35. 1. Suffix Array
2. Suffix Array
3.

36. Binary Search)
pivot)

50. Suffix Array
•
•
• Suffix Array
• Suffix Array
• Suffix Array

51. • (CSA,FMIndex
• LZ

52. ✓Suffix Array
✓Lucene × Suffix Array
✓Suffix Array
✓

53. • Hadoop
•
GZIP or xxxxLZ Compressed Suffix
Array FM Index
•
•