JavaEye architcture introduction

1. JavaEye
Robbin Fan

2. • JavaEye
• JavaEye
• JavaEye
•

3. JavaEye

4. JavaEye
• 1U
•1 Web Server, 1 DB Server
• 2.6
• 3

5. Web Server
• AMD Opteron 2.4GHz *2
• 8G
• 146G SCSI

6. DB Server
• AMD Opteron 2.0GHz *2
• 4G
• 73G SCSI

8. 150 /

9. JavaEye
10
20
RSS
RSS
20
100

11. ruby ......

12. Google Adplanner Data for JavaEye

13. Google Adplanner Data for JavaEye

14. IT
CSDN 260
JavaEye 74
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

15. IT
CSDN 260 3.5X
JavaEye 74
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

16. IT
CSDN 30+ Server 260 3.5X
JavaEye 74
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

17. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

18. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

19. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

20. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

21. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

22. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

23. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38
51CTO 23
infoq 8
0 75 150 225 300

24. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38 PHP
51CTO 23
infoq 8
0 75 150 225 300

25. IT
CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38 PHP
51CTO 23
infoq 8 Java
0 75 150 225 300

26. JavaEye

27. 2006.09
•1
•
•

28. 2006.09
• lighttpd
• ruby 1.8.4, rails 1.1.2, fastcgi
• mysql5.0

29. Single Server
lighttpd
fastcgi(ruby)
mysql

30. 2007.01
• 2
• web DB
• IO

31. Web Server DB Server
lighttpd
fastcgi(ruby) mysql

32. 2007.02
• posts
• posts select count 30
0.1

33. posts
• posts(id, ..., body)
• 2GB

34. posts
• posts(id, post_text_id,...) 50MB
• post_texts(id, body) 2GB

35. 2007.03
•
• memcached CachedModel
•
• 240 sql query/s 140 sql query/s
• memcached 75%

36. Web Server DB Server
lighttpd
fastcgi(ruby) mysql
memcached

37. 2007.05
•
•

38. 2007.09
•
• ruby ferret
•

39. 2008.01
• JavaEye
• cache_fu
• 84%
• sql query 50 /s

40. 2008.05
• rmmseg-cpp

41. 2008.10
• cache plugin
• 96%

42. 2008.10
• ferret
• Java lucene
• C/S

43. Web Server DB Server
lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql
memcached

44. 2008.11
• PDF

45. Web Server DB Server
lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql
PDF Creating
memcached (ruby, C)

46. 2009.03
• SNS feed
• twitter
• API

47. Web Server DB Server RSS/API
fastcgi(ruby)
lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql
PDF Creating
memcached (ruby, C)
twitter sync

48. 2009.03
• Google Analytics
•

49. Web Server DB Server RSS/API
fastcgi(ruby)
lighttpd SearchServer
analytics (lucene)
fastcgi(ruby) mysql
PDF Creating
memcached (ruby, C)
twitter sync
DNS Server/Email Server/
NFS Server DNS Server/Email Server

50. 2009.12
• Web IM
•
•

51. Backgroud Server
Web Server fastcgi(ruby)
IM Server
lighttpd PDF Creating
SearchServer
fastcgi(ruby)
DB Server
memcached
mysql

52. JavaEye
• CachedModel+ QueryCache
• cache_fu
• cache_plugin
• cache_money*

53. •
•
•
• page cache
• frgment cache

54. •
• n
• ORM

55. • Web
• IO
•

58. CachedModel
• , Hibernate
•
• select * from post_texts where id = ?
• 75%

61. CachedModel
• AR find_by_sql
•
•

62. cache_fu
• cache_fu AR
•
• cache_fu memcached
84%

63. cache plugin
• Rails Cache 60
• n:1
1:n
• memcached 96%

64. memcached

65. memcached

66. cache_money
• twitter
• ruby cache
•

67. • JavaEye
•
• ruby

68. JavaEye
• ruby db Web Server load
DB
• ruby

69. ruby
• ruby
• erb
•
• ruby

70. post
post html

71. post
post html

72. JavaEye
• memcached 96%
• cache get : sql query = 4 : 1

73. JavaEye
• ferret
• ferret+rmmseg-cpp
•

74. • ferret+rmmseg
• ferret

75. fastcgi json protocol SearchServer
(ruby) (java)
socket
long lived processes long lived threads

76. • DNS
• Web Server
• ruby

79. Thank you!