Leontinus Alpha Edison dalam webinar Startupbisnis x Codepolitan menjelaskan bagaimana Tokopedia.com melakukan scaling teknologinya

1. Scaling Tokopedia
Past, Present, Future

2. Once Upon a Time
In Jakarta, Jan 2009

5. 1 Product Guy
and 1 Half Engineer
as co-founder

6. Never have experience to
manage a high traffic website
Don’t have business
background AT ALL

7. Perl as back end
Build our own perl framework
Apache Mod Perl
Oracle Express Edition

11. Hm… looks like we need a
better front end designer

16. AwStats and
A little bit Google Analytic

17. CBN
apache server oracle server
Network
Topology
Apps
Topology
Internet
apache server
oracle server
http req http resp
sql

18. 2 co-founder
1 real engineer
1 cust care

21. Hooray, WE LAUNCH!!

22. IDR 33 Mio of GMV
in the first month

23. WE ARE SLOW!!!

24. * We didn’t have storage
* pictures uploaded is stored on the same machine
* Web page & static content is served by single apache
* We didn’t use CDN
* We didn’t even know what is CDN
WHY??

25. Network
Topology
CBN
apache app
server
oracle serverapache static
server

26. Apps
Topology
Internet
apache app server
oracle server
http req http resp
sql
Internet
apache upload / statis server
oracle server
http upload http resp
sql
Internet
apache upload / statis server
http req http resp
access web page upload pictures read static
pictures
css + js

27. We are back in business

28. BUT WE ARE SLOW AGAIN!!!

29. * Oracle express edition reach it’s limit
* No Partition
* No Replication
* Poor indexing
* Read/Write and Query on the same Master DB.
WHY??

30. SO WE
MIGRATE TO

31. Network
Topology
CBN
apache app
server
PostgreSQL Masterapache static
server
PostgreSQL Slave

32. Apps
Topology
Internet
apache app server
PostgreSQL Master
http req http resp
sql insert
sql update
sql delete
PostgreSQL Slave
sql iquery
WAL streaming
Replication

33. We did it again!!!!

34. DAMN SEARCH IS SLOW!!!

35. * We have a lot of new products every second
* We have to show search results in real time
* But every second the sorting keep changing
* PostgreSQL load is just too much!!!
WHY??

36. And Many
More……..

37. SEARCH IS EASY !!!!

38. Come on Man….
SLOW AGAIN??

39. * We were using apache + mod perl
* Apache consume a lot of resource
* Our code has a lot of memory leak
WHY??

40. * We found out about NginX is very light and fast
* We use nginx as load balancer
* Replace apache modperl with nginx-perl
* We have 1 nginx load balancer with several
nginx-perl servers
* For load balancing method, we mix round robin
and clustering
SOLUTION

41. siege -c100 -t5s -i -b -q 'http://www.tokopedia.com/ebenhaezer'
siege: invalid option -- 'q'
siege: invalid option -- 'q'
** SIEGE 2.72
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 14788 hits
Availability: 100.00 %
Elapsed time: 4.59 secs
Data transferred: 63.50 MB
Response time: 0.03 secs
Transaction rate: 3221.79 trans/sec
Throughput: 13.83 MB/sec
Concurrency: 87.52
Successful transactions: 7481
Failed transactions: 0
Longest transaction: 0.43
Shortest transaction: 0.00

42. Apps
Topology
PostgreSQL Master
sql insert
sql update
sql delete
PostgreSQL Slave
sql iquery
WAL streaming
Replication
Internet
http req http resp
NginX Load Balancer
nginx-perl #1 nginx-perl #2 nginx-perl #3 nginx-perl #n
proxy_pass
SOLR
Import
SOLR
query

43. Now what….
Storage??

44. * Hardware limitation
* We used SATA HDD not SSD
* Disk Utilities 100%
* No back up, No Failover
* Capacity is critical
* Users keep uploading pictures
WHY??

45. User

46. We also use CDN

47. AFTER ALL
WE ARE STILL SLOW!!!

48. SOLUTION
Internet
nginx-perl #1
PostgreSQL
Master
http req http resp
nginx-perl #2 nginx-perl #3 nginx-perl #n
NginX Load Balancer
proxy_pass
PostgreSQL
Slave
replication
MongoDB
primary
MongoDB
secondary
replication
SOLR
Redis
query & update
3rd Party API such as
Logistics, Banks,
Payment Gw
ETC
Internet

49. We Start To Know About
NginX, NoSQL
In-Memory Storage
GlusterFS Storage
Scale out (not scale up)
and many more…..
Lesson Learn??

50. Thanks to our
Awesome Engineers
and many more…

52. We are back in business

53. BUT …………..

54. For the first time in our
life we were doomed!!!

55. * One of our GlusterFS Server is
broken. Image read/write is super slow.
* We were using version of postgresql
which has some bugs on indexing.
WHY??

56. Another Awesome Engineers
Mixed with International Team

57. Current State

58. New VP of
Engineering

60. FUTURE

62. * Mobile First Company
* Zero Downtime
* Full to Cloud
* Re-architech to SoA
* Open API to Public
* Deploy New Tech, such as replace perl with Go Lang
* Advance Alert & Monitoring

63. * Redundancy and Failover
* Multiple 3rd party
* Datawarehouse such as Cubes, Pentaho etc
* Machine Learning, Business Intelligence
* Build things that can be share with others
* Really pay attention on security
* and many more……

64. What if the problems
come from ISP?
Unsolved Issues

65. * User cannot access Tokopedia
* Pictures are not showing
* css and js are not loaded
* Sometime it just show a blank page
* Some ISPs do Ads Injection
* ALL WITHOUT REASONS
FACTS

66. WHY??
WE DON’T KNOW
BUT SOMETHING HAPPEN
ON ISP SIDE

67. Works well
* Using NginX Geo Module
* All HTTPS since Q4 2014
* Try CDN Load balancing
Don’t work at all
* Talked to ISP
* “Fight” in idEA
What we’ve done

68. Don’t think “someone else
will join and take care of
this” — Mike Krieger of Instagram

69. Whether you think you
can, or you think you can’t,
you’re right — Henry Ford

70. THANK YOU
ANY QUESTIONS?