Contenu connexe Similaire à Github - Down the Rabbit Hole (20) Plus de Vagmi Mudumbai (11) Github - Down the Rabbit Hole4. DISTRIBUTED
FAST
TOTALLY AWESOME
@vagmi, @dharanasoft
8. Service Frontend / Proxy Balancer
Analytics/ App App App App
Server Server Server Server Queue
Monitoring
/
Services
Async
Persistence Persistence
Server
Cluster Cluster
GENERIC ARCHITECTURE
@vagmi, @dharanasoft
9. Smart HTTP HTTP
SSH Git
HTTP API Frontend
ONLY ONE PROBLEM
Multiple Service Points
@vagmi, @dharanasoft
10. • Git relies on filesystem to store its repository
• BLOB objects
• Packfiles
• Git GC
• SSH relies on filesystem (~/.ssh/authorized_keys)
• Ergo, GitHub is bloody hard to scale
TECHNICAL CHALLENGES
@vagmi, @dharanasoft
13. A really smart
ldirectord lb1a lb1b load balancer
8 core
fe1 fe2 fe3 fe4
16 GB
u0 u1 u2 u3 u4 u5 u6 u7 16 Unicorn
u8 u9 ua ub uc ud ue uf Processes
8 core, 16 GB
db1a db1b 15k RPM SAS drives
THE HTTP STACK
@vagmi, @dharanasoft
15. u0 u1 u2 u3 u4 u5 u6 u7 16 Unicorn
u8 u9 ua ub uc ud ue uf Processes
16 ProxyMachine
ProxyMachine fe1 fe2 fe3 fe4
Instances
Smoke, Chimney
Maps user/repo
db1a db1b
to machine/folder
8 Core, 16 GB
Ernie fs1a fs2a fs3a fs3b 6x300 GB
Grit fs1b fs2b fs3b fs4b 15k RPM SAS
RAID 10
@vagmi, @dharanasoft
17. u0 u1 u2 u3 u4 u5 u6 u7 16 Unicorn
u8 u9 ua ub uc ud ue uf Processes
memcache1 memcache2 memcache3 memcache4
fs1b fs2b fs3b fs4b
12 GB of memcached on each fs slave
@vagmi, @dharanasoft
19. A really smart
ldirectord lb1a lb1b load balancer
Custom SSHD
Gerve fe1 fe2 fe3 fe4
Chimney
db1a db1b
fs1a fs2a fs3a fs3b
git
fs1b fs2b fs3b fs4b
SSH STACK @vagmi, @dharanasoft
20. A really smart
ldirectord lb1a lb1b load balancer
ProxyMachine fe1 fe2 fe3 fe4
Chimney
db1a db1b
fs1a fs2a fs3a fs3b
gitd
fs1b fs2b fs3b fs4b
GITD STACK @vagmi, @dharanasoft
28. http://ruby-lang.org, http://rubyonrails.org/
https://github.com/mojombo/ernie,
http://www.erlang.org/, http://bert-rpc.org/
http://git-scm.com
http://nginx.org/
http://unicorn.bogomips.org/
http://mysql.com/
ldirectord http://horms.net/projects/ldirectord/
http://www.drbd.org/
@vagmi, @dharanasoft
Notes de l'éditeur \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n