1. Building Internet-scale Applications:
The Beginning
By @gaveen At #RefreshColombo
22 September 2011

2. gaveens@gmail.com
Works at Thinkcube
gaveen.owain.org
DevOps Gaveen Prabhasara
twitter.com/gaveen
SysAdmin
github.com/gaveen

3. Standing on the
Shoulders of Giants
“ If I claim to be a wise man, well
It surely means that I don't know
“
- Kansas
Carry On Wayward Son

4. This is a 25 minute talk
Which sounds like a collection of
Random thoughts about
Building Internet-scale Systems
You probably already know

5. What is it?
Internet-scale Applications
Does not always mean WebApps

6. Eg: Facebook,
YouTube, Twitter,
Gmail, Flikr
Internet-scale > Enterprise-scale
Case: Facebook
750M+ active users
350M+ daily users
Case: Akamai 100B+ daily hits
95800+ servers 30B+ monthly objects shared
1000+ networks 2.5M+ websites integrated
70+ countries 130TB+ daily logs
15-30% WWW traffic *Sources: http://goo.gl/zT1bL
*Source: http://goo.gl/yVmG9 http://goo.gl/cdFPI

7. Culture
Culture
Culture
How

8. Know Your Trade
Acquire domain-specific knowledge
Learn how people build similar services
Learn about Technologies
Stay updated
RTFM / Use Search Engines ;)
Learn

9. Get Right People
Find balance in the team
Know their strengths
Nurture them
Set a proper environment
Don't step on others toes
Team

10. Go for the Best Solution
Have a solid architecture
Find a good engineering design
Don't over-engineer / over-complicate
Avoid architecture degradation
Architecture & Design

11. Right Tool for the Right Job
You favourite tool can't do everything. Face it
'Prejudice against' & 'religious for' isn't good for you
Brand names aren't for worshiping
Cloud isn't a magical silver bullet
This is Spar... the Internet. It's big
Technology

12. Do Things Right
Use best practices
Automate. Automate. Automate.
Plan for scaling from the start
Don't let the process be a burden
Process
Little more detail...

13. Test All the Time or better,
do Continuous Integration
Testing isn't a QA job
Let Devs take responsibility
XP, TDD, BDD, Cucumber, etc.
Don't push untested code
Works-in-my-machine doesn't count
Process
Little more detail...

14. Fight the Smell, Fight Anti-Patterns
Re-factoring is good
Code smell isn't something to behold
Don't let the code age on you
Don't let the architecture degrade
Bad code is bad code
Process
Little more detail...

15. Use Version Control
Use a proper VCS
Git, SVN, Hg, Bzr, etc.
Helps automation
No version control, no deploying
Learn to use a VCS properly
Process
Little more detail...

16. Use Configuration Management
Infrastructure is Code, therefore Automate
User a Configuration Management System
Puppet, Chef, CFEngine, MCollective, etc.
One-off work is (usually) a myth
Managed Infra opposed to mucked-about infra
Process
DevOps Little more detail...

17. Security isn't a Task
You can't apply security later
Both infra & code needs to be secure
“Noboday wants to hack me” is a myth
HTTPS isn't a fad
# chmod 777 kills puppies
SELinux / AppArmor isn't your enemy
Process
Little more detail...

18. Scaling starts from the beginning
Plan early. Start from basics.
Balance load & plan for HA
Use Caching
Do stress / load testing
Not every technology scales gracefully
Use messaging, async jobs when needed
Etc., etc.
Process
Little more detail...

19. Monitor & Measure
Capture important metrics
Measure, analyze & improve
Monitoring sucks. But still...
Home grown a NMS aren't necessarily better
Have notification alerting in place
Process
Little more detail...

20. Q&A
Anybody who wants ISO's of the following Linux distros,
meet me with USB drives after the session.
BackTrack, CentOS, Debian, Fedora, Hanthana, MeeGo, Mint,
Ubuntu, Ubuntu Server