Yoni Goldberg presented on Gilt's transition to a microservices architecture. Some key points: - Gilt started in 2007 using Ruby on Rails but faced scaling issues as traffic grew. - They transitioned services to the JVM in 2009, starting with 10 initial services, which solved many problems but not developer pain points. - Gilt fully embraced microservices around 2012, breaking the monolith into hundreds of small, independent services using Scala and Play. - This empowered teams, enabled simpler deployments, and made the system more robust, but introduced new challenges around service discoverability, dependencies, and monitoring. - Gilt developed many tools to help manage microservices at scale, such

1. BUILDING A MODERN
MICROSERVICES
ARCHITECTURE @
QCON LONDON - 2015
YONI (JONATHAN) GOLDBERG

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/microservice-arch-gilt

3. Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. - GiltDirect, Sale Personalization, Loyalty, SEO,
Post-purchase, Login/Registration
- MIT CS BS/Meng | Google | IBM | IDF
- Israel | Brooklyn | Coffee | JS/Node | Arduino | Running |
Kite Surfing | Poker

7. THREE TAKE AWAYS
FROM GILT'S STORY
Which problems will be solved?
Which challenges will you face?
Is it the right choice for you?

8. Flash Sales Business
Founded in 2007
Top 50 Internet-Retailer
~150 Engineers
WHAT IS GILT?

10. ANOTHER WAY TO LOOK AT
GILT

11. THE CLASSIC STARTUP
STORY

12. THE EARLY DAYS
2007 - Ruby on Rails
the hottest new thing
The goal was to get to market fast

14. WE WERE ABLE TO HANDLE OUR
TRAFFIC PRETTY WELL

15. UNTIL LOUBOUTIN CAME TO
GILT

16. TECHNOLOGY PAIN POINTS -
2009
Spike required to launch 1,000s of ruby processes
Postgres was overloaded
Routing traffic between ruby processes sucked

17. DEV PAIN POINTS
1000 Models/Controllers, 200K LOC, 100s of jobs
Lots of contributors + no ownership
Difficult deployments with long integration cycles
Hard to identify root causes

18. WE NEEDED TO SOLVE
THE PROBLEM FAST

19. THREE THINGS HAPPENED
Started the transition to the JVM
M(a/i)cro-Service Era Started
Dedicated data stores

20. WHY JVM?
Widely adopted
Stable
Better support for concurrency
Better GC vs MRI

21. FIRST 10 SERVICES

23. WE SOLVED 90% OF OUR SCALING
PROBLEMS
BUT NOT THE DEVELOPERS PAIN
POINTS

24. SOLVED PAIN POINTS
Spike required to launch 1,000s of ruby processes
Postgres was overloaded
Routing traffic between ruby processes sucked

25. STILL OPEN PAIN POINTS
New services became semi-monolithic
1000 Models/Controllers, 200K LOC, 100s of jobs
Lots of contributors + no ownership
Difficult deployments with long integration cycles

26. WHY WE DOUBLED DOWN ON
MICRO-SERVICES
Empower teams and ownership
Smaller scope
Simpler and Easier deployments and rollbacks

27. WE BEGAN THE TRANSITION TO
SCALA AND PLAY
LOSA - LOTS OF SMALL (WEB) APPS
[SAME AS MICRO-SERVICES BUT FOR WEB-APPS]

29. AS OF LAST WEEK WE HAVE AROUND
300 SERVICES IN PROD

30. WHY THE INCREASE?

31. APP BOOTSTRAP
rake bootstrap:admin-web # Bootstrap a admin-web service
rake bootstrap:client-server-core # Bootstrap a client-server-core service
rake bootstrap:play # Bootstrap a play service
rake bootstrap:play-ui-build # Bootstrap a play-ui-build service
rake bootstrap:sbt-library # Bootstrap a sbt-library service
rake bootstrap:schema # Bootstrap a schema service

32. DEMO

40. HOW TO DEFINE A
MICROSERVICE?
FUNCTIONALITY SCOPE
NUMBER OF DEVS INVOLVED

41. "A SERVICE-ORIENTED ARCHITECTURE
COMPOSED OF LOOSELY COUPLED
ELEMENTS THAT HAVE BOUNDED
CONTEXTS"
- ADRIAN COCKCROFT

42. CURRENT CHALLENGES
Deployments and Testing (Functional/Integration)
Dev/Integration Environments
Service Discoverability
Who owns this service!?
Monitoring

43. MICRO SERVICE
DEPLOYMENT V3

44. CHALLENGES THAT WERE
SOLVED IN V2:
Frustrating to deploy semi-manually (Capistrano)
Scary to deploy other teams services
Hard to execute functional tests between services

45. SBT
Motivation: Scala adoption
Complex Scala syntax
Cool features: ~test, shell, console
Hard to debug

46. GILT-SBT-BUILD
Simple config for all the services
Pulls many plugins:
[nexus, testing, RPMs, run scripts, Monitoring, SemVer, ...]
Custom commands (e.g 'sbt release')

47. ION-CANNON + SBT
Run tests on dedicated Env
Dark canary releases
Easy rollbacks
Integrated health checks

49. DATA CENTER
LIMITATIONS
&&
MORE TECHNOLOGY
OWNERSHIP

50. IMMUTABLE
INFRASTRUCTURE

51. V3
Per department AWS account and budget
Longer deploys [New instances / CNAME swaps]
Each team has AWS expertise

53. API LOVE

54. WELL DEFINED REST APIS SOLVE
DISCOVERABILITY, DOCUMENTATION
AND INNER ADOPTION

55. APIS @ GILT
and
Describe the API in simple json
Auto generates versioned docs, routes and clients
Per team - API design committee
www.apidoc.me Swagger.io

59. DEPENDENCY FUN [DEMO]

64. "MID-TIER MICRO-SERVICE"
BIGGEST PERFORMANCE CHALLENGE
NETWORK IO

66. ON DEV/INTEGRATION ENVIRONMENTS
The hardware is not strong enough
No one wants to compile 20 services
Service Dependencies

67. EACH TEAM HAS A STAGING
ENV
SERVICE_PORTS=[
4001, #listing-service
8235, #svc-user-set
9420, #svc-free-fall
7895, #svc-Loyalty
8155, #web-loyalty
9410, #web inventory status
7898, #admin-loyalty
7899, #notification
7102, #rouge
9530, #svc-component
6802, #svc-waitlist-submit
4066, #svc-action-sale
....
PORT_FORWARD_ARGS=SERVICE_PORTS.map { |port|
['-L', "#{port}:localhost:#{port}"]

68. STAGING DIFFICULTIES:
Hard to keep all the services up to date
Maxed our staging env capacities
Shared data across legacy apps

69. WITH OUR MIGRATION TO AWS
[TEAM RESPONSIBILITY]
A pool of integration instances and dbs
Develop with prod instances [Not a fan]

70. ON OWNERSHIP
"code stays much longer than people" - SB

71. CODE OWNERSHIP

72. CURRENT APPROACH
Code Review!Code Review!Code Review!
Team owns services, not individual developers
Ownership transfer

75. DATA OWNERSHIP

76. WE TRANSITIONED TO
MICRO-DBS
MICROSERVICES NEEDS THEIR OWN
MONGODB | POSTGRES | RDS |
VOLDEMORT

77. MANAGE MICRO-RELATIONAL
DBS
SCHEMA EVOLUTION
MANAGER
https://github.com/gilt/schema-evolution-manager

78. PRINCIPLES OF
SCHEMA EVOLUTION
MANAGER
Independent from the service code
Manages the schema evolutions in a Git repo
Schema changes are deployed as tar flies
No rollbacks
Schema changes are required to be incremental

79. ON MONITORING

81. THE TOOLS WE USE
GRAPHITE / OPENTSDB

87. BUT IT WASN'T ENOUGH

88. CAVE
HTTP://CAVELLC.GITHUB.IO
CONTINUOUS | AUDIT | VAULT |
ENTERPRISE

89. ALERT EXAMPLES [SCALA]
orders [shipTo: US].sum.5m < 1000
response-time [svc: svc-team, env: prod].p999 > 1000

91. SUMMARY

93. WHAT'S NEXT ?
BUILD YOUR NEXT FEATURE
IN A NEW SERVICE

94. @yoni_goldberg
jgoldberg@gilt.com
QUESTION TIME
www.yonigoldberg.com

95. Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations/
microservice-arch-gilt