Architecting for Hyper Growth and Great Engineering Culture
1. Architecting for Hyper Growth and Great
Engineering Culture
Software Architect Indonesia Community #2 Meetup
Ifnu, 26 November 2016
2. Who am I?
• Ifnu Bima
• Ilmu Komputer IPB
• Join blibli in 2012
• Was working for Deustche Bank Singapore
• Employee no 69
• Engineers no ~13
• Development Manager
– blibli.com Web UI
– Mobile Apps : Android, iOS, Windows Phone
– Content Management System
– Search
– Product Discovery : Wishlist, Product Review
3. Architecture Goals
• Hyper Growth
– X times every year: traffic, transaction, feature,
codebase and head count
• Great Engineering Culture
– Foster innovation
– Iteration speed
4. Software Architecture
Software architecture refers to the fundamental
structures of a software system, the discipline of
creating such structures, and the documentation
of these structures
Software architecture is about making
fundamental structural choices which are costly
to change once implemented
Source : https://en.wikipedia.org/wiki/Software_architecture
5. Software Architect / Principal
• Design architecture
– Monolithycs vs Microservices
– SQL vs NoSQL
– Datacenter vs Cloud
• Pick Platform, library and provide template
– Java vs PHP vs Ruby vs Scala vs Go Lang
• Hiring standard
– Technical interview
– Hiring decision
– Training and Certification
6. Functional vs Non Functional Requirement
• Functional requirement in Bussiness
Requirement Documentation (BRD)
• Non Functional Requirement
– External Quality
• Speed, Performance
• Scalability, Elasticity, Reliability
• Security
– Internal Quality
• Unit testing coverage
• Code readability, design pattern
• Learning curve for new hire
7. Speed – Performance : Server Side
• HardDrive & Database is the slowest link
• Lots lots lots of caching :
– Redis : small but lots of item
– Guava : big and slow item, refresh cache with
background process
• SSD for IO intensive operation : Database,
static & image server
• Non blocking IO using reactive system
• Slow Query monitoring and indexing
8. Speed – Performance : Client Side
• Bandwith is the slowest link
• Small bandwith : gzip, image compression,
minimize css & js, image & html lazy load
• Render first, script later: defer 3rd party script
• Layout first, content later: layout only using
CSS, inline CSS for viewport
• Reduce page load & render from 12s to 4s
9. Scalability, Elasticity, Reliability
• Stateless application
• Externalize session to repository. Currently we use Redis
• Externalize scheduler
• Microservices
– http://martinfowler.com/articles/microservices.html
• Reactive System
– Non blocking IO
– Parallel execution
– Future / callback
• NoSQL
– Solr, elasticsearch
– Redis
– MongoDB
• Go lang ?
10. Performance Monitoring
• Newrelic.com
• AppDynamic
• Realtime and Archived performance monitoring
• Performance metrics
– Response time, both server and client side
– Drill down of time spent in application
– Slow query monitoring
– Cache monitoring. Cache hit ratio.
– Server monitoring : CPU, Memory, I/O and Network
11. Performance Testing and Profiling
• JMeter and WAPT, other alternative is Gatling
• Dedicated team to run it
• Java Profiling
– VisualVM
– XRebel
13. What we learn about customer
• Fast UI
– Speed & Performance
• Website stay up during promotion and flash sale
– Scalability & Elasticity
• Consistently good user experience
– Great UI/UX
– Timely order delivery
– Original and high quality product
• New features
– Rapid Deployment
– Continuous Delivery
14. Architecting for Great Engineering Culture
• Optimize for iteration speed
• Push relentlessly toward automation
• Build the right software abstractions
• Develop a focus on high code quality
with code reviews
• Maintain a respectful work
environment
• Build shared ownership of code
• Invest in automated testing
• Allot 20% time
• Build a culture of learning and
continuous improvement
• Hire the best
15. Optimize for iteration speed
• Microservices Architecture
• Blibli.com SOA architecture circa 2013
Store Front
Back officeMerchant
ESB
16. Monolithic vs SOA vs Microservices
Source : http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.html
22. Push relentlessly toward automation
• Compile, build, unit test every commit using maven / sbt run automaticly using
bamboo CI
• Functional test by QA, automated using JBehave and Appium
• API Test using JMeter
• Release versioned artifact (war / zip) to artifactory
• Deploy to UAT, Performance environment and Production using Chef and Jenkins
– Install latest java
– Setup properties
– Install latest tomcat
– Do additional steps : setup JCE, deploy static file to static server, run database change using
Flyway / MongoBee, etc
– Deploy application
Source : http://www.martinfowler.com/articles/microservices.html
23. Build the right software abstractions
• Microservices template
– Java with Spring
– PlayFramework Scala
– Java with Spring and RXJava
– Go Lang ??
• SSO using CAS
• OAuth API abstraction using CAS
• Example of software abstractions
– Google BigTable and MapReduce
– Facebook ReactJS and ReactNative
– Amazon AWS
24. Develop a focus on high code quality with code reviews
• Use GitFlow for code management
– http://nvie.com/posts/a-successful-git-branching-model/
– Every feature and bug will have their own branch
– Developer commit to their own branch and pull request to
feature / bug / develop branch
– Peer code review and successful build from bamboo is
mandatory to merge branch
• Pair Programming if possible
25. Develop a focus on high code quality with
code reviews
• TDD and Unit testing using JUnit and KarmaJS
• Static code analysis using Sonar Cube
– Calculate unit test coverage
– Calculate code complexity, less than 3
– Code smell
– Security analysis: SQL Injection vulnerability, XSS, etc
– Potential bug in runtime: NullPointerException,
Forever loop, etc
• 90% test coverage expected. Build failed if code
coverage drop compared to previous build. If
build failed, cannot merge to mainstream branch.
26. Maintain a respectful work environment
• Open culture
• Using Slack for open communication
• Share everything
• RESPECT as core value
– Risk taking
– Excellent
– Serving
– Passionate and Proud
– Employee great place to work
– Customer focus
– Teamwork
27. Build shared ownership of code
• Access to source code from day 1
• Some degree of rotation to make sure no part
of code that only known to 1 person
28. Invest in automated testing
• We do not have QA as title, it is Software
Development Engineer in Test (SDET)
• 2017 we really focus on automation
– Bamboo CI
– Chef and Jenkins
– Nightly Build
– Data Preparation
– JBehave & Appium
– JMeter, WAPT & gatling
– Monitoring and alert
– Elasticity on Datacenter
– Etc
29. Allot 20% time
• Internal coding competition
• Internal Hackaton
• Skunk Work
30. Build a culture of learning and continuous improvement
• Rotation for people to learn different things
• Survey and Management action for people who want to change their role or team
• Weekly sharing session : BrownBag
• Training
– Java Training and Certification, Sun Certified Enterprise Architect
– iOS development & testing
– SOLR training
– 7 Habit of effective people
– Presentation and public speaking
– Project management
– Leadership essential
– Management assessment
– Evident base interview
– User Experience from Somia
• Conference
– JavaOne in San Francisco
– Solr Revolution in Austin, Texas
– TechInAsia Jakarta
• Scholarship for Master Degree
31. Hire the Best
• Blibli.com future program
– https://www.blibli.com/page/future-program/
• Employee get Employee
– Best people will attract best people, law of
attraction
• Excruciating round of interview
• Very high hiring bar
