Speakers:
Shivan Bindal - Sr. Product Manager, RightScale
Chris Szymansky - Director of Engineering, The Resumator
Software is eating the world. And it is increasingly being delivered via the cloud. We’ll cover customer examples from demo and POC environments to single-tenant and multi-tenant SaaS applications powered by public and private clouds.
4. #4#4
#RightscaleCompute
• Set up Cloud Resources
dynamically at boot time
• Provide Monitoring and
Alerts
• Enable run-time
configuration w/ scripts or
recipes
• Facilitate dev-like workflow
ServerTemplates™
6. #6#6
#RightscaleCompute
User Model with Permissions
• Each account corresponds to a set
of cloud credentials
• Accounts have complete isolation –
reporting, configuration
assets, cost tracking, etc.
• User roles and access rights are
organized “Account down” as
opposed to “User up”
• A user can access 1 or more
environments with different roles
and access rights
• SSO supported
• Users are granted rights by Admin
via permanent / temporary
invitation process
9. #9#9
#RightscaleCompute
RightScale Example
• Developers have access to test account where they can create
deployments and servers as needed.
• QA has control on staging accounts where they do rolling code
updates from development and run automated regressions and
manual functional tests.
• Operations have access to separate environment with
production account that runs RightScale on RightScale.
10. #10#10
#RightscaleCompute
Non - Traditional Use Case
Customer 2
APPAPPAPPAPPAPPAPPAPPAPPAPPAPPAPP
DB
DB
LB
Customer 1
APPAPPAPPAPPAPPAPPAPPAPPAPPAPPAPP
DB
DB
LB
Customer X
APPAPPAPPAPPAPPAPPAPPAPPAPPAPPAPP
DB
DB
LB
11. #11#11
#RightscaleCompute
• Once on-prem legacy applications *can* be brought to cloud
• Several customers instantiate deployments with single-tenant
application stacks per customer
• Cost efficiency is irrelevant if you price your application
appropriately
• Requires ingenuity for operational management considering
vast number of customers and potential application stack
upgrades
SaaS-ification-ism
15. #15#15
#RightscaleCompute
Browsers
RightScale Dashboard
Internet
Cloud
boundary
Latency-based Routing
Load Balancing Service
F5 (Primary) F5 (Standby)
Autoscaling Server Tier
IIS & .NET
IIS & .NETTech Stack
TBD
Tech Stack
TBD
MySQL Database Server Tier
MySQL Server
(Master)
MySQL Server
(Slave)Replication
AWS US EastPrivate Cloud
ELB
Autoscaling Server Tier
IIS & .NET
IIS & .NETTech Stack
TBD
Tech Stack
TBD
DirectConnect
“Cloudbursting”
MySQL Database Server Tier
MySQL Server
(Slave)
Replication
VPC
28. #28#28
#RightscaleCompute
What’s next?
• Unknown is scary if not prepared
• Stay a step ahead of growth
• Benchmark trigger points for scaling
• Iterate on your architecture, don’t perfect it (or
you’re sacrificing product)
29. #29#29
#RightscaleCompute
Engineering at The Resumator
• 7 engineers + 1 systems admin
• Release often (sometimes daily)
• Generalists
• Scrum for feature dev
• “Lite” CD with RightScale
35. #35#35
#RightscaleCompute
Development and Deployment
• Jira issue –> Git commit –> Tagged release
• RightScript checks tagged release out to test environment
• Call test kickoff on test environment
• Log test results
• Proceed to check out tagged release to [Deployment]
40. #40#40
#RightscaleCompute
Document processing – old way
• Store file on S3
• Add record in MySQL database
• Run cron to find resumes with a certain magic
status (-4)
• Send resumes to 3rd party service to extract text
• Have webhook callback to update database
42. #42#42
#RightscaleCompute
Queue development
• Research alternatives (RabbitMQ, ZeroMQ, Redis)
• Chose Redis
• Simple, fast, and free
• Takes minutes to get up and running with
RightScale
• (Researching and testing still takes weeks!)
45. #45#45
#RightscaleCompute
Results
• Initial rollout: ~30,000 jobs per day
• End user and app server response time improved by
~25%
• Expanded queue to four types of jobs
• Handled traffic growth with essentially same infrastructure
46. #46#46
#RightscaleCompute
Lessons
• Set triggers for scaling
• Iterate on your infrastructure, don’t lock it in
• Small teams benefit from generalists
• Know when to shed old tech
• MySQL -> Redis and Solr
48. #48#48
#RightscaleCompute
Summary
• Cloud is *the* way to deliver software to customers going
forward
• Your complexity is in the app, you don’t need to extend
complexity upwards
• We see a lot of ways to be successful
• Simple management features are powerful
• Find what works for you and your business
• Learn from others’ experiences
Customers are businesses, also deal with end users (applicants).Software scales to not only number of customers, but also has to scale to how popular a customer is, as jokseekers flock to their careers pages.Primary customer is SMB
I’m going to quickly run through a few different parts of the SaaS product that we offer so you can get a little background on some of the technical challenges that comes with it.
2009Founded in Pittsburgh, PA1 employee1 server
Obviously you do not want to be caught napping and have to scale when its too late
Small team, but lessons can be applicable to any startup and most companies that are cash conscious
Small team, but lessons can be applicable to any startup and most companies that are cash conscious
Architecture
EachRightScale deployment environment has a copy of our architecture.Prod is the holy grailStaging is an exact replica of prodEach other deployment has its own characteristics for testing
App delivery
How we added a new component to our infrastructure
Common activities were bringing our systems to a standstill
Document Processing RequirementsTake in files (DOC, DOCX, PDF, RTF)Convert to PDFExtract text for searchStore on S3Display in browser
Document Processing RequirementsTake in files (DOC, DOCX, PDF, RTF)Convert to PDFExtract text for searchStore on S3Display in browser****All happening immediately as soon as a resume is submitted********1.1 million resumes processed during growth****
SolutionDesign a simple queuing systemSend a job with arbitrary data to the queueOther processes, like email, used 25,000 times per day can benefit from it(Let the software guys figure out what to do with it)
Right tool for the job: Solr, Redis, VarnishKnowing when to shed old techWant to lock in infrastructure but it won’t happenGeneralists help you understand problems from code level as well as systemsDon’t scale before you have to, but project worse cases and limitationsUnderstand limitations