2. A bit about Argyle Social What does Argyle do? Social Media management, engagement, & Analytics We are a small & very efficient dev team Adam (general), Mike (platform), & Josh (design)
3. AWS is Startup Friendly Feb ‘10 May ‘10 SimpleDB ‘The Server’ SimpleDB ‘The Server’ SQS MySql MySql Dec ‘10 Social RDS App Server SimpleDB MySql SQS
4. Servers Galore Jun ‘11 Social App Server URL Redir Social Social Stream Jobs MySql MySql RDS SimpleDB SQS S3 Support
5. Our current setup EC2 – 9 Servers SQS – 5 Queues SimpleDB – 1 Domain RDS – 1 DB; 5MM records Support & S3 – Just starting out Let’s dig into SQS, SimpleDB, & Support
6. SQS – Simple Queue Service Used for processing: Twitter Streams Email queue Filtering for significant items Log Processing for Goals Very easy to get started:
7. How it works Process Message Process Everything Queue Everything Process Message Process Message Process Message Queue
8. Argyle SQS Usage Twitter Site Streams Users Interweb Twitter Stream Reader Twitter Stream Reader Twitter Stream Reader Argyle Web Interface Twitter Processor Twitter Processor Twitter Processor Streams RDS Database Queue AWS Service Land
9. SQS – What we learned Not super fast Lag between pushing and seeing an item on pull Visibility timeout is key Make sure you have time to finish and delete ‘at least once delivery’ you may get something twice Not guaranteed FIFO Default size Restrictive Sizing can lead to errors 1kB to 64kB Getting Queue Url Create it and store it, don’t look it up
10. SQS - Conclusions Very easy to get bullet-proof queues No administration costs Expect some limitations that rolling or running your own won’t incur MSMQ, Spread, etc
11. Simple DB Easy storage & retrieval of Key/Value pairs Eventual consistent read vs. consistent read Grows with your data NoSQL N00bs: take your time This isn’t relational, which is obvious, but may take some time to really wrap your head around No JOIN or GROUP BY, etc
12. Argyle SimpleDB Usage Redirecting a URL Shortening a URL ar.gy/123 ar.gy/ftw ar.gy/12 Interweb Redirector Redirector Click log Cached URL Non-cached URL Memcached Memcached SimpleDB SimpleDB AWS Service Land
13. SDB – Scratch Pad *This makes your life easier, especially when starting out or debugging
14. SDB – Lessons Learned Be aware of the limits: Source: http://en.wikipedia.org/wiki/Amazon_SimpleDB#Limitations
15. SDB – Lessons Learned We’ve had some availability issues Sometimes no response Sometimes no write capabilities Storage of big items is possible, but must be linked out Can separate domains to increase performance
16. SimpleDB - Conclusions Great, easy storage Again, reduced maintenance costs Have to change your mindset from RDBMS
17. AWS Support We’re new to it We don’t expect much It can be helpful toget nagging questionsresolved This is your company on the line, right?
18. AWS Support We ‘lost’ a server; 4.5 hours after submitted: Clarification on “underlying host issue” pls; 40 minutes to reply
19. Support Conclusions Well worth the security Rumor has it that upgrading to higher levels isn’t worth it – any experiences out there? (Update: we got good feedback around higher levels of service from Adzerk) You still have to design for failure