Glynn Bird – Cloudant – Building applications for success. All too often, web applications are built to work in development but are not capable of scaling when success arrives. Whether the application is a log aggregator that can't deal with the throughput, a blog that can't handle traffic when it hits the heights of Google's rankings or a mobile game that goes viral, an application can become the victim of its own success. By building with Cloudant from the outset, and architecting the application to scale by design, we can build apps that scale as the traffic, data-volumes and users arrive. Using several real-life use cases, this talk will detail how Cloudant can solve an application's data storage, search and retrieval needs, scaling easily with success!

1. © 2014 IBM Corporation
Building Applications for Success
Glynn Bird, Developer Advocate

2. Agenda
• Application Development
• Scaling
• Real-world examples & how can Cloudant help?

3. Introduction
Glynn Bird
Developer Advocate
Cloudant
http://glynnbird.com
@glynn_bird glynn@cloudant.com

4. Development - Go-to tools
Basic
1980s 1990s 2000s 2010s
Pascal
C
C++
Forth
PHP4
Java
ObjC
PHP5
Node.js
File System
NoSQL SQL

5. Scalability?
• Pick your tools of choice
• PHP + MySQL, Ruby + Mongo
• Build on your machine
• Deploy!
• Scalability is an ops problem?
• Hint: “no”

6. Scaling tip #1 - Keep things separate
• Separate
• load balancing
• app
• database
• cache
• Separation of layers allows
independent scaling
Google’s first server rack
http://bit.ly/10xGe2Y

7. Scaling tip #2 - Keep app servers stateless
• Store all persistent data in a
database
• Don’t be tempted to store
sessions on local disk
• Stateless app servers can be
created and destroyed at will
IBM 305 RAMAC 5Mb Disk - 1957
http://bit.ly/10xGe2Y

8. Scaling tip #3 - queue-driven workflow
• Store tasks in queues
• Workers servers consume the
queue
• Queues allow workers to be
scaled at will
http://bit.ly/1Aycjse

9. Scaling tip #4 - Automate everything
• No need to manually
configure servers
anymore
• Workers servers
consume the queue
• It should be as easy to
deploy N servers as 1
server
http://bit.ly/1Aycjse

10. What does failure to scale look like?
http://bit.ly/1Aycjse

12. What does failure to scale look like?
• increased latency
• downtime
• high maintenance costs
• data loss
• disgruntled customers
http://bit.ly/1Aycjse

13. Cloudant Highlights
• Founded in 2009 by data scientists:
Hoffman, Miller & Kocoloski
• 20,000+ freemium and 225+ paying
customers
• Key collaboration & investment
agreement with Samsung Electronics
• HQ’d in Boston with offices/presence in
Seattle, San Francisco, Bristol, UK &
Seoul, SK
• 34 global locations on 4 hosting
providers
• Acquired by IBM in February 2014 into
the Information Management software
group

14. Cloudant NoSQL DBaaS
App Layer / Browser / Mobile App
REST (HTTPS)
Cloudant JSON API
Incremental
MapReduce Security Monitoring
Geo-Load Balancing
Indexed
JSON
Storage
Physical Storage @ IBM SoftLayer, Rackspace, Microsoft Azure, AWS
JSON
CRUD
Multi-
Master
Replication
Device
Sync
Lucene
Search
2D, 3D
Geospatial
IO
Control
Managed for you 24x7 by Cloudant developer-operators

15. How can Cloudant help you to scale?
Cloudant empowers clients to get their applications to market faster in a cost-effective,
hassle-free delivery model with guaranteed performance and support.
• Scales massively & elastically
• Handle millions of daily active users
• Guaranteed performance & up time
• Removes risk of project & SLA delivery failure
• More agile development for web & mobile
• No rigid schemas to slow development
• Managed for you 24x7
• Stay focused on new development, no DB administration

16. Example 1 - IoT Data Logging
• Temperature sensor £1.50
• Raspberry Pi £25
• Network attached
thermometer
DS18B20

17. DS18B20
Software
• Bash script
• Where should
we put the data
?
sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`
filename="/sys/bus/w1/devices/$sensorid/w1_slave"
T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`
t=${T:0:2}.${T:2:3}
unix=`date "+%s"`
d=`date "+%Y-%m-%d %H:%M:%S %Z"`
json="{ "ts": $unix, "date": "$d", "temperature": $t, "sensor_id":
"$sensorid" }"

18. DS18B20
Storage
• Local CouchDB
• Installed with
one line
sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`
filename="/sys/bus/w1/devices/$sensorid/w1_slave"
T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`
t=${T:0:2}.${T:2:3}
unix=`date "+%s"`
d=`date "+%Y-%m-%d %H:%M:%S %Z"`
json="{ "ts": $unix, "date": "$d", "temperature": $t, "sensor_id":
"$sensorid" }"
curl -X POST -H "Content-Type: application/json" -d "$json" "http://localhost:
5984/logger/"

19. Where does Cloudant fit in?
• Replicate to
Cloudant
• Scale by adding
more sensors
• Data collection
can be offline
• Sync when
connected

20. Demo
• http://celsius.mybluemix.net

21. Why is Cloudant a good fit?
• Replication
• collect data at the edge, replicate when online
• Cloudant can store data at massive scale with high
availability
• Time-series data is a great fit for Cloudant

22. Time-series data in Cloudant
• MapReduce indexes
on date field
• Allows range querying
with grouping e.g. find
min/max/avg between
two dates, grouped by
year/month/day/hour
etc.
{
"_id": "00339bc7",
"_rev": "1-0ef01b53",
"ts": 1411723490,
"date": "2014-09-26 09:24:50 UTC",
"temperature": 20.937,
"sensor_id": "28-000006746cde"
}
[2014,9,26,9,24,50] ---> 20.937

23. Example #2 - Mobile applications
• Mobile game
• Stores games state, usage stats,
achievements etc. locally
• Needs to be cross-platform

24. Option 1 - PouchDB + HTML5
• In-browser, cross-platform
datastore
• CouchDB compatible
• Syncs with Cloudant
http://pouchdb.com/

25. Option 2 - CloudantSync
• iOS and Android
native libraries
• Local storage + query
• Syncs to and from
Cloudant
https://cloudant.com/product/cloudant-features/sync/

26. One database per user

27. One database per user
• Each user’s data has it’s own access control
• Each database is relatively small; small enough to fit
on phone
• Scales massively; hundreds of thousands of
concurrent users
https://cloudant.com/product/cloudant-features/sync/

28. Demo
• Business Card collection app
• Native Android App
• Companion Web App
https://cloudant.com/product/cloudant-features/sync/

29. How did that work?
Android Phone App
pouchdb
IBM BlueMix
Web App

30. What about the code?
var db = new PouchDB("myfirstdatabase");
db.post( { a:1, b: "2", c: true } );
db.sync(URL, { live: true });
https://cloudant.com/product/cloudant-features/sync/

31. Why is Cloudant a good fit?
• Replication is a hard problem. PouchDB + Cloudant solve it
• One database per-user is a great way to scale mobile apps

32. Conclusion
• Building with Cloudant as your data layer has become a “go
to” development solution
• Building with Cloudant builds in scale from day one
• large data sets
• lots of smaller data sets
• Cloudant syncs!
https://cloudant.com/product/cloudant-features/sync/