1. Introduction into
Cloud Computing
philip.balinov@komfo.com
DevOps Engineer

2. SOME DEFINITIONS
def. CLOUD COMPUTING
def. BIG DATA

3. SOME DEFINITIONS
def. CLOUD COMPUTING
- distributed computing over a network
- the ability to run a program or application on
many connected computers at the same time.
def. BIG DATA
- data sets so large and complex that it becomes
difficult to process using traditional data
processing applications

4. SOME DEFINITIONS, CONTD.
Q: No, seriously. What is the Cloud?
A: Well, there are three types
 Infrastructure-as-a-Service
 Platform-as-a-Service
 Software-as-a-Service

5. THE CLOUD – PROS AND CONS
WHY CLOUD COMPUTING IS SO GREAT
 Better hardware utilization
 Economy of scale
 Usage-based pricing
 In-built resilience (here be monsters)
 No front-up costs
 No long-term contracts

6. Retail
Finance
E-Commerce
Telecommunicatio
n
B2B
Publishing/Media
Government & NGO
Automotive
Travel
KOMFO'S CLIENTS

7. KOMFO PLATFORM WORKFLOW OVERVIEW

8. EXTERNAL PROVIDERS

9. OUR TECHNOLOGY

PHP, Python, C

MySQL

MongoDB, Elasticsearch

Javascript (node.js)

Ruby

Freedom to use any tool fit for the job

10. KOMFO MAIN CHALLENGES
Continuously changing workload
Fast feature changes
BigData
(Predictive) Analytics
Security
Availability

12. HOW TO USE THE CLOUD

Automation

Horizontal scaling

Break tasks into many small sub-tasks

Synchronize all the workers

Write for eventual consistency

13. HOW TO USE THE CLOUD

Beware! There are traps*
*Minimally shared with fair weighting. Allows burst when
idle resources are available. In rare cases, resources
may be throttled back under heavy host contention.
–
**Disk I/O is shared across the host.
–
***A vCPU corresponds to a physical CPU thread.
–

14. HOW TO USE THE CLOUD

There are more traps*

Coordinate tasks – good messaging
system (AMQP, DB, MemCached)

Asynchronous task execution (see
above, also API Callback hooks)

Implement transactions in software

15. CLOUD ARCHITECTURE

Messaging

Ensure communication between dynamic
number of nodes

Message-oriented middleware

Exactly-once delivery

At-least-once delivery

Transaction-based delivery

Timeout-based delivery

17. MIX & MATCH

Crunch numbers in the cloud

Application servers

Slow running tasks

Temporary services

Test servers

Automation – automatic deployment of multi-
tiered environments

18. MIX & MATCH, CONTD.

Traditional servers for:

Incompatible apps (single-threaded, memory,
disk intensive, specialized hardware) do not
work well in cloud environments

Database servers are best kept on dedicated
machines

19. OK, so we have an (endlessly)
scalable cloud app now.
What are we forgetting?

20. DATABASES, NOSQL

def. NoSQL

a mechanism for storage and retrieval of data
that is modeled in means other than the tabular
relations used in relational databases.

21. DATABASES

Postgres, Hadoop, MongoDB, Cassandra, Riak

In-memory dataset for faster operation

No predefined structure

Integrated sharding, load-balancing and failover

Versatility - can be used for anything from data
storage to real-time messaging to search indexes

22. DATABASES

Use the best tool for the job depending on the task

NoSQL Advantages

Some sources generate a lot of data

Complex interconnections, cyclical
dependencies

Aggregations must be performed on both new
and old data

Structure of foreign sources may change on
short notice

23. DATABASES

Use the best tool for the job depending on the task

NoSQL Disadvantages (Classic SQL advantages)

Not ACID compliant

No transactions

No relations between data

Lack of structure means aggregations are slow

24. DATABASES, LONG TERM STRATEGY

Data quickly becomes irrelevant

Archive it, but keep it accessible

Online Data Warehouse solutions

Amazon Redshift

Keep Everything

Terabytes for pennies

25. Summary
The cloud rocks, mmkay?

26. Questions?