This document discusses Amazon SWF (Simple Workflow Service) and how it can be used with Ruby for task scheduling and workflow automation. It introduces the Gordon framework for building deciders and activities with Ruby. Key features of SWF are highlighted such as task lists, timeouts, and event history. The document concludes with information about job opportunities at Peritor GmbH.

1. Amazon SWF
Ruby User Group Berlin
Peritor GmbH

2. EC2 & Eucalyptus
Cluster Management
§ Automatic Configuration
§ Self Healing Clusters
§ Auto Scaling Clusters
§ One Click Deployment
§ One Click Cluster Cloning
§ Monitoring & Alerting
§ Logs & Accounting
§ Firewalls & Backups
§ Access & Rights Management
2

3. Architecture Web
App
DB
3

4. Architecture Web
App
EC2
S3
DB
Euca
4

5. Architecture Web
App
EC2
S3
DB
RabbitMQ
Euca
Agent
5

6. RabbitMQ
6

7. RabbitMQ
http://www.heavensend-design.com/happy/
7

8. Agent/Queue Requirements
§ Scalability
We manage thousands of instances
§ Availability Reliability
We need to be available even if an AWS region fails
8

9. DJ?
9

10. Resque?
10

11. Hmmmmm….
11

12. 12

13. SWF: Simple Workflow Service
From http://aws.amazon.com/swf/
§ Maintains application state
§ Tracks workflow executions and logs their progress
§ Holds and dispatches tasks
§ Controls which tasks each of your application hosts will
be assigned to execute
13

14. SWF: Simple Workflow Service
“Workflows as a Service”
§ Hosted
§ Scalable
§ Consistent (uses DynamoDB for state)
14

15. SWF: Pricing
§ $0.0001 per workflow execution
§ Until a Workflow Execution completes:
$0.000005 per workflow execution for every 24-hours
§ $0.000005 per retained workflow execution per day
+ traffic
15

16. SWF: Workflow
Activity 1
Activity 2
Activity 3
Start Observe Send
Instance
Start
Init JSON
Cleanup
Activity 4
16

17. SWF: Architecture
http://docs.amazonwebservices.com/amazonswf/latest/developerguide/swf-welcome.html?r=8856
17

18. SWF: Decider
§ Implements workflow logic
§ Gets history of events
§ Decides what to do next:
§ Call next activity
§ Abort workflow
§ Re-schedule
§ …
18

19. SWF: Worker
§ Implements activity logic
§ Gets input for current activity
§ Processes activity:
§ Complete
§ Fail
§ Timeout
§ Heartbeat
19

20. Separate Task Order/Graph
From Task Implementation
Worker
Decider
20

21. 21

22. Gordon
Ruby framework to use SWF
§ Syntactic sugar for creation of deciders and activities
§ Daemons for processing tasks
§ Unicorn-style master/child handling of processes
22

23. Gordon: Decider
23

24. Gordon: Decider
24

25. Gordon: Activity
25

26. Gordon: Start Workflow
26

27. Gordon: Processing
27

28. Gordon: Processing
28

29. Gordon: config.decider.yml
29

30. DEMO
30

31. Available soon at http://github.com/scalarium/gordon
Under Apache 2 license
31

32. SWF: Simple Workflow Service
Many nice features
§ Retention period
§ Task lists
§ Timeouts
§ Event history
§ Abort/cancel/re-schedule
§ Delay
§ Mark count
§ ...
32

33. We’re Hiring!
DevOps Ruby Developers
§ Scalarium operations § Rails frontend
§ Write and maintain § Ruby backend / agents
Chef cookbooks § Chef
§ Improve and scale backend § Scaling evented agents
§ Security § API
§ Help us support more OSs § Command line tools
§ Vagrant integration § EC2, S3, SWF, SQS, …
jobs@scalarium.com
33