Antons talking about patterns of Infrastructure Automation in the OpenSlava conference. Bratislava, October 2013

1. Antons Kranga
DevOps:Cloud Automation
October 11, 2013
Organizers
Top Media Partner
Media Partner
Supporter

2. Presenter
- Technology Architect at Accenture
- 10+ years experience in Enterprise Development
- Projects:
- Accenture Foundation Platform for Java
- Accenture Cloud Platform
- ALPJ
- Monsoon

3. Ask questions online
www.sli.do/openslava

4. Cloud
Copyright © 2013 Accenture All rights reserved.

5. This is cloud

6. Definition of Cloud Computing
Copyright © 2013 Accenture All rights reserved.

7. Cloud Computing
Copyright © 2013 Accenture All rights reserved.

8. Accenture Cloud Platform
Testing as Service
Monsoon
ALPJ
etc
Cloud Broker API
Copyright © 2013 Accenture All rights reserved.

9. Challenges of Cloud (What IaaS Offers)
Web Tier
Application Tier
Database
Copyright © 2012 Accenture All rights reserved.

10. Challenges of Cloud (What Users Want)
Web Tier
Application Tier
Database
Copyright © 2012 Accenture All rights reserved.

11. Infrastructure services
and patterns

12. example

13. OS Type: RedHat
OS Version: 6.2
Tomcat Home: /etc/tomcat
Tomcat Port: 8080
MySQL IP: 10.10.0.1
MySQL Root Password: password
include_recipe “java“
package "tomcat6“
artifact_deploy "tomcat6“ do
version "latest“
artifact_location "foo.war"
deploy_to node[:tomcat][:work_dir]
end
service "tomcat6" do
action [:enable, :start]
end
Attributes
Recipe
OS Specific Instructions

14. Chef Workstation
Copyright © 2012 Accenture All rights reserved.
Virtualized Infrastructure

15. Virtualized Infrastructure
Chef Workstation
Chef Server
REST
CMDB
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

16. Virtualized Infrastructure
Chef Workstation
Chef Server
REST
CMDB
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

17. Virtualized Infrastructure
Chef Workstation
knife
Uses SSH to
bootstraps Chef
client
Chef Server
REST
CMDB
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

18. Virtualized Infrastructure
Chef Workstation
knife
Chef Server
e
ch
t
en
li
fc
at6
c
tom
REST
CMDB
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

19. Virtualized Infrastructure
Chef Workstation
knife
Chef Server
e
ch
t
en
li
fc
at6
c
tom
REST
HTTP call signed
with certificate
CMDB
Cookbooks
Roles
...

20. Virtualized Infrastructure
Chef Workstation
knife
Chef Server
e
ch
t
en
li
fc
at6
c
tom
REST
CMDB
Cookbooks
Roles
...

21. Virtualized Infrastructure
Chef Workstation
knife
Chef Server
e
ch
t
en
li
fc
at6
c
tom
REST
CMDB
Cookbooks
t6
ca
m a
to jav les
ab
pt
i
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

22. Virtualized Infrastructure
Chef Workstation
knife
ohai
acts
oad f
upl
Chef Server
e
ch
t
en
li
fc
at6
c
tom
REST
CMDB
Cookbooks
t6
ca
m a
to jav les
ab
pt
i
Cookbooks
Roles
...
Copyright © 2012 Accenture All rights reserved.

23. Service discovery

24. Node 1
Node 2
Node 3
Node N

25. Node 1
Node 2
Register
Node 3
Register
Register
Database
Node N
Register

26. Node 1
Node 2
Register
Node 3
Register
Register
Discover nodes
Database
Node N
Register

27. Node 1
Node 2
Register
Node 3
Register
Register
Discover nodes
Database
Node N
Register

28. Load balancing

29. LB
Node 1
Node 2
Node 3
Node N

30. Single point of
failure
LB
Node 1
Node 2
Node 3
Node N

31. LB
Node 1
Node 2
Heartbeat
Node 3
LB2
Node N

32. Shared virtual IP
LB
Node 1
Node 2
Heartbeat
Node 3
LB2
Node N

33. Dealing with sticky sessions

34. Shared virtual IP
LB
Tomcat 1
Tomcat 2
Heartbeat
Tomcat 3
LB2
Tomcat N

35. Shared virtual IP
LB
Tomcat 1
Tomcat 2
Heartbeat
Tomcat 3
LB2
Tomcat N
Save session
Save session
Save session
Save session
Memcached 1
Memcached 2
Memcached 3
Memcached N
Same Server
Same Server
Same Server
Same Server

36. Shared virtual IP
LB
Tomcat 1
Save session
Tomcat 2
Save session
Heartbeat
Tomcat 3
Save session
LB2
Tomcat N
Save session
Memcached 1 Repl Memcached 2 Repl Memcached 3 Repl Memcached N
Same Server
Same Server
Same Server
Same Server

37. Shared virtual IP
If session repl not
hitting netw capacity
Tomcat 1
Save session
LB
Tomcat 2
Save session
Heartbeat
Tomcat 3
Save session
LB2
Tomcat N
Save session
Memcached 1 Repl Memcached 2 Repl Memcached 3 Repl Memcached N
Same Server
Same Server
Same Server
Same Server

38. Shared virtual IP
LB
Tomcat 1
Tomcat 2
Heartbeat
Tomcat 3
save session
Memcached
Large RAM VM
LB2
Tomcat N

39. Elasticity

40. Shared virtual IP
LB
Tomcat 1
Save session
Tomcat 2
Save session
Heartbeat
Tomcat 3
Save session
LB2
Tomcat N
Save session
Memcached 1 Repl Memcached 2 Repl Memcached 3 Repl Memcached N
Same Server
Same Server
Same Server
Same Server

41. Shared virtual IP
LB
Tomcat 1
Save session
Tomcat 2
Save session
Heartbeat
Tomcat 3
Save session
LB2
Tomcat N
Save session
Memcached 1 Repl Memcached 2 Repl Memcached 3 Repl Memcached N
Same Server
Same Server
Same Server
Auto scaling group
Same Server

42. Dealing with sticky sessions

43. Node 1
Node 2
Register
Node 3
Register
Register
Discover nodes
Database
Node N
Register

44. Node 1
Node 2
Node 3
Node N
publish facts
publish facts
publish facts
publish facts
Message Broker

45. Node 1
publish facts
subscribe events
Node 2
publish facts
subscribe events
Node 3
publish facts
subscribe events
Message Broker
Node N
publish facts
subscribe events

46. Node 1
publish facts
subscribe events
Node 2
publish facts
subscribe events
heartbeat
heartbeat
Node 3
publish facts
subscribe events
heartbeat
heartbeat
Message Broker
Node N
publish facts
subscribe events

47. Virtualized Infrastructure
Chef Server
Infra Management:
1. Provisioning
2. Service Discovery
CMDB
Message Broker

48. Operating Scenarios: Operations
Virtualized Infrastructure
Chef Server
Infra Management:
1. Provisioning
2. Patching
CMDB
Server Orchestration:
1. Cluster Orchestration
2. Scalability
3. Deployment Orchestration
mCollective
Multi zone example

49. Distributed logging

50. Node 1
Node 2
Node 3
Auto scaling group
Node N

51. Node 1
Node 2
Node 3
Auto scaling group
Cloud storage
Node N

52. Node 1
Node 2
Node 3
Log agent
Log agent
Log agent
Auto scaling group
Log collector
Cloud storage
Node N
Log agent

53. Phoenix Server

54. Node 1
Node 2
Node 3
Node N

55. Want to apply
change/patch
Node 1
Node 2
Node 3
Node N

56. Node 1
Node 2
Node 3
Node N
Node M

57. All changes
included
Node 1
Node 2
Node 3
Migrate data
Node N
Node M

58. Node 1
Node 2
Node 3
Shared data
Cloud storage
Node N
Node M

59. Test and learn

60. Virtualized Infrastructure
Chef Server
CMDB

61. Virtualized Infrastructure
Chef Server
Chaos Monkey
CMDB

62. Cookbook development

63. Stop forking, really!!!

64. community
cookbook
wrapper cookbook
wrapper cookbook adds new functionality
to community version of the cookbook
instead of changing it directly

65. include_recipe “tomcat“
community
cookbook
wrapper cookbook
chef_gem "chef-rewind“
rewind "template[catalina.xml]“ do
# add your custom staff
end

66. Cookbook Quality
•
Food Critic: Cookbook style analysis tool
•
URL: http://acrmp.github.com/foodcritic/
•
Integration: Travis CI
•
Cookbook Style Guide: https://github.com/ampledata/cookbook-style-guide
Copyright © 2012 Accenture All rights reserved.

67. Takeaways
- Accenture Cloud Platform
- Monsoon
- Accenture Lightweight Platform for Java
- Accenture Foundation Platform for Java
- Opscode Chef

68. Discussion
www.sli.do/openslava