Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Integration Testing for Salt
States Using AWS EC2
Container Service
Steven Braverman
DevOps Engineer at D&B
Graduated from UCLA with B.S. Cognitive Science
Developer Background
Love to surf
STEVEN BRAVERMAN
• pd2jira_python
• SIT
Created: Contributed to:
• SaltStack
• Troposphere
• Placebo
OPEN SOURCE PROJECTS
~175 year old company that prides on its capability to
nurture invaluable relationships in business by
uncovering truth an...
DEVOPS
It is important to test applying states to roles
efficiently, quickly, and securely
prior to applying them to production s...
MASTER
SALT MASTER/MINION SETUP
MINIONS
MASTER
SALT MASTER/MINION SETUP
MINIONS
BANANAGRAPESPEARWATERMELONAPPLE
MASTER
SALT MASTER/MINION SETUP
MINIONS
BANANAGRAPESPEARWATERMELONAPPLE
MASTER
SALT MASTER/MINION SETUP
MINIONS
BANANAGRAPESPEARWATERMELONAPPLE
MASTER
SALT MASTER/MINION SETUP
SALT MASTER/MINION SETUP
MASTER
MINIONS
BANANAGRAPESPEARWATERMELONAPPLE
MINIONS
MASTER
SALT MASTER/MINION SETUP
MASTER
PHP JENKINS NODES
SALT MASTER/MINION SETUP
JAVA DATABASE
MASTER
PHP JENKINS NODES
SALT MASTER/MINION SETUP
JAVA DATABASE
MASTER
PHP JAVA DATABASE JENKINS NODES
SALT MASTER/MINION SETUP
MASTER
PHP JAVA DATABASE JENKINS NODES
SALT MASTER/MINION SETUP
OPENVZ
VIRTUAL MACHINES
SALT MASTER/MINION SETUP
OPENVZ
VIRTUAL MACHINES
Pull-Request
Pull-request
builder
Results
PROBLEMS
• Linear integration testing is slow
• Have to maintain legacy virtual machines
PROBLEMS
• Linear integration testing is slow
• Have to maintain legacy virtual machines
No generic ways to run integratio...
Applying Salt states to a series of Docker images will:
HYPOTHESES
• Speed up the time it takes to apply state changes to ...
SALT INTEGRATION TESTING
MASTER
PHP JAVA DATABASE JENKINS NODES
SIT Flow
ASG
JENKINS NODES
SIT Flow
ASG
1. Create a pull-request
2. Pull salt repository and run unit tests
3. “Initiate” Jenkins node ...
INITIATION AND TEARDOWN
• Pull down your Salt repo into the workspace
• Run lint tests
• Run unit tests/coverage
Run SIT T...
INITIATION AND TEARDOWN
• SED to configure Jenkins node master config
• Start Salt-Master service
• Start Redis service
• ...
INITIATION AND TEARDOWN
Pre-SIT Tasks Run SIT Teardown SITInitiate SIT
1. SIT requests AutoScaling Group to provision a ne...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
1. SIT requests AutoScaling Group to provision a new instance
2. Instance provisions & SIT discovers it
3. ASG instance ge...
INITIATION AND TEARDOWN
• SED to configure Jenkins node master back to easily editable state
• Flush redis of data
• Remov...
VPC
Security Groups
EC2
AutoScaling Group
Launch Configuration
AWS INFRASTRUCTURE
Instance Profile
Policy
Role
ECS Cluster...
JENKINS NODES
Results
ASG
• Time saved drastically
• Maintenance
• Open-source generic SIT tool
DandB/Salt-Integration-Tes...
Competitors
?
TOPIC
Body text
THANKS TO:
Jeff Stibel
Aaron Stibel
Konstantin Antselovich
DevOps Team
Creative Team
TOPIC
Body text
THANK YOU
(you may now stand)
Use the SaltConf16 event app to provide feedback
for this presentation.
(we’re all ears)
QUESTIONS AND FEEDBACK
Integration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container service
Integration testing for salt states using aws ec2 container service
Prochain SlideShare
Chargement dans…5
×

Integration testing for salt states using aws ec2 container service

1 487 vues

Publié le

A SaltConf16 use case talk by Steven Braverman of Dun & Bradstreet. Testing configuration changes for multiple server roles can be time consuming when real instances or legacy container systems are used. Applying configuration changes to each role in parallel can be difficult. So what's the best way to test configuration changes efficiently, quickly, and securely prior to applying them? See how an integrated test setup using AWS EC2 Container Service (ECS), AWS AutoScaling Group, and SaltStack simplifies the application of configuration changes and allows you to test configuration changes in parallel to reduce the time spent testing.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Integration testing for salt states using aws ec2 container service

  1. 1. Integration Testing for Salt States Using AWS EC2 Container Service Steven Braverman
  2. 2. DevOps Engineer at D&B Graduated from UCLA with B.S. Cognitive Science Developer Background Love to surf STEVEN BRAVERMAN
  3. 3. • pd2jira_python • SIT Created: Contributed to: • SaltStack • Troposphere • Placebo OPEN SOURCE PROJECTS
  4. 4. ~175 year old company that prides on its capability to nurture invaluable relationships in business by uncovering truth and meaning from data.
  5. 5. DEVOPS
  6. 6. It is important to test applying states to roles efficiently, quickly, and securely prior to applying them to production servers. TESTING SALT STATES
  7. 7. MASTER SALT MASTER/MINION SETUP
  8. 8. MINIONS MASTER SALT MASTER/MINION SETUP
  9. 9. MINIONS BANANAGRAPESPEARWATERMELONAPPLE MASTER SALT MASTER/MINION SETUP
  10. 10. MINIONS BANANAGRAPESPEARWATERMELONAPPLE MASTER SALT MASTER/MINION SETUP
  11. 11. MINIONS BANANAGRAPESPEARWATERMELONAPPLE MASTER SALT MASTER/MINION SETUP
  12. 12. SALT MASTER/MINION SETUP MASTER MINIONS BANANAGRAPESPEARWATERMELONAPPLE
  13. 13. MINIONS MASTER SALT MASTER/MINION SETUP
  14. 14. MASTER PHP JENKINS NODES SALT MASTER/MINION SETUP JAVA DATABASE
  15. 15. MASTER PHP JENKINS NODES SALT MASTER/MINION SETUP JAVA DATABASE
  16. 16. MASTER PHP JAVA DATABASE JENKINS NODES SALT MASTER/MINION SETUP
  17. 17. MASTER PHP JAVA DATABASE JENKINS NODES SALT MASTER/MINION SETUP OPENVZ VIRTUAL MACHINES
  18. 18. SALT MASTER/MINION SETUP OPENVZ VIRTUAL MACHINES Pull-Request Pull-request builder Results
  19. 19. PROBLEMS • Linear integration testing is slow • Have to maintain legacy virtual machines
  20. 20. PROBLEMS • Linear integration testing is slow • Have to maintain legacy virtual machines No generic ways to run integration tests for SaltStack exist
  21. 21. Applying Salt states to a series of Docker images will: HYPOTHESES • Speed up the time it takes to apply state changes to roles • Allow for concurrent builds • Be easier to maintain
  22. 22. SALT INTEGRATION TESTING
  23. 23. MASTER PHP JAVA DATABASE JENKINS NODES SIT Flow ASG
  24. 24. JENKINS NODES SIT Flow ASG 1. Create a pull-request 2. Pull salt repository and run unit tests 3. “Initiate” Jenkins node to be a Master 4. Launch Integration Test 5. “Teardown” node back to a Minion 6. Rinse and Repeat
  25. 25. INITIATION AND TEARDOWN • Pull down your Salt repo into the workspace • Run lint tests • Run unit tests/coverage Run SIT Teardown SITInitiate SITPre-SIT Tasks
  26. 26. INITIATION AND TEARDOWN • SED to configure Jenkins node master config • Start Salt-Master service • Start Redis service • Install SIT requirements Run SITPre-SIT Tasks Initiate SIT Teardown SIT
  27. 27. INITIATION AND TEARDOWN Pre-SIT Tasks Run SIT Teardown SITInitiate SIT 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed
  28. 28. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST ASGJENKINS NODE Run SIT
  29. 29. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST ASGJENKINS NODE ASG INSTANCE Run SIT
  30. 30. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST Run SIT JENKINS NODE ECS CLUSTER
  31. 31. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST Run SIT JENKINS NODE ECS CLUSTER
  32. 32. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST JENKINS NODE Run SIT ECS CLUSTER
  33. 33. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST JENKINS NODE Run SIT ECS CLUSTER
  34. 34. 1. SIT requests AutoScaling Group to provision a new instance 2. Instance provisions & SIT discovers it 3. ASG instance gets registered into ECS cluster 4. ECS Tasks begin (runs Docker container); states are applied to the minions 5. Tasks return applied states’ results to the Jenkins node 6. When all tasks have stopped, ASG instance gets terminated 7. Results are analyzed INTEGRATION TEST JENKINS NODE Run SIT
  35. 35. INITIATION AND TEARDOWN • SED to configure Jenkins node master back to easily editable state • Flush redis of data • Remove Salt-keys • Stop redis service • Stop Salt-master Pre-SIT Tasks Teardown SITInitiate SIT Run SIT
  36. 36. VPC Security Groups EC2 AutoScaling Group Launch Configuration AWS INFRASTRUCTURE Instance Profile Policy Role ECS Cluster CloudFormation
  37. 37. JENKINS NODES Results ASG • Time saved drastically • Maintenance • Open-source generic SIT tool DandB/Salt-Integration-Testing
  38. 38. Competitors ?
  39. 39. TOPIC Body text THANKS TO: Jeff Stibel Aaron Stibel Konstantin Antselovich DevOps Team Creative Team
  40. 40. TOPIC Body text THANK YOU (you may now stand)
  41. 41. Use the SaltConf16 event app to provide feedback for this presentation. (we’re all ears) QUESTIONS AND FEEDBACK

×