Automating Yourself Out of Trouble

Automate yourself out of trouble
with Ansible
How we use Ansible at Dell EMC
Jose Delarosa / Sambhu Kalaga
Dell EMC
May 8th, 2019

Who are we
• Jose Delarosa
- Software Engineer
- Infrastructure solutions team
• Sambhu Kalaga
- Software Engineer
- OpenStack solutions team

Before we begin
1. Thank you for attending!
2. Please ask questions anytime
3. If time runs out, happy to talk to you afterwards

Why are we here today?
1. These days, automation is a necessity, not a luxury.
2. Ansible is a great tool for automation!
3. Dell EMC is active in the Ansible community and we
want to share with you some of the work we’ve done.

Agenda
1. Brief overview of Ansible
2. How Dell EMC uses Ansible
• Network switch configuration
• OpenStack configuration
• Out-of-band server management
• OpenShift deployment
3. Conclusion

Ansible in a nutshell
1. Agentless, no DB backend  Minimum footprint
2. Remote tasks are run in parallel  Fast & efficient
3. Only do things if needed  Idempotent
4. Easy to learn and use  Reduced learning curve

Some use cases for automation
OpenStack
• Compute nodes
• Storage nodes
• Controller nodes Container Management
• Stop & remove containers
• Refresh container images
• Deploy new containers
 1-to-n management
 Executes tasks in parallel
IT Security Hardening
• Firewall rules
• Remove unused packages
• Install security updates

Ansible components
Task: A Task is the smallest unit of work: “install a package” or “remove a user”.
Play: A Play is composed of tasks: The Play “prepare database” is composed of two
Tasks:
 Task 1: Create database
 Task 2: Give users access to database
Playbook: A Playbook is a collection of plays: The playbook “Setup my application” has
two Plays: 1) “prepare database” and 2) “prepare web server”.
Playbook: Setup my web application
Play 1: Setup database
Task 1:
Install mysql
package
Task 2:
Create database
customer_db
Play 2: Setup web server
Task 1:
Install httpd
package
Task 2:
Configure
site for TLS

Example
Say you provision 100 servers every
day and you run these commands
in each server:
The same
commands can be
placed in an Ansible
playbook and
executed in 100
servers.
daily_tasks.yml
$ ansible-playbook daily_tasks.yml

Ansible module
• An Ansible module is the code that implements all
the commands specified in playbooks.
• Modules can be written in any language, but most
popular is Python.
• If you are a system administrator, you will work
mostly with playbooks.
• If you are a developer, you will work mostly with
modules.

Ansible Template
Sample VLAN template –Dell EMC OS10 NOS
Template: A template is a file in
Ansible which is used to dynamically
create configurations using the
values from a variable file.

Ansible Roles
• Role: Dell EMC Networking modules are packaged
and distributed through Ansible-galaxy. These pre-
packaged units are known as roles. Ansible roles
have specific file structure and are easily shared
with other users. Roles help separate configuration
syntax from data.
Sample file structure:

Example: Network Switch Configuration

Stack ID
Stack ID
Stack ID
Stack ID
Stack ID
Dell EMC S5248
Network Switch with
OS10
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
Dell EMC S4048
Network Switch with
OS9
Dell EMC N4000
Network Switches with
OS6
YAML INVENTORY /
DATABASE
Configuration Modules
and Templates
Ansible
Create device configurations
Deploy configurations via SSH
Ansible concept
ACTLNK
1 2ACT
LNK
ACT
LNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119
26 323028
25 312927
34 403836
33 393735
42 484644
41 474543
ACTLNK
1 2ACT
LNK
ACT
LNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119
26 323028
25 312927
34 403836
33 393735
42 484644
41 474543
ACTLNK
1 2ACT
LNK
ACT
LNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119
26 323028
25 312927
34 403836
33 393735
42 484644
41 474543
ACTLNK
1 2ACT
LNK
ACT
LNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119
26 323028
25 312927
34 403836
33 393735
42 484644
41 474543
ACTLNK
1 2ACT
LNK
ACT
LNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119
26 323028
25 312927
34 403836
33 393735
42 484644
41 474543
Dell EMC R740XD
Rack Server

Dell EMC Networking Roles
• Dell EMC Ansible Network Roles:
• Dellos-acl
• Dellos-bgp
• Dellos-dcb
• Dellos-interface
• Dellos-lag
• Dellos-system
• Dellos-vlan
• Dellos-vlt
• Dellos-vrf
• Dellos-xstp
• Dellos-ecmp
• Dellos-prefixlist
Sample file structure:

Install Dell EMC Ansible Network Roles
• Enable Ansible repository:
• Install Ansible:
• Install Dell Networking Ansible Roles:

Example Leaf-Spine switch deployment
33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54
49 51 53
StackID
2 864
1 753
10 161412
9 151311
18 242220
17 232119 ACTLNK
2 864
1 753
10 161412
9 151311
18 242220
17 232119 ACTLNK
VLT
Spine
Leaf1 Leaf2
Sample playbook
Inventory file
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx

Dell EMC Ansible resources
• Dell EMC Ansible Galaxy documentation
• Dell EMC Network ansible examples
• Dell EMC Networking guides
• Dell EMC Fabric center

Example: Red Hat OpenStack
configuration

Dell EMC RedHat Open Stack
• JetPack is available here

Dell EMC RedHat Open Stack
Open Stack Use cases include:
• Projects
• Networks
• Users
• Security groups
• Flavors
• Instances
• and many more!!

Example: Out-of-Band Server
Management

Out-of-Band management
1. Server management independent of the operating system.
2. Provided by an embedded chip with its own Ethernet port.
3. Goes by many names: iDRAC, iLO, IMM, but commonly referred to as
BMC (Baseboard Management Controller).
4. Capabilities include:
 Component inventory
 Hardware failure detection & alerting
 Power management
 BIOS configuration

Communicating with the BMC
1. There are several legacy protocols that can be used: IPMI, WS-MAN,
racadm, etc. but these are not secure, hard to use, proprietary and
not very scalable.
2. Redfish addresses all these short-comings! It is the best method to
communicate with the BMC:
 Open source
 RESTful interface
 Secure
 Scalable
3. A Redfish request is sent as an URI, so a client could be any
application on a server, workstation or mobile device.

Ansible modules for Redfish
1. Dell created and merged upstream 3 modules for Redfish.
2. Everything you need to manage your servers:
• redfish_facts: status, hardware inventory, etc.
• redfish_command: power management, user management, etc.
• redfish_config: BIOS attributes, boot mode, etc.
3. Vendor-neutral, with ongoing contributions from developers
at the DMTF, Dell, HP, Lenovo and others.
4. Use these modules to manage your heterogeneous server
infrastructure from one Ansible controller.

Server management made easy
Management
Network
https://<BMC-IP>/redfish/v1/Systems/Systems.Embedded.1
{
Health OK
HealthRollup OK
}
1
2

Server management made really easy
Management
Network
{
Health OK
HealthRollup OK
}
1
2
Module: redfish_facts
Category: Systems
Command: GetSystemInventory
Module: redfish_config
Category: Systems
Command: SetBiosAttributes
Playbook Playbook

Example: Get system health
$ curl https://<BMC-IP>/redfish/v1/Systems/System.Embedded.1 --user root:password | jq .Status
{
"Health": "OK",
"HealthRollUp": "OK"
"State" : "Enabled"
}

Get system health with Ansible
Playbook
Results placed
In JSON file

Example: Get System Event Logs
$ curl https://<BMC-IP>/redfish/v1/Managers/iDRAC.Embedded.1/Logs/Sel --user root:password |
jq '.Members[] | {date: .Created, message: .Message, severity: .Severity}'
--- snip ---
{
"date": "2017-09-26T13:33:00-05:00",
"message": "Power supply redundancy is lost.",
"severity": "Critical"
}
{
"date": "2017-09-26T13:32:53-05:00",
"message": "The power input for power supply 2 is lost.",
"severity": "Critical"
}
{
"date": "2017-09-16T10:37:59-05:00",
"message": "Log cleared.",
"severity": "Ok"
}

Get System Event Logs with Ansible
Playbook
Results placed
In JSON file

Example: Set boot mode to UEFI and reboot
Change boot mode to UEFI
$ curl https://<BMC-IP>/redfish/v1/Systems/System.Embedded.1/Bios/Settings
--request PATCH
--header "Content-Type: application/json"
--data '{"Attributes":{"BootMode":"Uefi"}}'
--user root:password
Reboot
$ curl https://<BMC-IP>/redfish/v1/Systems/System.Embedded.1/Actions/ComputerSystem.Reset
--request POST
--header "Content-Type: application/json"
--data '{"ResetType":"GracefulRestart"}'
--user root:password

Set boot mode to UEFI and reboot with Ansible
Playbook
Full set of playbooks at
https://github.com/dell/redfish-ansible-module

Reference Architecture for OpenShift
1. Deployment was 95% automated with Ansible
2. Used Ansible to automate:
 Switch configuration (VLT, VLANs, LACP)
 Server BIOS configuration
 OS Provisioning via iPXE
 OpenShift deployment: master, infrastructure, application &
storage nodes
 GlusterFS pool

1. Download OpenShift RA here.
2. Roles and playbooks available here.
3. Future Reference Architectures between Dell and
Red Hat will continue to leverage Ansible.

Conclusion
1. These are just a few examples of how we use Ansible.
2. Code contributions are welcome!
3. Want feedback on what you’d like to see automated

Automating Yourself Out of Trouble

Automating Yourself Out of Trouble

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Automating Yourself Out of Trouble

Similar to Automating Yourself Out of Trouble (20)

Recently uploaded

Recently uploaded (20)

Automating Yourself Out of Trouble

Editor's Notes