2. About the Author
●
●
●
●
●
●
●
David Gil <dgil@a3sec.com>
Developer, sysadmin, project manager
Really believes in Open Source model
Programming since he was 9 years old
Ossim developer at its early stage
Python lover :-)
Debian package maintainer (a long, long
time ago)
● Sci-Fi books reader and mountain bike rider
4. Distributed Configuration
Management - Why?
● Control of all the infrastructure configuration
● Ensure configurations are the same on all
servers (or on a type of servers)
● Auto-restart services on configuration
changes
● Auto-Install required packages
5. Distributed Configuration
Management - Why?
● Control of all the infrastructure configuration
● Ensure configurations are the same on all
servers (or on a type of servers)
● Auto-restart services on configuration
changes
● Auto-Install required packages
Don't sysadmin, develop configuration states.
Make your life easier!
6. Remote Execution Platform
Why?
Examples:
● I want to restart apache2 on all my
frameworks
● I don't want to have to shell into every server
just to run an apt-get upgrade command
● I have {n} servers that I want to check what
version of {package} is installed
7. Remote Execution Platform
Why?
Examples:
● I want to restart apache2 on all my
frameworks
● I don't want to have to shell into every server
just to run an apt-get upgrade command
● I have {n} servers that I want to check what
version of {package} is installed
"ssh in a for loop is not a solution"
Luka Kanies, Puppet developer
8. What is Salt?
● FOSS project for distributed configuration
and remote execution
● Still young (2011) but active developed
● Other tools:
○ puppet (ruby): fedora, mozilla, sans
○ chef (ruby): openstack, cloudfoundry
● Salt is easier (IMHO) than chef and puppet
(steeper learning curve)
● Configure states by writing simple lists of
items (yaml), more readable for sysadmins
than vanilla python or ruby
9. Salt Keys - Just Easy
● Simple Architecture
○ Fits fine with Alienvault's Architecture
● Easy Installation
○ Squeeze debian packages, pip, bootstrap git
● Easy Configuration
○ No need to learn a new programming language
● Extensible
○ Develop Alienvault specific modules is quite easy!
10. Salt Keys - Scalable
● Able to manage tens of thousands of servers
● ZeroMQ based for messaging
● Persistent connections / Parallel execution
● MessagePack: fast and small message
format (fluentd, redis, etc.)
11. Salt Keys - Secure
Secure and encrypted communication transport
layer:
● Public Key for Master Authentication
● 256 bit AES for payload high speed
encryption
So, VPN configuration is not needed
12. Configuration State example
● You have 20 servers with exim4 as default
mail transport
● You want to get rid of exim4 and install
postfix instead
● In your 20 servers? One by one?
20. Security Deployment approach
● Always well-known behaviors
● Avoid accidental mistakes
● All configuration changes stored in a single
and unique place (master filesever)
● Private Git repository for Knowledge
Database (configuration states developed at
customers)
● Reusable configurations for other
deployments!
● Just code one time, test it and apply where
you need
21. Security Deployment approach
A few common and repetitive distribution tasks,
for every single server on your infrastructure:
● Hostname resolution
● Custom plugins distribution
● Remote code execution
● Snort threshold and rules
● Logrotate files
● Rsyslog filters
● Firewall rules
● etc.
22. Security Deployment approach
Hostname resolution
# address and name management in /etc/hosts file
mcafee-database:
host.present:
- ip: 192.168.0.42
Cron
# Clean user crontab management
ntpdate-debian > /var/log/syslog:
cron.present:
- user: root
- minute: 7
- hour: 2
24. Salt Architecture
Master (server)
● Approved minion keys for communication
● Send commands to minions
● Store configurations, files and resources for
minions
Minon (client)
● Connects to Master
● Listens for commands
● Downloads configurations from Master and
apply them (update config states)
29. Built-in State Modules (partial list)
●
●
●
●
●
●
●
●
●
●
cmd
cron
file
git
group
host
locale
mount
mongo*
mysql*
●
●
●
●
●
●
●
pkg
postgres*
rabbitmq*
service
ssh*
timezone
user
Custom modules can
be written in Python!
30. Built-in Exec Modules (partial list)
●
●
●
●
●
●
●
●
●
●
●
apache
apt
archive
at
cassandra
cluster
cron
disk
file
gem
git
● hosts
● iptables
● keyboard
● ldap
● locale
● network
All supported:
salt 'node' sys.doc
Custom modules can
be written in Python!
31. Targeting
● Match minions for cli commands and state
files
● Force different states for every type of server
● Alienvault profiles:
○
○
○
○
○
SIEM
Logger
Framework
Sensor
Database
34. Targeting
● By Hostname: Minion_id
'*', 'avsensor.*', 'av(siem|logger)', 'web1,web2,web3'
● By system information: Grains
grains['os'] == 'Debian' and 'Server' in grains
['av_profile']
● By defined groups: Node Groups
nodegroups:
'sensors': avsensor*.domain.com, foo.bar
'servers': av(siem|logger).domain.com, bar.foo
● Mix combination: Compound
'webserv* and G@os:Debian or E@web-dc1-srv.*'
35. Targeting - Grains
Static bits of information that a minion collects
about the system (inventory!)
cpu_model
cpuarch
domain
fqdn
gpus
host
kernel
kernelrelease
lsb_codename
lsb_description
lsb_os
lsb_release
manufacturer
mem_total
nodename
num_cpus
os
oscodename
osfullname
osrelease
path
productname
server_id
shell
virtual
etc.
36. Targeting - Grains
● Using grains to define states
● Jinja2 template markup
apache:
pkg.installed:
{% if grains['os'] == 'RedHat' %}
- name: httpd
{% elif grains['os'] == 'Ubuntu' %}
- name: apache2
{% endif %}
37. Extending - Custom grains
You can write your own custom grains:
# Return the host profile for Alienvault minions
def av_profile():
alienvault_config = '/etc/ossim/ossim_setup.conf'
if not os.path.isfile(alienvault_config):
return {}
config = ConfigParser.RawConfigParser()
config.read(alienvault_config)
profile = config.get('root', 'profile')
return {'alienvault_profile': profile}
38. Extending - Custom grains
And use them:
# salt -G 'os:Debian' test.ping
avsiem.client01.com: True
avlogger.client01.com: True
avsensor01.client01.com: True
avsensor02.client01.com: True
# salt -G 'av_profile:Server' cmd.run 'uname -r'
avsiem.client01.com: 2.6.32-5-amd64
avlogger.client01.com: 2.6.31.6
39. Extending - Custom modules
Build and use your custom exec modules:
# salt '*.alienvault' alienvault.profile
'server01.alienvault': 'Server,Database'
'logger01.alienvault': 'Server,Database'
# salt '*.alienvault' alienvault.snort_dbsize
'server01.alienvault': 457Mb
'logger01.alienvault': 0Mb
# salt '*.alienvault' alienvault.numalarms
'server01.alienvault': 393
'logger01.alienvault': 0
40. Extending - Custom modules
Example: Nagios configuration management
# salt ‘nagios01’ nagios.add_host
“salt-test-host” “127.0.0.1”
nagios01: [nagios] Added new host: salt-test-host
# salt ‘nagios01’ nagios.add_service
“salt-test-host” “HTTP” “check_http”
nagios01: [nagios] Added new service: HTTP for host salttest-host
41. Extending - Custom states
Custom states for
availability monitoring,
using exec modules
# add new host to nagios
ZZZ-salt-test-host:
monitoring.add_host:
- address: 127.0.0.1
# add new services to nagios
HTTP:
monitoring.add_service:
- checkname: check_http
- hostnames:
- ZZZ-salt-test-host
- YYY-salt-test-host
42. Dynamic Module Distribution
● Build your custom
grains, exec
modules and states
● Automatically
distributed to the
minions with the
salt-master file
server
● Directories are
prepended with an
underscore
/srv/salt/
|-- alienvault
|
|-- agent.sls
|
|-- database.sls
|
|-- plugins.sls
|
`-- server.sls
|-- core
|
|-- hosts.sls
|
|-- munin.sls
|
|-- tools.sls
|
`-- users.sls
|-- _grains
|-- _modules
|-- _states
`-- top.sls
44. Aggregation&Management UI
Build services on top of Salt architecture:
● MSSPs and SOCs
● Monitoring, Deployment and Management
off all the infrastructure in a single
dashboard
● Configurations backup on demand
● Patches and fixes distribution
● Full Inventory
● etc...
