Raphaël Pinson's talk on "Configuration surgery with Augeas" at PuppetCamp Geneva '12. Video at http://youtu.be/H0MJaIv4bgk Learn more: www.puppetlabs.com

1. Configuration surgery with Augeas
Raphaël Pinson
@raphink
LSM 2012, Geneva
2012-07-11
https://github.com/raphink/augeas-talks/

2. Tired of ugly sed and awk one liners?
or of using tons of different parsing libraries or
common::line tricks?
www.camptocamp.com / 2/38

3. Become a configuration surgeon with
Augeas
www.camptocamp.com / 3/38

4. What is the need?
● A lot of different syntaxes
● Securely editing configuration files with a
unified API
www.camptocamp.com / 4/38

5. A tree
Augeas turns configuration files into a tree
structure:
/etc/hosts -> /files/etc/hosts
www.camptocamp.com / 5/38

6. Its branches and leaves
... and their parameters into branches and leaves:
augtool> print /files/etc/hosts
/files/etc/hosts
/files/etc/hosts/1
/files/etc/hosts/1/ipaddr = "127.0.0.1"
/files/etc/hosts/1/canonical = "localhost"
www.camptocamp.com / 6/38

7. Augeas provides many stock parsers
They are called lenses:
Access Cron Host_Conf
Aliases Crypttab Hostname
Anacron debctrl Hosts_Access
Approx Desktop IniFile
AptConf Dhcpd Inputrc
Automaster Dpkg Iptables
Automounter Exports Kdump
BackupPCHosts FAI_DiskConfig Keepalived
cgconfig Fonts Keepalived
cgrules Fuse Login_defs
Channels Grub Mke2fs
...
www.camptocamp.com / 7/38

8. ... as well as generic lenses
available to build new parsers:
Build Sep Simplelines
IniFile Shellvars Simplevars
Rx Shellvars_list Util
www.camptocamp.com / 8/38

9. augtool lets you inspect the tree
$ augtool
augtool> ls /
augeas/ = (none)
files/ = (none)
augtool> print /files/etc/passwd/root/
/files/etc/passwd/root
/files/etc/passwd/root/password = "x"
/files/etc/passwd/root/uid = "0"
/files/etc/passwd/root/gid = "0"
/files/etc/passwd/root/name = "root"
/files/etc/passwd/root/home = "/root"
/files/etc/passwd/root/shell = "/bin/bash"
www.camptocamp.com / 9/38

10. The tree can be queried using XPath
augtool> print /files/etc/passwd/*[uid='0'][1]
/files/etc/passwd/root
/files/etc/passwd/root/password = "x"
/files/etc/passwd/root/uid = "0"
/files/etc/passwd/root/gid = "0"
/files/etc/passwd/root/name = "root"
/files/etc/passwd/root/home = "/root"
/files/etc/passwd/root/shell = "/bin/bash"
www.camptocamp.com / 10/38

11. But also modified
$ getent passwd root
root:x:0:0:root:/root:/bin/bash
$ augtool
augtool> set /files/etc/passwd/*[uid='0']/shell /bin/sh
augtool> match /files/etc/passwd/*[uid='0']/shell
/files/etc/passwd/root/shell = "/bin/sh"
augtool> save
Saved 1 file(s)
augtool> exit
$ getent passwd root
root:x:0:0:root:/root:/bin/sh
www.camptocamp.com / 11/38

12. Puppet has a native provider
augeas {'export foo':
context => '/files/etc/exports',
changes => [
"set dir[. = '/foo'] /foo",
"set dir[. = '/foo']/client weeble",
"set dir[. = '/foo']/client/option[1] ro",
"set dir[. = '/foo']/client/option[2] all_squash",
],
}
www.camptocamp.com / 12/38

13. It is better to wrap things up
define kmod::generic(
$type, $module, $ensure=present,
$command='', $file='/etc/modprobe.d/modprobe.conf'
) {
augeas {"${type} module ${module}":
context => "/files${file}",
changes => [
"set ${type}[. = '${module}'] ${module}",
"set ${type}[. = '${module}']/command '${command}'",
],
}
}
www.camptocamp.com / 13/38

14. mcollective has an agent
$ mco augeas match /files/etc/passwd/rpinson/shell
* [ ======================================> ] 196 / 196
...
wrk1
saja-map-dev
/files/etc/passwd/rpinson/shell = /bin/bash
wrk3
wrk4
/files/etc/passwd/rpinson/shell = /bin/bash
...
www.camptocamp.com / 14/38

15. ... and uses it for discovery
$ mco find -S "augeas_match(/files/etc/passwd/rip).size = 0"
www.camptocamp.com / 15/38

16. Bindings include Perl, Python, Java,
PHP, Haskell, Ruby...
require 'augeas'
aug = Augeas.open
if aug.match('/augeas/load'+lens).length > 0
aug.set('/augeas/load/'+lens+'incl[last()+1]', path)
else
aug.set('/augeas/load/'+lens+'/lens', lens+'.lns')
end
(From the mcollective agent)
www.camptocamp.com / 16/38

17. The Ruby bindings can be used in Facter
Facter.add(:augeasversion) do
setcode do
begin
require 'augeas'
aug = Augeas::open('/', nil, Augeas::NO_MODL_AUTOLOAD)
ver = aug.get('/augeas/version')
aug.close
ver
rescue Exception
Facter.debug('ruby-augeas not available')
end
end
end
(From the augeasversion fact)
www.camptocamp.com / 17/38

18. Or to write native types
def ip
aug = nil
path = "/files#{self.class.file(resource)}"
begin
aug = self.class.augopen(resource)
aug.get("#{path}/*[canonical =
'#{resource[:name]}']/ipaddr")
ensure
aug.close if aug
end
end
(See https://github.com/domcleal/augeasproviders)
www.camptocamp.com / 18/38

19. The case of sshd_config
Custom type:
define ssh::config::sshd ($ensure='present', $value='') {
case $ensure {
'present': { $changes = "set ${name} ${value}" }
'absent': { $changes = "rm ${name}" }
'default': { fail("Wrong value for ensure: ${ensure}") }
}
augeas {"Set ${name} in /etc/ssh/sshd_config":
context => '/files/etc/ssh/sshd_config',
changes => $changes,
}
}
www.camptocamp.com / 19/38

20. Using the custom type for sshd_config
ssh::config::sshd {'PasswordAuthenticator':
value => 'yes',
}
www.camptocamp.com / 20/38

21. The problem with sshd_config
Match groups:
Match Host example.com
PermitRootLogin no
=> Not possible with ssh::config::sshd, requires
insertions and looping through the configuration
parameters.
www.camptocamp.com / 21/38

22. A native provider for sshd_config (1)
The type:
Puppet::Type.newtype(:sshd_config) do
ensurable
newparam(:name) do
desc "The name of the entry."
isnamevar
end
newproperty(:value) do
desc "Entry value."
end
newproperty(:target) do
desc "File target."
end
newparam(:condition) do
desc "Match group condition for the entry."
end
end
www.camptocamp.com / 22/38

23. A native provider for sshd_config (2)
The provider:
require 'augeas' if Puppet.features.augeas?
Puppet::Type.type(:sshd_config).provide(:augeas) do
desc "Uses Augeas API to update an sshd_config parameter"
def self.file(resource = nil)
file = "/etc/ssh/sshd_config"
file = resource[:target] if resource and resource[:target]
file.chomp("/")
end
confine :true => Puppet.features.augeas?
confine :exists => file
www.camptocamp.com / 23/38

24. A native provider for sshd_config (3)
def self.augopen(resource = nil)
aug = nil
file = file(resource)
begin
aug = Augeas.open(nil, nil, Augeas::NO_MODL_AUTOLOAD)
aug.transform(
:lens => "Sshd.lns",
:name => "Sshd",
:incl => file
)
aug.load!
if aug.match("/files#{file}").empty?
message = aug.get("/augeas/files#{file}/error/message")
fail("Augeas didn't load #{file}: #{message}")
end
rescue
aug.close if aug
raise
end
aug
end
www.camptocamp.com / 24/38

25. A native provider for sshd_config (4)
def self.instances
aug = nil
path = "/files#{file}"
entry_path = self.class.entry_path(resource)
begin
resources = []
aug = augopen
aug.match(entry_path).each do |hpath|
entry = {}
entry[:name] = resource[:name]
entry[:conditions] = Hash[*resource[:condition].split(' ').flatten(1)]
entry[:value] = aug.get(hpath)
resources << new(entry)
end
resources
ensure
aug.close if aug
end
end
www.camptocamp.com / 25/38

26. A native provider for sshd_config (5)
def self.match_conditions(resource=nil)
if resource[:condition]
conditions = Hash[*resource[:condition].split(' ').flatten(1)]
cond_keys = conditions.keys.length
cond_str = "[count(Condition/*)=#{cond_keys}]"
conditions.each { |k,v| cond_str += "[Condition/#{k}="#{v}"]" }
cond_str
else
""
end
end
def self.entry_path(resource=nil)
path = "/files#{self.file(resource)}"
if resource[:condition]
cond_str = self.match_conditions(resource)
"#{path}/Match#{cond_str}/Settings/#{resource[:name]}"
else
"#{path}/#{resource[:name]}"
end
end
www.camptocamp.com / 26/38

27. A native provider for sshd_config (6)
def self.match_exists?(resource=nil)
aug = nil
path = "/files#{self.file(resource)}"
begin
aug = self.augopen(resource)
if resource[:condition]
cond_str = self.match_conditions(resource)
else
false
end
not aug.match("#{path}/Match#{cond_str}").empty?
ensure
aug.close if aug
end
end
www.camptocamp.com / 27/38

28. A native provider for sshd_config (7)
def exists?
aug = nil
entry_path = self.class.entry_path(resource)
begin
aug = self.class.augopen(resource)
not aug.match(entry_path).empty?
ensure
aug.close if aug
end
end
def self.create_match(resource=nil, aug=nil)
path = "/files#{self.file(resource)}"
begin
aug.insert("#{path}/*[last()]", "Match", false)
conditions = Hash[*resource[:condition].split(' ').flatten(1)]
conditions.each do |k,v|
aug.set("#{path}/Match[last()]/Condition/#{k}", v)
end
aug
end
end
www.camptocamp.com / 28/38

29. A native provider for sshd_config (8)
def create
aug = nil
path = "/files#{self.class.file(resource)}"
entry_path = self.class.entry_path(resource)
begin
aug = self.class.augopen(resource)
if resource[:condition]
unless self.class.match_exists?(resource)
aug = self.class.create_match(resource, aug)
end
else
unless aug.match("#{path}/Match").empty?
aug.insert("#{path}/Match[1]", resource[:name], true)
end
end
aug.set(entry_path, resource[:value])
aug.save!
ensure
aug.close if aug
end
end
www.camptocamp.com / 29/38

30. A native provider for sshd_config (9)
def destroy
aug = nil
path = "/files#{self.class.file(resource)}"
begin
aug = self.class.augopen(resource)
entry_path = self.class.entry_path(resource)
aug.rm(entry_path)
aug.rm("#{path}/Match[count(Settings/*)=0]")
aug.save!
ensure
aug.close if aug
end
end
def target
self.class.file(resource)
end
www.camptocamp.com / 30/38

31. A native provider for sshd_config (10)
def value
aug = nil
path = "/files#{self.class.file(resource)}"
begin
aug = self.class.augopen(resource)
entry_path = self.class.entry_path(resource)
aug.get(entry_path)
ensure
aug.close if aug
end
end
www.camptocamp.com / 31/38

32. A native provider for sshd_config (11)
def value=(thevalue)
aug = nil
path = "/files#{self.class.file(resource)}"
begin
aug = self.class.augopen(resource)
entry_path = self.class.entry_path(resource)
aug.set(entry_path, thevalue)
aug.save!
ensure
aug.close if aug
end
end
www.camptocamp.com / 32/38

33. Using the native provider for
sshd_config
sshd_config {'PermitRootLogin':
ensure => present,
condition => 'Host example.com',
value => 'yes',
}
www.camptocamp.com / 33/38

34. Errors are reported in the /augeas tree
augtool> print /augeas//error
/augeas/files/etc/mke2fs.conf/error = "parse_failed"
/augeas/files/etc/mke2fs.conf/error/pos = "82"
/augeas/files/etc/mke2fs.conf/error/line = "3"
/augeas/files/etc/mke2fs.conf/error/char = "0"
/augeas/files/etc/mke2fs.conf/error/lens =
"/usr/share/augeas/lenses/dist/mke2fs.aug:132.10-.49:"
/augeas/files/etc/mke2fs.conf/error/message =
"Get did not match entire input"
www.camptocamp.com / 34/38

35. Other projects using Augeas
● libvirt
● rpm
● Nut
● guestfs
● ZYpp
● Config::Model
● Augeas::Validator
www.camptocamp.com / 35/38

36. Future projects
● more API calls
● improved XPath syntax
● more lenses
● more native providers
● DBUS provider
● content validation in Puppet (validator)
● integration in package managers
● finish the Augeas book
● ...
● your idea/project here...
www.camptocamp.com / 36/38

37. Questions?
http://augeas.net
augeas-devel@redhat.com
freenode: #augeas
www.camptocamp.com / 37/38