This document discusses how to use Func to manage and run commands on multiple remote servers (minions) from a central location (overlord). Func provides an easy way to run commands across many servers and gather the output in a useful way. It uses secure XML-RPC communications and allows commands to be run from the command line, scripts, or other languages. The document also covers installing and configuring Func, using the Func API, writing custom modules, and other Func features.
7. The problem
●
And gather the output in a useful way*.
* How do you know it worked on the 1000 machines you ran it on?
8. Some history...
●
“masshost” script at UOL Web Services
●
Written in Expect (later wrapped with
Expect.pm)
●
SSH under the hood
●
Ran serially, host by host
●
Most big shops have something like this
9. You could write your own...
●
Deal with Expect (or Expect.pm)
●
Handle ssh 'fun'
●
Password auth or key auth
●
Missing keys (first login)
●
Changing keys (after re-imaging a machine)
●
Handling parallel/asynchronous execution if your
site is large enough
●
Handling input / output from all of the above
●
Bragging rights
13. What's Func?
●
Fedora Unified Network Controller
●
An easy way to run commands on lots of
remote machines
●
Written in
●
Bindings to talk to it for Java
●
But you can talk to it from anything that speaks
YAML or JSON
14. What's Func?
●
But it's much more:
●
Secure server communications platform
●
XML-RPC over SSL
●
You could build your own puppet or cfengine
with it.
●
Build a backend for a server-automation
system with a web interface
●
Automate 1000s of machines from a single
location
●
World domination!
16. Func Overview
●
Overlord machine(s) distribute commands
to Minions:
17. Func Overview
●
Overlord machine(s) distribute commands
to Minions: “Install the
vim-enhanced
package”
“Run /usr/bin/detonate”
18. Func Overview
●
Overlord machine(s) distribute commands
to Minions: “Stop the
countdown-sequence
daemon”
“Kill all 'intruder*'
Processes that are
running
19. Func Overview
●
Minions return results to the Overlord:
“Here's a list of running
processes”
“That command
Returned status 1”
21. Installation
●
Easiest way is with yum
●
On Fedora:
● yum install func
● Will pull in certmaster too
●
On CentOS (Use EPEL):
● rpm -Uvh
http://download.fedora.redhat.com/pub/
epel/5/i386/epel-release-5-
3.noarch.rpm
● yum install func
22. Installation
●
On Ubuntu, Debian and others:
●
Follow:
https://fedorahosted.org/func/wiki/InstallAndSetupGuide
23. Configuration
●
On the overlord:
● /sbin/chkconfig --level 345
certmaster on
● /sbin/service certmaster start
24. Configuration
●
On the minions:
●
/etc/certmaster/minion.conf:
[main] certmaster = overlord.example.org
log_level = DEBUG
cert_dir = /etc/pki/certmaster
●
Then:
– /sbin/chkconfig --level 345 funcd on
– /sbin/service funcd start
25. Configuration
●
Back on the overlord:
● certmaster-ca –list
● certmaster-ca –sign
minionN.example.com
●
All done!
●
Now start commanding your minions
28. Func from the command-line
func minion.example.com call
command exists "/bin/detonate"
29. Func from the command-line
Minion Hosts
{
func minion.example.com call
command exists "/bin/detonate"
{
{
{
Module Method Arguments
30. Func from the command-line
func minion.example.com call
command run
"/bin/detonate -c pull_the_lever"
31. Func from the command-line
Globbing to command multiple minions*:
func “db*.example.com” call
command run
"/bin/detonate -c pull_the_lever"
*This works best if you haven't named all your minions after Loony Tunes characters...
32. Func from the command-line
Globbing to command multiple minions:
func “db*.example.com” …
func “db*.example.com;mail.example.com” …
func --exclude=”db*” “*.example.com” …
func @webhosts …
(Define @groups in /etc/func/groups
Or extend via API to access remote dbs,
ldap, etc.)
33. Func from the command line
● func minion.example.com call
process info “rebel_commandos”
● func minion.example.com call
process kill rebel_commandos SIGKILL
● func *.example.com call
service start httpd
● func “*” ping # or...
● func “*” list_minions
### will show you all your minions.
34. Func from the command line
What modules are available on my minions?
func “*” call system list_modules
35. Func from the command line
What methods are available in a module?
func “minion.example.com” call
<modulename> list_methods
36. Other Func Modules
●
CopyFile ●
SysCtl
●
IPTables ●
Vlan
●
Mount ●
Bridge
●
Pullfile ●
Hardware
●
Reboot ●
S.M.A.R.T.
●
RPM ●
Build your own
●
Virt (Easy!)
●
(More on that later)
●
Yum
43. Func API
#!/usr/bin/python
import func.overlord.client as fc
client = fc.Client(“*.example.org”)
results = client.yumcmd.update(“httpd”)
results = client.service.start(“httpd”)
results =
client.sysctl.set('net.ipv4.tcp_syncookies',1)
44. Func API
#!/usr/bin/python
import func.overlord.client as fc
client = fc.Client(“*.example.org”)
results = client.yumcmd.update(“httpd”)
results = client.service.start(“httpd”)
results =
client.sysctl.set('net.ipv4.tcp_syncookies',1)
{
{
{
Module Method Arguments
45. Using Func
from Other
Add picture here
Languages
46. Using Func from other languages
● Use func-transmit
●
Good for anything that can grok JSON or
YAML 1.0
47. Using Func from other languages
●
Create a file or string of YAML or JSON:
clients: "*"
async: False
nforks: 1
module: command
method: run
parameters: "/bin/echo Hello World"
48. Using Func from other languages
●
Pipe it into func-transmit
● func-transmit --yaml < my.yaml
● func-transmit --json < my.json
●
Get results back in JSON or YAML
50. Writing your own modules
Write the code:
#!/usr/bin/python
import func_module
class Test(func_module.FuncModule):
version = "0.1.1"
api_version = "0.0.1"
description = "A sample module"
def mult(self, numb1, numb2):
return numb1 * numb2
51. Write your own modules
●
Drop your file into
/usr/lib/python$version/site-
packages/func/minion/modules
● Restart funcd on your minions
●
Done.
52. Writing your own modules
● func-create-module will generate a basic
module template
●
Package your modules into rpms for ease
of deployment (or push them out with func)
●
See existing modules for examples
53. Writing your own modules
●
Can't write modules in other than
Python...yet.
54. Other features
●
Delegation
●
Multiplexing
●
Asynchrony
●
Web interface
●
ACLs
55. Cobbler integration
●
Kickstart
●
Install Func
●
Point it to your Overlord
●
All from Cobbler!
●
See https://fedorahosted.org/cobbler/wiki/
FuncIntegration
60. Photo Credits*
●
“gargoyles” by Flickr user 'certified_su,' http://www.flickr.com/photos/certified_su/3502855962/, Creative Commons
License BY 2.0
●
“Cubicle Gargoyle” by Flickr user 'Dennis from Atlanta,' http://www.flickr.com/photos/dennis_matheson/3419153723/, CC
License BY-SA 2.0
●
“Stacked servers” by Flickr user 'redjar,' http://www.flickr.com/photos/redjar/360113468/, CC License BY-SA 2.0
●
“Happy Gargoyle” by Flickr user 'Wolfrage,' http://www.flickr.com/photos/wolfrage/3931044145/, CC License BY 2.0
●
“Gargoyles” by Flickr user 'Laertes,' http://www.flickr.com/photos/jonhurd/492395/, CC License BY 2.0
●
“blades” by Flickr user 'gothopotam,' http://www.flickr.com/photos/zagrobot/2731084578/, CC License BY 2.0
●
“Table Decoration” by Flickr user 'Kevin,' http://www.flickr.com/photos/kevincollins/75018222/, CC License BY 2.0
●
“Gargoyle” by Flickr user 'Joe Jakeman,' http://www.flickr.com/photos/jojakeman/2553116529/, CC License BY 2.0
●
“Exeter gargoyle” by Flickr user 'aaron.bihari,' http://www.flickr.com/photos/dakima-arts/3456721942/, CC License BY-SA
2.0
●
“Dark gargoyle” by Flickr user 'Bichuas (E. Carton),' http://www.flickr.com/photos/bichuas/442265525/, CC License BY-SA
2.0
●
“IMG_4743” by Flickr user 'beggs,' http://www.flickr.com/photos/beggs/88809549/, CC License BY 2.0
●
“Gothic Gargoyles” by Flickr user 'jakeliefer,' http://www.flickr.com/photos/jakeliefer/283541940/, CC License BY 2.0
●
“Hunting Gargoyles” by Flickr user 'moonrat42,' http://www.flickr.com/photos/moonrat/1444630026/, CC License BY-SA 2.0
●
“Kissing Gargoyle, Leon, Spain” by Flickr user 'AndrewGould,' http://www.flickr.com/photos/andrewgould/3643732025/,
CC License BY 2.0
●
“IMG_4781” by Flickr user 'beggs,' http://www.flickr.com/photos/beggs/88810165/, CC License BY 2.0
●
“Thann : gargouille / gargoyle 1” by Flickr user 'OliBac,' http://www.flickr.com/photos/olibac/3451405035/, CC License BY
2.0
●
“(gar)Goyle of my dreams...” by Flickr user 'IntangibleArts,' http://www.flickr.com/photos/intangible/1136971041/, CC
License BY 2.0
●
“Gargoyle overlooking Paris” by Flickr user 'aliciat89,' http://www.flickr.com/photos/aliciat/3138416652/, CC License BY 2.0
●
For more info on Creative Commons Licenses see http://creativecommons.org
●
This presentation is licensed under a Creative Commons Attribution-Share Alike 2.0 license:
http://creativecommons.org/licenses/by-sa/2.0/deed.en
*Thank you Creative Commons!
61. Shell tricks
●
My favorite shell trick (at no extra charge):
●
{} expansion
●
echo db{1,2,3,4}.example.com
= “db1.example.com db2.example.com ...”
●
Nestable:
– mkdir -p rpm/{RPMS/
{i386,i686,noarch},SRPMS,SPECS,BUILD,SOURCES}