Debugging (custom) Ansible Modules -- Presentation from Ansible meetup in London on 16th July 2015

1. Debugging custom Ansible modules

2. Who am I
Alex Leonhardt
Ansible user for ~2-3 months,
previously Puppet and some SaltStack
Ops guy for a very long time
Twitter: @alex_leonhardt

3. Elsevier
"Elsevier is an information solutions company with roots in publishing scientific, medical, and technical literature"
Publisher of scientific, medical and technical literature
ScienceDirect.com, part of Elsevier
AWS, Ansible, Packer
Python / BASH + GoCD for operations/automation
Microservice architecutre (mostly Java)
Know any/all of above? We're hiring!

4. Debugging Ansible modules
Ansible makes debugging somewhat problematic, expects valid json for
any output.
print('x has value: {0}').format(x) won't ever make it to the console.
So how do we check the state of a variable, e.g. when looping over a list?

5. Structure

6. Playbook

7. Be verbose, be very verbose
-vvv

8. Using AWS (who isn't) ?
~/.boto
Boto has a debug setting, use it!

9. Boto debug
HTTP POST output
Caution! Credentials are shown in clear text

10. Custom module
ec2_describe.py

11. Custom module
ec2_describe.py

12. Custom module
list ec2 instance ids
but 'print' won't work >:(

13. Custom module (run)

14. Custom module (run)

15. Some output
not great, but something

16. at least we now know the correct class
Some more output
to check for attributes, etc.

17. IDEs really help to find the correct
VIM vs IDE vs (other)
class(es) quickly, but use whatever you like

18. Fixed code
and the correct method used
... this time ;)

19. Final play

20. Final play (output)

21. But wait...
... what about print("hello world: {0}").format(x) ?

22. Use python logging
" p r i n t "

23. Use python logging
Setup logging in the main() function
CAUTION!
Thiswillalsoprintbotodebugoutputincludingyourplain-textkeys!

24. Finally.. we get some output!

25. Bonus points - more boto debug!

26. Use 'q'
"Quick-and-dirty debugging output for tired programmers"
Install it with pip install q( )
Output is sent to $TMPDIR/q
Use for normal output (replace 'print' with 'q')
Use as a decorator (@q)
https://pypi.python.org/pypi/q

27. A q(uick) example
Output
Decorator

28. A q(uick) example (output)
Output

29. A q(uick) example (output)
Decorator

30. check out the temporary directory:
Mooaaarr debugging!
tell ansible to not clean-up after itself:
$exportANSIBLE_KEEP_REMOTE_FILES=1
$play-Mmodules/-iinventorymyplay_describe.yml-vvv
(in our case locally as we dont run against remote hosts)
$cd ~/.ansible/tmp/ansible-tmp-1434809169.56-151563844546363
$ls-l
-rw-r--r-- 1ale staff 7265520Jun15:06ec2_describe

31. Mooaaarr debugging!
ec2_describe
[...]
state=module.params.get('state')
ifstate=='describe':
i_list,changed=get_instance_list(module,ec2)
else:
changed=False
i_list='Failed'
module.exit_json(changed=changed,instances=i_list)
#importmodulesnippets
#ThiscodeispartofAnsible,butisanindependentcomponent.
#Thisparticularfilesnippet,andthisfilesnippetonly,isBSDlicensed.
[...]
#==BEGINDYNAMICALLYINSERTEDCODE==
ANSIBLE_VERSION='1.9.1'
MODULE_ARGS='region=eu-west-1'
MODULE_COMPLEX_ARGS='{"region":"eu-west-1"}'

32. Use pdb (python debugger)
... to step through the code.
~/.ansible/tmp/ansible-tmp-1434809169.56-151563844546363$ python-mpdbec2_describe

33. Cleanup after yourself
Leaving debug logs around is dangerous, so don't forget!
rm-vf/tmp/ansible_debug.log$TMPDIR/q

34. Links / Resources
https://docs.python.org/2/library/logging.html
https://pypi.python.org/pypi/q
https://docs.python.org/2/library/pdb.html
http://docs.ansible.com/ec2_module.html
http://docs.ansible.com/developing_modules.html#testing-modules

35. Thank you!