Slideset describing a fictitious Cloud service like GAE or no.de but with python to demo the power of Python, OCCI and Solaris!

1. Create a simple Service Offering with Python & Solaris Python with Solaris Zones, Dtrace & OCCI MUCOSUG meeting, June 16th, 2011

2. Create a simple Service Offering with Python & Solaris Warning:Clouds ahead Python with Solaris Zones, Dtrace & OCCI MUCOSUG meeting, June 16th, 2011

3. Intention: present why Python and Solaris are a perfect fit… 2011 (c) Thijs MetschThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

4. This is just a showcase to demonstrate all necessary ingredients...

5. Why Python, OCCI and Solaris?

6. Why Python, OCCI and Solaris? Containers Robustness ZFS SMF DTrace Performance

7. Why Python, OCCI and Solaris? Innovation RESTful Python Powered Standard Cloud Protocol Containers Robustness ZFS SMF DTrace Performance

8. Why Python, OCCI and Solaris? Innovation RESTful Readability Simplicity Python Powered Maintainability BDFL Standard Cloud Protocol Development Speed Containers Robustness ZFS SMF DTrace Performance

9. Why Python, OCCI and Solaris? Innovation RESTful Readability Simplicity Python Powered Maintainability BDFL Standard Cloud Protocol Development Speed Fastest way to create a service and start earning money! Containers Robustness ZFS SMF DTrace Performance

10. Source: http://xkcd.com/353/

11. Source: lesher.ws/choose_python.pdf

12. Cutting edge,Cool & Robust Technology

13. General Flow

14. Big Picture* WSGIApp Monitoring ServiceOffering Deployment Developer Manage Access Enduser Solaris Zonewith WSGI App Enduser Solaris Zonewith WSGI App Enduser Solaris Zonewith WSGI App ... ...(Scaling) Monitoring (Analytics) Solaris Box Internet * Everybody needs one

15. Big Picture* pyOCCI WSGIApp Monitoring OCCI/Mercurial ServiceOffering Deployment Developer Manage Python Access Enduser Solaris Zonewith WSGI App Enduser Solaris Zonewith WSGI App Enduser Solaris Zonewith WSGI App ... ...(Scaling) Monitoring (Analytics) DTrace Solaris Box Website Internet * Everybody needs one

16. Step 1 – Service definition

17. A simple WSGI app fromwsgiref.simple_serverimportmake_server defapplication(env,response): response_body='Hello World' status='200 OK' response_headers=[('Content-Type','text/plain'), ('Content-Length',str(len(response_body))) ] response(status,response_headers) return[response_body] httpd=make_server('localhost',4711,application) httpd.handle_request()

18. Step 2 – Service deployment

20. During the creation of OCCI we updated implementations periodically

21. Complete implementation of the Specification ~1000 LOC

22. Testframe written in Python„The Open Cloud Computing Interface (OCCI) is a RESTfulProtocol and API for all kinds of Management tasks. OCCI was originally initiated to create a remote management API for IaaS model based Services. It has since evolved into a flexible API with a strong focus on integration, portability, interoperability and innovation while still offering a high degree of extensibility. The current release of the Open Cloud Computing Interface is suitable to serve many other models in addition to IaaS, including e.g. PaaS and SaaS.” – http://www.occi-wg.org

24. During the creation of OCCI we updated implementations periodically

25. Complete implementation of the Specification ~1000 LOC

26. Testframe written in Python„The Open Cloud Computing Interface (OCCI) is a RESTfulProtocol and API for all kinds of Management tasks. OCCI was originally initiated to create a remote management API for IaaS model based Services. It has since evolved into a flexible API with a strong focus on integration, portability, interoperability and innovation while still offering a high degree of extensibility. The current release of the Open Cloud Computing Interface is suitable to serve many other models in addition to IaaS, including e.g. PaaS and SaaS.” – http://www.occi-wg.org Python based OCCI implementation: pyOCCI

27. Step 3 – Backen Magic

28. Python is ideal to create a (Scalable/Cloud) Service -Very easy to integrate with other services (e.g. For Authentication, DBs, Messaging, SCM...)

29. Python is ideal to create a (Scalable/Cloud) Service -Very easy to integrate with other services (e.g. For Authentication, DBs, Messaging, SCM...) For Data transfer: e.g. Uploading the WSGI through mercurial

30. Integration with LDAP, CouchDB, AMQP importldap server=ldap.initialize('ldap://localhost:8080') server.search_s('dc=example,dc=com',ldap.SCOPE_SUBTREE,'(cn=foo*)',['cn','mail']) importcouchdb couch=couchdb.Server('http://localhost:5984/') db=couch['services'] doc={'user':'foo'} db.save(doc) importpika connection=pika.BlockingConnection(pika.ConnectionParameters(host='localhost')) channel=connection.channel() channel.queue_declare(queue='hello') channel.basic_publish(exchange='',routing_key='hello',body='Hello World!') connection.close()

31. Now let‘s handle Resource Pools & Zones (OS Integration) with Python

32. Wrapping zone.h with SWIG... $ make all swig -Wall -python pyzone.i gcc -I/usr/include/python2.6 -c pyzone_wrap.c ld -shared pyzone_wrap.o -o _pyzone.so $ cat test.py import pyzone zones = pyzone.zone_list() print len(zones) # equal to: pyzone.get_nr_of_act_zones() print zones $ python test.py 2 ['global', 'foo'] I can release this if there is interest…

33. Step 4 – Using the service

34. Batteries included (for the Client) ... importhttplib connection=httplib.HTTPConnection('localhost:8080') connection.request('GET','/') r1=connection.getresponse() printr1.status,r1.reason importjson printjson.dumps({'msg':'Hello World'},sort_keys=True,indent=2) Python also offers many GUI frameworks – also build-in (and therefore available on all platforms/clients) is Tk (Easy to use with pytkgen*)... * Creates Tk based GUIs from JSON definition files.

35. Step 5 – Monitoring

36. You can user Dtrace to monitor the ZFS, Zones, Network etc.

37. You can also create stuff like this... Source: http://wiki.joyent.com/display/node/Using+Cloud+Analytics

38. Or even trace Python itself...

39. Check if Python probes are available $ pfexec dtrace -ln 'python*:::' ID PROVIDER MODULE FUNCTION NAME 9398 python861 libpython2.6.so.1.0 PyEval_EvalFrameEx function-entry 9399 python861 libpython2.6.so.1.0 dtrace_entry function-entry 9400 python861 libpython2.6.so.1.0 PyEval_EvalFrameEx function-return 9401 python861 libpython2.6.so.1.0 dtrace_return function-return

40. Check if Python probes are available $ pfexec dtrace -ln 'python*:::' ID PROVIDER MODULE FUNCTION NAME 9398 python861 libpython2.6.so.1.0 PyEval_EvalFrameEx function-entry 9399 python861 libpython2.6.so.1.0 dtrace_entry function-entry 9400 python861 libpython2.6.so.1.0 PyEval_EvalFrameEx function-return 9401 python861 libpython2.6.so.1.0 dtrace_return function-return If you need more you could tinker around with ceval.c in the Python sources

41. Count calls #pragma D option quietpython*:::function-entry/(self->name = copyinstr(arg0)) != NULL && self->name >= "/export/home/tmetsch/data/workspace/pyssf/pyocci/" /{ @[copyinstr(arg0)] = count(); self->ts=timestamp;} $ pfexec dtrace -s ./test2.d /export/home/tmetsch/data/workspace/pyssf/pyocci/examples/vm_skeleton.py 9

42. Count calls #pragma D option quietpython*:::function-entry/(self->name = copyinstr(arg0)) != NULL && self->name >= "/export/home/tmetsch/data/workspace/pyssf/pyocci/" /{ @[copyinstr(arg0)] = count(); self->ts=timestamp;} Predicate $ pfexec dtrace -s ./test2.d /export/home/tmetsch/data/workspace/pyssf/pyocci/examples/vm_skeleton.py 9

43. Trace the flow... #pragma D option quietpython*:::function-entry/(self->name = copyinstr(arg0)) != NULL && self->name == "/export/home/tmetsch/data/workspace/pyssf/pyocci/core.py" /{ printf("step into: %s %s ", copyinstr(arg1), copyinstr(arg0)); self->traced = 1}python*:::function-return/self->traced/{ printf("back from: %s ", copyinstr(arg1)); self->traced = 0}

44. Trace the flow...(2) $ pfexec dtrace -s ./test4.d -c '/usr/bin/python ./pyocci/examples/vms_main.py' step into: <module> ./pyocci/examples/vms_main.py back from: <module> step into: Login ./pyocci/examples/vms_main.py back from: Login step into: MyService ./pyocci/examples/vms_main.py back from: MyService step into: __init__ ./pyocci/examples/vms_main.py back from: _load_ui_modules step into: start ./pyocci/examples/vms_main.py back from: __init__

45. Monitoring the OCCI service $ pfexec dtrace -s ./test2.d -c '/usr/bin/python ./pyocci/examples/vms_main.py' Creating the virtual machineStarting virtual machine with id/compute/c768773a-2b6e-ce34-f08a-be3fd3a3670c ./pyocci/examples/vms_main.py 5 Login 2193 MyService 3538 ...

46. Python + Solaris = Perfect Platform* Cool Technology * http://www.nohuddleoffense.de/2011/02/21/my-software-development-environment-for-python/

47. Also for Development BTW (See Blog post) Python + Solaris = Perfect Platform* Cool Technology * http://www.nohuddleoffense.de/2011/02/21/my-software-development-environment-for-python/

49. http://pyssf.sf.net

50. http://www.openindiana.org

51. http://wsgi.org/wsgi

52. http://www.occi-wg.org

53. https://github.com/tmetsch/pytkgen

54. http://mercurial.selenic.com/

55. Python

56. http://www.python.org

57. http://diveintopython3.org

58. Solaris

59. http://www.opensolaris.org

60. http://www.illumos.org

61. http://www.solarisinternals.comSource: amazon.de

62. Thank you! (c) 2011 - Thijs Metschtmetsch at opensolaris dot org http://www.twitter.com/befreax http://www.tmetsch.org