![Contents ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à FOSDEM 2012: Practical implementation of promise theory in CFEngine
Similaire à FOSDEM 2012: Practical implementation of promise theory in CFEngine (20)
Dernier
Dernier (20)
FOSDEM 2012: Practical implementation of promise theory in CFEngine
- 1. Practical implementation of promise theory in CFEngine Mikhail Gusarov CFEngine AS <mikhail.gusarov@cfengine.com>
- 7. Summary
- 8. Promise theory «Agent A promises to agent B to fulfill C»
- 11. Multiple MXes
- 14. Reports body common control { bundlesequence => {"mybundle"}; } bundle agent mybundle { reports: linux:: "Hello world!"; }
- 15. Reports body common control { bundlesequence => {"mybundle"}; } bundle agent mybundle { reports: linux:: "Hello world!"; }
- 16. Reports bundle agent mybundle { reports: linux:: "Hello world!"; }
- 17. Reports bundle agent mybundle { reports: linux:: "Hello world!"; }
- 18. Reports bundle agent mybundle { reports: linux:: "Hello world!"; }
- 19. Reports bundle agent mybundle { reports: linux:: "Hello world!"; }
- 20. Reports (translated) def mybundle() { if "linux" in global_contexts { report("Hello world!"); } } mybundle()
- 21. Reports (run) $ cf-agent R: Hello world! $
- 22. Reports (run) $ cf-agent R: Hello world! $ cf-agent $
- 23. Reports (run) $ cf-agent R: Hello world! $ cf-agent $ cf-agent $
- 24. Reports bundle agent mybundle { reports: linux:: "Hello world!"; "That's surprising"; }
- 25. Reports (run) $ cf-agent R: Hello world! $ cf-agent $ cf-agent $ cf-agent R: That's surprising $
- 26. Reports (run) $ cf-agent R: Hello world! $ cf-agent $ cf-agent $ cf-agent R: That's surprising $ cf-agent R: Hello world! $
- 27. def mybundle() { promises.add( report("Hello world!", activate = λ : "linux" in global_contexts)) } mybundle() run_all_relevant_promises()
- 28. «Ensure that specified message is reported to sysadmin as soon as possible, but no more than once per 5 minutes to the sysadmin using the specified reporting mechanisms»
- 29. «Ensure that specified message is reported to sysadmin as soon as possible, but no more than once per 5 minutes to the sysadmin using the specified reporting mechanisms» Convergence
- 30. «Ensure that specified message is reported to sysadmin as soon as possible , but no more than once per 5 minutes to the sysadmin using the specified reporting mechanisms» Embracing errors
- 31. «Ensure that specified message is reported to sysadmin as soon as possible, but no more than once per 5 minutes to the sysadmin using the specified reporting mechanisms » Autonomy
- 32. Processes bundle agent myprocesses { processes: "nginx" match_range => irange("1","20"); restart_class => "restart_nginx"; commands: restart_nginx:: "/etc/init.d/nginx start"; }
- 33. Processes bundle agent myprocesses { processes: "nginx" match_range => irange( "1","20" ); restart_class => "restart_nginx"; commands: restart_nginx:: "/etc/init.d/nginx start"; }
- 34. Processes bundle agent myprocesses { processes: "nginx" match_range => irange( "1","inf" ); restart_class => "restart_nginx" ; commands: restart_nginx:: "/etc/init.d/nginx start"; }
- 35. def myprocesses() { nginx_cnt = count_proc("nginx") if nginx_cnt == 0 { promises.add( command( "/etc/init.d/nginx start")) } elif nginx_cnt > 20 { Promises.add(kill("nginx", nginx_cnt – 20)) } } myprocesses() run_all_relevant_promises()
- 36. def myprocesses() { promises.add( processes("nginx", λ count: if count == 0 { define_context("restart_nginx")}) promises.add( commands( "/etc/init.d/nginx start", activate_if = λ : "restart_nginx" in global_contexts)) } myprocesses() run_all_relevant_promises()
- 37. «Ensure that amount of specified processes is kept in specified range, killing extra processes and communicating the need to restart if necessary»
- 38. «Ensure that amount of specified processes is kept in specified range , killing extra processes and communicating the need to restart if necessary» Convergence
- 39. « Ensure that amount of specified processes is kept in specified range, killing extra processes and communicating the need to restart if necessary » Embracing errors
- 40. «Ensure that amount of specified processes is kept in specified range, killing extra processes and communicating the need to restart if necessary» Autonomy
- 41. Packages bundle agent mypackages { packages: "libapache2-mod-wsgi" package_method => apt, package_policy => "addupdate", package_version => "3.3-4", package_select => ">="; }
- 42. Packages bundle agent mypackages { packages: " libapache2-mod-wsgi " package_method => apt , package_policy => " addupdate ", package_version => " 3.3-4 ", package_select => " >= "; }
- 43. Packages (translated) (Implementation is too large to fit in the margin slide)
- 44. «Ensure that specified package is installed or not installed according to the constraints specified. Do this by instructing local package manager to add, upgrade or remove package»
- 45. «Ensure that specified package is installed or not installed according to the constraints specified. Do this by instructing local package manager to add, upgrade or remove package» Convergent
- 46. «Ensure that specified package is installed or not installed according to the constraints specified. Do this by instructing local package manager to add, upgrade or remove package» Embracing errors
- 47. «Ensure that specified package is installed or not installed according to the constraints specified. Do this by instructing local package manager to add, upgrade or remove package» Autonomous
- 49. «Maintain in-agent binding of left-side name to the value of right-side expression»
- 50. Convergent, autonomous «Maintain in-agent binding of left-side name to the value of right-side expression»
- 51. Embracing errors bundle agent myvars { vars: "myvar" string => execresult ( "/usr/bin/testparm /etc/smb/smb.conf", "noshell"); }
- 53. Convergent
- 54. Error-embracing
- 55. Autonomous
- 56. Not the things from conventional languages
- 57. IRC: #cfengine @ FreeNode Web: http://cfengine.com/tech ML: https://cfengine.org/mailman/listinfo Questions?
Notes de l'éditeur
- Welcome and thank you for coming. My name is Mikhail Gusarov and I work for CFEngine AS, which is a company backing free software project CFEngine. I expect it to be most interesting to system administrators and software engineers who tried CFEngine and were puzzled by the semantics of operations, and also who read the works on promise theory by Mark Burgess and are trying to figure out how do the theoretical foundation are mapped to real-world CFEngine implementation.
- We will start with the quick recap of promise theory, and then proceed to dissect several promise types in CFEngine, trying to find how the fundamental principles are reflected in particular design.
- Let's start with a quick question: please raise your hand if you have used CFEngine before. And who has read the Mark Burgess' papers on promise theory? The promise theory states that in distributed (or multi-agent) systems you may not adequately describe reality by obliging actors to do something, and instead promise is the building block. Promise in computing is similar to the one in real life: it might be broken (voluntarily or forcedly), it might be fulfilled from time to time, so at the best we may only statistically say that the promise is kept.
- What does it mean from the software point of view? We need some way do describe and test promises, and the most useful way to do it is to formulate promises in the form of «desired state», with operators to test whether desired state is achieved, and to perform actions on the system to move closer to the desired state. We need to embrace errors, as it's quite rare situtaion when the promise we are trying to fulfill may be achived by the agent itself, so we need to be able to handle unfulfilled promises by any other agents we communicate with, as well as any internal inconsistencies. We need to be able to work autonomously, as we may not rely to another agents. At least we should be able to fail gracefully and wait until other agents start to fulfill their promises.
- Here's an example of a reliable system which uses all of those principles. E-mail «promises» are convergent, in the sense that single promise this system fulfills is to deliver mail to the given address, and every MTA knows how to verify whether this promise is fulfilled or not, and how to advance in achievement of this promise. E-mail embraces errors, and there is huge lot of error handling in all components of-email. E-mail is autonomous in the sense, that any MTA is able to work offline, and does not fail completely if there is no DNS service. Once external services is available, it fetches the necessary data, makes routing decision and contines with delivery.
- Let's start with a simple policy.
- First section declares what _bundles_ are to be taken into account during CFEngine run, and is not particularily interesting. I will skip this part of the policy for the rest of the talk.
- Let's have a look at second one. This is a bundle, that is, a building block for CFEngine policy, best approximated as a module in other languages.
- This module contains a single statement of type &quot;report&quot;.
- executed only in context &quot;linux&quot;, which is the case on Linux machines.
- This statement asks to report a string &quot;Hello world!&quot;.
- Let's use some imaginary imperative programming language to re-state the policy above. From the look of it, it looks similar to ... But does it behave this way?
- Let's run it. Fine nothing unexpected.
- Let's run it again. That's interesting.
- Let's run it again. Hmm...
- Let's add another report statement.
- And run it again.
- And one more time. So, the «reports» statement does not behave like a conventional «print» statement from imperative language.
- More correct restatement in imaginary language we have used before would be like the following.
- If we would specify the «reports» promise from CFEngine language in English, it would sound similar to the following.
- This type of promises is convergent: we know the end state, and how to get closer to it.
- This type of promises embraces errors, as we know that reporting mechanisms are not always available.
- And this this type of promise is autonomous in the sense that agent does not need any additional information to decide where to direct the report.
- Let's have a look at another promise.
- This declaration promises that agent will keep number of «nginx» processes in specified range.
- And if it is not, extra processes will be killed, or the need to spawn new processes will be communicated to another promise.
- This is a simplistic rewrite of the previous policy in our imaginary imperative language, but it is not accurate in one significant aspect: this code only checks for amount of nginx processes once, while agent is able to re-check it again, if other parts of policy do a corrections, and spawn or kill monitored processes.
- So the following rewrite would be more adequate, but as you easily see, it is just a restatement of policy in different syntax.
- Let's try to formulate what does this promise type say.
- This promise type is convergent, as we know the target state, and how to reach it.
- The mere existance of this promise type is admittance of the fact processes may run out of the control.
- And the promise type is obviously autonomous, as it does not need any external data to proceed.
- Let's consider the next promise:
- Agent promises to ensure that package libapache2-mod-wsgi from APT repository is installed and has version at least 3.3-4.
- I won't even try to translate this kind of promise, as it will be either very verbose, or just restate the policy language using DSL of some sort.
- The following description of package promise type is simplified (as it does not include large number of less-used features, like lack of local package manager, or rarely used operations), but nonetheless useful.
- Packages promise is obviously convergent, as we know the end state and how to reach it.
- It implicitly embraces errors, as implementation will patiently wait until local package manager is able to do what is requested (e.g. until network is available to fetch new packages)
- And the promise is autonomous in the sense that no extra information is needed to verify the state or to try to reach the promised state.
- Variables are special kind of promises. They belong to the agent itself, and not to the environment.
- And hence their semantics is pretty similar to the one you might find in functional languages.
- Variables are trivially convergent and autonomous by definition.
- But still, there is error-handling component. CFEngine variables will try to evaluate itself until it suceeds. For example, in this variable definition, execresult function may fail if testparm binary is not available, and variable will be evaluated as frequently as deemed necessary to try to get it value, e.g. If testparm binary is installed by another part of policy.
- So, we have got through three constructs in CFEngine language, and all of them shown the same unifying principles behind: convergency, embracing errors, and autonomy. This approach distinguishes CFEngine from configuration management tools, such as glorified parallel shell wrappers, where you need to think of convergence yourself, and the language does not help you to produce self-healing systems. But also this approach makes CFEngine language so different from conventional languages.
- As you just saw, the building blocks of CFEngine language are quite different from the functional or imperative languages. CFEngine agent does not execute sequence of commands, and does not evaluate a function changing world as a side-effect, but keeps a number of promises. There are roughly two ways of describing domain knowledge in machine-readable way: * eDSL for general-purpose language * Separate DSL Some promises may require a great deal of context to be bundled in order to be convergent, and do not docempose to more fundamental entities which still have the property of convergency. This makes describing policies as a eDSL pretty cumbersome - resulting eDSL would still have to have all the CFEngine language primitives, including classes and variables, so it would just add another layer of syntactic clutter on top of the same semantics. Considering this issue, CFEngine chose the approach of having a separate DSL.