Rudder 4.1 was released in March 2017 with: - an advanced feature to query external APIs and pull in node properties dynamically the ability to add "key=value" tags to all Rules and Directives in order to categorize them - a new API on relay servers to enable node-to-node file sharing and remote run in firewalled environments performance improvements - a new plugin package format Rudder 4.2 was released in September 2017 and includes the support for a new plugin that adds support for a new Windows DSC-based agent. Rudder 4.3 will include: - Parameters for Technique Editor techniques - ACLs on the API accounts - Many architecture improvements In parallel, new plugins are being developed: - A plugin to integrate data from external APIs - Monitoring integration with Centreon - CMDB integration with iTop - A reporting plugin for historized compliance This talk will introduce these new features and show how to use them, hopefully getting you as excited as we are! Then, we will move on to explain about longer-term feature ideas we have for Rudder, and the general vision linked to future developments. About Nicolas Charles Nicolas is a tinkerer who likes when things just work, and tries his best to reach this goal. He started as a developer 15 years ago, and often had to reach out of this role to solve issues. In 2010, he co-founded Normation, and he still enjoys fixing things in Rudder and at its users.

1. What’s new and what’s next
in Rudder
Nicolas CHARLES
Co-founder and COO
@nico_charles

2. 2
Agenda
Rudder news since last camp
What’s new in Rudder 4.1, 4.2 and 4.3
Future direction

3. 3
Rudder versions
Currently supported versions
●
4.1.x – current ESR* version
●
4.2.x – current latest version
– Will be supported for 3 months after next version is released
(so until May 2018)
●
4.3.x – next version
– Scheduled for release in February 2018
* ESR = Extended Support Release
2015 2016 2017 2018
Jun Jan Oct Mar Sep Feb
3.1 3.2 4.0 4.1 4.2 4.3

4. 4
Microsoft Powershell DSC Agent
4.2
Manage Windows Systems using Powershell DSC
Native agent for Windows
– Uses Microsoft Powershell DSC
– Requires Powershell 4 or more

5. 5
Microsoft Powershell DSC Agent
4.2
Generic methods for DSC, Classic or both Agents

6. 6
New generation method: Directive by Directive
●
Directives are not merged together anymore!
●
New Policy Generation type: SEPARATED
●
Mix audit and enforce mode for same Technique on a Node
●
Several versions at the same time on a node
4.3

7. 7
New generation method: Directive by Directive
●
Directives are not merged together anymore!
●
Most techniques have been rewritten to support this feature
– New version (but you can upgrade them one directive at a time)
<POLICYGENERATION>separated</POLICYGENERATION>
●
Generate one folder per directive
– Path: TechniqueName/TechniqueVersion_DirectiveID
4.3

8. 8
New generation method: Directive by Directive
● Technically, RudderUniqueID placeholder
●
In Technique bundle names/function names
●
In resulting class to avoid collisions
●
Hooks: One time action before and after Directives
– For global actions – like getting the repositories PGP keys only once
4.3

9. 9
Techniques Parameters
●
Defines parameters within the Technique Editor
●
Better re-usability
4.3

10. 10
Techniques Parameters
●
Defines parameters within the Technique Editor
●
Define parameter during Directive creation
4.3

11. 11
Node properties
●
Node properties can now be JSON values:
datacenter = {
"id": "FRA1",
"name": "Colo 1, Paris",
"location": "Paris, France",
"dns_suffix": "paris.example.com"
}
●
Access properties in any directive field:
${node.properties[datacenter]}
${node.properties[datacenter][id]}
●
Use default values:
${node.properties[datacenter][id] | default = "UK2" }
${node.properties[netbios_name] | default = ${rudder.node.hostname} }
${node.properties[datacenter][name] | default = """value with "quotes"
if necessary""" }
4.1

12. 12
Node properties
●
Import automatically properties on nodes from third-party
REST application
●
Datasource plugin
●
Drive behaviour from external source
●
Specific type of Node Property
4.1

13. 13
Node properties
●
Import automatically properties on nodes from third-party
REST application
4.1

14. 14
Node properties
●
Import automatically properties on nodes from third-party
REST application
●
Can be global or on a node by node basis
●
Add data in headers
4.1

15. 15
Node properties
●
Import automatically properties on nodes from third-party
REST application
●
Extract from received JSON relevant information
4.1

16. 16
Node properties
●
Import automatically properties on nodes from third-party
REST application
●
Customize update frequency
4.1

17. 17
Node properties
●
Import automatically properties on nodes from third-party
REST application
●
Define what happens when the API doesn’t answer
4.1

18. 18
Node properties
Agent searches for optional properties files
/var/rudder/local/properties.d/*.json
Add new properties or override existing properties defined on Rudder
Example:
Results in :
"sysctls_postgresql":{"kernel.shmmax":"5368709120"}
4.1
On the node side
"sysctls_postgresql": {
"kernel.shmmax":"5368709120"
}
On the server side
"sysctls_postgresql": {
"kernel.shmall":"903330",
"kernel.shmmax":"3700041320"
}
Override node properties locally

19. 19
JSON everywhere
ncf methods
variable_dict
variable_dict_
from_fle
Import JSON at runtime
4.1

20. 20
JSON everywhere
4.1
Merge JSON at runtime

21. 21
Tags everywhere!
4.1
Tags on Directives and Rules to classify and filter

22. 22
A new API on relay servers
Central server
Node Node Node
TCP communication (port 5309)
File metadata
File contents
Authentifcation + encryption (TLS)
TCP/UDP communication
(ports 443 and 514)
Protocols: HTTPS, syslog
Node Node Node
Isolated network zone
Relay server
Inventory
+ Reports
Confguration
policy

23. 23
... RELAY API
A new API on relay servers
Relay server
Node Node
UI REST API
... RELAY API
Central server
RELAY APIRELAY API
Trigger agent runScenario 1:
Trigger agent runs
remotely, including
via firewalls.
4.1

24. 24
... RELAY API
A new API on relay servers
Relay server
Node 1 Node 2
UI REST API
... RELAY API
Central server
RELAY APIRELAY API
File shared
with metadata
Scenario 2:
Share files from one
node to another.
In the same network
or not (via relays).
... RELAY API
Relay server
sharedfle_to_node(‘node 2’, ‘db.sql’,
‘/var/share/db.sql’, ‘3 days’)
sharedfle_from_node(‘node 1’,
‘db.sql’, ‘/var/share/db.sql’)
ncf methods
4.1

25. 25
Hooks
4.1
Customize behaviour on the server

26. 26
Hooks
4.3
Extend inventory agent side
●
Inventory runs hooks in /var/rudder/hooks.d or C:Program
FilesRudderhooks.d
●
Executable scripts, owned by current user or root, and not world writable
●
Script must return valid JSON
●
Added in inventory tag CUSTOM_PROPERTIES
●
Available in Node Properties on the Rudder Server
●
Can be used to create Groups
●
Available in API

27. 27
Improved performance
●
Improved UI performance
●
New graph rendering library
●
All Web resources are cached
●
Compress all data from Web Interface
●
Better Agent performance
●
40% faster in normal usage, up to 20 times faster with large policies
●
Slightly faster policy generation
4.1

28. 28
Agent
●
Lighter agent
●
Perl is no longer packaged within the Agent
●
SystemD support
●
Timing on the CLI output
●
Dropped the old cfengine network protocol
4.3

29. 29
Miscellaneous
●
Groups of groups
●
Node lifecycle
●
Renaming of ncf generic method
●
And a tool to automatically update the generic method call
●
Same versioning for Rudder & ncf
4.3

30. 30
Plugins
●
Branding: Customize Rudder UI
●
Backgrounds and font colours
●
Login page
●
Logos
●
Title text

31. 31
Plugins
●
Precise ACLs on API
●
Rights per token on any REST API endpoint
●
Token expiration date
●
Maps user permissions to tokens
●
What can we define?
●
AclPath : segments, separated by /
– Segment is either a String (api, nodes, rules, etc)
– Wildcard * , anywhere as a segment
– Double wildcard ** , only at the end, matches anything
●
HttpAction (GET, POST, PUT, DELETE)
●
Anything that is not authorized is denied
4.3

32. 32
Plugins
●
Examples
●
ALLOW api/nodes/** , GET
– Permits to read all in the nodes API
– But no changes at all
●
ALLOW api/nodes, GET
– Permits to list nodes (including searches), but not the pending nodes
●
ALLOW api/directives/7dd68892-6820-4f85-8e44-a7cc820dd06e , POST
– Edit only directive with id 7dd68892-6820-4f85-8e44-a7cc820dd06e
●
ALLOW api/directives/*/check, POST
– Only permits to valid that a change is valid
4.3

33. 33
Plugins
●
Centreon: Automatically configure monitoring on systems
Node
UI REST API
ncf RELAY API
Central server
RELAY APIRELAY API
1 - Synchronize all nodes
in Centreon
Plugin
2 - Configure
node
3 - Configure
hosttemplate

34. 34
Plugins
●
iTop: CMDB integration
●
Export inventories to iTop
●
Import properties from iTop
– Drive policies from CMDB and external data
●
Export Directives and Compliance
– Measure impact of non-compliance

35. 35
Plugins
●
Reporting
●
Generate compliance reports over a period of time
– In development progress

36. 36
Plugins
●
Reporting
●
Defines Rules/Groups/Nodes and a period
●
Select what to display

37. 37
Bug classification

38. 38
Bug classification – 3 parameters

39. 39
Bug classification – 3 parameters
●
User visibility: use case impacted by issue
●
First impression – even before Rudder installation
●
Getting started – during demo, first install or basic usage of simple Techniques
●
Operational – usage of Technique Editor, advanced Techniques, Rudder settings
●
Infrequent – complex configurations, third-party integration

40. 40
Bug classification – 3 parameters

41. 41
Bug classification – 3 parameters
●
Severity:
●
Critical – Prevent main usage of Rudder, can cause data loss – no workaround
●
Major– Prevent usage of a part of Rudder – no easy workaround
●
Minor – Something is misleading or with an easy workaround
●
Trivial – No functional impact, but it would be nicer if it were fixed.

42. 42
Bug classification – 3 parameters

43. 43
Bug classification – 3 parameters
●
Effort required:
●
Small – This issue can be solved in less than a day
●
Medium – It can be fixed in a reasonable amount of time
●
Large – This issue is complex, needs some thoughts and time (about a week)
●
Very large – This issue is so complex that we cannot estimate its duration
(several weeks to months)

44. 44
Bug classification - Priority
●
These information are reviewed, and a priority is computed
from these
●
From 0 (lower priority) to about 150 (the top priority)
●
Weighted based on user visibility and severity
●
Biased toward smallest effort and oldest bugs

45. 45
http://faq.rudder-project.org
New FAQ

46. 46
Agenda
The future

47. 47
Client – Server communication
●
Two steps policy update
●
Validation on the client side
●
Ensure complete consistency
●
Much faster policy generation

48. 48
Client – Server communication
●
Drop syslog protocol
●
Send reports via HTTPS
●
Minimize impact of agent on nodes
●
Improve performances and network usage

49. 49
Web Interface
●
Customize Dashboard
●
Customize columns in tables
●
Improve Group page
●
Improve search interface and group creation
●
Manage Users within the UI

50. 50
Future plugins (planned + ideas)
Sync data
between
Rudder servers
Ideas of plugins
Advanced access
control
(OrBAC)
High availability
for Rudder server
“Ramp up”
policies for
progressive
rollouts

51. 51
Rudder Ambassador Program
●
Rudder Ambassador
●
Program for exceptional Rudder contributors
●
To be announced

52. 52
Rudder.io
●
New Domain name
●
Rudder.io
●
Focus on the Rudder brand

53. What’s new and what’s next
in Rudder
Questions ?
Nicolas CHARLES
Co-founder and COO
@nico_charles