TOSCA in Practice with ARIA

TOSCA in Practice
with ARIA
Getting up to speed with
TOSCA Simple Profile in YAML
and its ARIA implementation
©2016 GigaSpaces

Thank you for silencing your phone.

What Is It?
Descriptive object-oriented domain-specific language (DSL)
Based on YAML (and XML) (ARIA supports Unicode)
For describing cloud topologies, meaning:
Hosts: bare metal, virtual machines, containers
Software components: applications, databases, middleware
Network components: load balancers, gateways, VNFs
Entry points for orchestration (per node; general workflows coming in v1.1)
Description is mostly static, though:

Context
Members (Nov 2016):
ASG, Bank of America, Brocade, CA, Cisco, Cloudsoft, CSI, FastConnect, Fujitsu, GigaSpaces, Google, HP, Huawei, IBM, iMinds,
Intel, JPMorgan Chase, Mitre, NetApp, NetCracker, NIST, Nokia, Politecnico di Milano, Primeton, Quali, Red Hat, SAP, SINTEF,
UNIPI, VMware, Vnomic, WIPRO, Yanna, Zenoss
Timeline:
1993: “SGML Open”; HQ in Massachusetts
1998: “OASIS”; change of emphasis to XML
1999: ebXML (with UN)
2005: ODF (clash with Microsoft’s Open Document)
2011: Call for participation for TOSCA
2012: First draft of TOSCA 1.0 (in XML)

YAML
1. YAML = YAML Ain’t a Markup Language
2. 2001: first proposed, as “Yet Another Markup Language”
3. 2009: version 1.2
4. Pretty much compatible with JSON
5. But intended for human readability

It Ain’t Over Until the Fat Lady Sings

Operas, Orchestras, and … Napoleon?
TOSCA = Topology and Orchestration Specification for Cloud Applications
Tosca was an Italian opera by Giacomo Puccini that premiered in 1900. Based on
Victorien Sardou’s French play about the Kingdom of Naples’ defense of Rome from
Napoleon in 1800. It contains depictions of torture, murder and suicide, as well as
some of Puccini’s best-known lyrical arias. (Wikipedia)
ARIA = Agile Reference Implementation of Automation
Aria is an expressive melody, usually, but not always, performed by a singer. A self-
contained piece for one voice, normally part of a larger work. (Wikipedia)

Links
TOSCA
https://www.oasis-open.org/committees/tosca/
http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
YAML/v1.0/cos01/TOSCA-Simple-Profile-YAML-v1.0-cos01.html
ARIA
http://ariatosca.org/
https://github.com/aria-tosca/aria-
ng/tree/master/src/tosca/aria_extension_tosca/profiles

Get ARIA
See http://ariatosca.org for up-to-date instructions. For today:
Requirements:
Python 2.7, pip, virtualenv
Install:
git clone https://github.com/aria-tosca/aria-ng
cd aria-ng
virtualenv env
. env/bin/activate
pip install .

Using ARIA
Quick start:
aria blueprints/tosca/node-cellar/node-cellar.yaml
Variations:
aria blueprints/tosca/node-cellar/node-cellar.yaml --graph
aria blueprints/tosca/node-cellar/node-cellar.yaml --json
aria blueprints/tosca/node-cellar/node-cellar.yaml --yaml
aria blueprints/tosca/node-cellar/node-cellar.yaml model
aria blueprints/tosca/node-cellar/node-cellar.yaml presentation

Meta
tosca_definitions_version: tosca_simple_yaml_1_0
description: >-
Node Cellar TOSCA service template.
metadata:
template_name: node-cellar
template_author: ARIA
template_version: '1.0.0'
aria_version: '1.0'
dsl_definitions:
blah_blah: &MYANCHOR
abc: 123

Repositories
Optional: Can be used for imports and for downloading artifacts
repositories:
openstack:
description: >-
Provides OpenStack standard types for TOSCA as
well as implementation scripts.
url: http://myorg.org/repository/openstack/
credential:
user: guest
token: anonymous123

Imports
Short form, implicit repository:
imports:
- types/openstack.yaml
- types/nodejs.yaml
Long form, specify repository:
imports:
- file: types.yaml
repository: openstack
- file: types.yaml
repository: nodejs
(Also supports namespace_prefix, but let’s ignore it for simplicity)

Types
1. Nodes, Interfaces, Artifacts, Capabilities, Relationships, Groups, Policies, Data
2. Inheritance via derived_from
Data types can also inherit from primitive types (more on this later)
3. Polymorphism
In most cases when a type is specified, a derived type could also work
ARIA makes sure you do not break this contract
4. Can override properties, attributes, and interface and operation inputs and
implementations

Example: Node Type
node_types:
mongodb.Server:
description: >-
MongoDB server application.
derived_from: tosca.nodes.DBMS
properties:
port: # override
type: integer
default: 27017
artifacts:
mongodb:
description: >-
MongoDB application package.
type: os.Archive
file: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1604-3.2.8.tgz
deploy_path: /opt/mongodb
capabilities:
host: # override
type: tosca.capabilities.Container
valid_source_types: [ mongodb.Database ]

Types: Best Practices
1. Prefer to inherit from a standard TOSCA type when possible
More portable
More compatible with tools
Some have special functionality
e.g. tosca.relationships.HostedOn is used by get_property HOST function
Take some time to familiarize yourself with them! (not in scope for this workshop)
2. Put your types in separate YAML files to be imported
3. Use namespaces in lowercase, and camel case for type names

Templates
1. Nodes and Relationships (but also: Topology, Groups, Policies)
2. Templates are instantiated in deployments
Nodes and Relationships are true templates: they can have many instances per deployment
Groups and Policies have only one instance per deployment plan
An instance of a topology is a deployment
3. One topology_template per service template
Container for all the other templates
4. Must have a type

Example: Node Templates
topology_template:
node_templates:
node_cellar:
type: NodeMongoApplication
requirements:
- database: node_cellar_database
capabilities:
app_endpoint:
properties:
protocol: udp
url_path: /nodecellar
node_cellar_database:
type: mongodb.Database
properties:
name: node_cellar
artifacts:
initial:
type: mongodb.DatabaseDump
file: node-cellar.json
repository: node_cellar

Copying Templates
loadbalancer_host:
type: openstack.Instance
properties:
flavor_name: m1.small
os_users:
root:
password: admin123
interfaces:
Standard:
inputs:
openstack_credential: { get_input: openstack_credential }
application_host:
copy: loadbalancer_host
capabilities:
scalable:
properties:
max_instances: 10

Types, Templates, and Values
1. Values: properties, inputs, attributes
2. In types we declare them, in templates we assign them
3. Cannot assign values not declared at the type (ARIA will let you know)
4. Required elements must be assigned (ARIA will let you know)
5. Elements might have default values if you do not assign them

Journey of a Value, Part 1: Definition
node_types:
tosca.nodes.DBMS:
derived_from: tosca.nodes.SoftwareComponent
properties:
root_password:
type: string
required: false
port:
type: integer
constraints:
- in_range: [ 1, 65535 ]
mongodb.Server:
properties:
port: # override
type: integer
default: 27017

Journey of a Value, Part 2: Assignment
topology_template:
node_templates:
main_database:
type: mongodb.Server
properties:
port: 7000
backup_database:
properties:
port: 7001

Interfaces and Operations
Entry points for orchestration: nodes, groups, relationships
Completely agnostic to technology:
Scripts (shell, Python, Perl, etc.)
Puppet/Chef/Fabric
Cloudify/Ansible/Juju
Etc.
The orchestrator will need to do its own validation
Strongly typed: interface types, declared in node type, specified in node template

Interface Types
interface_types:
tosca.interfaces.node.lifecycle.Standard:
derived_from: tosca.interfaces.Root
create: {}
configure: {}
start: {}
stop: {}
delete: {}

lifecycle.Stateful:
derived_from: tosca.interfaces.node.lifecycle.Standard
inputs:
term:
type: string
constraints:
- valid_values: [ short, long ]
reset: {}
store:
implementation: store_state.py
retrieve:
implementation:
primary: get_state.py
dependencies: [ state_configuration.ini ]
inputs:
validate:
type: boolean
default: true
Interface Types

Interface Declaration
node_types:
tosca.nodes.Root:
interfaces:
Standard:
type: tosca.interfaces.node.lifecycle.Standard
WebCache:
derived_from: tosca.nodes.Root
interfaces:
Stateful:
type: lifecycle.Stateful
reset: # override
implementation: reset_web_cache.py
retrieve:
inputs:
validate: # override
type: boolean
default: false

Interface Specification
topology_template:
node_templates:
user_frontend:
type: WebCache
interfaces:
Standard:
create: create_frontend.py
destroy: # long form
implementation:
primary: destroy_frontend.py
dependencies: [ user_frontend_config.ini ]
Stateful:
inputs:
term: short
retrieve:
inputs:
validate: true

Dynamics: Template Inputs, Outputs, Attributes
1. Inputs: a simple (too simple?) way to make service templates dynamic
2. Attributes: like parameters, but handled “live” by the orchestrator
3. Outputs: usually call the get_attribute function

Attributes
node_types:
mongodb.Server:
attributes:
admin_port:
type: tosca.datatypes.network.PortSpec
default:
protocol: tcp
source: 80
status: experimental
Available statuses:
supported (default), unsupported, experimental, deprecated

Dynamics: Putting it Together
topology_template:
inputs:
dbpassword:
type: string
value: ‘123’
node_templates:
database:
parameters:
password: { get_input: dbpassword }
outputs:
endpoint:
type: tosca.datatypes.network.PortSpec
value: { get_attribute: [ database, admin_port ] }

Requirements and Capabilities (“Reqs-and-Caps”)
1. TOSCA’s “secret sauce”
Warning: it’s a complex and complicated spec
The model annoyingly breaks the usual distinction between types and templates
ARIA covers it comprehensively
Still, you are advised to keep things as straightforward as possible in your service templates
2. Requirements can be general about some things, specific about others
3. Requirements are where relationships might come to play
Yes, you can use reqs-and-caps without ever using relationships!

Capabilities
1. Declared in node type
2. Properties and interfaces assigned in node template
3. Node types inherit their parents’ capabilities
Can override, but only with the same capability type or descendant
Need to completely change capabilities? Then you shouldn’t inherit from this node type!
4. Limit which source node are allowed to require us via “valid_source_types”
And their descendant types

Capabilities
capability_types:
tosca.capabilities.Endpoint:
derived_from: tosca.capabilities.Root
properties:
protocol:
type: string
default: tcp
port:
type: tosca.datatypes.network.PortDef
required: false
secure:
type: boolean
default: false
tosca.capabilities.Endpoint.Admin:
derived_from: tosca.capabilities.Endpoint
secure:
type: boolean
default: true
constraints:
- equal: true

Capabilities
node_types:
tosca.nodes.WebServer:
capabilities:
data_endpoint: tosca.capabilities.Endpoint
admin_endpoint: tosca.capabilities.Endpoint.Admin
host: # long form
valid_source_types: [ tosca.nodes.WebApplication ]
topology_template:
node_templates:
streaming_server:
type: tosca.nodes.WebServer
capabilities:
data_endpoint:
properties:
protocol: udp
port: 7500

Declaring Requirements
1. Much more complicated than capabilities
2. Each declared once in node type
Use “occurrences” range to limit the number of times it can be specified in node templates
If not specified, means 1 occurrence is required
The spec does not say what happens if you declare the same requirement name more than once
ARIA will only use the first
3. Must declare target capability type
4. Optionally declare target node type

Declaring Requirements
node_types:
tosca.nodes.Root:
requirements:
- dependency:
capability: tosca.capabilities.Node
node: tosca.nodes.Root
relationship: tosca.relationships.DependsOn
occurrences: [ 0, UNBOUNDED ]
Short form (implies 1 required occurrence):
ServerSoftware:
requirements:
- host: tosca.capabilities.Container
- os: tosca.capabilities.OperatingSystem

Specifying Requirements
1. Node template can specify each requirement zero, one, or more times
Depends on “occurrences” for the requirement as declared in the node type
If “occurrences” is not specified, the requirement will automatically be added once (ARIA’s
interpretation)
2. Can override required capability declared at node type
Must be descendent capability type
Or can be the name of the capability in a potential target node (see below)
3. Can override required node declared at node type
This time you can specify a node type or a node template

topology_template:
node_templates:
web_host:
type: openstack.Instance
capabilities:
installed_os: # tosca.capabilities.OperatingSystem
properties:
type: linux
registered_web_server:
type: ServerSoftware
requirements:
- host: openstack.Instance # node type
- os: openstack.Instance
anonymous_web_server:
requirements:
- host: web_host # node template
- dependency: registered_web_server
# “os” is implied by node type

admin_web_server:
requirements:
- dependency: anonymous_web_server
- host:
capability: openstack.SoftwareContainer # type: derives from tosca.capabilities.Container
- os:
capability: installed_os # name
node: openstack.Instance
node_filter:
properties:
- mem_size: { greater_or_equal: 4 GB }
- num_cpus: { in_range: [ 2, 4 ] }
capabilities:
installed_os:
properties:
- type: { equal: linux }

admin_web_server:
node_filter:
properties:
capabilities:
installed_os:
properties:
requirements:
- dependency: anonymous_web_server

Capability Occurrences
1. Demand a minimum number of sources (ARIA will let you know)
2. Set the maximum number of source requirements that we can satisfy
If a node reaches the maximum, perhaps a new node can be instantiated?
The spec is vague on this: it’s up to the orchestrator
node_types:
Router:
capabilities:
uplink:
valid_source_types: [ Gateway ]
occurrences: [ 2, UNBOUNDED ] # at least 2
No way to specify 2 and only 2 because in a range, max > min

Declaring Relationships
relationship_types:
tosca.relationships.ConnectsTo:
derived_from: tosca.relationships.Root
valid_target_types: [ tosca.capabilities.Endpoint ]
properties:
credential:
type: tosca.datatypes.Credential
required: false
Optionally declare relationship type in requirement
node_types:
Monitor:
requirements:
- link:
capability: tosca.capabilities.Endpoint
relationship: tosca.relationships.ConnectsTo

Specifying Relationships
1. Can override the requirement’s relationship declared at the node type
This time you can specify a relationship type or a relationship template
Note: this is the only place in TOSCA where you can refer to a relationship template!
2. Assign relationship properties and interface inputs
node_templates:
site_up_monitor:
type: Monitor
requirements:
- link:
relationship:
type: tosca.relationships.ConnectsTo
properties:
credential:
user: admin
token: password123

Relationship Templates
relationship_templates:
admin_connector:
properties:
credential:
user: admin
token: password123
node_templates:
site_up_monitor:
type: Monitor
requirements:
- link:
relationship: admin_connector

Interfaces in Relationships
interface_types:
tosca.interfaces.relationship.Configure:
derived_from: tosca.interfaces.Root
pre_configure_source: {}
pre_configure_target: {}
post_configure_source: {}
post_configure_target: {}
add_target: {}
add_source: {}
target_changed: {}
remove_target: {}

Interfaces in Relationship Templates
Can override or define extra interfaces beyond those in the relationship type
1. Overrides follow the same rules as in type inheritance
2. Extra interfaces? Why so complicated, TOSCA? This feature seems superfluous
3. We recommend that instead you derive a new relationship type
relationship_templates:
admin_connector:
properties:
credential:
user: admin
token: password123
interfaces:
Stateful:
type: lifecycle.Stateful
reset: # override
implementation: reset_admin_connection.py

Satisfying Requirements
A summary of how ARIA satisfies requirements (and builds a relationship graph):
1. Instantiate nodes with their requirements and capabilities
Derive from type hierarchies, apply template assignments, call functions
2. Iterate all requirements of all node instances
3. Find possible target nodes (by node type or node template)
4. Optionally apply node filter constraints
On properties of the node and/or its capabilities
5. Within the nodes, find the applicable target capability (by type or name)

Substitution Mappings
1. A complete service template can be encapsulated as a single node type
So clients can declare more than one node template of that type
2. Composing several services together via nesting
3. The orchestrator will have to know what to do
Instantiating this node type would involve deploying the entire service
The TOSCA spec says very little here
4. You probably want the node type to be defined in its own YAML file so that both
“server” and “client” blueprints can import it

Substitution Mappings: Type
node_types:
VirtualRouter:
derived_from: VNF
properties:
master:
type: boolean
attributes:
network_name:
type: string
capabilities:
virtual_link: tosca.capabilities.Endpoint
requirements:
- backend_lan:
capability: tosca.capabilities.network.Bindable
relationship: tosca.relationships.network.BindsTo

Substitution Mappings: Server
topology_template:
outputs:
network_name:
type: string
substitution_mappings:
node_type: VirtualRouter
capabilities:
virtual_link: [ loadbalancer, uplink ]
admin_endpoint: [ admin_server, endpoint ]
requirements:
backend_lan: [ analytics_collector, network ]

Substitution Mappings: Client
topology_template:
node_templates:
primary_router:
type: VirtualRouter
properties:
master: true
requirements:
- backend_lan: service_vpn
backup_router:
type: VirtualRouter
properties:
master: false
requirements:
- backend_lan: service_vpn

Groups
1. Join nodes together in order to:
Apply shared properties
Orchestrate together (operations)
Policies (we’ll get to that later)
2. Groups cannot be nested
A group contains nodes, not other groups
However, policies can be applied to several groups/nodes
3. Grouping at done at the node template level, not for node instances

Group Types
group_types:
openstack.Secured:
derived_from: tosca.groups.Root
members: # node types
- openstack.Instance
- docker.Container
properties:
security_scope:
type: string
constraints:
- valid_values: [ public, private ]
interfaces:
Standard:
create:
implementation: create_openstack_secured.py
inputs:
priority:
type: integer

Group Templates
topology_template:
groups:
vpn:
type: openstack.Secured
members: # node templates
- database_host
- application_host
- monitor_container
properties:
security_scope: private
interfaces:
Standard:
create:
inputs:
priority: 1

Policies
1. Metadata for the orchestrator
But no interfaces/operations for triggering them
2. Apply to nodes and/or groups, or generally
3. Policy type “targets” refer to node types and/or group types
For descendants, you can override the parent’s “targets”
4. Policy “targets” refer to node templates and/or groups
Must match related types in policy type “targets” or their descendants
If you don’t specify “targets”, then there are none

Policy Types
policy_types:
openstack.Scaling:
derived_from: tosca.policies.Scaling
properties:
bandwidth_threshold:
type: scalar-unit.size
default: 1 GB
targets: # node or group types
- openstack.Instance # node type
- openstack.Secured # group type

Policy Templates
topology_template:
policies:
handle_high_load:
type: openstack.Scaling
targets: # node templates or groups
- vpn # group
- load_balancer # node template
properties:
bandwidth_threshold: 1.5 GB

Data Types
1. Primitive types: “string”, “integer”, “boolean”, “null”
What is the point of “null”?
2. Scalar types: “scalar.size”, “scalar.time”, “scalar.frequency”
ARIA provides Python classes for these (comparison, sorting, etc.)
3. Special types: “timestamp”, “version”, “range”
ARIA provides Python classes for these
4. Types with entry schema: “list” and “map”
Entry schema can be any data type (primitive, special, complex)

Scalar Types
1. Float * unit -> normalized to base unit (integer or float)
2. The base unit is the default if not specified
3. Unit ignores uppercase/lowercase
type: scalar-unit.size -> integer of bytes
1.2 kb, 1.2 KiB, 12, 12 B, 0.5 B
type: scalar-unit.time -> float of seconds
0.01 s, 12.1, 60 m, 1.5 D
type: scalar-unit.frequency -> float of Hz
0.001, 12 kHz, 4.4 GHZ 12 Mhz

Timestamp
1. Date only: YYYY-MM-DD
Treated as UTC timezone
2. Date and time: YAML 1.1 (ISO8601 variant)
3. ARIA will normalize to YAML 1.1 canonical form
type: timestamp
2001-12-14 21:59:43.10 -5
1970-12-03
Canonical:
2001-12-15T02:59:43.1Z-05:00

Version
1. Integers only
2. At least major.minor
3. Optional: fix version, qualifier, and build version after dash
type: version
0.1
1.0
1.0.12.133-6
1.0.12.0-1
Invalid:
1.0.12-3

Range
1. List of two integers only
2. Max > min
3. Actually, max can be also be the string “UNBOUNDED” (but not min)
type: range
[ 0, 10 ]
[ -1, 1 ]
[ 10, UNBOUNDED ]
Invalid:
[ 5, 5 ], [ UNBOUNDED, 0 ]

Lists and Maps
1. Must specify entry_schema
2. Map keys are always strings
type: list
entry_schema: string
- the first string
- the second string
type: map
entry_schema: scalar-unit.size
big: 20 GB
medium: 3.5 GB
Small: 100 KB

Complex Data Types
data_types:
User:
properties:
name:
type: string
password:
type: string
default: 1234
home_directory:
type: string
required: false
quota:
type: scalar-unit.size
default: 1 MiB

Complex Data Types
data_types:
Configuration:
properties:
users:
type: map
entry_schema: User
password_change:
type: scalar-unit.time
default: 30 d
SystemConfiguration:
derived_from: Configuration
properties:
password_change: # override
type: scalar-unit.time
default: 1 d

Constraints
1. A mini-DSL inside TOSCA
Always logical “and” between all constraints
2. Applies to primitives, scalar, and special types
Including “properties” of complex types that can be any of these
3. Comparison: “equal”, “greater_than”, “greater_or_equal”, “less_than”,
“less_or_equal”, “in_range”, “valid_values”
Values to which we compare are validated against the property type
4. Regular expressions: “pattern”

Constraints in Data Types
Ad hoc:
data_types:
User:
properties:
name:
type: string
constraints:
- min_length: 2
- max_length: 18
You can inherit from a primitive to add constraints to it:
openstack.UUID:
derived_from: string
constraints:
- pattern: '^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$'

Constraints in Requirements
admin_web_server:
requirements:
- os:
capability: installed_os # name
node: openstack.Instance
node_filter:
properties:
capabilities:
installed_os:
properties:

Functions
1. Format: dict with single entry
2. Can be nested
3. Some depend on a deployment plan, with satisfied requirements
e.g. get_property: [ TARGET, flavor_id ]
4. Some are handled by the orchestrator
e.g. get_operation_output: [ TARGET, standard, create, ip_address ]
5. ARIA makes a best effort to keep calling functions until they work
But it’s up to you to make sure that you don’t cause dependency loops

Simple Functions
1. String manipulation:
concat: [ a, b, c … ] -> ‘abc’
token: [ ‘string|to.tokenize’, ‘,;.|’, 1 ] -> ‘to’
2. Template values:
get_input: <input>
get_property: [ <node template or relationship template>, … ]
3. Node properties:
get_property: [ SELF, … ]

Functions That Depend on a Service Instance
1. get_property: [ HOST | SOURCE | TARGET, … ]
2.get_nodes_of_type: <node type>

Packaging
CSAR (Cloud Service Archive)
Just a zip file, really: YAML plus artifacts
Currently not very well defined spec
Artifacts
Files that can be installed at nodes by the orchestrator
In the CSAR, or an online repository
Define them either at node type or at node template

Artifact Types
artifact_types:
tosca.artifacts.Implementation:
derived_from: tosca.artifacts.Root
tosca.artifacts.Implementation.Bash:
derived_from: tosca.artifacts.Implementation
mime_type: application/x-sh
file_ext: [ sh ]
mongodb.DatabaseDump:
derived_from: tosca.artifacts.Root
file_ext: [ json ]
properties:
index:
type: list
entry_schema: string

Artifacts at Node Type
node_types:
mongodb.Server:
artifacts:
installation_image:
type: TarBall
file: https://fastdl.mongodb.org/mongodb-x86_64-ubuntu1604-3.2.8.tgz
deploy_path: /opt/mongodb

Artifacts at Node Template
topology_template:
node_templates:
user_database:
type: mongodb.Database
properties:
name: users
artifacts:
initial:
type: mongodb.DatabaseDump
file: users.json
repository: service_desk

Putting It All Together
Let’s look at a complete service template

Thank You
Today’s presenter:
Tal Liron, GigaSpaces
tal@gigaspaces.com
For more information:
http://ariatosca.org/

TOSCA in Practice with ARIA

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à TOSCA in Practice with ARIA

Similaire à TOSCA in Practice with ARIA (20)

Plus de Cloudify Community

Plus de Cloudify Community (20)

Dernier

Dernier (20)

TOSCA in Practice with ARIA