A tutorial about the API for the description of a monitoring infrastructure currently discussed inside the OCCI working group. The slides start by giving the basic concepts, proceed with a description of the entities that implement the monitoring infrastructure, and conclude with a step by step definition of a non-trivial monitoring infrastructure.

1. OCCI Monitoring
Augusto Ciuffoletti
OCCI Monitoring
Extending the OCCI API with monitoring capabilities
Augusto Ciuffoletti
Dept. of Computer Science - Univ. of Pisa
September 13, 2013

2. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence

3. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring

4. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring
The user may be in its turn a service provider (inside
monitoring)

5. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring
The user may be in its turn a service provider (inside
monitoring)
The user may simply want to verify the quality of the
service (outside monitoring)

6. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring
The user may be in its turn a service provider (inside
monitoring)
The user may simply want to verify the quality of the
service (outside monitoring)
In both cases, the user needs the tools to define
resource monitoring

7. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring
The user may be in its turn a service provider (inside
monitoring)
The user may simply want to verify the quality of the
service (outside monitoring)
In both cases, the user needs the tools to define
resource monitoring
Keep into account the case of a composite service
(many providers)

8. OCCI Monitoring
Augusto Ciuffoletti
Motivation
SLA is a defined target to obtain user confidence
SLA is tightly related with monitoring, so we start from
cloud monitoring
The user may be in its turn a service provider (inside
monitoring)
The user may simply want to verify the quality of the
service (outside monitoring)
In both cases, the user needs the tools to define
resource monitoring
Keep into account the case of a composite service
(many providers)
A simple API aligned with OCCI

9. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
Monitoring is made of three basic activities

10. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
Monitoring is made of three basic activities
extract operational parameters from a Resource

11. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
Monitoring is made of three basic activities
extract operational parameters from a Resource
gather the operational parameters to obtain the
measure of a metric of interest

12. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
Monitoring is made of three basic activities
extract operational parameters from a Resource
gather the operational parameters to obtain the
measure of a metric of interest
deliver the measurement to the relevant party

13. OCCI Monitoring
Augusto Ciuffoletti
Aggregate and deliver
Distiguished activities that need the provision of
computing, storage, networking resources

14. OCCI Monitoring
Augusto Ciuffoletti
Aggregate and deliver
Distiguished activities that need the provision of
computing, storage, networking resources
Tightly integrated, under control of the provider

15. OCCI Monitoring
Augusto Ciuffoletti
Aggregate and deliver
Distiguished activities that need the provision of
computing, storage, networking resources
Tightly integrated, under control of the provider
Candidate for the introduction of a new kind of
resource: the Sensor

16. OCCI Monitoring
Augusto Ciuffoletti
Aggregate and deliver
Distiguished activities that need the provision of
computing, storage, networking resources
Tightly integrated, under control of the provider
Candidate for the introduction of a new kind of
resource: the Sensor
The user that wants to exert monitoring instantiates
(and pays for) a Sensor

17. OCCI Monitoring
Augusto Ciuffoletti
Aggregate and deliver
Distiguished activities that need the provision of
computing, storage, networking resources
Tightly integrated, under control of the provider
Candidate for the introduction of a new kind of
resource: the Sensor
The user that wants to exert monitoring instantiates
(and pays for) a Sensor
Note: a cost is associated to the Sensor, although this
is not explicit in the definition of monitoring

18. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time

19. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time
The native attributes of a Sensor are:

20. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time
The native attributes of a Sensor are:
How frequently it produces a new measurement

21. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time
The native attributes of a Sensor are:
How frequently it produces a new measurement
During which time lapse it performs the measurements

22. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time
The native attributes of a Sensor are:
How frequently it produces a new measurement
During which time lapse it performs the measurements
Too many variants for aggregation/delivery:
OCCI-mixins

23. OCCI Monitoring
Augusto Ciuffoletti
Basic functions
The monitoring function is controlled by time
The native attributes of a Sensor are:
How frequently it produces a new measurement
During which time lapse it performs the measurements
Too many variants for aggregation/delivery:
OCCI-mixins
Mixins are classified using tags (sort of subtyping)

24. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource

25. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)

26. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)
This is represented as a collector link from the sensor to
the resource

27. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)
This is represented as a collector link from the sensor to
the resource
The generic attributes of a collector correspond to the
sampling period:

28. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)
This is represented as a collector link from the sensor to
the resource
The generic attributes of a collector correspond to the
sampling period:
The sampling period

29. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)
This is represented as a collector link from the sensor to
the resource
The generic attributes of a collector correspond to the
sampling period:
The sampling period
The accuracy of the sampling period

30. OCCI Monitoring
Augusto Ciuffoletti
Extract measurements
This is an ability that a sensor has with respect to a real
resource
Capability of extracting measurements (e.g., through
resource instrumentation)
This is represented as a collector link from the sensor to
the resource
The generic attributes of a collector correspond to the
sampling period:
The sampling period
The accuracy of the sampling period
OCCI-mixins are introduced to specify the measurement
technique used

31. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label

32. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification

33. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification
By defining tagged mixins, the provider describes the
monitoring capabilities offered to the user

34. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification
By defining tagged mixins, the provider describes the
monitoring capabilities offered to the user
Next slides describe three such tags:

35. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification
By defining tagged mixins, the provider describes the
monitoring capabilities offered to the user
Next slides describe three such tags:
metric specific for collector links

36. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification
By defining tagged mixins, the provider describes the
monitoring capabilities offered to the user
Next slides describe three such tags:
metric specific for collector links
aggregator specific for sensor resources

37. OCCI Monitoring
Augusto Ciuffoletti
Mixin sub-typing: the tags
According with the OCCI core specification a mixin with
no attributes (a tag) can be associated to another mixin
as a sort of label
Syntactic and semantic restrictions can be introduced
for mixins with a certain label, thus supporting a
specification
By defining tagged mixins, the provider describes the
monitoring capabilities offered to the user
Next slides describe three such tags:
metric specific for collector links
aggregator specific for sensor resources
publisher specific for sensor resources

38. OCCI Monitoring
Augusto Ciuffoletti
The metric tag
The tag metric is associated with a measurement
technique, and is related with a collector link;

39. OCCI Monitoring
Augusto Ciuffoletti
The metric tag
The tag metric is associated with a measurement
technique, and is related with a collector link;
The mixin that has the metric tag has the following
attributes:

40. OCCI Monitoring
Augusto Ciuffoletti
The metric tag
The tag metric is associated with a measurement
technique, and is related with a collector link;
The mixin that has the metric tag has the following
attributes:
metric attribute strings that are used as an identifier to
refer to the output measurement stream

41. OCCI Monitoring
Augusto Ciuffoletti
The metric tag
The tag metric is associated with a measurement
technique, and is related with a collector link;
The mixin that has the metric tag has the following
attributes:
metric attribute strings that are used as an identifier to
refer to the output measurement stream
control attributes used to control the measurement
process (e.g. packet length of a ping)

42. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation

43. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation
an aggregator mixin is related with a sensor resource

44. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation
an aggregator mixin is related with a sensor resource
the mixin with the aggregator tag are characterized by:

45. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation
an aggregator mixin is related with a sensor resource
the mixin with the aggregator tag are characterized by:
input attributes strings that are used as an identifier to
refer to input measurement streams

46. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation
an aggregator mixin is related with a sensor resource
the mixin with the aggregator tag are characterized by:
input attributes strings that are used as an identifier to
refer to input measurement streams
control attributes that are parameters for the
aggregation function (e.g. the gain in an
EWMA)

47. OCCI Monitoring
Augusto Ciuffoletti
The aggregator tag
the tag aggregator is associated with a mixin that
implements an algorithm for monitoring data
aggregation
an aggregator mixin is related with a sensor resource
the mixin with the aggregator tag are characterized by:
input attributes strings that are used as an identifier to
refer to input measurement streams
control attributes that are parameters for the
aggregation function (e.g. the gain in an
EWMA)
metric attributes strings that are used as an identifier to
refer to the output measurement stream

48. OCCI Monitoring
Augusto Ciuffoletti
The publisher tag
the tag publisher is associated with a mixin that
implements a technique to deliver the data outside the
monitoring infrastructure

49. OCCI Monitoring
Augusto Ciuffoletti
The publisher tag
the tag publisher is associated with a mixin that
implements a technique to deliver the data outside the
monitoring infrastructure
a publisher mixin is related with a sensor resource

50. OCCI Monitoring
Augusto Ciuffoletti
The publisher tag
the tag publisher is associated with a mixin that
implements a technique to deliver the data outside the
monitoring infrastructure
a publisher mixin is related with a sensor resource
the mixin with a publisher tag are characterized by:

51. OCCI Monitoring
Augusto Ciuffoletti
The publisher tag
the tag publisher is associated with a mixin that
implements a technique to deliver the data outside the
monitoring infrastructure
a publisher mixin is related with a sensor resource
the mixin with a publisher tag are characterized by:
input attribute a string that is used as an identifier to
refer to input measurement stream

52. OCCI Monitoring
Augusto Ciuffoletti
The publisher tag
the tag publisher is associated with a mixin that
implements a technique to deliver the data outside the
monitoring infrastructure
a publisher mixin is related with a sensor resource
the mixin with a publisher tag are characterized by:
input attribute a string that is used as an identifier to
refer to input measurement stream
control attributes that are parameters for the publishing
function (e.g. the UDP port that is used
for streaming)

53. OCCI Monitoring
Augusto Ciuffoletti
The role of metric and input attributes
they describe the flow of monitoring data

54. OCCI Monitoring
Augusto Ciuffoletti
The role of metric and input attributes
they describe the flow of monitoring data
their values are unique identifiers in a scope
corresponding to a sensor and all outgoing collectors

55. OCCI Monitoring
Augusto Ciuffoletti
The role of metric and input attributes
they describe the flow of monitoring data
their values are unique identifiers in a scope
corresponding to a sensor and all outgoing collectors
the provider is free to implement the transport of the
measurement stream in the most convenient way

56. OCCI Monitoring
Augusto Ciuffoletti
Shortcuts for simple use cases
here metric mixin, can be associated directly to the
monitored resource and has a web service interface

57. OCCI Monitoring
Augusto Ciuffoletti
Shortcuts for simple use cases
here metric mixin, can be associated directly to the
monitored resource and has a web service interface
a tagged mixin can be associated with a generic
resource or link, avoiding the definition of a sensor or
collector

58. OCCI Monitoring
Augusto Ciuffoletti
Shortcuts for simple use cases
here metric mixin, can be associated directly to the
monitored resource and has a web service interface
a tagged mixin can be associated with a generic
resource or link, avoiding the definition of a sensor or
collector
shortcuts effectively help very simple use cases, but are
generally a source of inefficiency and complexity

59. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
The resources we want to monitor

60. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Create one Sensor resource

61. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Use two collectors to define the measurement activiy

62. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Associate two metric mixins to the Collector X

63. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
And another two metric mixins to the Collector Y

64. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Associate two aggregator mixins to the Sensor

65. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
One publisher is going to use raw data from the collector

66. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Another is going to receive measurements from the
aggregators

67. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
A frame for Collector X and its mixins

68. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
... one for Collector Y...

69. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
... one for the sensor

70. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
The scope of the Sensor (for metric streams ids)

71. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Feeding the aggregators: a,b,d are measurement stream ids

72. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Feeding publisher 2: aggregated (f,g) and raw data (e)

73. OCCI Monitoring
Augusto Ciuffoletti
Step by step design of a monitoring infrastructure
Feeding publisher 1: measurement stream b is multicast

74. OCCI Monitoring
Augusto Ciuffoletti
Also in the document
Conformance profiles: to accomodate the presence of
providers that do not implement a monitoring interface

75. OCCI Monitoring
Augusto Ciuffoletti
Also in the document
Conformance profiles: to accomodate the presence of
providers that do not implement a monitoring interface
Security issues

76. OCCI Monitoring
Augusto Ciuffoletti
Also in the document
Conformance profiles: to accomodate the presence of
providers that do not implement a monitoring interface
Security issues
A detailed example using the http rendering

77. OCCI Monitoring
Augusto Ciuffoletti
Also in the document
Conformance profiles: to accomodate the presence of
providers that do not implement a monitoring interface
Security issues
A detailed example using the http rendering
...and several bugs.

78. OCCI Monitoring
Augusto Ciuffoletti
Also in the document
Conformance profiles: to accomodate the presence of
providers that do not implement a monitoring interface
Security issues
A detailed example using the http rendering
...and several bugs.
That’s all...