PrismTech's Vortex is a platform that provides seamless, ubiquitous, efficient and timely data sharing across mobile, embedded, desktop, cloud and web applications. Today Vortex is the enabling technology at the core the most innovative Internet of Things and Industrial Internet applications, such as Smart Cities, Smart Grids, and Smart Traffic.
This two part tutorial presentation (1) introduces the key concepts of Vortex, (2) gets you started with using Vortex to efficiently exchange data across mobile, embedded, desktop, cloud and web applications, and (3) provides a series of best practices, patterns and idioms to get the best out of Vortex.
1. Angelo
Corsaro,
PhD
Chief
Technology
Officer
angelo.corsaro@prismtech.com
Vortex Tutorial
Part I
2. CopyrightPrismTech,2014
Vortex enables seamless,
ubiquitous, efficient and
timely data sharing across
mobile, embedded, desktop,
cloud and web applications
Vortex is based on the OMG
DDS standard
The Vortex Platform
Vortex Device
Tools
Integration
Vortex Cloud
MaaS
5. CopyrightPrismTech,2014
Support for transport level
security via TLS and DTLS
Support for fine grained
access control
Access control plug-in
compliant with DDS Security
Vortex Security Model
Arthur Dent
Arthur Dent
Ford Prerfect
Zaphod Beeblebrox
Marvin
Trillian
A(r,w), B(r)
A(r,w), B(r,w), X(r)
*(r,w)
*(r)
A(r,w), B(r,w), C(r,w)
Ford Prerfect
Zaphod Beeblebrox
Trillian
Marvin
A
B
A,B
X
*
*
A,B,C
Identity Access Rights
Sessions are authenticated
and communication is
encrypted
Only the Topic included as
part of the access rights are
visible and accessible
10. CopyrightPrismTech,2014
SESAR
PENS
PENS: Pan European Network Service
FDP: Flight Data Processor
CWP: Controller Working Position
IPv6 network SSM support
FDP
FDPFDP
CWP
Tower
Device
FDP
FDPFDP Tower
Device
CWP
FDP
FDPFDP Tower
Device
CWP
FDP
FDPFDP
CWP
Tower
Device
EU Nation 4
EU Nation 1
EU Nation 2
EU Nation 3
12. CopyrightPrismTech,2014
To explore the various features provided by the Vortex platform we will be designing and
implementing a micro-blogging platform called ChirpIt. Specifically, we want to support the
following features:
ChirpIt users should be able to “chirp”, “re-chirp”, “like” and “dislike” trills as well as get
detailed statistics
The ChirpIt platform should provide information on trending topics — identified by hashtags —
as well as trending users
Third party services should be able to flexibly access slices of produced trills to perform their
own trend analysis
ChirpIt should scale to millions of users
ChirpIt should be based on a Lambda Architecture
ChirpIt Requirements
15. CopyrightPrismTech,2014
DDS provides a Global Data Space
abstraction that allows applications
to autonomously, anonymously,
securely and efficiently share data
DDS’ Global Data Space is fully
distributed (decentralised), highly
efficient and scalable
Data Distribution Service (DDS)
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
16. CopyrightPrismTech,2014
DataWriters and DataReaders are
automatically and dynamically
matched by the DDS Discovery
A rich set of QoS allows to control
existential, temporal, and spatial
properties of data
Data Distribution Service (DDS)
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
17. CopyrightPrismTech,2014
Elegant and High Level Data Sharing Abstraction
Polyglot and platform independent
• Java, Scala, C, C++, C#, JavaScript, CoffeeScript etc.
• Android, Windows, Linux, VxWorks, etc.
Peer-to-Peer by nature, Brokered when useful
Key Highlights
18. CopyrightPrismTech,2014
Content and Temporal Filtering (both sender and receiver filtering supported)
Queries
20+ QoS to control existential, temporal, and spatial properties of data
Key Highlights
19. CopyrightPrismTech,2014
High Performance and Scalable
• ~30 usec latency*
• 7M msgs/sec node-to-node throughput*
Key Highlights
(*) Performance measured on Linux Host with i7 processor on a 1Gbps Ethernet network
22. CopyrightPrismTech,2014
DDS data lives within a domain
A domain is identified with a non
negative integer, such as 1, 3, 31
The number 0 identifies the default
domain
A domain represent an impassable
communication plane
Domain
DDS Domain
23. CopyrightPrismTech,2014
Partitions are the mechanism provided by DDS to organise information within a
domain
Access to partitions is controlled through QoS Policies
Partitions are defined as strings:
- “system:telemetry”
- “system:log”
- “data:row-2:col-3”
Partitions addressed by name or regular expressions:
- ”system:telemetry”
- “data:row-2:col-*”
Partitions
Partitions
25. CopyrightPrismTech,2014
A Topic defines a domain-wide information’s class
A Topic is defined by means of a (name, type, qos)
tuple, where
• name: identifies the topic within the domain
• type: is the programming language type
associated with the topic. Types are
extensible and evolvable
• qos: is a collection of policies that express
the non-functional properties of this topic,
e.g. reliability, persistence, etc.
Topic
Topic
Type
Name
QoS
...
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
26. CopyrightPrismTech,2014
As explained in the previous slide a topic defines a class/type of information
Topics can be defined as Singleton or can have multiple Instances
Topic Instances are identified by means of the topic key
A Topic Key is identified by a tuple of attributes -- like in databases
Remarks:
- A Singleton topic has a single domain-wide instance
- A “regular” Topic can have as many instances as the number of different key
values, e.g., if the key is an 8-bit character then the topic can have 256 different
instances
Topic and Instances
33. CopyrightPrismTech,2014
To make the system scale properly we need to ensure that the chirps, rechirps,
etc., are distributed only to the interested parties
As we have a single topic representing chirp actions, how can we ensure that
information is received only by followers?
DDS Provides two mechanisms:
- Partitions: Allow to create information scopes
- Content Filters: Allow to receive the data that satisfy certains predicates
Scoping Information
36. CopyrightPrismTech,2014
In summary partitions are used to scope information
Each user will “join” a partition for each followed party
Example:
- If @drx follows @magneto, @wolverine, @cyclops and @mistique then he will
receive ChirpActions from the partitions:
• chirp:@magneto, chirp:@wolverine, chirp:@cyclops and chirp:@mistique
- On the other hand, regardless of the people that follows @wolverine he will always
and only write ChirpActions in the partitions
• chirp:@wolverine
Using Partitions to Scope Chirps
38. CopyrightPrismTech,2014
A DataWriter (DW) is a strongly typed entity used
to produce samples for one or more instances of
a Topic, with a given QoS
Conceptually, the DataWriter QoS should be the
same as the Topic QoS or more stringent
However, DDS does enforce a specific relationship
between the Topic and DataWriter QoS
DataWriter
DW
Type
Topic
QoS
39. CopyrightPrismTech,2014
The DataWriter controls the life-cycle of Topic
Instances and allows to:
Define a new topic instance
Write samples for a topic instance
Dispose the topic instance
DataWriter
DW
Type
Topic
QoS
41. CopyrightPrismTech,2014
A DataReader (DR) is a strongly typed entity used to
access and/or consume samples for a Topic, with a
given QoS
Conceptually, the DataReader QoS should be the
same as the Topic QoS or less stringent
However, DDS does enforce a specific relationship
between the Topic and DataReader QoS
DataReader
DR
Type
Topic
QoS
43. CopyrightPrismTech,2014
DomainParticipant: Provides access to a data cloud -- called a domain in DDS
Topic: Domain-wide definition of a kind of Information
Publisher/Subscriber: Provide scope to data sharing through the concept of partitions
DataReader/DataWriter: Allow to read/write data for a given topic in the partitions their Subscriber/Publisher are associated with.
DDS Entities
Domain (e.g. Domain 123)
Domain
Participant
Topic
Publisher
DataWrter
Subscriber
DataReader
Partition (e.g. “Telemetry”, “Shapes”, )
Topic Instances/Samples
Ta
Tb
Tc
Tx
Ty
T1
T1 T3
45. CopyrightPrismTech,2014
ChirpOut App
//
Domain
Participant
defined
by
dependency
injection
//
Define
the
topic
val
topic
=
Topic[ChirpAction](Config.ChirpActionTopic)
//
Define
the
publisher
and
link-‐it
up
to
the
“chirp:@user”
partition
val
pubQos
=
PublisherQos()
.withPolicy(Partition(Config.ChirpRootPartition
+
user))
implicit
val
pub
=
Publisher(pubQos)
//
Define
the
data
writer
and
make
it
reliable
and
for
the
time
being
//
Transient
Local…
Will
refined
this
in
the
Part
II
val
dwQos
=
DataWriterQos().withPolicies(
Reliability.Reliable,
Durability.TransientLocal)
//
Create
the
DataWriter
val
dw
=
DataWriter[ChirpAction](pub,
topic,
dwQos)
//
Chirp!!!
val
chirp
=
…
dw.write(chirp)
46. CopyrightPrismTech,2014
ChirpIn
//
Domain
Participant
defined
by
dependency
injection
//
Define
the
topic
val
topic
=
Topic[ChirpAction](Config.ChirpActionTopic)
//
Define
the
subscribe
and
link-‐it
up
to
the
partition
associated
to
//
the
user
followed
val
subQos
=
PublisherQos()
.withPolicy(Partition(followerList.map(Config.ChirpRootPartition
+
_))
implicit
val
sub
=
Subscriber(subQos)
//
Define
the
data
reader
and
make
it
reliable
and
for
the
time
being
//
Transient
Local…
Will
refined
this
in
the
Part
II
val
drQos
=
DataReaderQos().withPolicies(
Reliability.Reliable,
Durability.TransientLocal)
//
Create
the
DataWriter
val
dr
=
DataReader[ChirpAction](sub,
topic,
drQos)
47. CopyrightPrismTech,2014
ChirpIn
//
React
to
new
Chirp
dr
listen
{
case
DataAvailable(_)
=>
{
//
take
vs.
read
will
become
clear
in
Part
II
val
samples
=
dr.take()
//
For
the
time
being
simply
print-‐out
the
new
chirps
samples
.filter(s
=>
(s.getData
!=
null))
.foreach(s
=>
displayChirp)
}
}
48. CopyrightPrismTech,2014
ChirpOut App
// Define the topic — assuming the topic is registered with the server…!
topic = new dds.Topic(0, 'Chirp', ‘com.ChirpIt’)!
// Define the data writer Qos!
dwQos = new dds.DataWriterQos(!
dds.Reliability.Reliable, !
dds.Durability.TranslientLocal,!
dds.Partition(userPartition))!
!
// Use an Option monad to maintain the writer!
chirpWriter = z_.None!
!
// Create the writer when the runtime is connected!
runtime.onconnect(() ->!
dw = new dds.DataWriter(runtime, topic, dwQos)!
chirpWriter = z_.Some(dw)!
)!
!
// …!
// Write chirps!
chirp = …!
chirpWriter.map((dw) -> dw.write(chirp))!
49. CopyrightPrismTech,2014
ChirpIn App
// Define the topic — assuming the topic is registered with the server…!
topic = new dds.Topic(0, 'Chirp', ‘com.ChirpIt’)!
// Define the data reader Qos!
drQos = new dds.DataWriterQos(!
dds.Reliability.Reliable, !
dds.Durability.TranslientLocal,!
dds.Partition(followedList))!
!
// Use an Option monad to maintain the reader!
chirpReader = z_.None!
!
// Create the writer when the runtime is connected!
runtime.onconnect(() ->!
dr = new dds.DataReader(runtime, topic, dwQos)!
// Attach a listener!
dr.addListener((chirp) -> diplayChirp(chirp))!
chirpReader = z_.Some(dr)!
)
52. CopyrightPrismTech,2014
$ ./chirpOut @wolverin
chirp>> @magneto will win!$ ./chirpOut @magneto
@magneto>> @wolverine chirping while asleep… again!
Download the package zip or tar.gz package available at:
- https://dl.dropboxusercontent.com/u/19238968/webcast/2014/10/chirpit.zip
- https://dl.dropboxusercontent.com/u/19238968/webcast/2014/10/chirpit.tar.gz
Extract the package somewhere in your file-system and open a terminal at that
location and run the app as shown below
ChirpIt Demo
$ ./chirpIn @magneto @wolverinse @mystique
@wolverin>> @magneto will win!
@magneto>> @wolverine chirping while asleep… again!
53. CopyrightPrismTech,2014
In Part II we extend our implementation to manage application configuration
storage and analytics
To do so, we will need to introduce additional features provided by the DDS and
the Vortex Platform, such as Durability and Integration with Big-Data Stores
What’s Next
54. CopyrightPrismTech,2014
In this presentation we have performed the first step toward implementing a
micro-blogging platform
The combination of Vortex Device and Vortex Cloud made it very easy to create
and deploy our Internet-Scale, multi-device micro-blogging platform
Some of the DDS QoS, such as Durability have already shown how historical
data, i.e. chirps made while we were offline — can be provided by the platform
The Part II will illustrate how the complete application fits together and what is
the role of Vortex in the Lambda Architecture
Summary