5. Communication: Foundations Layered Protocols
Layered protocols
The OSI referencemodel 5 /49
When process P wants to communicate with process Q,
it first builds a message in its own address space. Then it
executes a system call that causes the operating system
to send the message over the network to Q.
Although this basic idea sounds simple enough, in order
to prevent chaos, P and Q have to agree on the meaning
of the bits being sent.
6. Communications in Distributed Systems
At a very high level communication can be of two types
Unstructured Communication –
• It is essentially sharing memory or sharing data structures.
• Data can be put in shared buffer, another process can come and pick it up.
• But this is not a popular way of message passing in commercial
systems.
Structured Communication –
• Most of the communication in distributed system is structured.
• These constitute of IPC's (Inter-process communications), RPC's or sending
explicit message over sockets.
• Processes on same machine can use either of the above two methods.
• However, in Distributed Systems processes are on different machines.
• To communicate across machine, networking is required to pass
messages.
7. Communication: Foundations Layered Protocols
Basic networking model- OSI Ref Model
Physical
Data link
Network
Transport
Session
Application
Presentation
Application protocol
Presentation protocol
Session protocol
Transport protocol
Network protocol
Data link protocol
Physical protocol
Network
1
2
3
4
5
7
6
Drawbacks
Focus on message-passing only
Often unneeded or unwanted functionality
Violates access transparency
The OSI referencemodel 7 /49
9. Communication: Foundations Layered Protocols
Low-level layers
Recap
Physical layer: contains the specification and implementation of bits, and
their transmission between sender and receiver
Data link layer: prescribes the transmission of a series of bits into a frame
to allow for error and flow control
Network layer: describes how packets in a network of computers are to be
routed.
Observation
For many distributed systems, the lowest-level interface is that of the network
layer.
The OSI referencemodel 9 /49
10. Communication: Foundations Layered Protocols
Transport Layer
Important
The transport layer provides the actual communication facilities for most
distributed systems.
Standard Internet protocols
TCP: connection-oriented, reliable, stream-oriented communication
UDP: unreliable (best-effort) datagram communication
The OSI referencemodel 10 /
Many other protocols are in use like
Real time Transport protocol
Streaming Control Transmission Protocol etc.
11. Middleware protocols
• The middleware protocol provides higher level service to application which
is even much more advanced than session and presentation layer.
• For example, Multicast, Broadcast, Unicast services are provided by
the middleware.
• Application just needs to send a packet for broadcast to the
middleware and then Middleware takes care of sending
the message to the network.
• Essentially middleware abstracts the Application from the TCP / IP
and other networking aspects of the distributed system.
12. Client server TCP connection- 3 way handshaking
• In a typical client server architecture, client constructs a request and sends it
over to the server, server processes the request at its end and sends back the
response to the client.
• For a client to send a message to the server, the client has to
establish a TCP connection to the server first, i.e. a 3-way handshake
is required.
• The client sends a SYN packet to the server, which means that the
client wants to establish a TCP connection with the server.
• Server in turn sends back a SYN and a ACK(SYN) packet back to the
client.
• ACK(SYN) is the acknowledgement sent from the server to the client.
• The client then acknowledges the SYN request of server by sending a
ACK message back to the server.
• Thus this is a 3-way handshake mechanism to establish a TCP connection.
14. Push or Pull Architecture
Client - Pull Architecture:
• The client sends in request to the server to pull data from it.
• For example, HTTP is a client pull architecture protocol where a
client sends a webpage request to get a reply from the web server.
Pros:
• In this type of architecture the servers are stateless , i.e. servers don't need
to keep track of the clients its communicating with.
• From a failure prospective this architecture is more robust since no state of
clients is maintained, thus if the server crashes, the client just needs to send a
new request for the data.
Cons:
• This architecture is more scalable from memory point of view since states of
client need not be maintained on the server. But this architecture is not
efficient from communications point of view.
• To get one piece of data two messages are exchanged i.e. a request and a
reply, which results in a overhead.
15. Push or Pull Architecture
Server - Push Architecture:
• In this architecture, the client may not send explicit requests for data, server
asynchronously pushes data to the client.
• For example, in Video and Audio Streaming, the client just sends a one time
request for the data on the server, after that the server starts pushing data.
The client just listens to the socket and data keeps coming in from the
server.
Cons:
• In this type of architecture the servers are stateful, i.e. the servers keep
tracks of the clients its communicating with.
• This architecture is not robust from failure point of view. If the server
reboots, then all the state of clients is lost from memory
Pros:
• This architecture is less scalable from memory point of view since states of
clients needs to be maintained in the servers memory. But from
communication point of view this architecture is more efficient,
• since client just sends in one request and server starts pushing data
to the client which has less overhead than client - pull architecture.
16. Communication: Foundations Layered Protocols
Middleware layer
Observation
Middleware is invented to provide common services and protocols that can be
used by many different applications
A rich set of communication protocols
(Un)marshaling of data, necessary for integrated systems
Naming protocols, to allow easy sharing of resources
Security protocols for secure communication
Scaling mechanisms, such as for replication and caching
Note
What remains are truly application-specific protocols...such as?
Middleware protocols 16 /
DNS
18. Communication: Foundations Types of Communication
Types of communication
Viewing middleware as an distributed service in application-level communication
18 /
With persistent communication, a
message that has been submitted
for transmission is stored by the
communication middleware as
long as it takes to deliver it to the
receiver.
In this case, the middleware will
store the message at one or
several of the storage facilities
with transient communication, a
message is stored by the
communication system only as long
as the sending and receiving
application are executing.
More precisely, if the middleware
cannot deliver a message due to a
transmission interrupt, or because
the recipient is currently not active,
it will simply be discarded.
19. Communication: Foundations Types of Communication
Types of communication
Viewing middleware as an distributed service in application-level communication
19 /
The characteristic feature of
asynchronous communication
is that a sender continues
immediately after it has
submitted its message for
transmission. This means that
the message is (temporarily)
stored immediately by the
middleware upon submission.
With synchronous communication, the
sender is blocked until its request is
known to be accepted.
21. Communication: Foundations Types of Communication
Messaging
Message-oriented middleware
Aims at high-level persistent asynchronous communication:
Processes send each other messages, which are queued
Sender need not wait for immediate reply, but can do other things
Middleware often ensures fault tolerance
11 /49
22.
23. Remote Procedure Calls(RPCs)
• The goal of Remote Procedure Calls(RPCs) is to make distributed
computing look like centralized computing.
• It allows remote services to be called as procedures. Rather than
sending abstract messages between a client and a server and letting
the server process the message and send back the response,
• the idea was to directly let the client call functions on the
server.
• This makes things easier for the programmer as now he/she just have
to call a remote function on the server instead of sending explicit
message to the server.
24. Communication: Remote procedure call Basic RPC operation
Basic RPC operation
Observations
Application developers are familiar with simple procedure model
Well-engineered procedures operate in isolation (black box)
There is no fundamental reason not to execute procedures on separate
machine
Conclusion
Communication between caller & callee
can be hidden by using procedure-call
mechanism.
Call local procedure
and return results
Call remote
procedure
Return
fromcall
Client
Request Reply
Server
Time
Wait for result
12 /49
25. Communication: Remote procedure call Basic RPC operation
Basic RPC operation
Implementation
of doit
Client OS Server OS
Client machine Server machine
Client process Server process
1. Client callto
procedure
2. Stub builds
message
5. Stub unpacks
message
6. Stub makes
local call to“doit”
3. Message is sent
across the network
4. Server OS
hands message
to server stub
Server stub
Client stub
r = doit(a,b) r = doit(a,b)
proc: “doit”
type1: val(a)
type2: val(b)
proc:“doit”
type1: val(a)
type2: val(b)
proc: “doit”
type1: val(a)
type2: val(b)
1
2
3
4
5
Client procedure calls client stub.
Stub builds message; calls local OS.
OS sends message to remote OS.
Remote OS gives message to stub.
Stub unpacks parameters; calls
server.
Server does local call; returns result to stub.
Stub builds message; calls OS.
OS sends message to client’s OS.
6
7
8
9 Client’s OS gives message to stub.
10 Client stub unpacks result; returns to client.
13 /49
The steps involved in calling a RPC doit(a,b). The return path for the result is not shown.
Local Procedure call
newlist = append(data, dbList)
26. Client and server stubs
• The system generates stubs,
• these are essentially proxies at both ends that system generates for
the application that deals with setting up a network connection to
the server, constructing a message whenever a remote call is made,
sending the message, wait for reply, all of this is taken care by these
stubs.
• All of this is abstracted from the programmer with the help of stubs.
27. Communication: Remote procedure call Parameter passing
RPC: Parameter passing
There’s more than just wrapping parameters into a message
Client and server machines may have different data representations (think
of byte ordering)
Wrapping a parameter means transforming a value into a sequence of
bytes
Client and server have to agree on the same encoding:
How are basic data values represented (integers, floats, characters)
How are complex data values represented (arrays, unions)
Conclusion
The solution to this problem is to transform data that is to be sent to a machine- and
network-independent format, next to making sure that both communicating parties
expect the same message data type to be transmitted.
14 /49
Packing parameters into a message is called parameter marshaling
28. RPC- Application Based Support
Creation of stubs-
(a piece of code that converts parameters passed between client and server during a
remote procedure call (RPC))
after agreement on representation of simple data structures, an other
agreement on protocols and rules of communications is made. And now it
is implemented.
Fortunately, stubs for the same protocol but different
procedures normally differ only in their interface to the applications
Independent of specific programming language
29. RPC- Language based Support
We can also embed remote procedure calling into a language itself. The
main benefit is that application development often becomes much simpler.
Also, reaching a high degree of access transparency is often simpler as
many issues related to parameter passing can be circumvented altogether.
A well-known example in which remote procedure calling is
fully embedded is Java, where an RPC is referred to as a
remote method invocation (RMI).
In essence, a client being executed by its own (Java) virtual
machine can invoke a method of an object managed by
another virtual machine.
By simply reading an application’s source code, it may be hard
or even impossible to see whether a method invocation is to a
local or to a remote object.
31. Communication: Remote procedure call Variations on RPC
Asynchronous RPCs
Essence
Try to get rid of the strict request-reply behavior, but let the client continue
without waiting for an answer from the server.
AsynchronousRPC 16 /49
32. Communication: Remote procedure call Variations on RPC
Asynchronous RPCs
Problem
The problem with this approach is that when reliability is not guaranteed,
the client cannot know for sure whether or not its request will be processed.
Call local procedure
Call remote
procedure
Return
fromcall
Client
Request
Accept
request
Server Time
Wait for
acceptance
Callback to client
Return
results
AsynchronousRPC 16 /49
33. Communication: Remote procedure call Variations on RPC
Sending out multiple RPCs
Essence
Sending an RPC request to a group of servers.
Call local procedure
Call local procedure
Client
Call remote
procedures
Server
Server
Time
Callbacks to client
MulticastRPC 17 /49
Multicast RPC
What are the major points to ponder in
Multicast RPCs?
34. Binder: Port Mapper
• When the client and the server startup, and the client invokes a
message at the server, it needs to first find the server.
• This is done through Port Mapper, when the server starts up it
registers itself at the port mapper with the list of API's that it
exposes and the port that it is listening on.
• On start up, client first checks with the port mapper for the
server with the remote procedure that the client wants to
execute.
• From the port mapper it gets the server's port number, this is a
one time setup.
• Once this is done, the stub takes over and performs required
operations.
• Also it is assumed that the IP address of the serveris known but
which port it is listening on is unknown, which is then found
through port mapper.
35. Communication: Remote procedure call Example: DCE RPC
Client-to-server binding (DCE)
Issues (1) Client must locate server machine, and (2) locate the server.
Port table
Server
DCE
daemon
Client
1. Register port
4. Ask forport
5. Do RPC
Directory
server
2. Register service
Server machine
3. Look up server
Client machine
Directory machine
Binding a client to a server 19 /49
• When the client and the server startup, and the client invokes a message at the
server, it needs to first find the server.
• This is done through Port Mapper, when the server starts up it registers itself at the
port mapper with the list of API's that it exposes and the port that it is listening on.
• Therefore when the client starts up, it first checks with the port mapper for the server
with the remote procedure that the client wants to execute. From the port mapper it
gets the server's port number, this is a one time setup.
• Once this is done, the stub takes over and performs required operations.
37. Steps of a Remote Procedure Call
• The client calls a local procedure, called the client stub. To the client process,
this appears to be the actual procedure, because it is a regular local procedure.
• Network messages are sent by the client stub to the remote system (via a
system call to the local kernel using sockets interfaces).
• Network messages are transferred by the kernel to the remote system via
some protocol (either connectionless or connection-oriented).
• A server stub, sometimes called the skeleton, receives the messages on the
server. It unmarshals the arguments from the messages and, if necessary,
converts them from a standard network format into a machine-specic form.
• The server stub calls the server function (which, to the client, is the remote
procedure), passing it the arguments that it received from the client.
• When the server function is finished, it returns to the server stub with its return
values.
• The server stub converts the return values, if necessary, and marshals them
into one or more network messages to send to the client stub.
• Messages get sent back across the network to the client stub.
• The client stub reads the messages from the local kernel.
• The client stub then returns the results to the client function, converting them
from the network representation to a local one if necessary.
38. Marshelling
• Marshalling is a data presentation conversion, performed according to
special rules, usually for network transfer. The following data presentation
factors have to be took into account to perform marshalling.
• Different platforms use their own data formats. Byte order is also dependent on
processor (Big Endian or Little Endian). Standard representation of data across the
machines can resolve this issue. For example, External data representation (XDR)
protocol is useful for transferring data between different computer architectures
and has been used to communicate data between very diverse machines.
• Passing pointers between different machines is a problem, different machines
have different memory allocated to their variables. If the passed pointer tries to
deference it variable then it may get garbage value. We can overcome this
problem if the pointer points to a well-defined data structure, then pass a copy
and the server stub passes a pointer to the local copy.
• Another problem in marshalling is how to deal with pointers. Possible solutions
are either prohibit the use of pointers or convert the local client pointer into a
network pointer, also known as generalization of pointer which allow us to
deference the pointer over the network.
39. Binding
• An issue that we have glossed over so far is how the client locates the
server.
• One method is just to hardwire the network address of the server
into the client. The trouble with this approach is that it is extremely
inflexible.
• If the server moves or if the server is replicated or if the
interface changes, numerous programs will have to be
found and recompiled
To avoid all these problems, some distributed systems use what is called
dynamic binding to match up clients and servers.
• We use directory service where server register itself at start-up
WITH the directory service (in Unix also known as binder or RPC port
mapper) and the server tells the name of the server, the version
number, and a list of procedures provided by the server (read, write,
create, and delete).
• when client calls one of the remote procedures for the first time.
The client stub sees that it is not yet bound to a server, than find out
where the server is by contacting the directory service, the directory
service gives its handle and unique identifier to the client stub.
40. Binding: Comments
• The extra overhead of exporting and importing interfaces costs time.
• Since many client processes are short lived and each process has to
start all over again, the effect may be significant.
• In a large DS, the binder may become a bottleneck, multiple binders
are needed whenever an interface is registered or deregistered, a
substantial number of messages will be needed to keep all the binders
synchronized and up to date, creating even more overhead.
41.
42. Message-oriented communication
Remote procedure calls and remote object invocations contribute to hiding
communication in distributed systems, that is, they enhance access
transparency.
Unfortunately, neither mechanism is always appropriate.
In particular, when it cannot be assumed that the receiving side is
executing at the time a request is issued, alternative communication
services are needed.
The inherent synchronous nature of RPCs, by which a client is blocked
until its request has been processed, may need to be replaced by
something else.
43. Simple transient messaging with sockets
• Many distributed systems and applications are built directly on top of the
simple message-oriented model offered by the transport layer.
• Conceptually, a socket is a communication end point to which an
application can write data that are to be sent out over the underlying
network, and from which incoming data can be read. A socket forms an
abstraction over the actual port that is used by the local operating system
for a specific transport protocol.
44. Communication: Message-oriented communication Simple transient messaging with sockets
Message oriented communication
Transient messaging: sockets using TCP/IP
Operation Description
socket
bind
listen
accept
connect
send
receive
close
Create a new communication end point
Attach a local address to a socket
Tell operating system what the maximum number of pending
connection requests should be
Block caller until a connection request arrives
Actively attempt to establish a connection
Send some data over the connection
Receive some data over the connection
Release the connection
socket connect
receive
receive
send
send
accept close
close
Server
socket
Client
bind listen
Synchronization point Communication
20 /49
Connection-oriented communication pattern using sockets.
Simple transient messaging with sockets
45. Communication: Message-oriented communication Simple transient messaging with sockets
Sockets: Python code
Server
1 from socket import*
2 s = socket(AF_INET,SOCK_STREAM)
3 s.bind((HOST,PORT))
4 s.listen(1)
5 (conn, addr) = s.accept() #returns new socket and addr.client
6 while True: #forever
data = conn.recv(1024) #receivedatafromclient
7
8 ifnot data: break #stopifclientstopped
9 conn.send(str(data)+"*") #returnsentdataplusan" *"
10 conn.close() #closetheconnection
Client
1 from socket import*
2 s = socket(AF_INET,SOCK_STREAM)
3 s.connect((HOST, PORT))#connecttoserver(blockuntilaccepted)
4 s.send(’Hello,world’) #sendsamedata
5 data = s.recv(1024)
6 print data
7 s.close()
#receivetheresponse
#printtheresult
#closetheconnection
21 /49
46. ZeroMQ
One approach toward making network programming easier is based on the
observation that many messaging applications, or their components, can be
effectively organized according to a few simple communication patterns.
By subsequently providing enhancements to sockets for each of these
patterns, it may become easier to develop a networked, distributed
application.
This approach has been followed in ZeroMQ
Actual message transmission generally takes place over TCP connections
Communication is asynchronous: a sender will normally continue after having
submitted a message to the underlying communication subsystem.
47. Communication: Message-oriented communication Advanced transient messaging
Making sockets easier to work with
Observation
Sockets are rather low level and programming mistakes are easily made.
However, the way that they are used is often the same (such as in a client-
server setting).
Alternative: ZeroMQ
Provides a higher level of expression bypairingsockets: one for sending
messages at process P and a corresponding one at process Q for receiving
messages. All communication is asynchronous.
Three patterns
Request-reply ---
Publish-subscribe
Pipeline
Using messaging patterns: ZeroMQ 22 /49
48. Request Reply
• The request-reply pattern is used in traditional client-server
communication, like the ones normally used for remote procedure
calls.
• A client application uses a request socket (of type REQ) to send a
request message to a server and expects the latter to respond
with an appropriate response.
• The server is assumed to use a reply socket (of type REP). The
request-reply pattern simplifies matters for developers by
avoiding the need to call the listen operation, as well as the
accept operation.
• ZeroMQ assumes the client is waiting for a response from the
original recipient.
51. Publish-subscribe
In the case of a publish-subscribe pattern, clients subscribe to
specific messages that are published by servers (event-based
coordination).
• In effect, only the messages to which the client has subscribed will be
transmitted. If a server is publishing messages to which no one has
subscribed, these messages will be lost.
• In its simplest form, this pattern establishes multicasting messages from
a server to several clients.
• The server is assumed to use a socket of type PUB, while each client
must use SUB type sockets.
• Each client socket is connected to the socket of the server. By default, a
client subscribes to no specific message.
• This means that as long as no explicit subscription is provided, a client
will not receive a message published by the server.
52. Communication: Message-oriented communication Advanced transient messaging
Publish-subscribe
Server
#createapublishersocket
#howandwheretocommunicate
#bindsockettotheaddress
1 import zmq, time
2
3 context = zmq.Context()
4 s = context.socket(zmq.PUB)
5 p ="tcp://"+ HOST +":"+ PORT
6 s.bind(p)
7 whileTrue:
8 time.sleep(5)
9 s.send("TIME"+ time.asctime())
#waitevery5seconds
#publishthecurrenttime
Client
#createasubscribersocket
#howandwheretocommunicate
#connecttotheserver
#subscribetoTIMEmessages
1 importzmq
2
3 context = zmq.Context()
4 s = context.socket(zmq.SUB)
5 p ="tcp://"+ HOST +":"+ PORT
6 s.connect(p)
7 s.setsockopt(zmq.SUBSCRIBE,"TIME")
8
9 for i inrange (5): #Fiveiterations
time = s.recv() #receiveamessage
10
11 print time
Using messaging patterns: ZeroMQ 25 /49
53. Pipeline
• The pipeline pattern is characterized by the fact that a process
wants to push out its results, assuming that there are other
processes that want to pull in those results.
• The essence of the pipeline pattern is that a pushing process does
not really care which other process pulls in its results:
• the first available one will do just fine. Likewise, any
process pulling in results from multiple other processes
will do so from the first pushing process making its
results available.
• The intention of the pipeline pattern is thus seen to keep
as many processes working as possible, pushing results
through a pipeline of processes as quickly as possible.
54. Communication: Message-oriented communication Advanced transient messaging
Pipeline
Source
#createapushsocket
#checktasksourcehost
#checktasksourceport
#howandwheretoconnect
#bindsockettoaddress
1 import zmq, time, pickle, sys, random
2
3 context = zmq.Context()
4 me = str(sys.argv[1])
5 s = context.socket(zmq.PUSH)
6 src = SRC1 if me ==’1’ else SRC2
7 prt = PORT1 if me ==’1’ else PORT2
8 p ="tcp://"+ src +":"+ prt
9 s.bind(p)
10
11 for i inrange(100):
12 workload = random.randint(1, 100)
#generate100workloads
#computeworkload
13 s.send(pickle.dumps((me,workload))) #sendworkloadtoworker
Using messaging patterns: ZeroMQ 26 /49
56. MPI ( Message passing Interface)
Sockets are designed for simple send and receive primitives and one-to-
one communication using general purpose protocol stacks such as
TCP/IP.
What about parallel and intensive communications???
MPI is designed for parallel applications
MPI can be used for communication between clusters of clients and
servers,
which have intensive communication, and the overhead of
TCP/IP is very high for this scenario.
MPI also has more advanced primitives of different forms
of buffering and synchronization.
57. Communication: Message-oriented communication Advanced transient messaging
MPI: When lots of flexibility is needed
Representative operations
Operation Description
MPI bsend Append outgoing message to a local send buffer
MPI send Send a message and wait until copied to local or
remote buffer
MPI ssend Send a message and wait until transmission starts
MPI sendrecv Send a message and wait for reply
MPI isend Pass reference to outgoing message, and continue
MPI issend Pass reference to outgoing message, and wait until
receipt starts
MPI recv Receive a message; block if there is none
MPI irecv Check if there is an incoming message, but do not
block
The Message-Passing Interface (MPI) 28 /49
59. Message oriented persistent communications
• Important class of message-oriented middleware services, generally known as
message-queuing systems, or just Message-Oriented Middleware (MOM).
• Message-queuing systems provide extensive support for persistent
asynchronous communication.
• The essence of these systems is that they offer intermediate-
term storage capacity for messages, without requiring either the
sender or receiver to be active during message transmission.
• An important difference with sockets and MPI is that message-queuing systems
are typically targeted to support message transfers that are allowed to take
minutes instead of seconds or milliseconds.
With persistent communication, a message that has been submitted for transmission is
stored by the communication middleware as long as it takes to deliver it to the receiver.
In this case, the middleware will store the message at one or several of the storage facilities
and continue its work.
60. Communication: Message-oriented communication Message-oriented persistent communication
Message-oriented middleware
Essence
Asynchronous persistent communication through support of middleware-level
queues. Queues correspond to buffers at communication servers.
Operations- Basic interface to a queue in a message-queuing system
Operation Description
put Append a message to a specified queue
get Block until the specified queue is nonempty, and
remove the first message
poll Check a specified queue for messages, and remove
the first. Never block
notify Install a handler to be called when a message is put
into the specified queue
Message-queuing model 29 /49
61. Communication: Message-oriented communication Message-oriented persistent communication
General model
Queue managers
Queues are managed byqueue managers. An application can put messages
only into alocalqueue. Getting a message is possible by extracting it from a
localqueue only ⇒ queue managers need toroutemessages.
Routing
Local OS
Source queue
manager
Contact
address
Destination queue
manager
Address lookup
database
Look up
contact address
of destination
queue manager
Logical
queue-level
address (name)
Local OS
Network
General architecture of a message-queuing system 30 /49
62. Communication: Message-oriented communication Message-oriented persistent communication
Message broker
Observation
Message queuing systems assume a common messaging protocol: all
applications agree on message format (i.e., structure and data representation)
Broker handles application heterogeneity in an MQ system
Transforms incoming messages to target format
Very often acts as anapplication gateway
May providesubject-basedrouting capabilities (i.e.,publish-subscribe
capabilities)
Message brokers 31 /49
63. Communication: Message-oriented communication Message-oriented persistent communication
Message broker: general architecture
Local OS
Application
Interface
Local OS Local OS
Application
Interface
Broker plugins Rules
Queuing
layer
Source Destination
Message broker
Message brokers 32 /49
The general organization of a message broker in a message-queuing system
64. Advanced Message Queuing protocol- AMQP
AMQP evolves around applications, queue managers, and queues.
Taking an approach that is common for many networking situations, we
make a distinction between AMQP as a messaging service, the actual
messaging protocol, and, finally, the messaging interface as offered to
applications.
An overview of a single-server AMQP instance.
65. Communication: Multicast communication Application-level tree-based multicasting
Application-level multicasting
Essence
Organize nodes of a distributed system into anoverlay networkand use that
network to disseminate data:
Oftentimes atree, leading to unique paths
Alternatively, alsomesh networks, requiring a form ofrouting
37 /49
66. Communication: Multicast communication Application-level tree-based multicasting
Application-level multicasting in Chord
Basic approach
1
2
3
4
5
Initiator generates amulticast identifier mid .
Lookup succ(mid), the node responsible for mid.
Request is routed to succ(mid ), which will becometheroot.
If P wants to join, it sends ajoinrequest to the root.
When request arrives at Q:
Q has not seen a join request before ⇒ it becomesforwarder; P
becomes child of Q.Join request continues to be forwarded.
Q knows about tree ⇒ P becomes child of Q.No need to forward
join request anymore.
38 /49
67. Communication: Multicast communication Application-level tree-based multicasting
ALM: Some costs
Different metrics
Ra Rc
Re
D
Internet
Router
B
Overlay network
7
5
1 C
1
End host
A 1
1 Rb
1
30 20
40
E
Rd
Link stress: How often does an ALM message cross the same physical
link?Example:message from A to D needs to cross (Ra, Rb) twice.
Stretch: Ratio in delay between ALM-level path and network-level path.
Example:messages B to C follow path of length 73 at ALM, but 47 at
network level ⇒ stretch = 73/47.
Performance issues in overlays 39 /49
68. Communication: Multicast communication Flooding-based multicasting
Flooding
Essence
P simply sends a message m to
each of its neighbors. Each
neighbor will forward that message,
except to P, and only if it had not
seen m before.
Performance
The more edges, the more
expensive!
The size of a random overlay as
function of the number of nodes
50
100
150
200
250
300
pedge = 0.6
pedge = 0.4
pedge = 0.2
0
100 1000
500
Number of nodes
Number
of
edges
(x
1000)
40 /49
69. Communication: Multicast communication Flooding-based multicasting
Flooding
Essence
P simply sends a message m to
each of its neighbors. Each
neighbor will forward that message,
except to P, and only if it had not
seen m before.
Performance
The more edges, the more
expensive!
The size of a random overlay as
function of the number of nodes
50
100
150
200
250
300
pedge = 0.6
pedge = 0.4
pedge = 0.2
0
100 1000
500
Number of nodes
Number
of
edges
(x
1000)
Variation
Let Q forward a message with a certain probability pflood , possibly even
dependent on its own number of neighbors (i.e.,node degree) or the degree of
its neighbors.
40 /49
70. Communication: Multicast communication Gossip-based data dissemination
Epidemic protocols
Assume there are no write–write conflicts
Update operations are performed at a single server
A replica passes updated state to only a few neighbors
Update propagation is lazy, i.e., not immediate
Eventually, each update should reach every replica
Two forms of epidemics
Anti-entropy: Each replica regularly chooses another replica at random,
and exchanges state differences, leading to identical states at both
afterwards
Rumor spreading: A replica which has just been updated (i.e., has been
contaminated), tells a number of other replicas about its update
(contaminating them as well).
41 /49
71. Communication: Multicast communication Gossip-based data dissemination
Anti-entropy
Principle operations
A node P selects another node Q from the system at random.
Pull: P only pulls in new updates from Q
Push: P only pushes its own updates to Q
Push-pull: P and Q send updates to each other
Observation
For push-pull it takes O(log(N)) rounds to disseminate updates to all N nodes
(round= when every node has taken the initiative to start an exchange).
Information dissemination models 42 /49
72. Communication: Multicast communication Gossip-based data dissemination
Rumor spreading
Basic model
A server S having an update to report, contacts other servers. If a server is
contacted to which the update has already propagated, S stops contacting
other servers with probability pstop.
Observation
If s is the fraction of ignorant servers (i.e., which are unaware of the update), it
can be shown that with many servers
s =e−(1/pstop+1)(1−s)
Information dissemination models 45 /49
Notes de l'éditeur
Additional transport protocols are regularly proposed. For example, to support
real-time data transfer, the Real-time Transport Protocol (RTP) has been
defined. RTP is a framework protocol in the sense that it specifies packet formats
for real-time data without providing the actual mechanisms for guaranteeing
data delivery. In addition, it specifies a protocol for monitoring and controlling
data transfer of RTP packets [Schulzrinne et al., 2003]. Likewise, the Streaming
Control Transmission Protocol (SCTP) has been proposed as an alternative to
TCP [Stewart, 2007]. The main difference between SCTP and TCP is that SCTP
groups data into messages, whereas TCP merely moves bytes between processes.
Doing so may simplify application development.
In computer science, marshalling is the process of transforming the memory representation of an object into a data format suitable for storage or transmission
A method stub or simply stub in software development is a piece of code used to stand in for some other programming functionality.
A stub may simulate the behavior of existing code (such as a procedure on a remote machine; such methods are often called mocks) or be a temporary substitute for yet-to-be-developed code.
With persistent communication, a message that has been submitted for transmission is stored by the communication middleware as long as it takes to deliver it to the receiver.
In this case, the middleware will store the message at one or several of the storage facilities
