1. A System Management Framework
for Self-Optimizing P2P Systems
2nd Research Talk - 06/02/2008
Peers
Information
Management α
Architecture λ β
µ
httc –
Using Info. Analysis, Hessian Telemedia Technology
to Gain Modeling and Competence-Center e.V - www.httc.de
Efficiency Interpretation
KOM - Multimedia Communications Lab
Prof. Dr.-Ing. Ralf Steinmetz (Director)
Dept. of Electrical Engineering and Information Technology
Dept. of Computer Science (adjunct Professor)
Dipl.-Math., Dipl.-Inform. Kalman Graffi TUD – Technische Universität Darmstadt
Merckstr. 25, D-64283 Darmstadt, Germany
Kalman.Graffi@KOM.tu-darmstadt.de Tel.+49 6151 166150, Fax. +49 6151 166152
Tel.+49 6151 164959 www.KOM.tu-darmstadt.de
17. Februar 2011
© author(s) of these slides 2008 including research results of the research network KOM and TU Darmstadt otherwise as specified at the respective slide

2. Outline
Motivation
P2P systems in the future
Towards self-optimization
Self-optimization using a distributed control loop
Parameter specific QoS adaptation
In a peer: overlay bandwidth management
In the network: load balancing in heterogeneous P2P systems
System wide information management
Requirements and challenges
SkyEye.KOM – An information mgmt. over-overlay for structured P2P systems
Future Work
Distributed analysis and decision component
Evaluation
KOM – Multimedia Communications Lab 2

3. The Peer-to-Peer Paradigm
Peer-to-Peer Systems:
Users of a system provide the infrastructure of the system
Service is provided from users/peers to users/peers
Peer-to-Peer overlays:
virtual networks, providing new functionality H(„my
data“)
E.g. Distributed Hash Tables, Keyword-based Search = 3107 709
1008 1622 2011
2207
? 611
3485 2906
12.5.7.31
planet-lab.orgpeer-to-peer.info
berkeley.edu
Evolution of applications
61.51.166.150
95.7.6.10
86.8.10.18 7.31.10.25
File sharing:
No Quality of Service (QoS) requirements
Voice over IP
Real-time requirements
Video-on-demand
Real-time and bandwidth requirements
Online community platforms
Potential for high user interaction
See: K. Graffi, AsKo, et al. “Peer-to-Peer Forschung - Überblick und Herausforderungen” KOM – Multimedia Communications Lab 3
In: it - Information Technology (Methods and Applications of Informatics and Information Technology), vol. 46, no. 5, p. 272-279, July 2007

4. Towards the Modularization of P2P Systems
Live
Currently P2P applications Collaboration Collaboration
Environment
Very simple system design VoD Streaming
Group Distributed
New applications built from File sharing
Membership Data Structures
scratch Common API for Structured P2P Systems + Service Communicator
Enhanced Storage Layer and Enhanced Msg. Layer
E.g. file sharing Information Cache
Publish/Subscribe Full-
Vers.
Replicated Storage Text
Objects
Overlay Search
:
AccessControl Multicast: 1-to-n
Attribute- Capacity-
Internet based Object based Peer
Remote Storage Caching Messaging: 1-to-1 Search Search
Trend towards modularization DHT Message Dispatcher (Overlay MD)
Reusing of functional components Distributed Hash Table: Object to peer mapping
Easy to build applications Secure Direct Communication
Introducing role concepts
Ovrlay
P2P
Bootstrap Unstr. Overlay:
Structured Overlay: Key-based Routing
er Keyw.-bas. Search
Many interdependencies Overlay Message Dispatcher (Overlay MD)
Complex to optimize Wrapper of Transport Domain (WTD): Establishes and maintains connections
Internet
KOM – Multimedia Communications Lab 4

5. IMS
Self-optimization Control Loop
QoS Model
Information Management System Evaluation
Focus of
this talk
Gather system statistics on the distributed system: Measure in
Several metrics and parameters per module simulation:
Average values, standard deviations, confidence intervals scalability
Several (non-) functional requirements for information architecture efficiency
Supports capacity-based peer search inter de-
pendencies
P2P Systems
Observe in
QoS Improving Mechanisms Analytical Models
real app.:
more
Use information to improve Interpret system statistics
details
the quality of service Identify correlations between
feasibility
parameters
improve-
Optimize the resource usage metrics
ment speed
In the peer
limitations
In the network Consider optimization criteria
In the underlay Propose improved parameter
Partially done,
Focus of settings Partially done, future work
this talk
future work
KOM – Multimedia Communications Lab 5

6. IMS
Outline
QoS Model
Motivation
P2P systems in the future
Towards self-optimization
Self-optimization using a distributed control loop
Parameter specific QoS adaptation
In a peer: overlay bandwidth management
In the network: load balancing in heterogeneous P2P systems
System wide information management
Requirements and challenges
SkyEye.KOM – An information mgmt. over-overlay for structured P2P systems
Future Work
Distributed analysis and decision component
Evaluation
KOM – Multimedia Communications Lab 6

7. Main Question for QoS Improving Mechanisms
Can the system (self-)optimize using only local information
OR
Do we need global information for system wide self-optimization
On various layers to decide Peer
For the resource usage on each peer IN ^
1010011011000100110
OUT
P2P
For the resource usage in the P2P network Overlay
Focus on next slides
Optimized bandwidth usage on individual peers
Can we provide better QoS to specific message types?
Optimized task allocation in the P2P network
Can we distribute the load in the system and still consider peer capabilities?
KOM – Multimedia Communications Lab 7

8. Optimized Resource Usage in a Peer
Towards QoS for overlay flows
1 Peer 2
Types of flows in P2P systems IN ^ OUT
P2P
10100110110001000110
Overlay
Layer 4 traffic flows:
Underlay perspective
Out of scope
Direct P2P communication:
File transfer, application traffic, …
Few, but large data streams (elephants)
(Often) with low priority for the system
Overlay flows
Multi-hop: maintenance, user queries…
Many, small messages (mice)
Varying relevance for system
Provide QoS to overlay flow according to its relevance
See: K. Graffi, KP, NL, RST, “Taxonomy of [Active Queue Management , Scheduling] Strategies in Context– Multimedia Communications Lab
KOM of Peer-to-Peer Scenarios” 8
Technical Report, no. KOM-TR-2007-[01,02], January 2007.

9. Overlay Bandwidth Management
Novel substrate “Network Wrapper”
Between overlay and transport layer:
Queues messages
Applies Scheduling and AQM solution: HiPNOS.KOM
HiPNOS.KOM: Highest Priority First, No Starvation
Introduce message priorities for Loss and Delay
AQM: at congestion, drop message with lowest loss-prio.
Scheduling: at free bandwidth, send message with highest delay-prio.
Avoid starvation: Periodically increase delay-prio. of queued messages
Design decision based on observations
Overlay “flows” and traditional flows differ significantly
Stateless scheduler needed, QoS requirements stored in messages
See: K. Graffi, KP, SK, AsKo, NL, RST, “Overlay Bandwidth Management: Scheduling and Active Queue Management of Overlay Flows”
KOM – Multimedia Communications Lab 9
In: IEEE Local Computer Networks '07 (IEEE LCN’07), October 2007.

10. Overlay Bandwidth Management Results
Observation:
Proportional relations:
Delay to delay-priority
Loss to loss-priority
Results:
HiPNOS.KOM provides QoS
Regarding delay and loss
According to chosen priorities
See: K. Graffi, KP, SK, AsKo, NL, RST, “Overlay Bandwidth Management: Scheduling and Active Queue Management of Overlay Flows”
KOM – Multimedia Communications Lab 10
In: IEEE Local Computer Networks '07 (IEEE LCN’07), October 2007.

11. Bandwidth QoS applied for Emergency Call Handling
Emergency Call Handling is not supported in VoIP (Skype)
2009: mandatory for VoIP providers
P2P fits: all-IP, scalable,
but Quality of Service?
Requirements
1. Location critical service:
Find closest/responsible emergency station
Solved with adapted Globase.KOM
2. Quality of Service for P2P flows needed
QoS policy: low delay, low loss
contact emergency station as soon as possible
without message loss
Solved with adapted HiPNOS.KOM Source: NENA
Source: US census
High priorities for emergency calls
Snapshot of thedensity in Alabama
Population simulated scenarios
Alabama Emergency Zones
See: K. Graffi, AsKo et al. “ECHoP2P: Emergency Call Handling over Peer-to-Peer Overlays” KOM – Multimedia Communications Lab 11
In: International Workshop on Peer-to-Peer Network Virtual Environments (P2P-NVE ‘07), p. 58-67, December 2007

12. IMS
Outline
QoS Model
Motivation
P2P systems in the future
Towards self-optimization
Self-optimization using a distributed control loop
Parameter specific QoS adaptation
In a peer: overlay bandwidth management
In the network: load balancing in heterogeneous P2P systems
System wide information management
Requirements and challenges
SkyEye.KOM – An information mgmt. over-overlay for structured P2P systems
Future Work
Distributed analysis and decision component
Evaluation
KOM – Multimedia Communications Lab 12

13. Task / Load Allocation in the Network
Main question: Stores pointers
on resource
Optimized allocation of resources providers
according to specific criteria DHT
System wide metric:
Load balancing Peers interested
Consider heterogeneity in the res. ask for
providing peers
Provider
Heterogeneity: peer related information parameters dispatching
1. Allocated tasks for the system with focus on
load balancing
2. Estimation of local tasks / load
3. Bandwidth quality of the peer
4. Online time of the peer
5. … easily extendable
Scoring function determines most suitable provider
See: K. Graffi, SK, KP, AsKo, RST. “Load Balancing for Multimedia Streaming in Heterogeneous Peer-to-Peer Systems”.
KOM – Multimedia Communications Lab 13
In: 18th Int. Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV '08), ACM SIGMM, May 2008.

14. Task / Load Allocation in the Network
Evaluation:
Parameterized scoring function
Load derivation decreased by up to 53%
Up to 109% better results in comparison
to random task assignment
Conclusion
Various roles in the system can be assigned
Fair and load balanced
Considering peer heterogeneity
Easy to optimize according to
specific criteria, but what is important?
See: K. Graffi, SK, KP, AsKo, RST. “Load Balancing for Multimedia Streaming in Heterogeneous Peer-to-Peer Systems”.
KOM – Multimedia Communications Lab 14
In: 18th Int. Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV '08), ACM SIGMM, May 2008.

15. Lessons Learned for QoS in P2P Systems
Results for scheduling and AQM of resources in a peer
Delay and delay-priority, loss and loss-priority are proportional
Emergency calls have always highest priority, quality of service can be provided
Results for controlled task allocation in a P2P network
Trade-off between heterogeneity and load balancing in task allocation easy to do
Lessons learned:
IF … known:
what is the optimization criteria
set of all alternatives to choose from
THEN mechanisms for Quality of Service are easy to adopt
Question from beginning:
How to solve optimization problem: locally OR do we need system wide management?
Solve it system wide requires system wide information
Necessary for efficient decisions in distributed systems: system wide optimization criteria
Missing in Peer-to-Peer systems, need for Information Management System
KOM – Multimedia Communications Lab 15

16. IMS
Outline
QoS Model
Motivation
P2P systems in the future
Towards self-optimization
Self-optimization using a distributed control loop
Parameter specific QoS adaptation
In a peer: overlay bandwidth management
In the network: load balancing in heterogeneous P2P systems
System wide information management
Requirements and challenges
SkyEye.KOM – An information mgmt. over-overlay for structured P2P systems
Future Work
Distributed analysis and decision component
Evaluation
KOM – Multimedia Communications Lab 16

17. Requirements on an Information Mgmt. System
For all structured P2P overlays
Covered by DHT-function: route(msg, key), lookup(key)
Usable by all functional layers/modules in the P2P system
Function:
Generating system statistics
System wide average and absolute values, standard deviation, confidence intervals
Num. of peers, online times, hops per lookup, hit rate, relative delay penalty, node degree
Load (CPU, memory, storage space, bandwidth), additionally weighted with individual
peer capabilities
Capacity-based peer search
Search for a specified number of peers with specific capacities
E.g. give me 3 peers with:
Storage space > 20Mb
Bandwidth > 100kb/s
KOM – Multimedia Communications Lab 17

18. SkyEye: Information Management Over-overlay
For all structured P2P overlays Function:
Covered by DHT-function: Generating system statistics
route(msg, key), lookup(key) Capacity-based peer search
Usable by all functional layers in the
P2P system
Coordinator Support Peer Peer
Features:
Overlay-independency
Robustness (double-churn)
Load-balancing
Supporting peer heterogeneity
Scalability (# of info and peers) Unified ID space and abstr. functions
Adapting to usage patterns
API for DHT’s ID space and functions
Low overhead
DHT overlay
offering route(key,msg,nextHop), resp(key) .
Internet
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 18
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

19. Information Management Architecture
Concept of Over-overlay
Coordinator Support Peer Peer
Built on underlying structured overlay
Communicates via common API
route(msg, key)
Unified ID space [0,1] decouples
from specific DHT implementation
Unified ID space and abstr. functions
Information Domains: API for DHT’s ID space and functions
ID space separated in intervals (domains)
Peer responsible for a specific ID (e.g. middle) is responsible for ID domain
Peers in the domain send updates to this Coordinator
Coordinators may choose Support Peers to share the load/responsibility
Coordinators define maximum load to carry
Updates propagated upwards the tree
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 19
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

20. Efficiency Management Architecture Details
Update strategy
Each node publishes information Coordinator Support Peer Peer
updates in the architecture (bottom up)
Each node knows where to send updates to
Update consists of:
Non-aggregatable peer capacity information
Aggregatable systems statistics part
Updates are processed before forwarding
Update information has a limited lifetime
For SP: best 11-20 from below
Supporting Peers for Load Balancing
Coordinator may chose Supporting Peers best 1-10 best 1-10
Good peers chosen by e.g. 50/50 ratio
For SP: best 11-20 from below
Pick e.g. 20 best peers in the domain
Best 10 peers in domain advertised one level up
best 1-10 best 1-10
Second best 10 peers can be used as support
Workload can be delegated to supporting peers
For SP: best 11-20 For SP: best 11-20
Tree depth / peer load adjustable
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 20
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

21. Queries in the Information Mgmt. System
Query Type: Capacity-based Peer Search
Give me M peers
Fulfilling specific requirements on
Bandwidth, storage space, computational capabilities,
Online time, peer load, reputation
… (wide set of requirements definable)
Query processing
First sent to coordinator
Query traverses bottom-up, until M matching peers found
Result is sent then to requesting peer
Tradeoff:
Upper peers in tree know more
Load should be kept on lower levels of the tree
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 21
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

22. Very Simple Tree Maintenance
Join, Leave, Repair, Maintenance: Not needed
Join: Just send an update to Coordinator
After failure of “simple peer”: peer specific information times out
After failure of Coordinator / Support Peer:
route(msg, key) identifies new Coordinator
Coordinator picks a new Support Peer
New responsible peer is updated in the next update interval
No additional maintenance needed (done by structured overlay)
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 22
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

23. Evaluation of SkyEye: Scalability and
Information Freshness
Scalability Freshness
Tree-structure of coordinators form Freshness tightly related to tree depth
information architecture Information freshness:
Supp. peers: Strong peers take the load O(log N * updateInterval)
Query performance: O(log N) hops Conclusion:
Information is fresh even under churn
Failing peers are quickly replaced
It takes one update interval to recover
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 23
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

24. Evaluation of SkyEye: Quality of Information
Completeness of Knowledge Load on the Peers
Peers may define maximum load Constant update load (1 or 3 msg./intvl.)
Some information may be dropped Either just leaf or Coordinator as well
Completeness of knowledge: Ratio of Most of the peers are at level 8-9
considered peers in the corresp. domain Information density is highest at level 8-9
All >90%, some decline at level 9 Some information is dropped there
Quality of Information: due to load limit of peers
Peers in the results: >98% are online
Less useful peer information is dropped
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 24
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

25. Evaluation of SkyEye: Query Performance
Position of query resolving: Impact of query complexity
Remember: most peers in level 8-9 Query complexity ranging from 5 to 15
Query takes 2-4 hops to be resolved Small impact on position of resolving
Most of the queries resolved in level 4-5 Leaves room for optimization:
Leaves room for optimization: Resolving easier queries at less
Injecting queries lower in the tree loaded peers
Adapting tree height to query load
See: K. Graffi, A. Kovacevic “SkyEye: An Information Management Over-Overlay for Getting the Oracle ViewMultimedia Communications Lab 25
KOM – on Struct. P2P Systems”.
Submitted to: 8th International Conference on Peer-to-Peer Computing (IEEE? P2P '08), IEEE?, Sept. 2008.

26. Future Work on SkyEye.KOM
Some functional requirements and features addressed:
Gathering system statistics and capacity-based peer search solved
Overlay-independency
Unified ID space
Robustness (double-churn)
Usage of DHT-function route(msg, key), lookup(key)
Supporting peer heterogeneity
Load dispatching to Support Peers
Low overhead
Relying on robust underlying DHT, no maintenance
Room for optimization
Load-balancing
Several Support Peers per Coordinator: synchronization issues
Scalability (# of info and peers)
Limitations on the information size
Adapting to usage patterns
Injects queries and updates on various levels in the tree
KOM – Multimedia Communications Lab 26

27. IMS
Outline
QoS Model
Motivation
P2P systems in the future
Towards self-optimization
Self-optimization using a distributed control loop
Parameter specific QoS adaptation
In a peer: overlay bandwidth management
In the network: load balancing in heterogeneous P2P systems
System wide information management
Requirements and challenges
SkyEye.KOM – An information mgmt. over-overlay for structured P2P systems
Future Work
Distributed analysis and decision component
Evaluation
KOM – Multimedia Communications Lab 27

28. Distributed analysis and decision component
Input
System statistics, especially parameters and metrics
Optimization criteria (e.g. acceptable metric range, desired system state)
Black box functionality
Calculate interdependencies between metrics and parameters
Output
Optimized parameter settings used as input for QoS improving mechanisms
Future work
Describe P2P systems using mathematical (algebraic, stochastic) models
Use data mining libraries (Weka1) to identify correlations
1 http://www.cs.waikato.ac.nz/ml/weka/ KOM – Multimedia Communications Lab 28

29. Evaluation of the
Self-optimization Control Loop
Using PeerfactSim.KOM
Introduce common API, to make SkyEye applicable on all implemented DHTs
Used modules should publish their settings and metrics to SkyEye
And adopt the proposed parameter settings.
In QuaP2P reference scenario PIPE
PIPE: P2P-based Integrated Project support Environment
Collaborative work of QuaP2P group
Project benefits from SkyEye and self-optimization
In LifeSocial(.KOM) – a P2P-based online community platform
Online communities like Facebook, MySpace and StudiVZ are highly attractive
P2P-based solution fits more to the nature of an online community
LifeSocial offers a platform to test SkyEye in a large-scale testbed
KOM – Multimedia Communications Lab 29

30. Questions?
KOM – Multimedia Communications Lab 30