Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

On QoE Metrics and QoE Fairness for Network & Traffic Management

401 vues

Publié le

*** QoE Metrics ***
While Quality of Experience (QoE) has advanced very significantly as a field in recent years, the methods used for analyzing it have not always kept pace. When QoE is studied, measured or estimated, practically all the literature deals with Mean Opinion Scores (MOS). The MOS provides a simple scalar value for QoE, but it has several limitations, some of which are made clear in its name: for many applications, just having a mean value is not sufficient. For service and content providers in particular, it is more interesting to have an idea of how the scores are distributed, so as to ensure that a certain portion of the user population is experiencing satisfactory levels of quality, thus reducing churn. We put forward the limitations of MOS, present other statistical tools that provide a much more comprehensive view of how quality is perceived by the users, and illustrate it all by analyzing the results of several subjective studies with these tools.

*** QoE Fairness ***
User-centric service and application management focuses on the Quality of Experience (QoE) as perceived by the end user. Thereby, the goal is to maximize QoE while ensuring fairness among users, e.g., for resource allocation and scheduling in shared systems. Although the literature suggests to consider consequently QoE fairness, there is currently no accepted definition of QoE fairness. The contribution of this paper is the definition of a generic QoE fairness index F which has desirable key properties as well as the rationale behind it. By using examples and a measurement study involving multiple users downloading web content over a bottleneck link, we differentiate the proposed index from QoS fairness and the widely used Jain’s fairness index. Based on results, we argue that neither QoS fairness nor Jain’s fairness index meet all of the desirable QoE-relevant properties which are met by F. Consequently, the proposed index F may be used to compare QoE fairness across systems and applications, thus serving as a benchmark for QoE management mechanisms and system optimization.

Publié dans : Sciences
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

On QoE Metrics and QoE Fairness for Network & Traffic Management

  1. 1. Prof. Dr. Tobias Hoßfeld Chair of Modeling of Adaptive Systems (MAS) Institute for Computer Science and Business Information Systems (ICB) University of Duisburg-Essen www.mas.wiwi.uni-due.de On QoE Metrics and QoE Fairness for Network & Traffic Management Tobias Hossfeld Lea Skorin-Kapov Poul Heegaard
  2. 2. QoE Metrics and QoE Fairness 10/14/2016 2 System Fairness of system is evaluated by considering all QoE values f(QoS). Subjects evaluate test conditions e.g. on a 5-point scale. QoE Model e.g. f(QoS) = MOS QoE Fairness QoS Measurement QoE Metrics How to define and calculate QoE fairness? Which QoE metrics are of interest for providers?
  3. 3. MOS AND QOE Hoßfeld, Tobias, Poul E. Heegaard, Martín Varela, and Sebastian Möller. "QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS." Quality and User Experience, no. 1(2) (2016). http://link.springer.com/journal/41233
  4. 4. Quality of Experience • From Quality of service (QoS) to Quality of Experience (QoE) – QoS: packet loss, delay, jitter, … – QoE: subjective experience/satisfaction of users of a service • Example: video user interested in video quality and smooth video playout without interruptions • QoE model required for evaluation, improving QoE by proper monitoring and management… Input video (known reference) 10/14/2016 4
  5. 5. Mean Opinion Score (MOS) • Mean Opinion Score (MOS): numerical indication of the perceived quality of received media after compression and/or transmission Excellent Good Fair Poor Bad 5 4 3 2 1 Imperceptible Perceptible Slightly annoying Annoying Very annoying MOS Quality Impairment Excellent! Bad! Fair!Good! Poor!  Fair = 3 10/14/2016 5
  6. 6. Same MOS but Different Distributions 10/14/2016 6
  7. 7. 1 2 3 4 5 0 0.2 0.4 0.6 0.8 1 MOS probabilityofdissatisfieduser q =3 QoE Beyond the MOS -20° 80° On average, it’s fine! But still in pain! On average, it’s fine! Still, some suffer! MOS: Fair = 3 MOS > 3: 75% dissatisfied !  Other metrics! 10/14/2016 7
  8. 8. QoE Metrics • MOS: average user rating for one test condition • SOS: user diversity for that test condition • Theta-Acceptability: prob. that opinion score is above certain threshold • %GoB: the percentage of users rating Good-or-Better (%GoB) • %PoW: the percentage of users rating Poor-or-Worse (%PoW) • Quantile: user rating of fraction of (satisfied, dissatfied) • Probability distribution: complete information • SOS parameter a – quantifies user diversity for one application – relates SOS and MOS 𝑆 𝑥 = 𝛼 −𝑥2 + 𝑁𝑥 . – SOS parameter is scale independent 10/14/2016 8 Hoßfeld, Tobias, Poul E. Heegaard, Martín Varela, and Sebastian Möller. "QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS." Quality and User Experience, no. 1(2) (2016). http://link.springer.com/journal/41233
  9. 9. QOE FAIRNESS Tobias Hoßfeld, Lea Skorin-Kapov, Poul E. Heegaard, Martin Varela. Definition of QoE Fairness in Shared Systems. IEEE Comm. Letters, accepted Oct. 2016
  10. 10. QoE Metrics and QoE Fairness 10/14/2016 10 System Fairness of system is evaluated by considering all QoE values f(QoS). Subjects evaluate test conditions e.g. on a 5-point scale. QoE Model e.g. f(QoS) = MOS QoE Fairness QoS Measurement QoE Metrics How to define and calculate QoE fairness? Which QoE metrics are of interest for providers?
  11. 11. QoE Fairness • How to define and calculate QoE fairness? System Fairness of system is evaluated by considering all QoE values f(QoS). QoE Model e.g. f(QoS) = MOS QoE Fairness QoS Measurement 10/14/2016 11
  12. 12. Which system is better? • A) 10% experience best QoE; 90% worst QoE • B) 90% experience best QoE; 10% worst QoE 10/14/2016 12 QoE on 5-point scale • L=1: worst QoE • H=5: best QoE Normalized QoE (linear transformation) • L=0: worst QoE • H=1: best QoE
  13. 13. Which system is fairer? • A) 10% experience best QoE; 90% worst QoE • B) 90% experience best QoE; 10% worst QoE 10/14/2016 13 QoE on 5-point scale • L=1: worst QoE • H=5: best QoE Normalized QoE (linear transformation) • L=0: worst QoE • H=1: best QoE
  14. 14. For which x is the system maximal unfair? • In the system – x% of users experience maximum (best) QoE: 𝑯 = 𝟓 – 100-x% of users experience minimum (worst) QoE: 𝑳 = 𝟏 10/14/2016 14 Jain’s fairness index 𝐽 = 1 1+𝑐2 = 𝐸 𝑌 2 𝐸[𝑌2]
  15. 15. Desirable Properties of a QoE Fairness Index • (a) Population size independence • (b) Scale and metric independence • (c) Boundedness [0;1] • (d) Continuity • (e) Intuitive 10/14/2016 15 Jain‘s fairness index designed for those properties
  16. 16. For which x is the system maximal unfair? • In the system – x% of users experience maximum (best) QoE: 𝑯 = 𝟓 – 100-x% of users experience minimum (worst) QoE: 𝑳 = 𝟏 10/14/2016 16
  17. 17. Desirable Properties of a QoE Fairness Index • (a) Population size independence • (b) Scale and metric independence • (c) Boundedness [0;1] • (d) Continuity • (e) Intuitive • (f) Deviation symmetric • (g) QoE level independence • (h) Valid for multi-applications 10/14/2016 17 Jain‘s fairness index designed for those properties Specific to QoE Fairness
  18. 18. Definition of QoE Fairness Index • QoE model maps QoS parameters x to QoE in [𝐿; 𝐻] 𝑄: 𝑥 ↦ 𝑦 = 𝑄 𝑥 ∈ [𝐿; 𝐻] – E.g. 𝑄(𝑥) is the MOS value on a 5-point scale, 𝐿 = 1, 𝐻 = 5 • In a system with 𝑛 users, QoE values are random variable 𝑌 • Maximum standard deviation of 𝑌 𝜎 𝑚𝑎𝑥 = 1 2 (𝐻 − 𝐿). • Fairness index 10/14/2016 18 𝐹 = 1 − 𝜎 𝜎 𝑚𝑎𝑥 = 1 − 2𝜎 𝐻 − 𝐿
  19. 19. Illustration 10/14/2016 19 𝐹 = 1 − 𝜎 𝜎 𝑚𝑎𝑥 = 1 − 2𝜎 𝐻 − 𝐿 1 =L 5 =H 2 3 4 5-point scale x x x xx xxx xx xx x xx xx xx x Avg. QoE 𝝁 𝟐𝝈 x single user experience
  20. 20. Issues with Jain‘s Fairness Index • Coefficient of variation: only useful for ratio scales – Requires natural zero point – No meaning for data on interval scale • QoE is given on interval scales – Coefficient of variation is not a valid measurement 10/14/2016 20 1 =L 5 =H 2 3 4 5-point scale x x x xx xxx xx xx x xx xx xx x
  21. 21. Comparison: Jain and QoE Fairness Index • Jain’s fairness index 𝐽 = 1 1+𝑐2 = 𝐸 𝑌 2 𝐸[𝑌2] • QoE fairness index 𝐹 = 1 − 2𝜎 𝐻−𝐿 10/14/2016 21 Jain’s J violates (f) Deviation symmetric (g) QoE level independence Jain’s J is not very sensitive. J for max. standard deviation: 𝑱 𝒎𝒊𝒏 = 𝟎. 𝟔𝟗 (for 5-point scale)
  22. 22. Some numbers Scenario~ Description J F 1 All users experience 1. 1 1 2 50% experience 1 and 50% experience 2. 0.90 0.75 3 50% experience 1 and 50% experience 3. 0.80 0.50 4 50% experience 1 and 50% experience 4. 0.74 0.25 5 50% experience 1 and 50% experience 5. 0.69 0.00 6 50% experience 2 and 50% experience 4. 0.90 0.50 7 50% experience 2.9 and 50% experience 4.9. 0.94 0.50 8 Uniform distribution 𝑌 ~ 𝑈(𝐿; 𝐻). 0.75 0.42 10/14/2016 22
  23. 23. QoS Fairness != QoE Fairness 10/14/2016 23
  24. 24. References • Hoßfeld, Tobias, Poul E. Heegaard, Martín Varela, and Sebastian Möller. "QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS." Quality and User Experience 1, no. 1 (2016): 2. – Open access: http://link.springer.com/journal/41233 – Scripts: https://github.com/hossfeld/QoE-Metrics/wiki – Formal Definition of QoE Metrics: http://arxiv.org/abs/1607.00321 • Tobias Hoßfeld, Lea Skorin-Kapov, Poul E. Heegaard, Martin Varela. Definition of QoE Fairness in Shared Systems. IEEE Comm. Letters, accepted Oct. 2016 – http://ieeexplore.ieee.org/document/7588099/ 10/14/2016 24

×