We've figured out how to send physical goods around the world: aggregate them into containers. We're still struggling how to do digital good, which we disaggregate into packets. Here's the answer.
How to Troubleshoot Apps for the Modern Connected Worker
Digital supply chain quality management
1. Digital Supply Chain
Quality Management System
How to deliver assured application performance
and fit-for-purpose cloud access services?
February 2018
2. NETWORK DEMAND
• Weakly characterised performance envelope
• Poorly defined network demand using
averaged scalar metrics (throughput, loss,
delay, jitter) – but there no quality in averages
• Focus on demand for bandwidth/speed
(quantity) but not on stationarity (quality)
App performance QoE demand = network QoS requirement
NETWORK SUPPLY
• Emergent performance and highly variable
quality (by geography, bearer, product, line)
• Unstable quality (‘non-stationary’), so apps fail
• Poor product performance metrics (mostly
‘speed’ – a weak proxy for QoE and quality)
• Not assured; no quality floor
vs
UNWANTED
RESULT:
• Cannot compare & predict fitness-for-purpose; blame game with app/network suppliers
• All performance risk transferred from vendor(s) to client
• Performance failure only becomes visible in deployment; comes & goes without warning
• Business benefit of move to cloud/SaaS/UC/virtual working lost
• Self-insurance cost (plan Bs) and uncontained customer brand/employee goodwill impact
REQUIREMENT: RUN DISTRIBUTED APPLICATION(S) WITH A BOUNDED FAILURE RATE
2
3. TYPICAL ROOT CAUSES OF DIGITAL EXPERIENCE QUALITY FAILURE
UNMANAGED APPLICATION PERFORMANCE FAILURE
Why so?
PEOPLE
Skills gap: wrong belief QoE
engineering is not even
possible, or can be done
using weak QoE metrics
PROCESS
Lack a scientific management
framework for digital
experience quality; failure to
apply proven knowledge
TECHNOLOGY
Immature science and
engineering of quality in
digital supply chains;
wrong resource model
Acquire language of performance
hazards and skills to reason
about supply and demand in
digital supply chains
Apply existing variation
management science (theory of
constraints, lean, six sigma) to
digital experience quality
Use quality attenuation analytics
and high-fidelity network
measures to see what is really
happening & model/predict
3
4. THE ‘QUALITY ATTENUATION’ FRAMEWORK OFFERS A HOLISTIC SOLUTION
FOUNDATIONS
Mathematics
∆Q calculus provides rational
unified resource model for
distributed computing
Science
New ’quality attenuation’
metrics and models for
demand and supply
Engineering
Performance hazard
modelling adapted from
safety-critical systems
TECHNOLOGY
Measurement
Instantaneous performance
captures by high-fidelity
measurement (space & time)
Models
Robust predictive models
based on “performance
budgets” for supply chains
Mechanisms
High-frequency resource
trading adapted from
supercomputing
CAPABILITIES
Calibration
QoE-centric network data
with strong causality model
Coordination
Contract technical
performance at boundaries in
digital supply chains
Control
New approach to scheduling
and quality assurance that
delivers predictable
performance and QoE
For more technical detail, visit qualityattenuation.science. 4
5. THE DIFFERENCE THE QUALITY ATTENUATION FRAMEWORK APPROACH MAKES
Predictable performance with a managed QoE “safety margin”
It’s only ordinary science and engineering using proven management methods.
There is no magic involved! Just new mathematics and metrics.
Perform robust product
feasibility analysis in
advance of deployment
Size individual
deployments and
quantify QoE risk
Isolate performance
problems using the
scientific method
The concept of a “performance hazard” allows us to relate the supply to demand and quantify the risk
of QoE failure via a “performance contract”. This technical contract between supply and demand is
called a Quantitative Timeliness Agreement (QTA). Meeting the QTA bound on loss and delay is both
necessary and sufficient to deliver the application performance outcome.
5
6. HOW IT WORKS: COMPARE DEMAND TO SUPPLY (YES, IT IS THAT SIMPLE!)
NETWORK DEMAND NETWORK SUPPLY QoE ‘SLAZARD’
High-fidelity network measures
6
7. OUR SPECIAL NETWORK “X-RAY VISION SPECTACLES” LET US SEE SUPPLY VS DEMAND
See performance from
the user perspective
DEMAND
See performance from
the network perspective
SUPPLY
Relate demand to supply using quality attenuation analytics,
high-fidelity network measurement,
and the ∆Q calculus to predict performance
Rather than 3D movie glasses, it
is really a well-proven
measurement system developed
over 10 years. It injects low
bitrate ‘golden packet’ flows with
special statistical properties.
The timing of those ‘golden’
packets are observed as they pass
in each direction, and that data is
analysed to construct an high-
definition ‘image’ of the network
in space and time.
This system has been deployed
many times at tier 1 fixed and
mobile operators, and is now
available for end user use in a
compact virtual machine form.
7
8. EXAMPLES OF HIGH-FIDELITY MEASURES (MANY MORE ARE AVAILABLE)
Round Trip (run e4e53aec-4045-4d2f-96e7-67ebf1307ce2)
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 50 100 150 200 250 300
delay(s)
run time (s)
Observed Delay against Experiment Run Time
boris-s001->london->NHC->london->boris-s001
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 200 400 600 800 1000 1200 1400 1600
delay(s)
packet size (octet)
Observed Delay against Packet Size
boris-s001->london->NHC->london->boris-s001
G=8.69e-3s, S=2.52e-7s/octet, MSE=2.08e-7
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 50 100 150 200 250 300
delay(s)
run time (s)
Observed Delay Variability (V) against Experiment Run Time
boris-s001->london->NHC->london->boris-s001
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 200 400 600 800 1000 1200 1400 1600
delay(s)
packet size (octet)
Observed Delay Variability (V) against Packet Size
sample variability (V): mean=1.20e-2s, std dev=2.75e-2
boris-s001->london->NHC->london->boris-s001
0.8
1
vedservice
Cumulative Distribution of V
fraction lost=5.40e-2
boris-s001->london->NHC->london->boris-s001
1
veservice
scale
Inverse Cumulative Distribution of V
boris-s001->london->NHC->london->boris-s001
4G home gateway + WiFi in Lithuania ATLAS experiment at CERN
(video at 40 million frames/sec)
Satellite in southern Spain Peak hour DOCSIS cable + WiFi in Ireland 8
9. ∆Q METRICS HAVE AN ALGEBRA FOR TRUE ENGINEERING (AND NOTHING ELSE DOES!)
∆Q(A) ∆Q(B) ∆Q(C)
VA
SA
GA
VB
SB
GB
VC
SC
GC
+
+
+
+
+
+
=
=
=
∑V
∑S
∑G
SUPPLIER A SUPPLIER B SUPPLIER C
∆Q(∑ A+B+C)
9
Variable delay due to load
Size of packet delay
Geographic delay
∆Q|G
∆Q|S
∆Q|V
Packet size
One-waydelay
G/S/V are independent probability
functions using improper random
variables or improper cumulative
distributions. These can be
(de)convolved and “budgeted”
along the supply chain using
(de)composable “quality contracts”.
10. Single class of service
Unpredictable ‘best effort’ quality
Poor resource utilisation
Complex to manage
Multiple classes of service
Predictable ‘just right’ quality
Excellent resource utilisation
Simple to manage
“BEST EFFORT”
Quality with a quantity
Low value, high cost
QUALITY ASSURED
Quantity of quality
High value, low cost
NEW SCHEDULING MECHANISMS DELIVER ASSURED QUALITY TO A ∆Q-BASED SPEC
By applying the concepts of ‘lean’ and ‘just in time’ to packet networks we
can achieve a transformation in economics and deliver ‘assured cloud access’. 10
11. INTENTIONAL
SEMANTICS
DENOTATIONAL
SEMANTICS
OPERATIONAL
SEMANTICS
What QoE you wanted What QoS you asked for What you got
What it maybe useful for What QoE it might give What QoS has happened
F4P:
P4F:
Quality Assured: Fitness-for-purpose (F4P)
Best Effort: Purpose-for-fitness (P4F)
– Engineered performance
with predictable quality floor
Emergent performance
with unpredictable quality floor –
THIS IS A PARADIGM CHANGE FROM EMERGENT TO ENGINEERED PERFORMANCE
11
This change is the basis for an telco/cloud industry transformation comparable to how
physical transport switched from break bulk shipping to intermodal container logistics.
12. To learn more about the science visit
qualityattenuation.science.
To discuss how we can work together contact
mail@martingeddes.com.