On June 26, 2014, Hala talked about how a multi terabit network like Verizon EdgeCast supports a mega massive live event streamed by thousands or millions of customers worldwide.
Video of the presentation: http://youtu.be/NtMurrCpvlk
2. Why worry about
web performance?
The appetite for online media
streaming continues to grow.
With it the expectation
for streaming stability and
enhanced quality grows as well.
3. The challenge
How does a multi
terabit network like
EdgeCast support a
mega massive live
event streamed by
thousands or
millions of
customers
worldwide?
Hi, my name is Hala Al-Adwan and I’m the VP of Engineering at EdgeCast
I’m hear to talk to you about the importance of Web Performance at a CDN
So why is web performances so important? If we look at the landscape of internet usage right now
IP video traffic constitutes 66% of all consumer internet traffic by 2018 it’ll be up at 79%
internet video to TV doubled to date and consumer VOD is projected to double by 2018
By that time, Content Delivery Networks (CDNs) are expected to be delivering over half of all internet video
And as we see an increase in consumption, what comes with it is an increase in end-user expectations of reliability and enhanced quality
(4k video is coming slowly but surely)
And I think we’ve all experienced the frustrating of buffering or black screen right as someone is about to score a goal
Initially the topic for this talk was about the importance of Web Performance,
but given that we are in the midst of the biggest sporting event in the world right now
I thought I would hijack it a bit
and talk about how we, at EdgeCast, approach Web Performance, in this world of streaming video, for our customers who are supporting a huge event such as the world cup
First step, or challenge, with a large video event is that the requirement is often given to us in terms of end user numbers - “we need to scale to a billion users!” - but, we’re a bandwidth business
to address this, we rely heavily on our data - we do an analysis of our streaming usage and our traffic patterns to translate the user number to bandwidth per user
in addition to that, we analyze the traffic distribution across all of our points of presence,
to identify which of these pops need to be scaled for network capacity
And once we have this information in hand - our network engineering team gets busy!
The second step in optimizing the stream experience, is looking at the first mile,
some of our customers will opt to use their own origin especially in the case of live events,
and we need to make sure we’re able to access their content as swiftly as possible
for this, we do route analysis to ensure we are taking the optimal path to their content consistently
With network optimizations under way, whether it’s first mile or within our own network, the next step is to focus on application optimization
For these events, we formed an internal working group of SMEs from every team across the Engineering and Ops
we had weekly meetings to review analysis results and track progress
and these meetings became daily as the event neared to ensure nothing gets missed
we tackle this challenge at every tier,
whether it’s configuring appropriate HTTP rules or setting up specific cache-fill optimizations for each customer
We also look dog fooding our own products, in this case we used our DNS Route product to leverage it’s built in defensive DNS rules such as
scheduling traffic distribution based on expected game time load
As well as the ability to shed traffic to near pops in case of pop saturation
And by the way, we stop phutzing with the code 2-3 weeks prior to the event to ensure stability
Continuous optimizations is not a one time thing, it’s an on going process
we continue to monitor, collect data, learn, optimize and do it over again
it’s a constant state of learning, tuning and optimizing
Maybe you caught our Chief Architect, Rob Peters’ presentation yesterday on “Measurement Maturity Model” and later today we have Amir’s presentation on “Ghostfish: How to Replay Production Traffic in Real-time” - both are great examples of our continued focus on collecting, analyzing and learning from our data
Finally, last but not least is our customer!
first and foremost our focus is OUR CUSTOMER, everything we do is geared towards providing the best customer to end-user experience
We maintain focus on the conversation with the customer throughout the event planning as well as during the events
We make sure they are aware of what we’re doing for them, and we’re aware of what their needs and future requirements are
We increase our on-call presence during the events and set up war rooms with a live bridge with the customers
After every event we have a post mortem with lessons learned that we take forward into optimizing for our next event
Being able to apply our scale methodologies to something as large and global as the world cup is super exciting! We have had and will continue to have, lessons learned as the games go on
And we hope we can share with with you at the next velocity conference