1. Art
of
Using
Xen
at
Scale
Pradeep
Vincent
Principal
Engineer,
Amazon
Web
Services

2. Agenda
• Amazon
Web
Services
-­‐
Intro
• Virtualiza@on
in
the
Cloud
• Scale
Impact
on
Virtualiza@on
• Xen
Enhancements..
• Future
Features
–
Some
Thoughts

3. AMAZON
WEB
SERVICES
-­‐
INTRO

4. Amazon
Web
Services
“..delivers
a
highly
scalable
cloud
compu@ng
plaIorm
with
high
availability
and
dependability,
and
the
flexibility
to
enable
customers
to
build
a
wide
range
of
applica@ons.”

5. Amazon
Web
Services
• Includes
more
than
20
Services
– Compute
Services
• EC2
– Storage
Services
• S3
• EBS
– Database
Services
– Content
Delivery
Services
– Messaging
Services
– And
many
more..
• More
Details
:
aws.amazon.com

6. EC2
Instance
Types
High
High
Cluster
Standard
Micro
Memory
Compute
Compute
Instances
Instances
Instances
Instances
Instances

7. VIRTUALIZATION
IN
THE
CLOUD

8. Virtualiza@on
and
Cloud
• Virtualiza@on
a
key
enabler
for
Cloud
Compu@ng
• Virtualiza@on
Provides..
– Secure
Control
Plane
– Fine
grained
resource
alloca@on
– Cloud
Specific
Business
Logic
in
Dom-­‐0

9. Virtualiza@on
in
the
Cloud
• Guest
OS(Kernel)
chosen
by
customer
– Large
and
ever
changing
set
• Aggressive
inges@on
of
Security
fixes
– Security
is
paramount
• Unique
Challenges
at
Scale..

10. SCALE
IMPACT
ON
VIRTUALIZATION

11. Scale
Ma_ers..

12.
Intra
Host
‘Scaling’
Mul@-­‐tenancy

13. Mul@-­‐tenancy
• Scaling
Challenges
– Performance
Scaling:
I/O
performance
etc
– Tools
Scaling:
XM
etc
• Isola@on
Challenges
– Performance
Isola@on
– VM
Fault
Isola@on

14. Fault
Isola@on

15. VM-­‐level
Fault
Isola@on
• Failure
related
to
one
VM
isolated
from
Peer
VMs
• Good
Hypervisor
Level
Fault
Isola@on
in
Xen
• End-­‐to-­‐End
Fault
Isola@on
Important
– Including
Dom-­‐0
I/O
Stack
– Including
Control
Plane
(e.g.
Xenwatch)

16. Performance

17. Consistency
of
Performance
Performance
Isola@on

18. Performance
Scaling
• Cri@cal
for
Mul@-­‐tenancy
• Par@cularly
I/O
Performance
• Good
in
Xen
• Deserves
con@nued
Focus..

19. I/O
Performance
Direct
AHached
Device
Performance
Good
But..
Loss
of
Control
and
Flexibility
Not
Great

20.
Para-­‐virtualized
I/O
Performance
Important

21. Reduce
Cost

22. Hardware
Enablement
• Xen
Enablement
bit
‘Slow’
for
some
hardware
• Faster
‘Produc@on’
quality
enablement
in
Xen
– Help
Cloud
adopt
new
hardware
faster

23.
Upstream
Linux
IntegraOon
GREAT
!!

24. Interface
Compa@bility

25. Interface
Compa@bility
• Applies
to
Many
Types
of
Interfaces
– Paravirtualized
Driver
Interface
• Blkfront
–
Blkback
• NeIront
–
Netbk
– Hypercall
Interface
– Control
Plane
Interface
(XM
vs
XL)
• Has
Been
Good
in
Xen
– Deserves
con@nued
focus

26. XEN
ENHANCEMENTS..

27. Serve
Spiky
Workloads
For
Less

28. Micro
instance
in
EC2

29. Micro
Instance
and
CPU
Hogs

30. Large
‘CPU
Time’
Bucket
Bigger
than
10s
of
millisecs
10s
of
milliseconds

31. Some
Thoughts
on
FUTURE
FEATURES

32. Mul@-­‐@er
Scheduling
?
• Lots
of
use
cases
in
the
Cloud
• Classes
with
priori@za@on
– Beyond
‘CPU
Pools’
– Similar
to
Linux
Scheduler
?
– Real
Time
Scheduling/Advanced
Func@onality
• Preserve
exis@ng
Scheduler
Seman@cs
to
the
extent
possible

33. Dom-­‐0
Resource
Accoun@ng
• CPU
consumed
in
Dom-­‐0
for
each
VM
– Isn’t
accounted
against
the
VM
per
se
– Fairness
Issues
– Priori@za@on
Issues

34. Dom-­‐0
Resource
Accoun@ng
• Possible
Solu@ons
– Group
Scheduling
in
Dom-­‐0
–
Par@al
Solu@on
– Xen
and
Dom-­‐0
Scheduler
Coopera@on
??
• Hypervisor
–
Dom-­‐0
Interface
Blurred..
– Something
else
??
• Scheduler
deals
with
Threads..
– One
Thread
serving
Mul@ple
VMs
not
great
– ‘Thread
Aligned’
VM
Resource
Consump@on
Good

35. Key
Takeaways
• Virtualiza@on
Key
For
Cloud
• End-­‐to-­‐End
Fault
Isola@on
Important
• Performance
Isola@on
And
Performance
Consistency
Important
• Para-­‐virtualized
I/O
Performance
Important
• Faster
‘Produc@on’
Quality
Hardware
Enablement
• Interface
Compa@bility
Cri@cal

36. SHAMELESS
MARKETING

37. We
are
Hiring
!

38. We
are
Hiring
!
We
are
Hiring
!

39. We
are
Hiring
!
We
are
Hiring
!
We
are
Hiring
!
aws.amazon.com/careers
We
are
Hiring
!

40. THANK
YOU
• EMAIL:
PRADEEPV@AMAZON.COM
• MORE
INFO:
WWW.AWS.AMAZON.COM