Join this session to learn how AWS customers are using Spot Instances to save up to 90% off On-Demand price when running frontend and web applications, big-data and analytics, Batch and HPC, dev/test, and containerized workloads

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SSID: Guest
Password: Cube@11999
BEST PRACTICES FOR OPTIMIZING YOUR EC2
COSTS WITH SPOT INSTANCES
Ran Sheinberg
Specialist Solutions Architect – EC2 Spot

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• EC2 Spot Instances overview
• Pricing model – no bidding
• Interruptions
• Spot orchestration options (Spot Fleet, EC2 Auto Scaling Groups)
• Use-cases and workload types for Spot Instances:
Containerized workloads (ECS, EKS/K8S, Batch/HPC)
Stateless Web/API frontends, also applicable for queue workers
Analytics and Big-Data (EMR / Spark)
CI/CD and Dev/Test

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
On-Demand
Pay for compute capacity
by the second or hour
with no long-term
commitments
For spiky workloads or to
define needs initially
Amazon EC2 Consumption Models
Reserved
Significant discount compared
to On-Demand instance
pricing
Steady state applications or
predictable usage, databases
Spot
Spare EC2 capacity for up to
90% off the On-Demand price.
For fault tolerant, instance
flexible or time-insensitive
workloads

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EC2 Spot Instances
Spare EC2 Capacity that AWS can reclaim with a 2 minute notice
Low Cost Easy access Usage flexibility

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Very Simple Rules of Spot Instances
The price changes infrequently
based on supply and demand of
spare capacity
Just request capacity and pay the
current rate. When we need the
capacity back you’ll get a 2 minute
warning

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spare capacity at scale
Clemson university – 1.1 Million cores
https://tinyurl.com/clemson-spot

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How customers are using Spot instances
Big Data Containers & CI/CD HPC & Batch Stateless Web Services
• Benefits: Provides
acceleration, scale, and deep
cost savings to hyper-scale
workloads
• Features: Scale to large
numbers of parallel nodes via
Spot Fleet
• Integrations: EMR, Hadoop or
Spark; Specify 5 instance types
with weighted capacities via
EMR Instance Fleets
• Benefits: Deploy stateless
containerized workloads and
manage clusters at any scale at
a fraction of the cost
• Features: Create EC2 Spot
Fleet directly from the ECS
console; Hibernate CI/CD
instances
• Integrations: ECS, EKS, K8,
KOPS, Kube-AWS, Jenkins,
Bamboo
• Benefits: Run multiple projects
simultaneously; launch & de-
commission 1000s of nodes
• Features: Spot Auto Scaling
groups; F1(FPGA), eg1 (Elastic
GPUs), Cluster GPU instances
to accelerate processing
• Integrations AWS Batch,
CloudFormation, Spot Fleet,
ECS
• Benefits: Scale, throughput
and deep cost savings for
large-scale web operations
• Features: launch and manage a
collection of diversified Spot
Instances across pools Spot
Fleet
• Integrations: Spot Fleet,
CloudFormation, EC2 Auto
Scaling

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Israeli customers’ Spot Instances case studies

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS
CloudFormation
AWS
OpsWorks
Amazon
EMR
Amazon
ECS
AWS Data
Pipeline
AWS BatchAuto Scaling
Amazon EC2 Spot works with…

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Flexibility is crucial
Instance flexible Time flexible Region flexible

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C4
$1.76
On
Demand
$0.88
$0.44
$0.22
$0.11
EC2 Spot pools – instance flexibility
Each instance family
Each instance size
Each Availability Zone (55)
In every region (18)
Is a separate Spot pool
R5 M4
C5
I3 M5d
R4 D2
C4

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Time flexibilityexamples
• Model training
• Genomics
• Development
• Testing
• One-time queries
• Time sensitive workloads• Time insensitive workloads
examples
• Web services
• APIs
• Analytics
• Grid computing
• Containers

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Low, Predictable Prices Eliminate the bid!
New Pricing Model

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
New Pricing Model
New smooth pricing

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot Instance Pricing
• You will never be
charged more than
the maximum price
you specified
• While your instance
runs, you are
charged the Spot
price that is in effect
for that period
You pay
the market
price

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What about interruptions?
Over the last 3 months, 95% of Spot Instance interruptions were
from a customer manually terminating the instance because the
application had completed its work

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot Instance Advisor

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Interruption notices via CloudWatch Events
https://tinyurl.com/spot-cwe
Or poll the instance metadata internally

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot Blocks
• Defined duration workload without interruptions
• 1-6 hours
• Lower discounts compared to Spot

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Is my workload Spot Ready?
Stateless Fault-Tolerant Flexible: Multi-
AZ and Instance
Flexibility
Loosely Coupled
Looks familiar?

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Containerized workloads on Spot Instances
• Best practices overlap: cattle, not pets
• Cluster instances (worker nodes) are conceptually redundant
and ephemeral in containerized workloads
• Instance type flexibility is easy

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot with Amazon ECS

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS - Provisioning and scaling
• Integrated directly into ECS console,
or use CloudFormation, Terraform
• ECS creates a Spot Fleet in your account
• Scaling with Target Tracking is easy
ECS service: CPU/MEM, HTTP requests
Spot Fleet on aggregate metric - CPU, MEM
• Interruptions are handled automatically via
scripts installed in User Data
ECS: automatically Drain the instance
Spot Fleet: automatically replace the instance

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS on Spot – architecture diagram

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot
with
Amazon EKS

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Availability
Zone 1
etcd
Master
etcd
Master
Availability
Zone 2
Availability
Zone 3
etcd
Master
EKS – Master & etcd vs Worker nodes
Worker nodes
Your AWS account
Managed master &
etcd
Not visible in your
AWS account

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kubernetes and EKS considerations
• Use Spot Instances as worker nodes
• Run a DaemonSet on every Spot worker node (example script exists on GitHub)
 Catches the interruption notice from the instance metadata
 Drains the node (evicts all pods from the node)
• Use labels to identify Spot nodes
 To schedule the DaemonSet
 To avoid scheduling stateful applications

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kubernetes and EKS scaling
• HPA (horizontal pod autoscaler)
 Autoscales the number of pods in a Deployment/ReplicaSet
• CA (cluster-autoscaler)
 Autoscales the number of worker nodes in the cluster when:
o Pods cannot be scheduled due to lack of compute resources
o Nodes are underutilized and important pods can be rescheduled elsewhere
• use multiple ASGs today for Spot instance diversification, use one ASG later this year
when Spot Fleet is integrated with ASG

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
mycluster.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl
m4.large Spot ASG
Min: 1 Max: 10
m5.large Spot ASG
Min: 1 Max: 10
c4.xlarge O-D ASG
Min: 1 Max: 10
Kuberentes: Multiple ASGs
{For EKS or
non-EKS

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Spot with Amazon EMR

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Use Spot in EMR, unless heavily time-constrained
• Spot best practice – diversify and be instance type agnostic
Use Instance Fleets - up to 5 instance types
Improved chances of getting capacity,
decreased impact from interruptions
Spark will automatically recover from
instance failures/interruptions
Enable Dynamic allocation of executors (default in EMR)
• Decouple storage from compute (from HDFS to S3 EMRFS)
• Defined duration (Spot blocks) and fallback to On-Demand
• If you need auto-scaling (i.e for Presto), use uniform groups
EMR – provisioning and instance types
r3.4xlarge
i3.4xlarge
r4.4xlarge
m4.10xlarge

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
# Parallelized
nodes
Time
# Parallelized
nodes
Time
Job running time: 1 hourJob running time: 10 hours
Parallelization with Spot Instances

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Stateless web application or API frontend
• Spot diversification before all else
• Ensure low bootstrap time
• To mitigate risk, launch On-Demand in a
different pool if Spot capacity is insufficient
• Availability and performance should not
be impacted
• Same concept for queue type workers
https://tinyurl.com/SpotAppnextBlog

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dev/Test workloads
• Jenkins build slaves – use Spot Fleet plugin
 Jenkins will automatically increase Spot Fleet target capacity upon pending jobs and scale in when no queue
• Running periodic benchmarks or load tests – use Spot Blocks
 To ensure that load does not decrease against app under test when Spot is interrupted
• Cannot be stateless and fault-tolerant? Running a Monolith or in-mem cache on the instance?
 Running Dev/Test on Spot could still be successful as an interim step, risk vs cost calculation, choose low interruption
rate instance types

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
General best practices
• Start with Launch Templates sooner than later, these will be required for new features
• New instance families generally have higher interruption rates
• Be instance type agnostic and let Fleet provide the required capacity at the lowest price

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank You!
ranshein@amazon.com
Today at 15:00 – hands-on workshop
Web application with Spot Fleet or CI/CD pipelines with Jenkins

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.