Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Costs as You Scale on AWS
Keith Jarrett
WW Cost Optimization Lead
Amazon Web Services
E N T 3 0 2
Megan Kennedy
Program Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard

You’re using AWS
and you like it!

But maybe you are spending more
than you planned …

Or you’d just like to spend less

With the AWS cloud,
you only pay for what
you use

How do you
ensure that you
only pay for what
you need?

What to expect
We will introduce our framework for optimizing
and managing your AWS spend by focusing on
paying only for what you need
Hear lessons learned directly from those who have
been successful building cost management and
optimization capabilities: Vanguard Investments
And leave you with actionable steps to take back
to the office to implement immediately

The cloud has changed the way
IT and engineering deliver value

Embracing cloud economics
“We’ve seen a 52% total cost of ownership reduction.”
Ben Wilson,CTO GE Oil & Gas
Regarding their 58% cost reduction,
“People who conclude the cloud is too expensive are
doing it wrong.” Jake Burns, VP of Cloud Services
“40% decrease in usage cost per hour,
while growing usage by 70%.”

But it doesn’t just happen

The five pillars of cost optimization
Right-sizing your
instances
Pick the right
pricing model
Increase elasticity
Measuring &
monitoring
Match usage to
storage class

The cloud has changed the way
IT & finance operate

Traditional planning & procurement
The disruption of IT procurement

Traditional planning & procurement
The disruption of IT procurement
Disruption of procurement process

Re: Designing the process

Cloud financial management framework
Cost
optimization
• Cost aware architecture
• Right-sizing & elasticity
• Storage optimization
• Reserved instances
• Spot usage
Measurement &
accountability
• Account strategy
• Tagging strategy &
maintenance
• Cost visibility & analysis
• Metrics tracking &
monitoring
Planning &
forecasting
• Business planning
• Usage forecasting
• Cost forecasting
• Budgeting
Cloud financial
operations
• Governance & controls
• Procurement strategy
• Automation
• Build your cloud team
• Tool adoption & training
• Establish culture of cost
accountability

Getting started measurement & accountability

Step 1: Establish your account strategy

Using AWS Organizations to automate account
creation and consolidated billing
Policy-based management for multiple AWS accounts
Control AWS service
use across accounts
Consolidate billing
and usage reporting
Automate account
creation
https://aws.amazon.com/blogs/security/how-to-use-aws-organizations-to-automate-end-to-end-account-creation/

Business unit (BU)
AWS Organizations: Account strategies

Environment lifecycle

Project-based

Cost allocation tags
Environment
Project
Team
Application ID
Cost center

AWS Organizations + budgets for monitoring
Alerts can provide
notifications when budget is
reached
Cost allocation tags
Environment
Project
Team
Application ID
Cost center

Step 2: Establish your tagging strategy

What’s in a tag?

Using tags to establish visibility & chargebacks
https://aws.amazon.com/answers/account-management/aws-tagging-strategies/
Stakeholders Examples
• Cost center
• Application or workload
• User
• Expiration date
• Automation support

AWS Tools to help with missing or misspelled tags
2. AWS Config + Lambda1. AWS Tag Editor
https://bit.ly/co17_tag2
https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/scenario-implementing-tagging.html

Step 3: Set up cost reports in AWS Cost Explorer

Re: Introducing AWS Cost Explorer
AWS Cost Explorer lets you dive deeper into your
cost and usage data to identify trends, pinpoint
cost drivers, and detect anomalies
Get started quickly!
• Monthly and daily cost by service or linked
account
• Monthly EC2 running hours cost and usage
• Reserved instance reports
• RI utilization
• RI coverage
NOTE: All functionality is also available
programmatically via the AWS Cost Explorer API

Cost and usage report
Group by: Instance type, linked account,
purchase option, region, service, plus
many more …
Select daily or monthly
view and time horizon
Spend (above—here shown grouped by
instance type) and usage (below)
Filters to remove data not
relevant to your analysis
And then export your analysis to CSV
Advanced options to do things
like include or exclude costs
related to credits, refunds,
support, and so on
All functionality is available programmatically via API

Using Cost Explorer to understand true cost
It’s an
accrual
world.
– J.R. Storment
Cloudability Co-Founder
Understanding “true” AWS spend:
• Blended (removes cost of RI hours)
• Unblended (shows how RIs were applied)
• Unblended amortized (smooths upfront RI payment)

It’s an
accrual
world.
– J.R. Storment
Upfront RI purchase
Non-amortized Amazon EC2 spend

It’s an
accrual
world.
– J.R. Storment
Upfront RI purchase
Non-amortized Amazon EC2 spend
Amortized Amazon EC2 spend
Amortized
payment

AWS Cost Explorer | RI Reporting
Reserved Instance (RI) Reporting
• [New! Reserved Instances dashboard in Cost
Explorer]
• RI savings-related information now available in
the RI utilization and coverage reports
• Availability of amortized RI cost data
• Supports Amazon EC2, Amazon RDS, Amazon
Redshift, Amazon ElastiCache, and Amazon
Elasticsearch reservations

AWS Cost Explorer | RI purchase recommendations
Reserved Instance purchase recommendations
• Available to both payer and linked accounts
• RI purchase recommendations for the following
reservation models
• Amazon EC2
• Amazon RDS
• Amazon Redshift
• Amazon ElastiCache
• Amazon Elasticsearch

Step 4: Leverage partners to accelerate optimization

Resource and cost management competency
Partner solutions designed to provide spend visibility, resource utilization, chargebacks, and workload costs
(Optimize Now)
https://aws.amazon.com/products/management-tools/partner-solutions/

Open invitation to join me
re:Invent 2019

Megan Kennedy
Program Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard

What to expect
Introducing Vanguard
Cloud strategy & Cloud Business Office (CBO)
overview
Cloud financial management approach

Vanguard
One of the world’s largest investment management companies
19 16.5K
20+ M 170 $5.2T

CLIENTS
PROFITS
OWN OWN
VANGUARD
FUNDS
How we are different—operate at cost
Vanguard

2016 2017 2018 2019…
Vanguard’s cloud journey
Public
cloud
MVP
web apps
MVP
analytics
Web apps
1.0
100% New
analytics apps
75%
Existing
web apps
100%
New
web apps
100%
Packages
Packages
1.0
100% Existing
web apps
Analytics
1.0
Analytics
2.0
100%
Existing
analytics
ENABLEMENTMIGRATION
MVP
packages

Cloud Business Office (CBO)
CBO
Empowerment hub supporting cloud-based application development
Software delivery
lines of business
Information security
Legal
Risk
Compliance
Procurement
Cloud operations
Cloud engineering
Finance
Security, controls, risks
monitoring
Client engagement
Product management

Cloud growth: The path we were on
0.5
1.0
1.5
2.0
2.5
MillionEC2hours
Actual EC2 hours Forecasted EC2 hours
EC2 usage growth

0.5
1.0
1.5
2.0
2.5
MillionEC2hours
EC2 usage growth and migration count
10
0
200
300
400
Migration count
Migrationcount

0.5
1.0
1.5
2.0
2.5
MillionEC2hours
10
0
200
300
400
Migration count
Migrationcount
CBO
EC2 usage growth and migration count

Phase I:

Phase I: React!
Purchasing
Governance
Education

Phase I: Results
250
50
0
750
1,000
1,500
Dec-17
EC2 hours
EC2 usage growth and RI coverage
Nov-17Oct-17Sep-17Aug-17
1,250
ReservedOn-demand
EC2hours(000s)
33% RI coverage

Phase II:

Phase II: Manage growth
Governance
Education
Transparency
Purchasing

$0.20
$0.40
$0.80
$1.00
$1.40
May-17
LOB EC2 cost per hour
Apr-17Feb-17Dec-17Oct-17
$1.20
EC2costperhour
$0.60
Mar-17Jan-18Nov-17
Phase II: Manage growth
*46% decrease
in five months!
$1.20 / hr
$0.65 / hr

Some good news stories …
• Have to look at the bill every day
instead of monthly
• We were having new conversations
and challenging SLAs
• Fundamental shift from a
provisioned to consumption model
$500
$1,000
$1,500
$2,000
$2,500
Amazon CloudWatch spend (daily)$3,000
Phase II: Results

Some good news stories …
• Have to look at the bill every day
instead of monthly
• We were having new conversations
and challenging SLAs
• Fundamental shift from a
provisioned to consumption model
Line of business (LOB) ingest cluster hours
3500
3000
2500
2000
1500
1000
1-Jan 8-Jan 15-Jan 22-Jan 29-Jan 5-Feb 12-Feb
Phase II: Results

Phase III:

Phase III: Proactive management
Optimization
Governance
Accountability
Purchasing

Phase III: Results
What did we learn?
• Accountability requires a strong
tagging policy
• Policy Enforcement
• Up-front automation
Support email tag
(% of assets tagged)
70%
90%
60%
50%
40%
80%
30%
20%
10%

What did we learn?
• Implemented policy and
automation on sizing and run
schedules
• It pays to be proactive
0%
20%
40%
60%
80%
100%
0 20 40 60 80 100 120
%MemUsed
$Max CPU %
AUG—RDS utilization
Phase III: Results—RDS utilization (%)

What did we learn?
• Implemented policy and
automation on sizing and run
schedules
• It pays to be proactive
0%
20%
40%
60%
80%
100%
0 20 40 60 80 100 120
%MemUsed Max CPU %
SEPT—RDS utilization
Phase III: Results—RDS utilization (%)

70%
90%
60%
50%
40%
80%
30%
20%
10%
100%
Phase III: Results—Coverage by payment option
Oct-17 Nov-17 Dec-17 Jan-18 Feb-18 Mar-18 Apr-18 May-18 Jun-18 Jul-18 Aug-18 Sep-18
ReservedOn-demand Spot
100% 100% 96% 97%
88%
64%
60%
37% 34% 37% 44% 43%
25%
35%
29%
25% 23%
31% 34% 31% 34%
10%
19% 20%
38%
3%4%
17% 20%
2%
Line of business usage by payment type

Lessons learned: What would we do differently?
• Tagging: Enforce tagging early and often—it makes everything easier
• RI purchases: Do the math and see what makes sense for you
• Set expectations: Enterprise discount programs and third-party tool
implementation take time—plan accordingly
• Be proactive: Iterate & automate

Re: Designing the journey
• Cash view: OD • Cash view: OD + RI • Opex view: RI + OD
(amortization)
• Monthly cost data
• Align billing to org
• Central
infrastructure
• Manual reporting
• Show-back and
education
• LOB self service
• Charge-back
• LOB ownership
• Monthly approval RoB
• Train CBO
• Predefined approvals
• Cost workshops
• EDP coverage
• Cost opt tooling
• Reactive
• Manual shutdowns
• Conservative RI
Purchase
• Adv. RI purchase
• Automation
• Proactive
• Policy driven
automation & alerts
Phase 1 Phase 2 Phase 3

Where to start
• Visit cost optimization resource center @
https://aws.amazon.com/pricing/cost-optimization/
• Enable cost explorer for your organization
• Leverage cost management partners
https://aws.amazon.com/products/management-tools/partner-
solutions/
• Join us this week to learn more!

Want to learn more?
Wednesday, Nov 28
[WIN201] How to Optimize Your Microsoft Windows Workloads on AWS (ft. Xero)
11:30 a.m. – 12:30 p.m. | Venetian, Level 4, Delfino 4005
Thursday, Nov 29
[ENT 309] Building a World Class Cloud Center of Excellence (ft. Zendesk & CloudHealth)
3:15 p.m. – 4:15 p.m. | Venetian, Level 4, Delfino 4002
Tuesday, Nov 27
[ENT224] End Extra Spend: Hunting for Increased Value through Cost Optimization
3:15 p.m. – 5:30 p.m. | MGM, Level 3, Premier Ballroom 320

Thank you!
Keith Jarrett
WW Cost Optimization Lead
Amazon Web Services
Megan Kennedy
Program
Manager
Vanguard
Tim Treston
Cloud Economist
Vanguard

Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018

Similaire à Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018 (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Optimizing Costs as You Scale on AWS (ENT302) - AWS re:Invent 2018