A presentation which is used as GIP participants’ progress report. This presentation will explain how the custom-built AWS cost visualizer works.

1. AWS COST VISUALIZER
HDE GIP - MTS JUNE 2014
Presented by Iskandar Setiadi & Saskya Irena

2. AWS COST VISUALIZER
OVERVIEW
1. Problem Description
2. Analysis & Implementation
3. Result
4. Demonstration

4. ENORMOUS NUMBER OF PRODUCTS

5. UNDETAILED BILLING REPORT (UI)

6. COMPLICATED BILLING REPORT

7. CSV DETAILS
InvoiceID,PayerAccountId,LinkedAccountId,RecordType,RecordId,ProductName,RateId,SubscriptionId,Pr
"Estimated" ,"LineItem","21610877191184803585998665","AmazonElastic
"Estimated" ,"LineItem","21942771010558985838334611","AmazonElastic
BUT THERE ARE 8.000.000 (八百万)
ENTRIES PER MONTH!

8. HOW MUCH IS THE EXPENSE FOR:
1. Each Customer
2. Each S3 Bucket
3. Each Infrastructure
4. Each Project (Tapirus, Nobita)
5. Other Expenses
- DAILY / MONTHLY -
- INSTANCE, ETC -

9. SOLUTION
CREATE YOUR OWN TOOL!

10. IMPLEMENTATION
ENVIRONMENT
1. Python 2.6.9
2. Flask Framework & Redis for Database
3. Linux/UNIX based Operating System
4. Amazon EC2 - t1.micro -> c3.large
5. Github (HDE/aws-cost-visualizer)

11. FIELDS OF DATA (CUSTOMER / CI /
PROJECT)
1. Company Name
2. EC2 Instance Reserved
3. EC2 Instance Non-Reserved
4. EC2 EBS Standard
5. EC2 EBS PIOPS
6. EC2 Data Transfer
7. EC2 Others
8. S3 Usage Rate *
9. S3 **
10. Others
*= CUSTOMER ONLY
** = CI & PROJECT ONLY

12. FIELDS OF DATA (S3)
1. Bucket Name
2. Total Cost

13. FIELDS OF DATA (NON-CUSTOMER)
1. Product Name (SQS, etc)
2. Total Cost

14. PARSING CONVENTION
EC2 Instance Reserved: BoxUsage, HeavyUsage
EC2 Instance Non-Reserved: BoxUsage, SpotUsage
EC2 EBS Standard: VolumeUsage, VolumeIOUsage,
SnapshotUsage
EC2 EBS PIOPS: VPIOPS, VolumeUsage
EC2 Data Transfer: Regional, In-Bytes, Out-Bytes
Others: Idle EIP, LoadBalancerUsage, Monitoring (Alarm,
etc), DataProcessing, etc

15. 26 DAYS STATISTICS
7.220.000++ LINES
1147 SECONDS
(19 MINUTES 3 SECONDS)

16. KEY IDEAS
CHECK THE LATEST 2 MONTHS REPORT
RUN IT ONCE / TWICE PER DAY
(11 P.M. / 5 A.M.)
USE SCRIPT / CRON TO HANDLE FAILURE [PRODUCTION]
IF AWS MONTHLY REPORT IS GETTING BIGGER (5 TIMES)
-> MIGRATE TO BETTER EC2 INSTANCE

17. NAMING CONVENTIONS / RESERVED
KEYWORDS
THERE'S NO UNDERSCORE '_' ON S3 BUCKET NAME
PLEASE APPEND KEYWORD LIST AS BRIEF AS POSSIBLE

18. ASSUMPTIONS
All conventions in AWS Billing Report, Usage Report, and
Company List are constant
But the nature of production environment is always
changing, so it's needed to maintain the application
If there's no specified date in AWS Billing Report, this
application will map it to 1st date

19. DEMONSTRATION TIME!

20. ANY QUESTION?
THANKS FOR YOUR ATTENTION!
ありがとう ございました