From the 2017 HPCC Systems Community Day: Amazon Web Services (AWS) is the premier IaaS provider. It leads the pack by offering more and better services at lower prices. Furthermore, AWS continuously improves and innovates to stay in front. There are numerous reasons to use an IaaS for HPCC Systems instead of dedicated hardware, especially if the workload does not execute 24/7. AWS has developed several features and tools for launching clusters. CloudFormation provides users a tool to make creating and managing an AWS resources much easier. Foremost it consists of a template (CFT) that defines resources required. The template is parameterizable and flexible so that a single CFT can launch an HPCC Systems cluster with an arbitrary number of nodes, various amount of memory per node, and other configuration options. Second, an Amazon Machine Image (AMI) contains the information needed to launch a compute node, with appropriate software, and configure it for a specific operation. We developed a CFT and an AMI for HPCC Systems. Additionally, we developed a reference architecture for HPCC Systems in AWS. It is a typical N+1 cluster, N worker nodes and one node (or mode) for cluster wide services such as Dali. The architecture also has storage (i.e., EBS volumes) and networking (i.e., VPN) resources. Significant effort was expended to determine the best set of resources for HPCC Systems clusters. Furthermore, we created a program to create and manage HPCC Systems clusters in AWS from the command line. This talk will present the tools we created. It also explains and justifies the reference architecture and many of the configuration options. Vince Freeh Associate Professor, North Carolina State University Vincent W. Freeh is an associate professor of computer science at North Carolina State University. He received his Ph.D. in 1996 from the University of Arizona. His research focus is high-performance system software, with emphasis on filesystems, parallel and distributed systems, power-aware computing, and storage systems. Prof Freeh teaches courses in the above research areas as well as in compilers. He has more than 55 referred publications in numerous computer science conferences and scientific journals. He received an NSF CAREER Award and several IBM Faculty Development Awards. He was a captain in the US Army Corps of Engineers before entering graduate school for his MS. Chin-Jung Hsu PhD Student, North Carolina State University Chin-Jung Hsu is a Ph.D. candidate in Computer Science at North Carolina State University. His primary research interests include distributed systems, storage systems, and performance optimization. He interned at NetApp and AT&T Research Lab, where he applied machine learning techniques to distributed storage systems for ensuring performance guarantees. Chin-Jung is currently working on how to efficiently run HPCC Systems applications on the public clouds such as AWS and Azure.

1. October 4, 2017
Vincent W. Freeh and Chin-Jung Hsu
HaaS: HPCC Systems as a Service
Bring your own data

2. 1
Why?
• Cost effective
• Pay-as-you-go
• Elastic
• Trade capex for opex

3. Create nodes
Provision EC2 instances
Define VPC
Configure storage
Define security
2
Argh……

4. 3
Configuring a Cluster
CloudFormation Template (CFT) is the recipe
provision and configure infrastructure
Amazon Machine Image (AMI) is the environment
customize machines

5. 4
Infrastructure
• EC2 (virtual machines)
• Instance types
• Storage options
• Auto Scaling Group
• Virtual Private Cloud (VPC)
• Subnets
• InternetGateway
• SecurityGroups
Image
• HPCC Systems
• HaaS support
• Third-party libraries
• Generate environment.xml

6. Create command-line interface for HPCC Systems on Cloud
Expert users & shell scripts (programmatic)
Provision infrastructure
Manage HPCC Systems cluster
Periodical workloads
Save and restore checkpoints (data)
Efficient & cost effective
Do not reimplement
Use existing tools: ecl, dfuplus, aws, ...
5
Goals

7. Phases
• Execute
• Idle
• Restore
• Save
Contain costs
• Shorten time to save & restore
• Reduce offline costs
• Increase execution efficiency
(reduce runtime)
Restore Save
6
Execute ExecuteIdle

8. Architecture
7
master slave
auto scaling groupauto scaling group
VPC
CloudFormation Template
• instance type
• amount of storage
• custom AMI
• # of thor nodes
• # of roxie nodes
• # of support nodes
Template parameters:

9. CFT Parameters HPCC Cluster
create
delete
list
events
...
haas stack
Provision Cluster
8

10. start
stop
status
haas cluster
Manage HPCC Cluster
9

11. 10
Execute ExecuteIdle
Checkpoint Data
save
restore
haas data
S3 Bucket

12. 11
Resources
• DFS
• Workunit
• Dropzone
Regular Expression
haas data save 10nodes
[dfs|wu|dz]
--regex “*.csv”
Flexible Checkpoints

13. Resize Cluster
12
DISTRIBUTE(recordset,…)
hass stack create 4nodes
hass data restore 4nodes
(from 2nodes)
hass data resize 4nodes

14. Elastic Block Storage
(EBS)
Scalable Storage Service
(S3) + EBS
Execute ExecuteIdle
$0.023*
$0.023
Storage Cost Analysis
13
$0.10
offline online
$ per GB-month
Elastic File System
(EFS)
$0.30 $0.30
$0.023 +
$0.10x
Trestore Tsave

15. 14
save restore
1GB 8.9 3.8
2GB 14.3 5.5
4GB 27.8 13.3
8GB 58.1 41.5
16GB 109.3 72.0
32GB 204.5 141.4
Evaluation of checkpointing (4 nodes)
Exploit S3
hpcc4bhab-10nodes-dfs-1.tar.gz
is24lezpr-10nodes-dfs-2.tar.gz
awsome09x-10nodes-dfs-3.tar.gz
.
.
unit:seconds

16. 15
Minutes

17. 16
Usage: haas [OPTIONS] COMMAND [ARGS]...
A command line tool for HPCC-as-a-Service (HaaS)
stack provision an HPCC cluster on the cloud
|------ create :: initialize a configurable HPCC cluster with CFT
|------ events :: show events of stack creation
|------ list :: list provisioned clusters
|------ delete :: delete an existing cluster
cluster operate the provisioned HPCC cluster
|------ start :: start the target cluster
|------ stop :: stop the cluster
|------ status :: check the status
data checkpoint an HPCC cluster to AWS S3
|------ save :: save the current cluster state
|------ restore :: restore from a previous checkpoint
|------ resize :: redistribute data after restoring from
different size cluster

18. https://github.com/vin0110/haas
How can you help?
• Use it
• Report errors
• Suggest features
• Contribute
Sponsored by
https://goo.gl/Kknz6w
17