The document discusses Amazon EC2 F1 instances for accelerating C/C++ applications using FPGAs. It provides an overview of F1 instances and capabilities like packet processing and multiple AFIs. Updates include more regions, sizes, and tools. Examples demonstrate financial computing, genomics, and video acceleration. The workshop will use precompiled AFIs and F1.2xlarge instances to introduce FPGA development.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerate Your C/C++ Applications with
Amazon EC2 F1 Instances
Kristopher King
Engineering Manager
AWS
C M P 4 0 5

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerate Your C/C++ Applications with
Amazon EC2 F1 Instances
The workshop requires an F1 (f1.2xl) instance limit 1 or more
Default AWS Account limits on F1 Instances are 0
To immediately increase your F1 instance limit:
1. Please submit your AWS Account number through this website:
http://bit.ly/F1Limit
2. Wait 5-10 mins before starting an F1 instance
C M P 4 0 5
Workshop prerequisite:

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• F1 instance overview/updates
• Developer tools/kit
• Application acceleration
• Software defined acceleration
• Workshop

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Related breakouts
Thursday, November 29
MFG401: Rightsizing Your Silicon Design Environment: Elastic Clusters for EDA Workloads
3:15PM | MGM, Level 1, Grand Ballroom 118

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FPGAs—The ultimate parallel processing device
2.5 million system logic cells
6,800 DSP engines
345 Mb on-chip memory
No predefined instruction set or underlying architecture
Developer customizes the architecture to his needs
• Custom datapaths
• Custom bit-width
• Custom memory hierarchies
Excels at all types of parallelism
• Deeply pipelined (e.g., video codecs)
• Bit manipulations (e.g., AES, SHA)
• Wide datapath (e.g., DNN)
• Custom memory hierarchy (e.g., data analytics)
Adapts to evolving algorithms and workload needs

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F1 instances—Optimized for hardware accelerators
• Xilinx 16nm UltraScale+ VU9P FPGA – Up to 8 FPGA(s) per instance
• Each FPGA includes 64 GiB DDR4 ECC
• Up to full bandwidth PCIe x16
• 2.5 million logic elements and 6,800 Digital Signal Processing engines
• Three different instance sizes with up to 64 vCPUs per instance
• 32:1 memory to core ratio and up to 1 TB of RAM
• Includes local NVME storage
F1
Model FPGA vCPU Memory (GiB)
Instance storage
(GiB) Networking performance EBS bandwidth
f1.2xlarge 1 8 122 1 x 470 NVMe SSD Up to 10,000 Mbps 1,700 Mbps
f1.4xlarge 2 16 244 1 x 940 NVMe SSD Up to 10,000 Mbps 3,500 Mbps
f1.16xlarge 8 64 976 4 x 940 NVMe SSD 25,000 Mbps 14,000 Mbps

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Machine
Image (AMI)
Amazon FPGA
Image (AFI)
CPU
Application
DDR-4
Attached
Memory
PCIe
An F1 instance can
have any number of
AFIs
An AFI can be loaded
into the FPGA in
secondsLaunch F1 instance
and load AFI
AMI
FPGA acceleration using F1
F1
DDR
Controllers
DDR-4
Attached
Memory

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What were we up to since last re:Invent?
More regions
IAD, PDX, GovCloud,
DUB
New: FRA, LHR, SYD
and BJS
New instance size
f1.4xlarge provides a
significant performance
boost compared to
f1.2xlarge
3 FPGA instance types
New features
Virtual Ethernet
networking packet
processing use-cases
Performance
improvements
Local instance NVMe
storage is now encrypted
with 256-bit encryption
Upgraded tools
Faster compile times
Higher frequencies and
improved timing
closure
Multiple bug fixes
F 1
F 1
F 1
F 1

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F1
Use cases:
• Packet processing
• Security appliances
• Virtual appliancesEC2 Elastic
Network
Interface
(ENI)
Network
packets
streaming
to FPGA
F1 new capabilities: Packet processing

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F1 new capabilities: Accelerating pipelines
Single FPGA utilizing multiple AFIs at runtime
• Simplify dependencies in third-party IP
• Accelerate development of complex designs
AFI 1 AFI 2 AFI 3 AFI N
FPGA DRAM FPGA DRAM FPGA DRAM

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F1 new capabilities: New tools
New FPGA developer AMI
- Faster compile times
- Higher frequencies
- Improved timing closure
- Supports Vivado 2018.2

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer tools
GitHub:
aws/aws-fpga

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Development environments
SDAccel/
GitHub:
aws/aws-fpga/
hdk/

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Amazon FPGA Image (AFI)
F1 Ingestion Service
Encrypted
DCP
id AFI
id AFI
id AFI
AWS AFI Storage
Developer S3
CLI:
AWSXCLBIN (agfi-0123456789abcdefg)

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Loading Amazon FPGA Image (AFI)
AWS AFI Storage
CLI:
x86 CPU
Host
Application
Drivers
Runtime Library
Acceleration APIs
id AFI
id AFI
id AFI
AWSXCLBIN+

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer kit examples
HDK
DMA (Direct Memory
Access)
URAM
VHDL
SDAccel
RTL Kernel
Security (AES, RSA, SHA1)
Computer vision (Affine, Convolve,
Huffman, IDCT)
File compression (Gzip)
WebP image compression
Financial
…80+ examples
HLx
HLS - DMA using IP Integrator
HLS – Digital up converter

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application acceleration
func2
func1 func2 func3 func4
func1 func3 func4
CPU
CPU
FPGA
FPGA handles compute-intensive, deeply pipelined,
massively parallel operations. CPU runs the
application code.With FPGA acceleration
Without acceleration

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Remember Amdahl’s Law
Identify performance bottlenecks in the application
• Use profiling tools, analyze the “roof line” of a Flame Graph
func1 func2 func3 func4CPU
It’s better to accelerate
func2 by 2x…
…than to accelerate
func3 by 50x !
When NOT beneficial
– Little to no parallelism
• Algorithm is highly sequential over
multiple data
• Tasks are highly dependent
When May Not be beneficial
– Small problem size
– Cost of Host to Device transfers
outweighs benefit
When to USE
– Algorithm allows for parallelization
– Many similar tasks

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Twitch VP9 live encoding using F1
Saving 25% on cost / bandwidth

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F1 use cases and partners
 Financial computing
 Genomics sequencing
 Engineering simulations
 Image and video processing
 Big data and machine learning
 Security, compression
 …and more

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture of FPGA acceleration
PCIe
x86 CPU
Host Application
Drivers
Runtime Library
Acceleration APIs
FPGA
Accelerated Functions
Direct Memory Access
Engine
Global Memory
Configurable Interconnect
User
Application
Code
Acceleration
Platform

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RTL, C/C++ or
OpenCL C
Link
C/C++ with
OpenCL API
Link
Host Application
Executable FPGA
Binary
(.awsxclbin)
Compile
Compile
Host
Application
FPGA
Kernels
Source Code
Development overview
AFI
Generation
.xclbin
gcc
SDx

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The FPGA kernel compilation
.xo
RTL
package_xo
.xo
C/C++
xocc -c
.xo
OpenCL
xocc -c
xocc -link PLATFORM
.xclbin

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Development flow
Software Emulation Hardware Emulation Hardware Execution
Host application runs with
a C/C++ or OpenCL model
of the Kernels
Host application runs with
a simulated RTL model of
the Kernels
Host application runs with
actual FPGA
implementation of the
Kernels
Confirm functional
correctness of the system
Test the host / kernel
integration, get
performance estimates
Confirm system runs
correctly and with desired
performance
Fastest turnaround time Best debug capabilities
Accurate performance
results

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop
Start with introduction module:
https://github.com/awslabs/aws-fpga-app-
notes/blob/master/reInvent18_Developer_Workshop/README.md
FAQ
Q: Why are precompiled Amazon FPGA images (AFIs) being used in the workshop?
A: To reduce the length of the workshop we precompiled AFIs and made them public.
Q: Why does the workshop use F1.2xlarge instances?
A: Normally a developer would use non-F1 instance for development. To avoid switching
between instances during the workshop, we will use F1.2xlarge only.
Q: Why do I need to exit the terminal between modules?
A: Exiting the terminal is a required step that clears the environment variables between
modules.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s next?
Resources:
Github: https://github.com/aws/aws-fpga
Developer Forum: https://forums.aws.amazon.com/forum.jspa?forumID=243
FPGA Developer AMI: https://aws.amazon.com/marketplace/pp/B06VVYBLZZ
F1 SDAccel On-demand lab: https://www.xilinx.com/products/design-tools/cloud-based-
acceleration/aws-f1-developer-lab-form.html

33. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kristopher King

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.