From One to Many: Evolving VPC Design (ARC309-R1) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
From One to Many:
Evolving VPC Design
Androski Spicer
Solutions Architect
Amazon Web Services
A R C 3 0 9

A m a z o n V i r t u a l P r i v a t e C l o u d ( A m a z o n V P C ) D e s i g n
Simplicity

A m a z o n V P C d e s i g n
Rethinking connectivity

Subnet
Route Table
Elastic Network Interface
Internet Gateway
Virtual Private
Gateway
VPN Connection
Network ACL
Security group
EC2 instances
VPC peering
AWS Direct Connect (DX)
Availability Zone
VPC Endpoints
Amazon VPC
Region
DX gateway
Customer
Datacenter
SHARED SERVICES

One VPC
W E A L L S T A R T W I T H
Subnet - A
Network ACL
Availability Zone - A
Region
Subnet - B
Network ACL
Availability Zone - B
Subnet - C
Network ACL
Availability Zone - C

us-east-2
VPC
VPC
VPC
NA
HQ
Chicago
DX
London
DX
ap-northeast-1
VPC
VPC
VPC
VPC
EU
HQ
us-west-2
VPC
VPC
VPC
eu-east-2
VPC
VPC
VPC
VPC
VPC
VPC
Tokyo
DX
Amazon Public
Services
GLOBALLY
PRIVATE VIF
PUBLIC VIF
Shared
Services
VPC
Shared
Services
VPC
Shared
Services
VPC
DX GATEWAY
TO
MANY
APAC
HQ
INTER REGION
VPC PEER
VPC PEER
DX GATEWAY
DX GATEWAY

VPC IP space design
Don’toverlapIPspace
Considerconnectivitytocorporatenetworks
PlanforexpansiontoadditionalAvailabilityZonesorregions
Subnet
Availability Zone A
IPv4 IPv6
OptionallyenableIPv6onAmazonVPC
/56ofAmazon’sGlobalUnicastAddress(GUA)perAmazonVPC
/64CIDRblockpersubnet
IPv6completelyindependentfromIPv4
Enabledpersubnetorperinstance(perENI)
SupportedbySecurityGroups,RouteTables,NACLs,VPCPeering
IGW,DX,FlowLogs,andDNSResolution
Choose A CIDR
/16
/28
(65,536 IPs)
(16IPs)

VPC ID : abc-de-fg-7
Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.3.0.0/16
Secondary CIDR :
10.4.0.0/16
Secondary CIDR :
10.5.0.0/16
Primary CIDR :
10.1.0.0/28
Main Route Table
Destination Target
10.1.0.0/28 Local
10.2.0.0/16 Local
10.3.0.0/16 Local
10.4.0.0/16 Local
10.5.0.0/16 Local
US-WEST-2
VPC resizing
Primary CIDR
10.3.0.0/16
• CIDR Block/s cannot overlap
• Existing CIDR Blocks cannot change
• CIDR block must not be the same or larger than the CIDR
range of a route in any of the VPC route tables

VPC ID : abc-de-fg-7
Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.3.0.0/16
Primary CIDR :
10.1.0.0/28
US-WEST-2
VPC resizing
Primary CIDR
10.3.0.0/16
Secondary CIDR Blocks can be removed
Primary CIDR Blocks cannot be changed
Unchangeable!
Secondary CIDR :
172.16.0.0/16
Secondary CIDR :
192.168.0.0/16 Primary CIDR Range Dictates which other RFC1918 Ranges
can be used
For example, if you use 10.0.0.0/8, then your additional
CIDRs must be from the RFC1918 10. space

Subnet creation
Availability Zone A
Evendistribution ofIPspaceacrossAZs
Useatleast2AZs
SubnetsareAZspecific
Howbig?Howmany?Subnet
Availability Zone B
Subnet
Availability Zone C
Subnet
/20
10.0.0.0/20
10.0.0.0:Networkaddress.
10.0.0.1:ReservedbyAWSfortheVPCrouter.
10.0.0.2:ReservedbyAWS:
10.0.0.3:ReservedbyAWSforfutureuse.
10.0.0.255:Networkbroadcastaddress.
TheIPaddressoftheDNSserverisalwaysthebaseoftheVPCnetworkrangeplustwo

Subnet creation
Availability Zone A
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
/16

VPC subnet design
Traditionalswitchinglimitationsdonotapply
Considerlarge,mixed-usesubnets
Usesecuritygroupstoenforceisolation
Usetagsforgroupingresources
Usesubnetsascontainersforroutingpolicy

IPv4 VPC subnet design
/16
Availability Zone A
Private Subnet
Public subnet
VPN Only Subnet
Availability Zone B
Public subnet
VPN Only Subnet
Availability Zone C
Public subnet
VPN Only Subnet
/22 /22 /22
/20
/20
/20
/20
/20
/20
4091 IPs
1019 IPs
4091 IPs
Private Subnet Private Subnet

What about IPv6 design
Availability Zone A
Public subnet
Private subnet
Availability Zone B
Public subnet
Private subnet
Availability Zone C
Public subnet
Private subnet
/64
/56
/64 /64
/64 /64 /64
18 QUINTILLION
18 QUINTILLION
18 SEXTILLION

INTERNET
ACCESS

Evolving design requirements
One Amazon VPC
One AWS account
One AWS Region
VPN connectivity to private-only VPC
Private IP Egress to Internet
Private IP access to AWS Public Services

LOCAL ROUTING POLICY
Availability Zone A
Private subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
Primary VPC CIDR 10.1.0.0/16
Availability Zone B
Private subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Availability Zone B
Public subnet
VPN only subnet
.1
.1
SECONDARY CIDR 10.2.0.0/16
US-WEST-2

Availability Zone A
Private subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
0.0.0.0/0 igw-a1234567
Availability Zone B
Private Subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Availability Zone B
Public subnet
Private subnet
.1
.1
US-WEST-2
INTERNET
AMAZON
PUBLIC SERVICES
INTERNET GATEWAY
Public subnet routing policy

Availability Zone A
Private subnet
Public subnet
Private subnet
.1
.1
.1
VPC CIDR 2001:db8:1234:1a00::/56
Availability Zone B
Private subnet
Public subnet
Private subnet
.1
.1
.1
US-WEST-2
AMAZON
PUBLIC SERVICESPublic Route Table
Destination Target
10.1.0.0/16 Local
2001:db8:1234:1a00::/56 Local
0.0.0.0/0 IGW
::/0 eigw-0ab0
INTERNET GATEWAY Global Unicast Address (GUA)
INTERNET
EGRESS ONLY

Availability Zone A
Private Subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
172.16.0.0/16 vgw-a1234567
Availability Zone B
Private Subnet
Public subnet
VPN Only Subnet
.1
.1
.1
Availability Zone B
Public subnet
Private subnet
.1
.1
US-WEST-2
REMOTE OFFICE BUILDING
VIRTUAL GATEWAY (VGW)

Routing in the private subnet
Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT instance
Corp CIDR VGW
VPN Only subnet
Availability Zone A
Public subnet
VPN Only Subnet
INTERNET
GATEWAY
INTERNET
NAT INSTANCE
Private subnet Private subnet

Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT instance
Corp CIDR VGW
VPN Only subnet
CORPORATE DATA
CENTER
Availability Zone A
Public subnet
VPN Only subnet
INTERNET
GATEWAY
INTERNET
NAT INSTANCE
NAT INSTANCE
Private Subnet Private subnet

Deploy a NAT gateway
Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-
1
Corp CIDR VGW
VPN Only subnet
CORPORATE
DATA CENTER
Availability Zone A
Public subnet
VPN Only subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
• Still need IGW
• Separate subnets
• Requires EIP
• AZ specific
• Burst to 10 Gbps

Private subnet
Availability Zone A
Private subnet
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-2
Corp CIDR VGW
VPN Only subnet
CORPORATE DATA
CENTER
Availability Zone A
Public subnet
VPN Only subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
NAT GATEWAY

Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-2
Corp CIDR VGW
VPN Only subnet
CORPORATE DATA
CENTER
Availability Zone A
Public subnet
VPN Only subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
NAT GATEWAY

Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-2
Corp CIDR VGW
VPN Only subnet
Availability Zone A
Public subnet
VPN Only subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
NAT GATEWAY

Bring Your Own IP

Bring Your Own IP
IN THE BEGINNING
AMAZON
PUBLIC IPS
DYNAMIC
PUBLIC IPs
ELASTIC
IPs
TODAY
BRING YOUR OWN
PUBLIC IP
NLBEC2 NAT GW
Advertised to the
Internet by AWS
Appears as an
address Pool
Create Elastic IPs
from address Pool

Bring Your Own IP Requirements
Requirements Registeredwithyourregionalinternetregistry(RIR)
Themostspecificaddressrangethatyoucanspecifyis/24.
Youcanbringeachaddressrangetooneregionatatime.
Youcanbring5addressrangesperregiontoyourAWSaccount.
TheaddressesintheIPaddressrangemusthaveacleanhistory.
AmericanRegistryforInternet
Numbers(ARIN)
orRéseauxIPEuropéensNetwork
CoordinationCentre(RIPE).

Private subnet
Internal app
Availability Zone A
AWS Oregon (us-west-2) Region
ROUTE TABLE
0.0.0.0/0 – > NAT GW
Public Subnet
Public Infrastructure
Availability Zone A
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW
NAT GATEWAY (NAT-GW)
ROUTE TABLE
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 IGW
PRIVATE
PUBLIC
IGW
18.219.170.117

1. We have
1. Our Private, Hybrid & Public
Subnets
2. Routes to the Internet & On-
premises
So Where Are We?

Private subnet
Virtual Private
Gateway
Internal apps
Amazon S3 Internet
Internet
Customer
network
Amazon DynamoDB
Availability Zone A
Amazon Kinesis
VPN connection
PUBLIC AWS SERVICES
INTERNET
GATEWAY
NAT
GATEWAY

Private subnet
Internal app
Amazon S3
Amazon DynamoDB
Availability Zone A
Amazon Kinesis
PUBLIC AWS SERVICES
VPC ENDPOINT
GATEWAY
ROIUTE TABLE
GET REQUEST TO AMAZON S3
PUT REQUEST TO DYNAMODB

Private subnet
Internal app
Amazon S3Amazon DynamoDB
Availability Zone A
GATEWAY VPC ENDPOINTS
VPC ENDPOINT
GATEWAY
ROUTE TABLE
GET REQUEST TO AMAZON S3
PUT REQUEST TO DYNAMODB
Route Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
Prefix List for S3 us-west-2 VPC-Endpoint
Prefix List for DynamoDB- us-west-2 VPC-Endpoint
aws ec2 create-vpc-endpoint
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-2.s3
--route-table-ids rtb-2ae6a24f rtb-61c78704
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-
2.dynamodb
aws ec2 describe-vpc-endpoint-services
{ "ServiceNames": [
"com.amazonaws.us-east-1.s3",
"com.amazonaws.us-east-1.dynamodb"
] }
Amazon S3
Amazon DynamoDB
Add Endpoint Hostnames to Security Group Outgoing Rules

{
"Statement": [
{
"Sid": "vpce-restrict-to-backup-bucket",
"Principal": "*",
"Action": [ "s3:GetObject", "s3:PutObject” ],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::reinvent-docs", "arn:aws:s3::: reinvent-docs /*"]
} ] }
{
"Statement": [
{
"Sid": "bucket-restrict-to-specific-vpce",
"Principal": "*",
"Action": "s3:*",
"Effect": "Deny",
"Resource": ["arn:aws:s3:::backups-reinvent", "arn:aws:s3:::backups-reinvent/*"],
"Condition": {"StringNotEquals": {"aws:sourceVpce": "vpce-bc42a4e5” } }
} ] }
Amazon Simple Storage Service
(Amazon S3) bucket policy
VPC Endpoint AWS Identity and Access
Management (IAM) access policy
Private subnet
Internal app
Availability Zone A
VPC ENDPOINTGATEWAY
ROUTE TABLE
GET REQUEST
TO S3
VPC Endpoint
IAM Access Policy
S3
Bucket Policy

{ "Statement": [ {
"Sid": "AccessToSpecificTable",
"Principal": "*",
"Action": [
"dynamodb:Batch*",
"dynamodb:Delete*",
"dynamodb:DescribeTable",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Update*"
],
"Effect": "Allow",
"Resource": "arn:aws:dynamodb:us-east1:123456789012:table/StockTable"
}]}
VPC Endpoint IAM Access Policy
Private subnet
Internal app
Availability Zone A
VPC ENDPOINTGATEWAY
ROUTE TABLE
GET REQUEST
TO DYNAMODB
IAM Access Policy
1. Route table association
2. VPCE policy
3. Bucket policy
4. Security groups with prefix list
Recap on security layers:

1. Route table association
2. VPC-E policy
3. Bucket policy
4. Security groups with prefix list
Recap on Security Layers:

Interface VPC endpoint
P O W E R E D B Y
A W S
P R I V A T E L I N K

INTERFACE
VPC ENDPOINTS
S E R V I C E O W N E D B Y Y O U , O T H E R A C C O U N T S
O R A M A Z O N P A R T N E R S
E n a b l e s p r i v a t e c o m m u n i c a t i o n b e t w e e n
A W S s e r v i c e s u s i n g a n e l a s t i c n e t w o r k
i n t e r f a c e w i t h p r i v a t e I P s i n y o u r
A m a z o n V P C
1 0 . 1 . 1 0 . 5 0
S U B N E T - 1 0 . 1 . 1 0 . 0 / 2 4
Availability Zone A
O N E E N I P E R A Z F O R A S P E C I F I C S E R V I C E
1 0 . 1 . 2 0 . 5 0
S U B N E T - 1 0 . 1 . 2 0 . 0 / 2 4
Availability Zone B
R e q u e s t e r - m a n a g e d
n e t w o r k i n t e r f a c e
1 0 G B P S E A C H I N T E R F A C E E N D P O I N T

S U B N E T - 1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
--vpc-id vpc-ec43eb89
--vpc-endpoint-type Interface
--service-name com.amazonaws.us-east-1.kinesis
--subnet-id subnet-abababab subnet-catbatratsat
--security-group-id sg-1a2b3c4d
S E C U R I T Y G R O U P
S U B N E T - 1 0 . 1 . 2 . 0 / 2 4
Availability Zone B
Amazon Kinesis
1 0 . 1 . 2 . 1 0
1 0 . 1 . 1 . 1 0
v p c - i d v p c - e c 4 3 e b 8 9
k i n e s i s . u s - e a s t -
1 . a m a z o n a w s . c o m
Enable Private DNS Name
o AWS Services
o AWS Marketplace
Services
Kinesis.putRecord
DNS Resolution
Virtual Private Gateway
Customer
network
IPSec VPN
NO ROUTES IN YOUR ROUTE TABLE
SUPPORTS TCP ONLY
aws ec2 describe-vpc-endpoints
vpce-0fe5b17a0707d6abc-29p5708s.kinesis.us-
west-2.vpce.amazonaws.com
ENDPOINT-SPECIFIC REGIONAL DNS HOSTNAME
vpce-0fe5b17a0707d6abc-29p5708s-us-west-
2a.kinesis.us-west-2.vpce.amazonaws.com
vpce-0fe5b17a0707d6abc-29p5708s-us-west-
2b.kinesis.us-west-2.vpce.amazonaws.com
ENDPOINT-SPECIFIC ZONAL DNS HOSTNAME
kinesis.us-west-2.amazonaws.com
PRIVATE DNS ENABLED

//Lambda Handler Function aka Main
exports.handler = (event, context, callback) => {
event.Records.forEach(function(record) {
var esDoc = new Buffer(record.kinesis.data, 'base64');
esDoc = esDoc.toString();
client.index({
index: process.env.esIndex,
id: record.kinesis.sequenceNumber,
type: process.env.esType,
body: {
"Kinesis-Shard-Event-ID":
record.eventID,
"Time-Written-To-Kinesis-Shard":
record.kinesis.approximateArrivalTimestamp,
"Message-Data": esDoc.toString(),
}
},function(err,resp,status) {
console.log(resp);
});
});
};
W r i t e s d a t a f r o m
K i n e s i s S t r e a m t o
E l a s t i c s e a r c h C l u s t e r
P r i v a t e D N S N a m e E n a b l e d
S U B N E T - 1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
1 0 . 1 . 1 . 1 0
v p c - i d v p c - e c 4 3 e b 8 9
k i n e s i s . u s - e a s t -
Kinesis.putRecord
Application Log
Data
Application
Server
ElasticSearch
Cluster
Amazon Kinesis
M a k e s L a m b d a
S e r v i c e A w a r e o f t h e
P U T e v e n t
Writes to ES
Endpoint
VPC Endpoint
IAM Access Policy
S3
Bucket Policy
Writes logs
S3
DNS Resolution
IP add :
10.1.1.10
returned

S U B N E T - 1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
1 0 . 1 . 1 . 1 0
v p c - i d v p c - e c 4 3 e b 8 9
d b A P I . e x e c u t e - a p i . . u s - e a s t - 1 . a m a z o n a w s . c o m
HTTPS GET
DNS Resolution
IP add : 10.1.1.10
returned
App - B
Resource Policy
HTTPS PUT
App- A Apps - C
Virtual Private Gateway
Customer
network
DX Gateway
API Gateway
P r i v a t e D N S N a m e E n a b l e d
v p c - a b c d - 1 2 0 7
u s - w e s t - 2
u s - w e s t - 2
Network Load Balancer
VPC Link

Securing access to Amazon Interface VPC endpoints
S u b n e t 1 0 . 0 . 1 . 0 / 2 4
Availability Zone A
S u b n e t : 1 0 . 0 . 2 . 0 / 2 4
Availability Zone B
S u b n e t : 1 0 . 0 . 3 . 0 / 2 4
Availability Zone C
S e c u r i t y G r o u p
V P C C I D R : 1 0 . 0 . 0 . 0 / 1 6

VPC Endpoint Services
via
amazonprivatelinks

S U B N E T - 1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
v p c - i d v p c - e c 4 3 e b 8 9
Application
Servers
CREATES VPC ENDPOINT SERVICE
WHITELIST ACCOUNTS FOR ACCESS
ASSOCIATE VPC ENDPOINT SERVICE WITH NLB
aws ec2 create-vpc-endpoint-service
--whitelist-account-ids 123456789012,210987654321
--network-load-balancer-ids nlb-aaaaaaaa
SERVICE ARN : aws::us-east-1::service-12345678
SERVICE DNS NAME : service-12345678.vpc-Endpoints.aws
u s - w e s t - 2
1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
S E C U R I T Y
G R O U P
1 0 . 1 . 1 . 1 0
v p c - i d v p c - b e s v p c e v r
d b A P I . e x e c u t e -
a p i . . u s - e a s t -
HTTPS GET
DNS Resolution
IP add : 10.1.1.10
returned
HTTPS PUT
Intranet App- A
u s - w e s t - 2
LISTING AVAILABLE SERVICE OVER VPC ENDPOINTS
P R O D U C E R C O N S U M E R
--vpc-id vpc-ec43eb89
--vpc-endpoint-type Interface
--service-name com.amazonaws.us-east-1.elasticloadbalancing
--subnet-id subnet-abababab subnet-catbatratsat
--security-group-id sg-1a2b3c4d
DNS Names
elasticloadbalancing.us-west-2.amazonaws.com (Z35DVM6FZNQKU5)
General DNS Names
vpce-030344adc43a00bdb-45ltt7jj.elasticloadbalancing.us-west-2.vpce.amazonaws.com (Z1YSA3EXCYUU9Z)
Zonal DNS Names
vpce-030344adc43a00bdb-45ltt7jj-us-west-2b.elasticloadbalancing.us-west-2.vpce.amazonaws.com
vpce-030344adc43a00bdb-45ltt7jj-us-west-2a.elasticloadbalancing.us-west-2.vpce.amazonaws.com
vpce-030344adc43a00bdb-45ltt7jj-us-west-2c.elasticloadbalancing.us-west-2.vpce.amazonaws.com
A c c o u n t
– A
A c c o u n t
– B

S U B N E T - 1 0 . 1 . 1 . 0 / 2 4 Availability Zone A
v p c - i d v p c - e c 4 3 e b 8 9
RDS Microsoft SQL Server
u s - w e s t - 2
1 0 . 1 . 1 . 0 / 2 4
Availability Zone A
S E C U R I T Y
G R O U P
1 0 . 1 . 1 . 1 0
v p c - i d v p c - b e s v p c e v r
d b A P I . e x e c u t e -
a p i . . u s - e a s t -
HTTPS GET
DNS Resolution
IP add : 10.1.1.10
returned
HTTPS PUT
Intranet App- A
u s - w e s t - 2
LISTING AVAILABLE SERVICE OVER VPC ENDPOINTS
A c c o u n t
– A
A c c o u n t
– B
RDS-FAILURE-EVENT
SNS

ONE
AMAZON
VPC
TWO
AMAZON
VPC
THREE
AMAZON
VPC

So why not one big Amazon VPC?
Subnet
Availability Zone A
IPv4
Subnet
Availability Zone B
IPv4
Subnet
Availability Zone C
IPv4
Subnet
Availability Zone A
IPv4
Subnet
Availability Zone B
IPv4
Subnet
Availability Zone C
IPv4

PROD DEV
LOGGING &
MONITORING
Consideration for one or many Amazon VPCs
PCI HIPAA NON-REG. APPS
LEGAL MARKETING SALES
DR

Centralize network connectivity to and from cloud
Centralize management, security, and common
services
Account owners in control of own VPC resources
Many AWS accounts
Many Amazon VPCs
One region

Region
Customer
network
Internal apps
DNS
Directory
Logging
Monitoring
Security
Public apps
14 Amazon
VPCs

HA VPN
to
Amazon VPC
VPC
HA VPN Pair
Availability Zone A
iBGP
eBGP
Customer CIDRs or Default Route
eBGP
AWS ASN 7224
Re-advertise VPC CIDR via IGP
VGW
VPC CIDR
Customer ASN (Public or Private)
CGW1 CGW2
VPN1
Tun1
VPN1
Tun2
Availability Zone A
VPN2
Tun1
VPN2
Tun2
Reuse your CGW Public IP
to connect to more Amazon VPCs
Customer
network
MED
MED
REGION

HUB & Spoke
VPC Peering
VPC
Shared services
VPC
Shared
services
Amazon
VPC
VPC
Customer
network
Spoke Amazon
VPC
REGION
VPC
VPC
VPC
VPC
VPC
VPC
Spoke Amazon
VPC
Spoke Amazon
VPC
Spoke Amazon
VPC
Spoke Amazon
VPC
Spoke Amazon
VPC

VPC Peering
Customer
network
VPC
Hub Amazon
VPC
Private subnet
VPC
Spoke Amazon VPC
Public subnet
10.2.0.0/1610.1.0.0/16
Private subnet
Shared services
10.2.22.0/24
10.1.11.0/24
REGION
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.11.0/24 PCX-1

HQ
OREGON
VPC
Shared services
Amazon VPC
VPC
Customer
network
OREGON REGION
VPC
VPC
VPC
VPC
IRELAND REGION
VPC
Shared services
Amazon VPC
VPC
VPC
VPC
VPC
NOOVERLAPPINGIP
ADDRESSSPACE
SHARED SERVICES
A M A Z O N B A C K B O N E
I N T E R - R E G I O N
V P C P E E R
C R O S S - R E G I O N
P E E R E D C O N N E C T I O N
E N C R Y P T E D
SINGAPORE REGION
VPC
VPC
VPC
Shared services
Amazon VPC
VPC
V P C P E E R

VPC
Shared services
Amazon VPC
VPC
Spoke Amazon
VPC
OREGON REGION
VPC
Spoke Amazon
VPC
VPC
Spoke Amazon
VPC
VPC
IRELAND REGION
VPC
Shared services
Amazon VPC
VPC
Spoke Amazon
VPC
VPC
Spoke Amazon
VPC
VPC
A M A Z O N B A C K B O N E
• N o s u p p o r t f o r s e c u r i t y g r o u p r e f e r e n c i n g o v e r c r o s s -
r e g i o n p e e r i n g l i n k s
• N o s u p p o r t f o r D N S r e s o l u t i o n o v e r c r o s s - r e g i o n p e e r i n g
C u s t o m e r s c a n s t i l l u s e A m a z o n R o u t e 5 3 p r i v a t e
h o s t e d z o n e s t o a c h i e v e t h i s
• N o s u p p o r t f o r i p v 6
• N o s u p p o r t f o r j u m b o f r a m e s
CONSIDER
V P C P E E R

AWS Direct Connect

AWS DIRECT CONNECT
Private subnet
Internal app
Availability Zone A
ROUTE TABLE
AWS Public Services
ACCOUNT - A10.1.0.0/16
vpc-aa-000
Customer Router
PRIVATE VIF
Oregon (us-west-2) Region
DX Location
DX Router
Virtual Gateway
PUBLIC VIF
Amazon
DynamoDB
Amazon S3 Amazon Kinesis Amazon API
Gateway
AWS CloudFormation
1Gigabit Ethernet = Single-mode fiber / 1000BASE-LX (1310nm) transceiver
10 gigabit Ethernet = 10GBASE-LR (1310nm) transceiver
BGP - BGP MD5 (Auth)
10Gbps 10Gbps
LAG
VLAN -1
VLAN -2
20Gbps

us-east-2
VPC
VPC
VPC
VPC
Transit VPC
VPC
us-west-2
VPC
VPC
VPC
eu-west-1
VPC
VPC
VPC
VPC
Transit VPC
VPC
AWS Network
Backbone
Provider
MPLS
Network
Branch Branch
NA
HQ
VPC
VPC
VPC
VPC
VPC
VPC
Chicago DX
AP
HQ
London DX
ap-northeast-1
VPC
VPC
VPC
VPC
Transit VPC
VPC
EU
HQ
Tokyo DX
DO NOT DO THIS

AWS Direct Connect Gateway

AWS Direct Connect Gateway
• # of DX Gateways (Global) : Default: 200
• # of VIF attachments per DX Gateway : Default: 30
• # of VGW associations per DX Gateway : Default: 10
LIMITS
• “Global” Object
• Logical grouping of VGW/VPC attachments and private
virtual Interfaces
• VGWs and VIFs can be in any region
• Provides connectivity between each VIF and all attached
VPCs

VPC
US-EAST-1
DX Connection
VPC
VPC
PRIVATEVIF
VPC
DX Location in Oregon
US-EAST-1
DX Gateway
VPC
VPC
PRIVATE VIF
VPC
EU-WEST-1
VPC
EU-WEST-1
• 1 PVIF Configuration needed to reach multiple VPCs
• No limitation on PVIF Creation
• BGP Session between customer network and GW
• 1 PVIF per VPC VGW
• PVIF limited by Bandwidth
• BGP Session between VGW & Customer Network
Associated

AWS
Direct Connect Gateway
AMAZON VPC
ROUTES
ONLY
CORP NETWORK ROUTES
172.16.0.0/16
PRIVATE VIF
ALL AMAZON VPC
ROUTES
CORPORATE
NETWORK
Main Route Table
Destination Target
10.30.0.0/16 Local
172.16.0.0/16 Local
10.30.0.0/1610.20.0.0/1610.10.0.0/16
Main Route Table
Destination Target
10.10.0.0/16 Local
172.16.0.0/16 Local
Main Route Table
Destination Target
10.20.0.0/16 Local
172.16.0.0/16 Local
10.10.0.0/16
10.20.0.0/16
10.30.0.0/16
172.16.0.0/16
10.10.0.0/16

Oregon DX Location
N.Virginia
Customer Router
Ireland DX Location
Customer Router
Singapore DX Location
Customer Router
Oregon
Singapore
London
Ireland
Germany
ACCOUNT-A
ACCOUNT-B
PRIVATE VIF
HOSTED
PRIVATE VIF
Oregon Canada
Reference
Architecture
ACCOUNT-C
HOSTED
PRIVATE VIF
Central
Oregon
30VIFS
10VGW
200DXGW

EU-WEST-1
PRIVATE VIF
DX Gateway
ACCOUNT-A
ACCOUNT-A
INTER-REGION VPC PEERING
CONNECTION
PRIVATE VIF
Oregon DX Location
ACCOUNT-B
VLAN 1
VLAN 2
Customer Router
US-WEST-2 ACCOUNT-A
SHARED SERVICES VPC
SHARED
SERVICES
AMAZON VPC
US-WEST-2 ACCOUNT-BUS-EAST-2
PRIVATE VIF
Interface Endpoints
3
Ireland DX Location
Customer Router

• Leverages Amazon Global Network Backbone
• Multiple VIF attachments to a gateway
• Multiple VGW/Amazon VPC attachments to a gateway
• VIFs and VGWs can be in any region
• Single account at launch
• VIF, DX Gateway, and VGW must have same account
• VPC CIDRs cannot overlap
• A VGW can only be associated to a single DX Gateway
• AWS VPN CloudHub is not supported
• VPN Failover is supported
Pro & Con: DX gateway

AWS Direct Connect Global Public VIFS
N E W
C R E A T E P U B L I C V I F S T O P U B L I C A W S S E R V I C E S G L O B A L L Y

D X
L O C A T I O N
P U B L I C V I F
N. Virginia
Ohio
N. California
SAO PAULO
Frankfurt
IrelandLondon
A M E R I C A
A S I A
Singapore SydneyMumbaiTokyo Seoul
Oregon
S . A M E R I C A
Montreal
Customer North American Office
B G P A S N
6 5 5 1 5
O h i o R o u t e s A d v e r t i s e d

DX Public Virtual Interface & BGP Community Tags
You can provide BGP Communities to Indicate how far to propagate your prefixes in the Amazon network
You can use the following BGP communities for your prefixes:
o 7224:9100—Local AWS Region
o 7224:9200—All AWS Regions for a continent (for example, North America–wide)
o 7224:9300—Global (all public AWS Regions)
AWS Direct Connect also provides BGP community tags on advertised Amazon routes which enables you to create
filters based on these community tags.
AWS Direct Connect applies the following BGP communities to its advertised routes:
7224:8100—Routes that originate from the same AWS Region
7224:8200—Routes that originate from the same continent
No tag—Global (all public AWS Regions).
The communities 7224:1 – 7224:65535 are reserved by AWS Direct Connect

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
Region
VPC Spoke Amazon VPC
Transit Amazon VPC
VPC
Spoke Amazon VPC
VPC
Spoke Amazon VPC
Transit
Amazon
VPC
EC2 VPN EC2 VPN

Transit Gateway
SIMPLICITY

TransitGatewayIsARegional,NativeAWSService
AllowsYoutoInterconnectThousandsofVPCsThatExistWithinTheSameAccountOrDifferentAccounts
Today,ATransitGatewayconnectstoYourDatacentersviaanIPSecTunnelOnly
Supportsupto 10,000Routes
NetworkSegmentationIsAchieved ByCreatingMultipleRouteTablesinaTransitGatewayandAssociate
VPCs&VPN
On-DemandBandwidthtoMoveLargeAmountsofData
AWS Transit Gateway
InterconnectingVPCsatscale:TransitGatewayisbestsuitedfor
customerswhohavemultipleVPCsandwanttoconnectthem.
Edgeconsolidation:TransitGatewayallowscustomerstosharea
commonVPNacrossalltheirVPCs.
GlobalConnectivity:TransitGatewayscanbepeeredacrossregionsusing
thesecureAWSbackboneallowingcustomerstobuildaglobalnetwork
thatconnectstheirVPCsandon-premisesnetworksworldwide.
What it is Use cases

Private Subnet -A
TGW- ENI
Availability Zone A
ROUTE TABLE
ROUTE TABLE
aws ec2 create-transit-gateway
Whitelist other account(s) using the Cross-account resource sharing API:
create-resource-share --name "Network Ops resource share"
-–principals [‘account-2’, ‘account-3’] //same OU
--resource-arns ["arn:aws:ec2:us-east-1:12345678901:tgw/ tgw-
0ea7775074e8d0683"]
Account-2
Account-3
aws ec2 describe-transit-gateways
tgw-0ea7775074e8d0683
Discover the TGWs being shared aws ec2 create-transit-gateway-vpc-attachment
--transit-gateway-id tgw-
14324bbc412a43243
--vpc-id vpc-2321314314
--subnet-ids subnet-12312312,subnet-
41343432
Associate VPC with the TGWs being shared
aws ec2 describe-transit-gateway-vpc-
attachments
--transit-gateway-id tgw-14324bbc412a43243
--filters “Name=transit-gateway-attachment-
state, Values= pendingAcceptance”
Discover & Accept Associations
How it works
Private Subnet -A
TGW- ENI
Availability Zone A

Routing & The Transit Gateway
TGWsupportsdynamicandstaticroutingbetweenattachedVPCs&VPN
Bydefault,VPCsandVPNsareassociatedwiththedefaultroutetable
RouteSegmentationcanbeachievedbycreatingadditionalroutetablesandassociateVPCsandVPNwithit.
RoutescanpointtoaVPCoraVPNconnection.
Thereare2wayswhereroutesgetpropagatedintheTransitGateway:
o Routespropagatedto/fromon-premises-networks/ Site-to-siteVPN
o Routeswillbepropagated/advertisedbetweentheTGWandyouron-premisesrouterusingBorder
GatewayProtocol(BGP)
o RoutesPropagatedto/fromVPCs.
o WhenyouattachaVPCtoaTransitGatewayorresizesanattachedVPC,theVPCCIDRswillbe
propagatedintotheTransitGatewayroutetableusinginternalAPIs(notBGP).
o RoutesintheTransitGatewayroutetablewillnotbepropagatedtotheVPC’sroutetable.
o VPCownerneedtocreatestaticroutetosendTraffictotheTransitGateway.
aws ec2 create-transit-gateway-route
--transit-gateway-route-table-ids
tgw-rtb-abc3232
--destination-cidr-block 10.1.0.0/16
--target-vpc-id vpc-34234322
CreatingRoutesStaticallyin the TGW

Let’s look at a routing scenario
Customerhas:
ThreeAccounts: [Account–A,Account–B,Account–C]
Three(3)VPCs: [ vpc-aa-00 | vpc-bb-00 | vpc-cc-00 ]
One(1)Datacenter: [ DC–1 ]
Customerneedsto:
InterconnectallthreeVPCs
vpc-aa-00 & vpc-bb-00 Shouldroute ALLInternetRequestthroughaNATGatewayin VPCvpc-cc-00
Establish multipleIPSectunnels toacentralpointandpropagateitsroutestoitsAmazonVPCs
vpc-aa-00, vpc-bb-00 & vpc-cc-00shouldbeabletocommunicatewithusersandresourcesinDC-1aws ec2 create-transit-gateway

Private subnet
Internal app
Availability Zone A
ROUTE TABLE
Private subnet
Internal app
Availability Zone A
ROUTE TABLE
TRANSITGATEWAY
TGW-XYZ-123
ACCOUNT - A
ACCOUNT - B
Customer network
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 TGW-XYZ-123
10.3.0.0/16 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
Transit Gateway Default Route Table
Destination Target Route Description
10.1.0.0/16 Vpc-aa-00 Route to reach VPC - A
10.2.0.0/24 Vpc-bb-00 Route to reach VPC - B
10.3.0.0/24 Vpc-cc-00 Route to reach VPC - C
172.16.0.0/16 IPSEC-VPN-CONN-XXXXXX Route to on-premises DC
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.0.0/16 TGW-XYZ-123
10.3.0.0/16 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
Availability
Zone A
ROUTE TABLE
ACCOUNT - EPrivate Route Table
Destination Target
10.3.0.0/16 Local
10.2.0.0/16 TGW-XYZ-123
10.1.0.0/16 TGW-XYZ-123
10.1.0.0/16
10.2.0.0/16
10.3.0.0/16
vpc-aa-000
vpc-bb-000
vpc-cc-000
DC -1
172.16.0.0/16
Bandwidth per tunnel 1250Mbps
Maximum Tunnels : 30
IPSEC VPN
ECMP

Private subnet
Internal app
Availability Zone A
ROUTE TABLE
Private subnet
Internal app
Availability Zone A
ROUTE TABLE
TRANSITGATEWAY
TGW-XYZ-123
ACCOUNT - A
ACCOUNT - B
Customer network
Private Route Table
Destination Target
10.1.0.0/16 Local
10.30.0.0/16 TGW-XYZ-123
0.0.0.0/0 IPSEC-VPN-CONN-XXXXXX Route to on-premises DC
Private Route Table
Destination Target
10.2.0.0/16 Local
10.30.0.0/16 TGW-XYZ-123
AZ-A
ROUTE TABLE
ACCOUNT - EPrivate Route Table
Destination Target
10.30.0.0/16 Local
10.1.0.0/16 TGW-XYZ-123
10.1.0.0/16
10.2.0.0/16
10.30.0.0/16
vpc-aa-000
vpc-bb-000
SHARED SERVICES -vpc-cc-000
DC -1
172.16.0.0/16
Bandwidth per tunnel 1250Mbps
Maximum Tunnels : 30
IPSEC VPN
ECMP
VPC-A to VPC –B Route Table
DNS
Directory
Logging
Monitoring
Security

Private subnet
Internal app
Availability Zone A
ROUTE TABLE
Private subnet
Internal app
Availability Zone A
ROUTE TABLE
TRANSITGATEWAY
TGW-XYZ-123
ACCOUNT - A
ACCOUNT - B
Customer network
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 TGW-XYZ-123
10.3.0.0/16 TGW-XYZ-123
0.0.0.0/0 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
10.3.0.0/24 Vpc-cc-00 Route to reach VPC - C
172.16.0.0/16 IPSEC-VPN-CON Route to on-premises DC
0.0.0.0/0 Vpc-cc-00 Routed to Palo-Alto ENI
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.0.0/16 TGW-XYZ-123
10.3.0.0/16 TGW-XYZ-123
0.0.0.0/0 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
Availability Zone A
ROUTE TABLE
ACCOUNT - E
Private Route Table
Destination Target
10.3.0.0/16 Local
0.0.0.0/0 Palo-Alto-ENI
10.1.0.0/16 TGW-XYZ-123
10.1.0.0/16
10.2.0.0/16
10.3.0.0/16
vpc-aa-000
vpc-bb-000
vpc-cc-000
DC -1
172.16.0.0/16
IPSEC VPN
ECMP
INTERNET
TGW- ENI
IGW

Private subnet
Internal app
Availability Zone A
ROUTE TABLE
us-eAst-2
Region
Private subnet
Internal app
Availability Zone A
ROUTE TABLE
TRANSITGATEWAY
TGW-XYZ-123
ACCOUNT - A
ACCOUNT - B
Customer
network
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
172.16.0.0/16 IPSEC-VPN-CONN Route to on-premises DC
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.0.0/16 TGW-XYZ-123
172.16.0.0/16 TGW-XYZ-123
10.1.0.0/16
10.2.0.0/16
vpc-aa-000
vpc-bb-000
172.16.0.0/16
DX INTEGRATION
IPSEC VPN
A Z - A
ACCOUNT - B10.22.0.0/16
Transit - vpc-bb-000
A Z - B
PRIVATE
VIF
IPSEC VPN
CROSS REGION PEERING COMING SOON!!!!!!!

Key benefits of Transit Gateway
SimplifiedNetworking
EasytoManageSetUp
HigherVPNBandwidth
Reliability
SinglepointtoconnectVPCs(whethersameaccountoracrossaccounts)andSite-to-SiteVPNsfor
simplifiedmanagement
Reducesthetime tosetupnewVPCsneedingedgeconnectivity
Reduces operationalburdeninmanagingedgeconnectivityforalargenumberofVPCs
OffersthesamereliabilityastherestofAWSplatform.Itusesacellular,scalable,andresilient
platformthatrunswithinAmazon’sproveninfrastructure
Achievebandwidthrangefrom1.2Gbpsto >60Gbps by
leveragingECMPacrosstwoto50Site-2-SiteVPNtunnels
Integration
ControloverinterconnectivitypoliciesbetweenVPCsandon-
premisesnetworkswhichimprovetheirnetworksecurity.
Security
managementandmonitorTransitGatewayswith
CloudFormation,CloudWatchandVPCFlowLogs.

In
conclusion

Thank you!
Androski Spicer
Solutions Architect
Amazon Web Services

From One to Many: Evolving VPC Design (ARC309-R1) - AWS re:Invent 2018

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à From One to Many: Evolving VPC Design (ARC309-R1) - AWS re:Invent 2018

Similaire à From One to Many: Evolving VPC Design (ARC309-R1) - AWS re:Invent 2018 (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

From One to Many: Evolving VPC Design (ARC309-R1) - AWS re:Invent 2018