Many customers, especially those in highly regulated industries, require real-time fraud detection capabilities for their transactional web applications. In most cases, such solutions rely on a physical network tap off of a switch. Since a physical tap is not available in the cloud, an alternative solution is needed. During this session, Larry Grant, Chief Cloud Architect at Vanguard, will share the initial approaches considered along with their pros and cons. Larry will then dive deep into Vanguard’s innovative solution leveraging AWS native services including Amazon Kinesis and Amazon DynamoDB. This highly scalable and resilient solution can scale out horizontally to handle thousands of web servers providing full packet capture for real-time fraud detection use cases.

1. AWS New York Lo- Security Week
Fraud Analy+cs at Cloud Scale
Larry Grant
Chief Cloud Architect, Office of the CTO, Vanguard

2. Agenda
What/Why Fraud Analy=cs
Requirements / Challenges
Vanguard Physical – On Premises Solu=on
Commercial Vendor Solu=ons
Home Grown Ideas
Final Vanguard Solu=on
What’s Next

3. What/Why Fraud Analy<cs
Web Threat Detec=on and Protec=on
Prevent Account Takeover AOacks
Block Automated AOacks
Detect various forms of online fraud (without affec=ng performance)
Use behavioral analy=cs to dis=nguish fraud from legi=mate use
Integra=on with Customer Experience Management Systems to see
the whole customer experience
Legal requirements and Controls

4. Requirements/Challenges
All Data Encrypted in Transit
All Data Encrypted at Rest
10’s/100’s (micro Accounts) of accounts
Mul=-AZ
Mul=-Region
Minimize/eliminate KEY management
Scalable across 100’s or 1000’s of web servers
Guarantee Web Session replay order
Achieve near real-=me streaming of data
Integrate with exis=ng legacy Vanguard Fraud systems

5. App Server
Physical Switch
Physical TAP Switch
DMZ
Private App Network
Private Security Network
l
t
• The Tap Switch is a special de
replay raw network packets
• L2 broadcast over promiscuou
• One or more receivers
Promiscuous
Mode
Network
SSL
Vanguard Physical - On Premises Solu<on
Web Servers
Fraud Analy<cs System

6. ELB
AZ 1 – Subnet 1
Fraud Relay
Server
IP Tables on EC2
to clone traffic
(requires Layer 2)
IP Sec Tunnel encrypts
traffic to des<na<on
fraud analy<cs system
ELB
SSL
SSL
Any AZ / Subnet
Repeat for every Account, AZ, Subnet
Vendor 1
SL
SSL
Physical TAP Switch
Private Security Network (On P
Fraud Analy<cs Sy
Web Servers
App Servers

7. AZ 1 – Subnet 1
IP Sec Tunnel encrypts
traffic to des<na<on
fraud analy<cs system
ENI 2
SSL
SSL
Any AZ / Subnet
Repeat for every Account, AZ, Subnet
Vendor 2
SL
SSL
Physical TAP Switch
Fraud Analy<cs Sy
Agent on
Web Server
Comm
Service
Mgmt.
Server
Private Security Network (On P
Uses non-
routable layer 2
to send a copy
of packet to
relay server
ENI 3
ENI 1
ENI 2
ENI 3
ENI 1
Web Servers
App Servers
ELB
ELB
Fraud Relay
Server

8. AZ 1 – Subnet 1
IP Sec Tunnel encrypts
traffic to des<na<on
Docker Receiver
Container
SSL
SSL
Any AZ / Subnet
Repeat for every Account, AZ, Subnet
Vendor 3
SL
SSL
Docker Agent
Container on
Web Server
Private Security Network (On P
Docker Cluster
(Container Receiver)
Fraud Analy<cs Sy
Web Servers
App Servers
ELB
ELB

9. AZ 1 – Subnet 1
SSL
SSL
Any AZ / Subnet
Home Grown 1
SL
SSL
TCPDump
running as an
agent
Private Security Network (On P
Local AWS CLI script to copy
new S3 files, then TCPReplay to
send them to Fraud System
Fraud Analy<cs Sy
AWS Security Account
Cross Account Instance Profile
AWS CLI copy to S3 or S3FS
SSL
STS temp
creden<als
refreshed every
30 min
AWS CLI pull/copy
from S3
Amazon S3
ELB
ELB
Web Servers
App Servers
SSL

10. AZ 1 – Subnet 1
SSL
SSL
Any AZ / Subnet
Home Grown 2
SL
SSL
Custom agent
storing data to
Kinesis
Private Security Network (On P
Custom service to
pull from Kinesis
Stream and replay to
local Fraud Systems. Separate
thread or process per shard
Fraud Analy<cs Sy
AWS Security Account
Cross Account Instance Profile
Agent Writes packets to Kinesis Stream
SSL
STS temp
creden<als
refreshed every
30 min
Amazon
Kinesis
Put records
(batch)
SSL
Amazon
DynamoDB
Distributed lock
management for
automated HA
Web Servers
App Servers
ELB
ELB
SSL
Promiscuous
Mode
Network

11. Amazon Security
Account
Public Internet Amazon LOB Account
nd:
DMZ
w Existing
Manual
Restart
Client Traffic
Push Kenisis
Pull Kenisis
ProductionVGI DMZ
Business Client
AWS Web Traffic Capture OER
Web
Browser
SSL Cert
Websense
Elastic Load
Balancer
Application
Load
Balancer Web Servers
HTTP-Proxy
Kinesis
1
2
3
5
6
7
1
2
3
2
4
Auto Scaling Cluster
TomCat
App Servers
8
9
Apache
ModProxy
Apache
ModProxy
Network
WAF
3rd
Party WAF
Auto Scaling Cluster
4
Physical
Switch
Web
Browser
Web Servers
Apache
SSL Cert
Physical TAP
AWS
Network
TCP 443/SSL
TCP 8444
SSL
TCP 443/SSL
TCP 443/SSL
Encryption is
terminated on F5
Device. Data going to
APCon is unencrypted
Tap Network devices
must be in
Promiscuous mode.
TCP 443 c
Non-SSL
Receiver
Receiver
Fraud Analytics Vendor 2
Fraud Analytics Vendor 1
6
6
TAP Replay Server
AWS Replay
Service
1
5
VMWare
TAP Switch
VMWare Host
Apache virtual hosts are used to
configure a mod proxy to redirect
unencrypted traffic to a local web
listener on the loop back, so no
data is ever unencrypted to the
host or wire. From the loopback,
a second mod proxy re-encrypts
the data before sending it off to a
separate app server
The agent is sniffing all traffic to
the loop back and filtering based
on port number
3
AWS Privatelink
DynamoDB
Used for Lock Manager. Has
the health monitor to see
who is the primary server for
querying Kinesis
SSL Cert
TCP 443/SSL
TCP 443/SSL
Tap Network
devices must
be in
Promiscuous
mode.
Receiver
Fraud Analytics Vendor 1
TCP 443/SSL
10
Lambda function inserting a
header (TCPClientIP) to identify
endusers IP Address to work
around a bug with Silvertail
Each Web Session,
identified by the high
port, must write to the
same shard for replay
consistency. We use the
high port + uuid for the
partition shard to
ensure even
distribution while
keeping a session fully
intact to a shard
rem Tap
CloudFront
Agent
Lambda IP
Function
5
SSL
SSL
SSL
SSL
SSL
SSL Cert
SSL Cert
SSL Cert
SSL Cert
The replay port
and protocol are
whatever the
original tapped
traffic was
Vanguard On-Premises

12. What’s Next
Lambda Parallel Streaming to any target
• S3
• Redshi`
QuickSight – Business Analy=cs
Athena – Parallel SQL Querying
EMR – Hadoop