In this session you will learn how AWS Deep Learning can be used to automate security operations and build a machine learning system to protect your applications. This session focuses on early detection and containment of security threats, and shows you how to use semi-supervised machine learning to continuously contain these threats.

1. S U M M I TS U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Automate security event management
using trust-based decision models
Deenadayaalan Thirugnanasambandam
Senior Cloud Architect
Amazon Web Services Proserv
Vinod Subramaniam
Senior Cloud Advisory
Amazon Web Services Proserv

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Automate
Key take away

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What to expect
Build security event management system using rules based approach
Build machine learning pipeline for security usecases
Improve anomaly detection using machine learning

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Today’s use case
Zooropa Enterprise wants to host an internet facing SFTP Server on EC2 for
Business to Business (B2B) file transfer
Events/Attributes Rule Based Scenario
Location (IP) of user User Alice accessing from an IP range: xx.yy not
in approved database
Authentication successes and failures Knowledge Database of rules
1. More than 2 unsuccessful attempts today
2. User successfully logged in during holidays
3. Excessive user activity during long weekend

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What is the business problem?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Protect system from insider attack
Protect system from external attack

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Where do we look for?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What do we look for?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Application Errors and Failures
Authentication & Authorisation patterns
Location of User
Access pattern – high risk functionality,
excessive usage

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Decision Management System – Rule Based
Amazon
CloudWatch
Amazon EC2
AWS Lambda
AWS
CloudTrail
Amazon
GuardDuty

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we prepare data?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Logs formatterAmazon
CloudWatch
Amazon
S3

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we prepare data?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Logs formatter
Enhance data
Amazon
CloudWatch
Amazon
S3
Data enhancerAmazon
S3

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we write rules?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Rule 1: User Alice accessing from an IP range
a.b.c.d/y not in approved database
Limitations of rules
1. A database of known IP ranges needs to be
maintained
2. Writing rules customised for a user is
cumbersome and not scalable

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we write rules?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Rule 2:
a) More than 2 unsuccessful attempts
today
b) User successfully logged in during
holidays
c) Excessive activity during long weekend
Limitations of rules
1. All such scenarios will need to be discovered
and rule written
2. No ability to detect and respond to a new
scenario or un-discovered scenario

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Decision management system – rule based
Amazon
CloudWatch
Amazon EC2
AWS Lambda
AWS Step Functions workflow
AWS
CloudTrail
Amazon
GuardDuty

15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Today’s use case
Zooropa Enterprise wants to host an internet facing SFTP Server on EC2 for
Business to Business (B2B) file transfer
Events/Attributes
Limitations of rule based
approach
Location (IP) of user 1. A database of known IP ranges
needs to be maintained
2. Writing rules customised for a
user is cumbersome and not
scalable
Authentication successes
and failures
1. All such scenarios will need to
be discovered and rule written
2. No ability to detect and respond
to a new scenario or un-
discovered scenario
ML based scenario
Detect anomalous login based on
attributes like historical login data +
1. IP, UserID
2. UserID, time of login
3. UserID, login attempts

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Choose the right tool for the job
1
2
3
Clustering and
segmentation
Anomaly/Outlier Association Rule
Mining
Prediction
SVM – One
Classifier
IP Insights K Means,
K NN
Isolation Forest -
RCF

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How we prepare data?
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
User ID Timestamp Client IP
deenadat 19/Feb/2019:23:29:30 54.240.193.1
Alice 6/Feb/2019:23:19:12 200.240.193.1
deenadat 6/Mar/2019:18:49:48 54.240.193.1
Alice 9/Mar/2019:18:22:25 200.240.193.1
User ID Date Value
deenadat 19/Feb/2019 4
Alice 6/Feb/2019 20
deenadat 6/Mar/2019 4
Alice 9/Mar/2019 15
IP
Insights
RCF

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Build the model
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Hyperparameter : Model
Entity Vectors : 2x Unique values

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Build the model
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
Hyperparameter : Model
Vector dimensions : 128

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Evaluate the model
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
IP Insights
model
IP : 200.240.193.1
UserID : Alice
Get anomaly
score

22. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Model deployment
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
IP Insights
model

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Decision management system – ML
Amazon
CloudWatch
Amazon EC2
AWS Lambda
AWS Step Functions workflow
AWS
CloudTrail
Amazon
GuardDuty

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Model evaluation for anomalous data
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment RCF

27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Model tuning
Business
Understanding
Data Understanding
Data Preparation
ModellingEvaluation
Deployment
from sagemaker.tuner import HyperparameterTuner,
IntegerParameter
# Configure HyperparameterTuner
ip_insights_tuner = HyperparameterTuner(
estimator=ip_insights, # previously-configured
Estimator object
objective_metric_name='validation:discriminator_auc',
hyperparameter_ranges={'vector_dim':
IntegerParameter(64, 1024)},max_jobs=4,
max_parallel_jobs=2)

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Retrain the model
Amazon
CloudWatch
Amazon EC2
AWS Lambda
AWS Step Functions workflow
AWS
CloudTrail
Amazon
GuardDuty

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What did we just learn?

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Try it out

31. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deenadayaalan Thirugnanasambandam Vinod Subramaniam