Open Source technologies are being widely adopted to help SOC / DevSecOps teams in day to day operations. We'll be showcasing how we've built our SIEM using Apache Metron with a custom SOAR layer - Blitz over it to alert and respond to threats in real time. We'll deep dive into the architecture of both platforms and demonstrate various use cases covering cloud infra, endpoint devices, outbound traffic and perimeter security threats. We'll also present how to automate remediation to alerts and scale the setup for orchestration and threat hunting.

1. SACON 2020

2. SACON 2020
Who are we anyway?

3. SACON 2020

4. SACON 2020
Some subject gyan

5. SACON 2020
We’ve got the stuff!

6. SACON 2020
And…..

7. SACON 2020
Proprietary SIEM challenges
● Device Integration and log parsing
● Log enrichment
● Log correlation from multiple sources
● Cost
● Scaling SIEM components
● Updates
● Customisations

8. SACON 2020
We needed something more…
o ~4000 events per second (this is going up) – parsed, enriched and indexed!
o ~30 million events per day - long term data retention and analytics
o ~150 alerts post correlation
o Horizontally scalable platform
o Attribute level statistical profiling
o Threat visualization
o Threat hunting

9. SACON 2020
Statistical Profiling

10. SACON 2020

11. SACON 2020
Tech Stack - Metron, Homegrown, AWS
ParseIngest Enrich
& Index
STELLAR
Alerting
Framework
Block
Escalate
Scan
JIRA
Alert &
Action
Kafka
Filebeat
Analytics, ML,
Visualisation
AWS Athena
Zeppelin

12. SACON 2020
Scale

13. SACON 2020
Architecture
HTTPD
Firewall
IDS/IPS
VPN
Application
Proxy
Mail
Network
Enrichment
(Storm)
Indexing
(Storm)
Profiler
(Storm)
Blitz
( Alerting &
Ticketing )
Parsing
(Storm)
S3
Elastic
Search
Kibana
Redash
Athena
Zeppelin
AWS
Biz Events
SageMaker

14. SACON 2020
Metron - Enrichment & Profiler
Destination Geo
Profiling
For each
geolocation,
aggregate the
bytes sent out
Profile expiry : 1
day
Check if current
volume is greater
than mean
volume
Lookup
Enrichment
Check if
destination
host/port is
whitelisted or
destination is a
legit SMTP
No
Set alert = True for
the eventPush to indexing
IPS
Logs

15. SACON 2020
Some Use Cases

16. SACON 2020
Objective:
● to flag change in server behavior basis it’s producer / consumer ratio
● to detect server compromise and bypass of security controls
● to detect data exfiltration
Profile Created:
● Server’s Producer / Consumer Ratio
Rules:
● If abs(current avg PCR – previous avg PCR) > 0.5
Change in server behaviour basis PCR

17. SACON 2020
Objective:
● to detect malware injected in server / code
● to detect security control bypass
Profile Created:
● Known User Agents per server or VLAN
Rules:
● Flag any new User Agent that a server or VLAN has never used before
Un-usual User Agents used by a server or VLAN

18. SACON 2020
Objective :
● to detect misconfiguration & security bypass
● to detect insider threat
Profile Created:
● Unique AWS Events triggered per bucket retained over a period of X days
Rules:
● Flag any event triggered for an S3 resource that has not been observed
for it in the past X days
AWS Anomalous S3 Activity

19. SACON 2020
Objective :
● to detect credential compromise
● to detect credential sharing
Profile Created:
● User agent, IP address and geolocation of each user logging in for X days
Rules:
● Alert if a new user agent for the user is observed
● Alert if a new IP address for the user is observed
● Alert if a new geolocation (country) is observed
AWS Anomalous User Login

20. SACON 2020
AWS Anomalous Activity
Objective :
● to detect lateral movement of threat
● to detect misconfiguration and security bypass
Profile Created:
● For all combinations of account, event source and aws_region, profile
events, user agent and geolocation (country)
Rules:
● Alert if a new user agent for the profile is observed
● Alert if a new event for the profile is observed
● Alert if a new geolocation (country) is observed

21. SACON 2020
SOARing over threats

22. SACON 2020
What is this … Blitz ?
Well, now that Metron has helped you identify an anomaly, WHAT IF
● You can get the anomaly alert details on a neatly generated email
/ JIRA ticket?
● You can custom enrich alert information with data from internal
and external sources ?
○ WHOIS info
○ Known Hosting IP
○ Reverse DNS Info
○ Customer Reputation & History

23. SACON 2020
● And have action buttons in the alert itself to respond in real time ?
○ Block IP or source
○ Raise a ticket
○ Forward to concerned team
○ Remediate endpoint

24. SACON 2020
Blitz - Overview
● Open Source incident response automation framework aimed at
accelerating incident triage, tracking and response capabilities
● Ingests device agnostic alert data structured in JSON format
● Enriches alerts and makes them actionable using embedded
custom response buttons
● Easily integrated with Metron using Nifi.
● Code and deployment instructions can be found at :
https://github.com/makemytrip/blitz

25. SACON 2020
Blitz - Architecture
Enrichments
Configuration
Output
Templates
Core Driver
&
Helper
Modules
Alert
Output
Modules

26. SACON 2020

27. SACON 2020
Blitz - Components
● Core Driver (Logic Engine) - processing alert data, calling
enrichments and building output
● Enrichment Engine - Container for enrichment modules that
fetch information from other sources/APIs.
● Device Configuration - Configure enrichments, actions, tokens
for all integrated devices
● Output Templates - Building and adjusting your alert UI
● Output Integrations - choosing SOC alert integration : email,
JIRA, Http, JSON/File

28. SACON 2020
Building Configuration

29. SACON 2020
Routing alert data to Blitz -
Parsing / Deduplicating

30. SACON 2020
Blitz - In Action

31. SACON 2020
Blitz - In Action

32. SACON 2020
Visualizations on Data Lake
Using Redash

33. SACON 2020
Privileged Activity & Sensitive Asset Monitoring
Redash - Data Lake (PAM/SAM Reports)

34. SACON 2020
Redash - Data Lake (WAF Events)

35. SACON 2020
Helium Charts Redash - Data Lake (AWS Events)

36. SACON 2020
Helium Charts Redash - Data Lake (IPS Events)

37. SACON 2020
Helium Charts
Redash - Data Lake (End User Events)

38. SACON 2020
Serverless Security Data Lake
Indexing Topology

39. SACON 2020
Anatomy of a Hunt - ML aided
Security Data
Lake
Feature
Extraction
Model
Persistence/Up
dates
Model Training
Anomaly Detection
&
Analysis

40. SACON 2020
Thank you
Questions? Fire !

41. SACON 2020
Backup Slides

42. SACON 2020
Metron Profiler Explained