Selection criteria for today’s ICS cybersecurity technology presented at S4 2019. Includes: - Recommendations for best practices before evaluating an industrial cybersecurity solution in OT environments - Outline of different ICS cybersecurity technologies such as the differences between active and passive scanning, anomaly detection, threat behavior analytics - What’s important in an industrial control systems cybersecurity platform - Practical guide to pilots and bake-offs To learn more read the whitepaper Key Considerations For Selecting An Industrial Cybersecurity Solution for Asset Identification, Threat Detection, and Response https://dragos.com/resource/key-considerations-for-selecting-an-industrial-cybersecurity-solution-for-asset-identification-threat-detection-and-response/ For more about Dragos and the 2019 S4 Detection challenge, read the blog and watch the video here: https://dragos.com/blog/industry-news/dragos-results-of-s4-industrial-cybersecurity-detection-challenge-contest/ More info: www.dragos.com Follow us on LinkedIn: https://www.linkedin.com/company/dragos-inc./ Follow us on Twitter: https://twitter.com/dragosinc

1. ICS Cybersecurity
Technology Selection
Selection criteria and considerations for today’s
ICS cybersecurity technology
MATT COWELL
mcowell@dragos.com 1/15/19

2. Monitoring software that utilizes threat
analytics to identify threats as they occur
and playbooks for guided investigations
Integrated Solution that has three
components: Threat Hunting,
Incident Response and Training
Threat
Operations
Threat Operations Value:
Services work informs the creation of
playbooks to make analysts efficient
THE DRAGOS OFFERING
Technology, Intelligence, Expertise
Expertise and knowledge in
ICS threat identification and
understanding in the form of
intelligence reports
WorldView Value:
Insights create threat analytics to
drive effective detection with context
WorldView

3. AGENDA
01
02
03
04
Capabilities & Objectives
Technology Overview
Criteria considerations
Summary and recommendations

4. WHY DID WE DO THIS?
Increasing awareness to secure
ICS (Executives, IT, OT)
ICS Technology confusion
Absence of independent testing
& reports.
? Increase in Pilots and proof of
concepts (POC’s)
Receiving many new RFP’s from
potential customers

5. BEFORE EVALUATING TECHNOLOGY
Assess Capabilities Define Objectives
o Asset discovery
o Vulnerability assessment
o Threat hunting
o SOC & IR
o In-house resources/departments
o Out-sourced resources
o Current technology usage

6. ASSESS CURRENT CAPABILITIES
https://www.sans.org/reading-room/whitepapers/ActiveDefense/sliding-scale-cyber-security-36240
https://dragos.com/media/ARCViewDragos-01.pdf

7. “There is no silver bullet. There are
always options and the options have
consequences.” - Mark Horowitz
TECHNOLOGY ALONE IS NOT ENOUGH

8. DEFINE PRIMARY OBJECTIVES
01 TRAINING & AWARENESS
Classes, accreditation, exercises, testing
02 KNOW YOUR NETWORK
Asset discovery, network protocols, internal & external
connections, crown jewel analysis
03 NETWORK SEGMENTATION
Creating zones & conduits, 0 trust networks, firewalling
04 PATCH MANAGEMENT
Device vulnerabilities, vulnerable services, available
patches
05 THREAT DETECTION
Network compromises, C2, untargeted malware,
targeted malware, malicious behavior, insider
06 INCIDENT RESPONSE
07 THREAT INTELLIGENCE
Collection of threat tradecraft, analysis of
tradecraft, IOC’s
08 RETALIATION/OFFENSE
Pursuing adversaries, “hacking back”
Planning, Logs & forensics, imaging, containment,
analysis, recovery/backup

9. ICS CYBERSECURITY TECHNOLOGIES
1. Anti-virus
2. Endpoint security/whitelisting
3. Switches/Firewalls/SDN
4. Data Diodes
5. Deception/Honeypot/Honeynet
6. Asset Discovery & Management
7. Patch Management
8. Anomaly Detection
9. Threat Detection
10. SIEM
11. Removable media scanning
12. Backup/Recovery

10. INDEPENDENT TECHNLOGY EVALUATION
Source: ForresterSource: Gartner

11. DIGITAL BOND – DETECTION CHALLENGE
2018
• 4 x vendors
• Offline analysis - pcap
• 2 phases – Asset Discovery,
Threat Detection
• Unclear scoring system
2019
• 2 vendors (1 open source team)
• Offline analysis – pcap
• Improved criteria and scoring
• More challenging & realistic
objectives (10x larger pcap)
https://s4xevents.com/challenge/
“To identify the capabilities and limitations of the passive monitoring solutions
to create an asset inventory and detect cyber incidents. To identify the market
leaders in these two areas of this highly competitive technology.”

12. NIST NCCOE USE CASES
https://www.nccoe.nist.gov/projects/use-cases
Energy Sector
1. Asset Management
2. Identity and Access Management (IAM)
3. Situational Awareness
Manufacturing Sector
1. Behavioral Anomaly detection

13. PILOTS & BAKE OFF’S
• Evaluate with existing systems
• Recommend controlled environment (non-production ideal)
• Smaller but realistic data set – easier to evaluate
• Define evaluation scope & time period
• Live or PCAP offline analysis (lower cost alternative)
• Evaluation include:
• Technology deployment process
• Product support
• Documentation
• Capabilities based on YOUR data
• Requires defined criteria to evaluate

14. ICS CYBERSECURITY TECHNOLOGIES
1. Anti-virus
2. Endpoint security/whitelisting
3. Switches/Firewalls/SDN
4. Data Diodes
5. Deception/Honeypot/Honeynet
6. Asset Discovery & Management
7. Patch Management
8. Anomaly Detection
9. Threat Detection
10. SIEM
11. Removable media scanning
12. Backup/Recovery

15. PRE-REQUISITES
• Network Architecture - IP networks, segments, throughput, serial
networks
• Network Infrastructure – Available SPAN ports, TAP’s, switch
capacity, firewall rules
• Physical Access – restricted locations, change processes
• Environmental – power, mounting, temperature etc.
• Stakeholders - Relevant IT & OT contacts

16. RECOMMENDED CRITERIA CATEGORIES
1. Architecture and Deployment
2. Collection/Ingestion
3. Asset Inventory
4. Detection
5. Response
6. User interface/Ease of use
7. Management
8. Reporting
9. 3rd party integrations
10. Commercial
11. Support
12. Advanced user

17. EXAMPLE CRITERIA 1
1. Architecture and Deployment
• On prem, Cloud
• Hardened, Enterprise
• Agent, Network sensor
2. Collection
• Passive/Active monitoring
• SPAN, PCAP, Log’s
• Max. throughput (scale)
3. Asset Inventory
• IP, MAC, Name
• Device type characterization
• OS Fingerprint
4. Detection
• Anomaly/Change detection
• Known malicious behaviors
• IOC & YARA detection

18. ACTIVE VS. PASSIVE DISCOVERY
ACTIVE PASSIVE
PROS Specific queries on demand No risk of disruption to operations
Quicker results Observe peer to peer comms
More product details Thorough threat detection
CONS Potential service disruption Results take time
Unsupported by vendors Requires SPAN ports/TAP’s
Limited threat detection Visibility dependent upon location

19. EXAMPLE CRITERIA 2
5. Response
• Case management
• Dataset querying
• Playbooks/guidance
6. User Interface
• Map visualization
• Dashboards
• Command line
7. Management
• Role based access
• Status monitoring
• Patching
8. Reporting
• Report format type
• Asset inventory
• User activity

20. EXAMPLE CRITERIA 3
9. Integrations
• Asset enrichment
• Events/notifications (SIEM)
• Network level actions
10. Commercial
• Hardware costs
• Licensing
• Maintenance
11. Support
• User guides
• Application support
• Online training
12. Advanced/Power user
• Custom data filtering
• Scripting data (i.e. python)
• Custom analytics

21. RESOURCES
Whitepaper: Key Considerations for Selecting an
Industrial Cybersecurity Solution for Asset
Identification, Threat Detection, and Response
https://dragos.com/resource/key-considerations-for-
selecting-an-industrial-cybersecurity-solution-for-asset-
identification-threat-detection-and-response/

22. RESOURCES
RFP template: Suggested evaluation criteria for
selecting an industrial cybersecurity platform.
Available soon

23. CONCLUSION
1. Obviously bias in the suggested criteria but useful data points when
combined with other sources to find what's right for YOU.
2. Align technology requirements to existing capabilities and end goals
3. Establishing evaluation criteria against YOUR objectives is essential
before you begin evaluating technology.
4. Importance of testing technology alongside YOUR existing systems &
data is a true test of the value of the solution but understand how it
scales.
5. Testing criteria is also useful in defining an effective RFP

24. Thank you
Questions?
mcowell@dragos.com
@m_p_cowell