Scalability with Omni Distributed Analysis Platform

Scalability and OmniPeek

Jim MacLeod Jay Botelho
Show us your tweets!
Product Manager Director, PM Use today’s webinar hashtag:
WildPackets WildPackets #wp_scalability
jmacleod@wildpackets.com jbotelho@wildpackets.com with any questions, comments, or feedback.
Follow me @shewfig Follow me @jaybotelho Follow us @wildpackets

© WildPackets, Inc. www.wildpackets.com

Administrivia
• All callers are on mute
‒ If you have problems, please let us know via the Chat window
• There will be Q&A at the end
‒ Feel free to type a question at any time
• Slides and recording will be available:
‒ Via a follow-up email

#wp_scalability © WildPackets, Inc. 2

Agenda
• Scalability and network analysis
• Challenges with high-speed networks (10G/40G/etc)
• Best practices in network analysis scalability
• Eliminate choke points without eliminating visibility
• Distributed network analysis scenarios
• About WildPackets
• Product line overview


Scalability and Network Analysis

4

Scalability
• The most overloaded term in IT???
• Scalability on 2 levels
‒ Network scalability
‒ Network analysis scalability
• Scalability is the ability of a [network] [network
analysis solution] to
‒ handle a growing amount of work in a capable manner or
‒ its ability to be enlarged to accommodate that growth1
• The key – “growing amount of work”

1 AndréB. Bondi, 'Characteristics of scalability and their impact on performance', Proceedings of the 2nd international workshop
on Software and performance, Ottawa, Ontario, Canada, 2000, ISBN 1-58113-195-X, pages 195–203


Network Growth
• Driving network growth:
‒ Number of users/connections
‒ Number of applications
‒ Amount of network traffic
‒ New technologies
‒ New data types
‒ New locations
• Network growth drives network analysis growth:
‒ Increased analytical throughput
‒ Increased analytical scope
‒ Increased data storage
‒ Distributed analysis


A Classic Example – The 10G Transition

Alerts/ User
Let It Roll!
Alarms Complaints

NO
Problem?

YES

Connect the Reproduce
Start a Trace
Analyzer if Necessary


1Gig Is Easy - Now
• Use almost any NIC
• Use almost any computer
• Capture and analyze all in real-time
• Little or no special hardware needed (taps, etc.)
• Little to no impact on existing network traffic
• “Analysis on the fly” still feasible

#wp_scalability © WildPackets, Inc.

Challenges with High-Speed
Networks

9

Challenges with Network Analysis
• Capturing at high speeds
‒ Hard to do without dropping packets
‒ Hard to analyze in real time
‒ Hard to store at high speed
• Analyzing increasing volumes of data
‒ Needle in a haystack
‒ Where to store it
‒ How to access it
• More links, less overall visibility
‒ Easier to add links than capture points


How Is 10G Being Utilized?
Example 1: Heavy Mfg Example 2: Cisco
• Major traffic driver: backups • Focus now on large, flat
• Current challenge: 2x and 4x 10G data center fabrics
1G EtherChannel on backup • Fabric Path / TRILL
servers is saturating “standard”
• New architecture spec for 10x • Nexus 7000 offers 32 ports
1G EtherChannel of 10G
• What’s coming: virtualized • Driving need: Constant
server clusters growing – one demand for 1G aggregation
has 360 VMs!


Strategy for Monitoring 10G Ethernet
Which of the following apply to your strategy for monitoring 10G
segments? (Select all that apply)

Our tools don't support 10G
41.1%

Want to keep 1G tools as long as possible 32.9%
Can't afford upgrading tools to 10G 21.9%

All set - tools already support 10G 21.9%
All set - our mirroring sol'n converts 10G to
1G
14.4%

Other 4.1%

0%
20%
SOURCE: Benchmarking Network and Security Operations: Tools, Processes, and
40%
Enabling Technologies Study, 2009, Enterprise Management Associates. n=124
60%

The 10G Challenge
• Traditional NICs not up to the task
• Processing power is a limiting factor
• Storage capacity is a limiting factor
• I/O bus and disk write speeds are a limiting factor
• 10G forces clarity in analysis
• At 10G, it truly is looking for a needle in a haystack
• 40G is another leap for network analysis


10G Compromises
• 10G to 1G taps
• Apply pre-capture filters or triggered captures to
selectively stream to disk
• 10G NIC upgrades in architectures designed for
multi-port 1G deployments


10Gig Network Analysis Workflow

Identify Key Deploy 24x7 Alarms/
Analysis Pts Monitoring Alerts

NO
Problem?

YES

Rewind Tune if
Analyze
Data Necessary


Changing Methods - Data Recorders
1. Alerts/alarms from network
monitoring appliance
3. Real-time and post-
incident analysis

IDS/IPS System

Servers
2. Incident data recorded and
ready for analysis


Meeting the 10G Challenge – TimeLine
• Fastest network recording and real-time statistical
display — simultaneously
‒ 12Gbps sustained capture with zero packet loss
‒ Network statistics display in TimeLine visualization format
• Rapid, intuitive forensics search and retrieval
‒ Historical network traffic analysis and quick data rewinding
‒ Several pre-defined forensics search templates making
searches easy and fast
• A natural extension to the WildPackets product line
• Turnkey bundled solution
‒ Appliance + OmniEngine, OmniAdapter, OmniPeek Connect


Best Practices in Network Analysis
Scalability

18

Network Analysis Scalability
• Architecture for sustainable growth
‒ Pod design to horizontally scale compute resources
‒ Extensible backbone or meshed core
‒ Automation, e.g. DHCP  IPAM
• Openness to new technologies
‒ Higher speed interfaces, e.g. 10G
‒ Non-hierarchical / multi-path, e.g. Fabric Path / TRILL
• Resource planning – Start with Network Analysis
‒ Baseline measurements as a starting point
‒ Track growth in traffic pattern


Best Practices
• Be ready
‒ Be specific regarding your analysis requirements
‒ Know your network
‒ Analyze the essentials
• Know your limits
‒ Filter and slice (whenever possible)
‒ Anticipate hardware resource needs
• Be reasonable
‒ Equipment to capture everywhere (CapEx)
‒ Human time required for analysis (OpEx)


Challenges in Scalability
• Choke points
‒ Will higher speed links suffice?
‒ Will your equipment support those links?
• Mesh
‒ Do you know where your packets are?
‒ Can you still capture essential traffic?


Architect for Visibility
• Know what traffic is important
‒ Are you looking at servers?
‒ Are you looking at user PCs?
‒ Are you looking at administrative protocols?
• Know where to find that traffic
‒ High-Speed capture for high-speed links
‒ Multiple capture points for meshed networks
• Build visibility into the infrastructure
‒ Manual placement of portables increases MTTR
‒ Pre-set capture points provide instant information


Backbone Visibility
• Capturing isn’t enough
‒ Lots of data = big haystack
• High speed is typically aggregated low-speed
‒ Inter-switch links or VM host w/ multiple guests
‒ Very few 1Gbps+ flows
‒ Lots of slower flows from different nodes
• Not just more bandwidth
‒ More PPS, more nodes, more flows


Scale Visibility
• Add packet capture to each pod
‒ Uplink for North-South traffic
‒ Internal visibility for East-West traffic
• APM: slow traffic between servers affects user experience
‒ Soft taps for VM servers
• See the inter-VM traffic
• Monitor high-speed links
‒ Inter-switch links have aggregated traffic
‒ Good visibility with reduced complexity and cost
• Monitor meshed backbones
‒ Capture at many points
‒ Aggregate on the console


Full Packet Path Visibility
• Pre-install multiple capture points
‒ Cheaper to install a capture server than to send an engineer
‒ Data moves faster than people: reduce MTTR
• Take it to the next level
‒ Multi-Segment Analysis
‒ Identify slow and lossy links
• Multiple simultaneous captures show delays and loss
• Leverage the power of infrastructure-based capture
‒ Faster MTTR for simple problems
‒ End-to-end analysis for more complicated issues


TimeLine for High Speed
• Hard to capture with commodity hardware
‒ OmniAdapter: 10Gbps
‒ Commodity card & driver: 3Gbps
• Hard to record at speed
‒ “Fast” hard drive is 6Gbps
‒ “Fast” network is 10Gbps+
‒ TimeLine for dedicated 10G capture at speed
• Hard to store high-speed traffic
‒ 10Gbps links contain lots of traffic
‒ Need many TB to provide a reasonable analysis window
‒ TimeLine includes up to 48TB


Understanding High Speed
• Identifying traffic on aggregated links
‒ Potentinally 1000s of servers
‒ Each server 1000s of client connections
• Needs automation
‒ Historical method: SNMP alerts and user complaints
‒ Distributed problem may not be apparent from single source
‒ Must look at the flows themselves, packets don’t lie
• Find the problem
‒ Requires synergy of filtering and visualization
‒ Leverage filters with triggers and alerts


Eliminating Choke Points
without Eliminating Visibility

28

Are Choke Points bad?
• Foundation of classic network architecture
‒ Know the packet path from the network tree
• A single link is “easy” to upgrade
‒ Higher speed, e.g. 1Gbps to 10Gbps
• Provides a point of traffic control
‒ Firewalls, IDS/IPS, DLP, etc.
• Still single point of failure
‒ If it goes down, large impact on business


Alternatives to Choke Points
• Bonded links
‒ Originally for speed, e.g. LACP
‒ Now redundancy among multiple switches, e.g. VPC
• Active-active redundancy
‒ Routing: ECMP
‒ Switching: FabricPath, TRILL, SPB
• “Virtualized network”
‒ MPLS/VPLS – not just for WAN
• Now it’s unclear what the packet path is


Visibility in Mesh Environments
• More links = more potential packet paths
‒ Result: out-of-order delivery?
• Common practice: deterministic load splitting
‒ Hash L3 (sometimes L4) to determine which link to use
‒ Maintain continuity among nodes, or within each flow
• Visibility options
‒ Use switch SPAN ports to aggregate
• Classic bandwidth problem: 2 data links  1 monitoring link
‒ Tap each link
• Additional cost per link
‒ “Key” points, plus agent on end nodes
• Aggregate multiple captures on the console

Distributed Network Analysis
Scenarios

32

Distributed Analysis


Distributed Monitoring


Q&A

Show us your tweets!
Use today’s webinar hashtag: Follow us on SlideShare!
Check out today’s slides on SlideShare
#wp_scalability www.slideshare.net/wildpackets
with any questions, comments, or feedback.
Follow us @wildpackets


WildPackets Corporate Overview

Optimizing Network and Application Performance


Corporate Background
• Experts in network monitoring, analysis, and troubleshooting
‒ Founded: 1990 / Headquarters: Walnut Creek, CA
‒ Offices throughout the US, EMEA, and APAC

• Our customers are leading edge organizations
‒ Mid-market and enterprise lines of business
‒ Financial, manufacturing, ISPs, major federal agencies,
state and local governments, and universities
‒ Over 7,000 customers / 60+ countries / 80% of Fortune 1,000

• Award-winning solutions that improve network performance
‒ Internet Telephony, Network Magazine, Network Computing awards
‒ United States Patent 5,787,253 issued July 28, 1998
• “Apparatus and Method of Analyzing Internet Activity”


Business Opportunity
• Growing $5B network management market (Gartner)

• VoIP, video, and other converged networking
applications are saturating network bandwidth

• Increasing network speeds create a discontinuity
‒ 1 Gig  10 Gig  40 Gig  100 Gig networks

• Users and business will not tolerate downtime
Need for better real-time network visibility in addition to
network forensics and DPI


Unprecedented Network Visibility

NETWORK HEALTH
GLOBAL WatchPoint can manage and report on key
devices’ performance and availability across
the entire network, from anywhere on the network.

UNDERSTAND END-USER PERFORMANCE
TimeLine and Omnipliance network recorders monitor
DISTRIBUTED and analyze performance across critical network
segments, virtual environments, and remote sites.

PINPOINT NETWORK ISSUES ANYWHERE
Omnipliance Portable can rapidly identify and troubleshoot
PORTABLE issues before they become major problems—wired or
wireless—down the hall or across the globe.

ROOT-CAUSE ANALYSIS
OmniPeek network analyzer performs deep packet inspection
DPI and can reconstruct all network activity, including e-mail and
IM, as well as analyze VoIP and video traffic quality.


A History of Innovation

2001 2005 2009 2011
• First 802.11 Combined distributed Innovative dashboard • Total visibility with
wireless analyzer network and VoIP with drill-down for VoIP zero packet loss
• First network network analysis and video • First wireless
analyzer with network analyzer to
automated expert support capture and
analysis analysis of 802.11n
3-stream wireless

2003 2008 2010 2012
Distributed real-time Enterprise-wide First to achieve 11 Gbps • Capture, record, and
troubleshooting Monitoring and Reporting sustained capture-to-disk analyze from 40G
network segments
• First wireless network
analyzer to support
801.11ac, k, r, u, v, w


Real-World Deployments
Education Financial Government

Health Care / Retail Telecom Technology

WildPackets Proprietary

Product Line Overview


Omni Distributed Analysis Platform
OmniPeek
Enterprise Packet Capture, Decode and Analysis
• Ethernet,1/10 Gigabit, 802.11, and voice and video over IP
• Portable capture and OmniEngine console
• Aggregate analysis data across multiple capture points

Omnipliance / TimeLine
Distributed Enterprise Network Forensics
• High-performance packet capture and real-time analysis
• Stream-to-disk for forensics analysis
• Integrated OmniAdapter network analysis cards up to 40G

WatchPoint
Centralized Enterprise Network Monitoring Appliance
• Aggregation and graphical display of network data
• WildPackets OmniEngines
• NetFlow and sFlow

Omni Distributed Analysis Platform
Software and Turnkey Solutions
• Enterprise monitoring and reporting
‒ WatchPoint Server
‒ OmniFlow, NetFlow, and sFlow Collectors
• Software probes and network recorders
‒ Omnipliance network recorders – Edge, Core
‒ TimeLine network recorders
‒ OmniAdapter analysis cards
• Distributed analysis software
‒ OmniPeek – Enterprise, Professional, Basic, Connect
‒ OmniEngine – Enterprise, Desktop, OmniVirtual
• Portable solutions
‒ OmniPeek network analyzer
‒ Omnipliance Portable

Key New Features in v7
• 40G network support
• Analyze issues from end to end:
Multi-Segment Analysis (MSA)
• Collect data from non-technical end users:
OmniPeek Remote Assistant (ORA)
• Single, interactive dashboard for
utilization, top talkers, top protocols,
latency, Experts, flows, and wireless
signal strength
• New wireless specifications
‒ 802.11ac 802.11k
‒ 802.11r 802.11u
‒ 802.11v 802.11w


OmniPeek Network Analyzer
• Distributed analysis manager
– Connect to and configure distributed OmniEngines, Omnipliances,
and TimeLines
• Comprehensive dashboards present network traffic in real-time
– Vital statistics and graphs display trends on network and application
performance
– Visual peer-map shows conversations and protocols
– Intuitive drill-down for root-cause analysis of performance bottlenecks
• Visual Expert diagnosis speeds problem resolution
– Packet and payload visualizers provide business-centric views
• Automated analytics and problem detection 24/7
– Easily create filters, triggers, scripting, advanced alarms, and alerts


Omnipliance Network Recorders
• Captures and analyzes all network traffic 24x7
– Runs WildPackets OmniEngine software probe
– Generates vital statistics on network and application performance
– Intuitive root-cause analysis of performance bottlenecks
• Expert analysis speeds problem resolution
– Fault analysis, statistical analysis, and independent notification
• Multiple issue digital forensics
– Real-time and post capture data mining for compliance and troubleshooting
• Intelligent data transport
– Network data analyzed locally
– Detailed analysis passed to OmniPeek on demand
– Summary statistics sent to WatchPoint for long term trending and reporting
– Efficient use of network bandwidth
• User-extensible platform
– Plug-in architecture and SDK

TimeLine Network Recorder
• Continuous network recording and comprehensive
real-time statistical display — simultaneously
‒ 12Gbps sustained capture with zero packet loss
‒ Network statistics display in TimeLine visualization format
• Rapid, intuitive forensics search and retrieval
‒ Historical network traffic analysis and quick data rewinding
‒ Several pre-defined forensics search templates making
searches easy and fast
• A natural extension to the WildPackets product line
• Turnkey bundled solution
‒ Appliance + OmniEngine, OmniAdapter, OmniPeek Connect


WildPackets Network Recorders
Price/Performance Solutions for Every Application

Portable Edge Core TimeLine
Ruggedized Small Networks Datacenter Workhorse Enterprise, Highly-
Troubleshooting Remote Offices Easily Expandable Utilized Networks
Aluminum chassis / 17” LCD 1U rack mountable chassis 3U rack mountable chassis 3U rack mountable chassis
Dual 2.13 GHz Quad-Core Intel Quad-Core Intel Xeon X3460 Dual Intel Xeon Quad Core Dual Intel Xeon Quad Core
Xeon L5630 "Westmere" 2.80Ghz E5530 2.4GHz X5560 2.8GHz
24GB RAM 4GB RAM 6GB RAM 18GB RAM
2 PCI-E Slots 2 PCI-E Slots 4 PCI-E Slots 4 PCI-E Slots
2 Built-in Ethernet Ports 2 Built-in Ethernet Ports 2 Built-in Ethernet Ports 2 Built-in Ethernet Ports
6TB SATA storage capacity 1TB SATA storage capacity 8/16TB SATA 8/16/32/48TB SATA
storage capacity storage capacity
4.5Gbps CTD 1.1Gbps CTD 3Gbps CTD 12Gbps CTD


WatchPoint
Centralized Monitoring for Distributed Enterprise Networks

• High-level, aggregated
view of all network
segments
– Monitor per campus, per
region, per country
• Wide range of network
data
– NetFlow, sFlow, OmniFlow
• Web-based, customizable
network dashboards
• Flexible detailed reports
• Direct link to detailed,
packet-based analysis


Comprehensive Support and Services
Standard Support Premier Support
 Maintenance and upgrades  24 x 7 x 365
 Telephone and email contacts  Dedicated escalation manager
 Knowledgebase  2 customer contacts per site
 MyPeek Portal  Plug-in reconfiguration assistance

WildPackets Training Academy
 Public, web-based, and on-site classes
 Complete curriculum: technology and product focused
 Practical applications and labs covering network analysis,
wireless, VoIP monitoring and advanced troubleshooting

Consulting and Custom Development Services
 Deployment, configuration, and assessment engagement
 Systems integration and testing
 Application integration, driver, decode, interface development


24x7 Network Monitoring,
Analysis, and Troubleshooting


WildPackets Key Differentiators
• Visual Expert intelligence with intuitive drill-down
– Let computer do the hard work, and return results, real-time
– Packet /payload visualization is faster than packet-per-packet diagnostics
– Experts and analytics can be memorized and automated
• Automated capture analytics
– Filters, triggers, scripting, and advanced alarming system combine to provide
automated network problem detection 24x7
• Multiple issue network forensics
– Can be tracked by one or more people simultaneously
– Real-time or post capture
• User-extensible platform
– Plug-in architecture and SDK
• Aggregated network views and reporting
– NetFlow, sFlow, and OmniFlow


Thank You!

WildPackets, Inc.
1340 Treat Boulevard, Suite 500
Walnut Creek, CA 94597
(925) 937-3200


Scalability with Omni Distributed Analysis Platform

Recommended

Recommended

More Related Content

More from Savvius, Inc

More from Savvius, Inc (20)

Recently uploaded

Recently uploaded (20)

Scalability with Omni Distributed Analysis Platform