The document summarizes past surveys on tools used by Network Operations Centers (NOCs) and outlines plans for a third survey. It reviews the first two surveys from 2011 and 2016, which covered NOC responsibilities, functions, and tools. It then discusses the structure and content planned for the third survey, including the key areas to cover, functionalities to include, and specific tools to ask about for each functionality based on responses from previous surveys. The goal is to gather input from NOCs on the most important and effective tools they use to fulfill various responsibilities and functions.

1. SIG-NOC Tools Survey 2019
Maria Isabel Gandía Carriedo
9th SIG-NOC Meeting
ARNES, Ljubljana, 09-04-2019

2. 1st NOC Survey: 2011
Run from July to October 2011, published in December.
Wide scope covering:
• NOC taxonomy
• Network and Services
• NOC tools (per 14 functionalities)
• Communication and front end
• Collaboration and best practices
54 questions.
43 valid answers.
Many open text boxes that didn’t do the great job they were
supposed to do.

3. 2nd NOC Tools Survey
Run from December’15 to February’16, published in June’16.
Focused on Tools according to 15 functions that the NOCs
may be responsible for, including DDoS mitigation.
One section about standards and industry best practices as
well as training activities.
Tools scored in two ways: Importance / Quality.
We included the tools that had more than one answer in the
last survey and also a generic "in-house" (asking the
possibility of open-sourcing it) and "other".
Answerable via “click-click”.
“35” questions (we nearly doubled the number of questions,
up to 66, to make it “SurveyMonkey friendly”).
64 valid answers.

4. What we will review today: the 3rd SIG-NOC Tools Survey
 For all the survey:
• The “big blocks” and how to sort them:
– Basic information
– Tools
– Standards
• Missing questions or answers
 The functionalities we will include.
 For each functionality:
• The name of the functionality
• The definition
• The tools that we will include
• Missing tools

5. Basic information
1. Name (acronym) of your organisation *
2. Type (range) of the network that your organisation is responsible for*
(Please choose all that apply):
• Wide area network among several countries
• National research and education network (NREN)
• Regional metropolitan network
• Campus university network
• Specific research network (any range)
• Commercial network (any range)
• Internet exchange
• Other:

6. Functionalities: Responsibilities and Functions of the NOC in 2016

7. Functionalities to cover (15 or 16) sorted by number of answers in 2016
 Monitoring,
 Problem management,
 Ticketing,
 Performance management,
 Reporting & statistics,
 Configuration management & backup,
 Communication, coordination & chat,
 Knowledge management/documentation,
 Change management,
 Out-of-band-access (?)
 Security management,
 Inventory management,
 DDoS mitigation,
 Resource management,
 Automation and Orchestration,
 Data aggregation, representation & visualization

8. NOC tools
 You will be asked to indicate whether or not your NOC is responsible for
each particular function. If you answered 'Yes', you will be redirected to
the tools page where you can indicate the importance and usefulness of
the particular software tools for the given NOC function. Please, only
mark those tools that your NOC has first-hand experiences
 For each one of the functionalities:
• Short definition of <functionality>
• Is your NOC responsible for <functionality>
• What tools do you use for <functionality>
Tool
How important is
this tool for your
NOC?
How would you rate
this tool for the
<functionality>?
TOOL1
TOOL2
…

9. For each functionality we will review
 This funcionality stands for...
Answers from one organization in 2011/2016:
Tool9
Tool10 (add?)
Tool11
Tool12
Tool13
Tool14
Tool15
• Tool1
• Tool2 (drop/change?)
• Tool3
• Tool4
• Tool5
• Tool6
• Tool7
• Tool8
Tools:
In red, tools to erase from
the survey, mostly because
they were mentioned only
once in 2016
Short definition
Tools mentioned only once
in 2011 / 2016. In green,
possible answers to rescue
In green, possible answers to include in
the survey, because they were
mentioned more than once in 2016 / new
suggestions

10. NOC tools – Monitoring
Observation and measurement of several parameters (from the link status or traffic
on an interface or line to physical parameters, like temperature, humidity, etc).
1-answer-2016:
AS-Stats
LibreNMS
CENTREON
Swatch
Ciena
OneControl
IBM Tivoli
Netdisco
Net-minder
Speedtest
Puppet
Racktables
Patchmanager
Splunk
Net Polygraph
NMS DWDM
vendors
GINS
minemon
Rancid frontend
MRTG front-end,
Netflow analyser
RRDtool
FTAS, G3
SMARTxAC
Turbo Krt
ViaIpe
1-answer-2011:
Alcatel NMS
BCNET CMDB
Beacon
Bigbrother
Ciena NMS
Ciena Preside
Cisco IP SLA
Cisco EEM
Dude
Equipt sp NMS
Fluxoscope
FSP NeManager
GARR mon. sui.
Hobbit
iBGPlay
ICmyNet.Flow
ICmyNet.IS
Kayako
LambdaMonitor
MonaLisa
Munin
NAV
NetCool
Netscout
Network Node
Mgr
NFA
NMIS
NTOP
Observium
OpManager
Racktables
SMARTxAC
Splunk
Trapmon
WuG
Zabbix
Arbor/Netscout
Peakflow/SP
BGPMON
CACTI
CRICKET
ELK STACK
ICINGA
INTERMAPPER
LOGGING
LOOKING-GLASS
MRTG
MUNIN (3)
NAGIOS
NAV (2)
NETFLOW
NFDUMP
NFSEN
OBSERVIUM
OPENVIEW
PERFSONAR
PMACCT
RANCID
RIPE Atlas / Stats
RIPE RIS / BGPlay
SILK
SMOKEPING
SNMP
SPECTRUM
SPLUNK
SYSLOG
WEATHERMAP
ZABBIX (4)
ZENOSS
ZINO
Suggested tools:

11. NOC tools – Monitoring – First 2 questions
We mixed methods with tools in the previous surveys. The idea is to separate them
and know the importance an frequency of usage for each one of them:
What Monitoring tools are of most importance for you? Please rate from 1 (less
important) to 6 (more important)
• SNMP-based tools
• Flow Monitoring-based tools (Netflow, sFlow, etc)
• Syslog handling tools
• Active monitoring probing, like delay (owamp, twamp) or throughput tests.
• External tools (like looking-glasses, RIPE NCC tools, BGPmon, etc)
• Other tools (like RANCID)
How often are these tools used in your NOC (Daily/Few times a week/≈ Once a
week/≈ Once a month/Only in case of incident)
• SNMP-based tools
• Flow Monitoring-based tools. (Netflow, sFlow, etc)
• Syslog handling tools
• Active monitoring probing, like delay (owamp, twamp) or throughput tests.
• External tools (like looking-glasses, RIPE NCC tools, BGPmon, etc)
• Other tools (like RANCID, Splunk…)

12. NOC tools – Problem management / Incident management?
The primary objectives of Problem Management are to prevent problems and resulting
incidents from happening, to eliminate recurring incidents and to minimize the impact
of incidents that cannot be prevented. Following the necessary steps in order to
identify the source of a problem. Getting warnings about problems or incidents.
Please note communication and collaboration tools will be in a different section.
Answers from one organization in 2011 / 2016:
2016:
GN6
Kibana
HP Openview
SpiceWorks
Observium
Munin
2011:
ARS
CA spectrum
Hobbit
HP insight
mgr
HP service
cter
HP service
mgr
Icinga
ICmyNet.IS
ITIL
Jira
Monitor One
Proprietary
NMS
Service now
Splunk
Vigilant_cong
estio
Wiki
Zenoss
CONFLUENCE
ELK stack
JIRA (3)
NAGIOS
NLNOG RING
OTRS
REQUEST TRACKER
RIPE Atlas / Stats
RIPE RIS / BGplay
SPLUNK
ZABBIX
ZINO
Tools:

13. NOC tools – Ticketing
Process for the tracking of incidents, problems or tasks.
2016:
MANTIS Bug Tracker
TRAC
HP Openview SD
VC4 IMS
Clocking
SpiceWorks
GLPI
GN6
Ticketing for Drupal
2011:
BMC service express
Easyvista
HP Service center
HP Service Manager
HP Service desk
Kayoko Help Desk
Answers from one organization in 2011 / 2016:
Tools:
ARS (Remedy)
JIRA
OTRS
REQUEST TRACKER
SERVICE NOW
TTS (2)

14. NOC tools – Performance management
Following a procedure in order to identify the source of a performance problem
on the network.
2016:
RRD
Spirent appliances
Speedtest
NAV
2011:
Atlas
BC NET CMDB
CISCO IP SLA
DynaTrace
IPPM
Jitter
MGEN
Munin
Nagios
NFDUMP
Netflow
Netminder
Ops Mgr
Owamp
PING
Prosilent
QoS
SpeedTest
Storsentry
Traceroute
TCPDUMP
Wireshark
Zenoss
Answers from one organization in 2011 / 2016:
Tools:
BWCTL
HADES
IPERF
MGEN (2)
MRTG
NDT
NLNOG RING tools
OWAMP/TWAMP
PERFSONAR
RIPE Atlas
SMOKEPING
WIRESHARK
ZINO

15. NOC tools – Reporting and statistics
Representation of data sources for reference and statistics. It includes capacity
reporting and SLA reporting.
2016:
Kibana
LibreNMS
Torrus
RRDtool
infovista
sanet
NAV
NFDUMP
2011:
BCNET CMDB
Business object
datamarts
Confluence
Cricket
Excel
GINS
HO service desk
Hobbit
Icinga
ICmyNet.IS
Infovision
Jira
MonaLISA
MSR reporter
Netflow
Smokeping
Splunk
Stager
StorSentry
Zabbix
Answers from one organization in 2011 / 2016:
Tools:
ARBOR
CA SPECTRUM
CACTI
GRAFANA
MRTG
MUNIN
NAGIOS
NFSEN
RT (2)
SPLUNK
TABLEAU
ZABBIX (2)
ZENOSS
ZINO

16. NOC tools – Configuration management and backup
Control and backup of the configuration for the routers switches and other pieces
of equipment.
2016:
RCS (4)
CA Spectrum and FTP
Puppet
Backuppc
etckeeper
Racktables
Patchmanager
Ciena NMS
SCCS
veeam
Imc
Rancid
emc
2011:
CiscoWorks
Cfengine
Netbackup
viewvc
Answers from one organization in 2011 / 2016:
Tools:
CVS
GIT
IMS
OXIDIZED
RANCID
SUBVERSION

17. NOC tools – Communication coordination chat
Communication with people, either in the same institution or in other institutions.
2016:
Asterisk
Kamailio
CallDav
ServiceInfo (webbased
sender for mailing lists)
SharePoint
2011:
Adobe connect
Davical
Desktop video
EVO
Gtalk
HP Service Center
HP Service Manager
iChat
MSN
Phone
Pidgin
Sametime
Scopia Desktop
VoIP
Webex
Answers from one organization in 2011 / 2016:
Tools:
E-mail
IM
IRC
JABBER
Landline
MAILING LISTS
Mobile
SKYPE
SLACK
TWITTER
WHATSAPP
WIKI

18. NOC tools – Knowledge management/documentation
Storing and sharing knowledge information to improve the efficiency in an
organization.
2016:
MoinMoin Wiki
TRACwiki
TiddlyWiki
FosWiki
Drupal CMS
File server
Subversion
Plone
ownCloud
SURFdrive
OneDrive
KIND
GIS
Comunitats
2011:
Editgrid
HP service center
Intranet (Web)
Joomla
Moinmoin
Plone
Sharepoint
Silverstripe
Telemator
Twiki
Wordpress blog
Answers from one organization in 2011 / 2016:
Tools:
Cloud storage*
CONFLUENCE
DOCUWIKI
MEDIAWIKI
OTRS
REQUEST TRACKER
SHAREPOINT
WIKI

19. NOC tools – Change management
Controlling and recording of changes in values technologies etc.
2016:
Redmine
gitlab
Racktables
Patchmanager
HP Openview Sdesk
pymetric
Wiki
GN6
2011:
EditGrid
HP-SM
Rancid
Redmine
Savannah
Sharepoint
Telemater
Trac
VC-4 CMDB
Answers from one organization in 2011 / 2016:
Tools:
CONFLUENCE
JIRA
GITLAB
REQUEST TRACKER
OTRS

20. NOC tools – Out-of-band Access (?)
Access to the network devices of an institution from an external network.
2016:
DWDM OSC
POTS
2011:
DRAC
IPMI
Answers from one organization in 2011 / 2016:
Tools:
ADSL
CONSOLE SERVER
DRAC
HP ILO
ISDN
KVM (not the hypervisor)
Landline
Mobile tech

21. NOC tools - Security management
Control of IT resources to avoid third parts from attacking the resources
of an institution.
2016:
RTIR
Netflow analyser
2011:
2-factor token
Bastion host
Copp
Cyclops
DNSSEC
Drupal based TTS
fwbuilder
iBGPlay
Icmynet.low
keepass
LDAP
NfSen
OTRS
Radius
Routing authentication
Rtconfig
RTIR
VPN
Answers from one organization in 2011 / 2016:
Tools:
ACL
BGMON
E-MAIL SECURITY APPLIANCE
FIREWALL
FIREWALLBUILDER (2)
FREERADIUS
IPS
IDS
KERBEROS
RADIATOR
REQUEST TRACKER
RSA Software
TACACS+
WEB SECURITY APPLIANCE

22. NOC tools – Inventory management
Organization and control of information about an institution's devices materials
and products.
2016:
Patchmanager
HP Openview SDesk
HP Openview NNM
Ciena NMS
filemaker
netdisco
Observium
KIND
CMT
GarrDB
GIS
GN6
2011:
BCNET CMDB
BDcops
Editgrid
HP Service desk
Inflow
Insight manager LDAP
MOT2
Navision
NOClook
Rancid
Telemator
VC-4 CMDB
Wiki
Answers from one organization in 2011 / 2016:
Tools:
EXCEL
IMS
IIR (2)
RACKTABLES (2)
RANCID
WIKI

23. NOC tools – DDoS Mitigation
DoS mitigation is a set of techniques for resisting distributed denial-of-service (DDoS)
attacks on networks attached to the Internet by protecting the target and relay networks.
FastNetMon
Answers from one organization in 2016:
Tools:
ACL
Akamai Solutions
Arbor/Netscout Cloud solutions-
Arbor/Netscout Hardware Solutions
(Peakflow, SP, TMS,…)
Blackholing
Cloudflare solutions.
Dyn solutions
Firewall on Demand
BGP FLOWSPEC
RATE-LIMITING
Traffic-washing
UTRS
What kind of DDoS Mitigation solutions do you use?
• Hardware platforms on-site in-line
• Hardware platforms on-site off-line
• Cloud solutions
• No special tool, but manual backholes, ACLS, etc

24. NOC tools – Resources management
Organization and control of logical resources such as IP addresses, AS numbers
circuit numbers, topology, documentation etc.
2016:
GIS
GestioIP
HP Openview SDesk
Phpipam
KIND
GIS
BDCOM
2011:
Access
BCNET CMDB
Bdcops
IP-range
MOT2
Pinger
Racktables
Telise
Text files
Answers from one organization in 2011 / 2016:
Tools:
6CONNECT
BLUECAT
CONFLUENCE
EXCEL
INFOBLOX
IPPlan
OMNIGRAFFLE (2)
RACKTABLES (4)
VISIO
WIKI

25. NOC tools - Data aggregation, representation, visualisation
Aggregate live data from various tools and represent/visualize them in a human
readable way.
2016:
MRTG, Tivoli maps,
Juniper RIM
Zino
Zenoss
CheckMK
Observium
2011:
CMDB
Google-maps
IMs
Monalisa
Munin
NAV
Netflow
Splunk
Stager
Zenoss
Zino
Answers from one organization in 2011:
Tools:
CACTI
ELASTICSEARCH
GRAFANA
KIBANA
LOGSTASH
SPLUNK
WEATHERMAP

26. NOC tools – Automation and Orchestration
Automation describes a task or function accomplished without human intervention in a
network, like configuration, management, testing, deployment, and operations of
physical and virtual devices.
Orchestration describes the arranging and coordination of automated tasks, ultimately
resulting in a consolidated process or workflow to automatically define and modify the
behavior of the network.
Tools:
Ansible
Chef
Dockers
Jenkins scripting
Kubernetes
Puppet
Python scripting
Salt
Terraform
What kind of tasks do you automate?
• Provisioning
• Routing Configuration
• Network discovery
• Other (please specify as many tasks as you miss here)

27. Standards
 What is the estimated level of adoption of the given standard or
methodology at your NOC? + How many of your NOC people are
certified or trained (in percentage)?
• ISO
• ISO 20000
• ISO 27001
• eTOM
• ITIL
• NITS
• FIPS
 What technical training(s) are provided either in-house or contracted to
your NOC personnel? Please, describe them all.

28. The way it will look like (SurveyMonkey-friendly style):
Easy to click, although it “doubles” the number of questions.

29. Some examples of graphs from the last survey
https://wiki.geant.org/display/SIGNOC/SIG-NOC+Tools+Survey+2016

30. Thank you!
Questions? Suggestions?
Please, send us your feedback!
sig-noc-al@lists.geant.org
mariaisabel.gandia@csuc.cat