Blackboard DevCon 2012 - How to Turn on the Lights to Your Blackboard Learn Environment with Zabbix
1. How to Turn the Lights
on Your Blackboard
Learn with Zabbix
Noriaki Tatsumi
Manager, Performance Engineering
Blackboard, Inc.
2. What’s Zabbix?
http://www.zabbix.com/
• Distributed monitoring solution
• GNU General Public License (GPL) version 2
• Alerts, advanced monitoring, trend analysis, data
visualization, actions, RC, JSON API
• Stores data away from production
• MySQL
• PostgreSQL
• SQLite
• Oracle / IBM DB2
2
6. Notify you when there’s an issue
• Be the first to know about down time
• Identify problems before the users report
• Get alerted for potential security attack
6
7. Troubleshoot your environment
• Reduce/eliminate guess work during root cause
analysis
• Minimize time to identify problem
• Measure improvement after introducing a fix
7
11. Why Zabbix and not another monitor like Nagios?
• Requires smaller effort to setup
• Better UI monitoring and graphing capabilities
• Template based
• Built-in capability to monitor JMX, log files, Unix,
Windows, SNMP, database, URL, and IPMI
• Easily extensible via agents, plugins, and APIs
• Other cool features like maps, screens, slide shows
11
18. How does Blackboard use Zabbix?
• Monitoring and trending operations and statistical data
for SaaS solutions like SafeAssign
• Performance and scalability forensics
• Internal server infrastructure monitoring
18
19. Zabbix Blackboard Learn Template Suite
• Easy to install
• Requirements
• Zabbix 2.0 and up
• Zabbix Java Gateway
• Extended Monitoring B2
• Available for download on edugarage
http://tinyurl.com/bbzabbix
19
20. Zabbix Blackboard Learn Template Suite
• Application
• Java
• Linux / Windows
• Tomcat
• Cache
• ActiveMQ
20
21. Zabbix Blackboard Learn Template Suite
Application
• Statistics
• Sessions (active, inactive)
• Users &Courses (available, unavailable)
• Assessment attempts / hour
• Database connection pool usage
• Security
• Config changes in conf/ folder & system registry database table
• Creation of privileged accounts, Logins from unexpected terminal
• Command-line authentication tool usage
21
32. Zabbix Blackboard Learn Template Suite
Tomcat
• Request processor
• Average processing time
• Requests per minute
• Errors per minute
• Thread pool
• Busy
• Current
• Max
32
38. Zabbix Blackboard Learn Template Suite
Data sampling and retention
• Sampling interval
• 30 ~ 3600 (1 hour)
• History (raw data)
• 7 days ~ 14 days
• Trend (min, max, avg)
• Stats: 5 years
• Others: 3 months
38
39. Zabbix Blackboard Learn Template Suite
Deployment
1. Download at http://tinyurl.com/bbzabbix
2. Install Monitoring Extension B2 for statistics and
security data
3. Upload the templates via Zabbix GUI
4. Deploy agents using the provided script
5. Add hosts and apply the templates in Zabbix via GUI
39
40. Zabbix server deployment options
• Compile from source
• VMware / VirtualBox (.vmdk)
• Open virtualization format (.ovf)
• Live CD/DVD (.iso)
• Preload ISO
• USB stick / hard disk image
• Xen guest
40
41. Zabbix Best Practices
• Define an objective for each item you’re monitoring
• Don’t sample data more than required
• Don’t keep history and trend data longer than
necessary
• Use percentile line on charts where applicable
• Graph all functions (min, avg, max) to indicate outliers
• Don't create triggers that gets ignored
• Self monitor Zabbix server
• Have a maintenance and a backup/restore plan
41