2. Introduction
1. University of Nottingham background
2. Local Hosting Services
3. Global Service Provision
4. Oracle Commercial Offerings
1. Oracle Support
2. Enterprise Server options
3. MySQL Enterprise Monitor
5. MySQL Cluster / NDB
6. Next Steps
7. Questions and close
3. Background – Key Facts
• Ranked in the UK and European top 30 and in the top 1 per
cent of all universities worldwide
• Russel Group member – one of 24 ‘leading UK universities
which are committed to maintaining the very best research, an
outstanding teaching and learning experience and unrivalled
links with business and the public sector’
• Significant local economic impact
• 44.5K students, UK and overseas
• 6.8K staff members
4. n
stopping there…….
Background - Systems
Libraries, Research
and Learning Services:
• Research Applications
• Research Data Management
• Collaboration
• Lecture Capture
• High Performance Computing
• Business Intelligence
• Student Experience
Core Systems:
5. Local ‘database hosting’ services
2007 September – ‘Research’ and ‘Corporate’ hosts created.
Pre 2007 – multiple stand alone physical databases
created primarily by Schools.
• Solaris host
• MySQL 5.0.45
• SAN storage
• Centos host
• MySQL 5.0.2
• nfs storage
A success! But …
• Little central focus given
• Ownership of DBs transient
• No quality control of application or DBs
• The years pass …
6. Global Service Provision
2012 – Key Virtual Learning Environment (VLE) service
moved from WebCT on Oracle to Moodle on MySQL
• Solaris hosts with RSF
failover
• MySQL 5.5 Community
Server
• SAN storage
• NFS storage
• Initial application setup trouble free
• Poor application scalability
• Erratic application performance
• Employed DBA contractors, who refined key tuneables
• Issues with server resources and disks on shared hosts
• Engaged with Oracle Consultancy, then increased
engagement with Oracle Support
• Moodle application preferred
• General preference for Open Source
• Dislike of Oracle database from both
Microsoft and Open Source lobbies
• Confidence in MySQL
• Feeling of better fit for Moodle
Did it “just work”?
7. Global Service Provision
VLE Database architecture from 2014 onwards
• Physical Dell Ubuntu boxes
• 32 core
• 128 GB RAM
• Internal spinning and SSD disks
• Enterprise Server MySQL 5.6.x
with Enterprise Thread Pool and
Enterprise Audit
• tmpdir to ramdisk
• Master Slave replication,
between data centres.
• Initially with Corosync / Heartbeat /
Pacemaker failover management
• Monitored by MySQL Enterprise
Manager
• Other non DB architectural features
needed to meet scaling needs of
Service
8. Oracle Support
• Require 24x7 Support on key services
• Prefer to deal with vendors of specific product
• Enterprise Server (plugins and hot fixes)
• MySQL Enterprise Monitor
• My Oracle Support
• Consultative Support
• Cost Effective
Verdict?
9. MySQL Enterprise Manager
• Easy to install
• Non invasive on production
services
• Pre-loaded with best practice
based alerts
• Time Series graphs
• Query Analyzer
• Can look back into the past
10. MySQL Cluster / NDB
2015 - Online Assessments service
Previous architecture: Solaris Zone, with RSF and SAN storage
Needed to be:
• Robust
• Performant
• Scalable
• Fully supported (hardware, OS, database, apache)
• Quickly implemented
11. MySQL Cluster / NDB
Online Assessments service
New architecture: Centos VMs split over 2 sites
12. MySQL Cluster / NDB
Commercial MySQL Cluster:
• Met requirements
• 24x7 Support
• Likely to need Oracle Support
• MySQL Cluster Manager
• Can patch with seconds of downtime
Be aware:
• Different technology from normal MySQL
• Needs solid administrators
• Needs solid network infrastructure with low latency
• Short transactions by PK essential
• Loading/restoring SQL data very slow, can appear to crash
• Need to script backups, OOM rating and stats gathering
Verdict?
13. Next Steps
• MySQL Enterprise Backup
• VLE service to 5.7.x DB
• Assessments to Cluster 7.5.x/MySQL 5.7.x
• Hosting Services to MySQL 5.7.x
• MySQL Router / MySQL Group Replication
• Trial Cloud platforms
• Security
• Transparent Data Encryption
• Encryption
• Enterprise Firewall
When we get the time!
Local economic impact £677 million, support 16K local jobs, across UK generate over £1 billion. Gold Rated in Teaching Excellence Framework. 2016 income £635 million, from tuition fees, research grants, , funding body grants and other income such as operating profits. Surplus £18 million. 5 faculties, 43 schools.
Rogō is an e-assessment system, also used for progress tests and surveys, which is open sourced.
Moodle is single robust, secure and integrated system to create personalised learning environments.
ExLibris – 5 products, 4 oracle, 1 mysql. Remote international installs.
Over 1700 servers/VMs. Over 1000 DBs.
Mention security vulnerabilities in apps, that a long select followed by an insert/update/delete will block other sessions in MyIsam.
Oracle had bought Sun in 2010.
Moodle used by all staff and students, needed to be accessible from overseas and be available 24x7.
Things improved, but the game had to be upped.
Key point here is we had to up our game – the core of the product if properly configured was good enough to meet our requirements. Each moodle login can hit db with 50 queries. tmpdir holds myisam tables (for big unions, view, sub queries, distincts etc) defaults to /tmp, then we used SAN, then ramdisk then tmpfs. Suspect moodle users mostly much smaller than us , so smaller datasets. Also mention problems with automating failover.
Verdict – include normally adequately responsive with good results, normally keen to help. Compare with traditional Oracle DB support.
In query analyser example we are seeing a day at the start of November, note the first seen column. Note you need to setup ‘consumers’ in performance schema to get example queries.
Needs to work as failure of exams is not acceptable to schools, reputational damage, students can complain and ask for marks to be upped. Moving into phase of being used in Malaysia and China as well as UK.
Skim over this, just draw attention to DB components and MCM.
Initially 7.4.8 cluster, bundled with 5.6.27? Of DB, plus MCM 1.2. now 7.4.16, 5.6.37, 1.4.2
Need control of your application or to understand it well. Queries innodb or myisam which wiz through can grind on Cluster.
Normal resolution to mysql issues – restarting service and watching error log did not work well.
By loading data being slow I mean 4 G DB import to NDB taking 20 hours, rather than 2 to InnoDB. Native NDB restore takes 30 mins, but then need to apply binary logs.