Présentation 'SkySQL Cloud MariaDB Data suite' effectuée le 12 décembre 2012 au Meetup LeMug.fr par Serge Frezefond

1. V1212.01
SkySQL Data Suite
A New Open Source Approach to MySQL Distributed Systems
Serge Frezefond

2. 2
Agenda
• SkySQL Cloud Data Suite Architecture
• SkySQL Cloud Data Suite on Amazon EC2
• Components for automated provisioning in AWS
• Components for high availability in EC2
• The Administration Console
• Automated Backup/Restore with S3

3. 4
Intro to SkySQL Data Suite

4. 6
SkySQL Cloud Data Suite
SkySQL Data SuiteSkySQL Data SuiteSkySQL Data SuiteSkySQL Data Suite
High
Availability
High
Availability
Full-TextFull-Text
ReplicationReplication
GaleraGalera
DRBDDRBD
AnalyticsAnalyticsAnalyticsAnalytics
Carrier GradeCarrier GradeCarrier GradeCarrier Grade
Online Apps.Online Apps.Online Apps.Online Apps.
Shared
Storage
Shared
Storage

5. 7
Cloud Data Suite vs Enterprise Data Suit
Cloud Data SuiteCloud Data Suite
•Deployed in Amazon AWSDeployed in Amazon AWS
•Automatic configuration and provisioningAutomatic configuration and provisioning
•MariaDB 5.5MariaDB 5.5
•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync
•Administration Console with Backup/RestoreAdministration Console with Backup/Restore
•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial
•PhpMyAdminPhpMyAdmin
•Community SupportCommunity Support
•Development ReleaseDevelopment Release
•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)
Cloud Data SuiteCloud Data Suite
•Deployed in Amazon AWSDeployed in Amazon AWS
•Automatic configuration and provisioningAutomatic configuration and provisioning
•MariaDB 5.5MariaDB 5.5
•MySQL Replication with Pacemaker/CorosyncMySQL Replication with Pacemaker/Corosync
•Administration Console with Backup/RestoreAdministration Console with Backup/Restore
•SkySQL Enterprise Monitoring aka MONyog - TrialSkySQL Enterprise Monitoring aka MONyog - Trial
•PhpMyAdminPhpMyAdmin
•Community SupportCommunity Support
•Development ReleaseDevelopment Release
•Fully OSS (with MONyog Trial)Fully OSS (with MONyog Trial)
Enteprise Data SuiteEnteprise Data Suite
•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS
•Standard download packagesStandard download packages
•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers
•MySQL Replication with MHA, DRBD, Galera orMySQL Replication with MHA, DRBD, Galera or
shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync
•ZManda Recovery ManagerZManda Recovery Manager
•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog
•SQLyogSQLyog
•Enterprise Technical SupportEnterprise Technical Support
•Production ReleaseProduction Release
•Mix of OSS and CommercialMix of OSS and Commercial
Enteprise Data SuiteEnteprise Data Suite
•Deployed in Clouds, virtualised env. and bare OSDeployed in Clouds, virtualised env. and bare OS
•Standard download packagesStandard download packages
•MariaDB, MySQL, Percona ServersMariaDB, MySQL, Percona Servers
•MySQL Replication with MHA, DRBD, Galera orMySQL Replication with MHA, DRBD, Galera or
shared storage and Pacemaker/Corosyncshared storage and Pacemaker/Corosync
•ZManda Recovery ManagerZManda Recovery Manager
•SkySQL Enterprise Monitory aka MONyogSkySQL Enterprise Monitory aka MONyog
•SQLyogSQLyog
•Enterprise Technical SupportEnterprise Technical Support
•Production ReleaseProduction Release
•Mix of OSS and CommercialMix of OSS and Commercial

6. 8
SkySQL Cloud Data Suite Sep 2012
1111
2222
3333
1.Configure your DB system on
config.skysql.com or cloud.skysql.com
2.Deploy the DB system:
• Online
• Configuration pull from AWS
• Manually
1.Connect the DB system to
your application

7. 9
SkySQL Cloud Data Suite Dec 2012
• Rel. 1.1 with bug fixing
• New Administration Console
with:
• rpm package and installation on Linux OS
• Administration and monitoring of
replicated and standalone servers
• Customisable graphs
• Customisable commands

8. 10
SkySQL Cloud Data Suite
Architecture

9. 11
SkySQL Cloud Data Suite Configurator
Configuration
Engine
Rules
Repository
Provisioning
Engine
Software
Repository
Deployment
Engine
Provisioning
Queue
Deployment
Queue
AMI
userdata.sh
User Account
Deployment triggered by theDeployment triggered by the
first boot of each instancefirst boot of each instance
Deployment triggered by theDeployment triggered by the
first boot of each instancefirst boot of each instance

10. 12
SkySQL Cloud Data Suite
Terminal
Admin Console
PhpMyAdmin
MONyog

11. 13
SkySQL Data Suite Building Block
Replicator
Data StoreAdmin
Console

12. 14
SkySQL Cloud Data Suite Sep 2012
Technologies and Software
Replicator
Data StoreAdmin
Console
FRONT ENDFRONT END
ApacheApache
TomcatTomcat
FRONT ENDFRONT END
ApacheApache
TomcatTomcat
BACK ENDBACK END
SQLiteSQLite
BACK ENDBACK END
SQLiteSQLite
MariaDBMariaDBMariaDBMariaDB
Linux 3.2Linux 3.2
(Amazon(Amazon
Distro)Distro)
Linux 3.2Linux 3.2
(Amazon(Amazon
Distro)Distro)
MONyogMONyog
TrialTrial
MONyogMONyog
TrialTrial
PHPMyAdminPHPMyAdminPHPMyAdminPHPMyAdmin
MySQLMySQL
ReplicationReplication
MySQLMySQL
ReplicationReplicationGlusterFSGlusterFSGlusterFSGlusterFSPacemakerPacemaker
CorosyncCorosync
PacemakerPacemaker
CorosyncCorosync XtrabackupXtrabackupXtrabackupXtrabackup

13. 15
SkySQL Cloud Data Suite Sep 2012
Data Store
Data Store
PublicPublic
SkySQL AMISkySQL AMI
PublicPublic
SkySQL AMISkySQL AMI
MariaDB 5.5MariaDB 5.5
MyISAM, InnoDB, XtraDB,MyISAM, InnoDB, XtraDB,
Enhanced Optimizer, VirtualEnhanced Optimizer, Virtual
ColumnsColumns
MariaDB 5.5MariaDB 5.5
MyISAM, InnoDB, XtraDB,MyISAM, InnoDB, XtraDB,
Enhanced Optimizer, VirtualEnhanced Optimizer, Virtual
ColumnsColumns
From standalone toFrom standalone to
1 Master + 3 Slaves1 Master + 3 Slaves
From standalone toFrom standalone to
1 Master + 3 Slaves1 Master + 3 Slaves
System and DataSystem and Data
file systemsfile systems
System and DataSystem and Data
file systemsfile systems
Pacemaker controls thePacemaker controls the
automatic failoverautomatic failover
Pacemaker controls thePacemaker controls the
automatic failoverautomatic failover

14. 16
SkySQL Cloud Data Suite Sep 2012
Administration Console
Admin
Console
GUI to monitor theGUI to monitor the
status of thestatus of the
ClusterCluster
GUI to monitor theGUI to monitor the
status of thestatus of the
ClusterCluster
ExtendedExtended
Monitoring fromMonitoring from
MONyog as a trialMONyog as a trial
ExtendedExtended
Monitoring fromMonitoring from
MONyog as a trialMONyog as a trial
PhPMyAdmin asPhPMyAdmin as
graphical user toolgraphical user tool
PhPMyAdmin asPhPMyAdmin as
graphical user toolgraphical user tool
Backup/RestoreBackup/Restore
to/from S3 usingto/from S3 using
XtrabackupXtrabackup
Backup/RestoreBackup/Restore
to/from S3 usingto/from S3 using
XtrabackupXtrabackup
Start/Stop, Isolate,Start/Stop, Isolate,
Promote nodesPromote nodes
Start/Stop, Isolate,Start/Stop, Isolate,
Promote nodesPromote nodes
mysql commandmysql command
line interfaceline interface
mysql commandmysql command
line interfaceline interface
Automatic failover for theAutomatic failover for the
admin and monitoringadmin and monitoring
serversservers
Automatic failover for theAutomatic failover for the
admin and monitoringadmin and monitoring
serversservers

15. 17
SkySQL Cloud Data Suite Sep 2012
Replicator
Replicator
Standard MySQLStandard MySQL
Replication in asynchronousReplication in asynchronous
and semi-synchronousand semi-synchronous
modemode
Standard MySQLStandard MySQL
Replication in asynchronousReplication in asynchronous
and semi-synchronousand semi-synchronous
modemode
GlusterFS on ext4 to replicateGlusterFS on ext4 to replicate
the data collection andthe data collection and
monitoringmonitoring
GlusterFS on ext4 to replicateGlusterFS on ext4 to replicate
the data collection andthe data collection and
monitoringmonitoring
Automatic failover of databaseAutomatic failover of database
servers and consoleservers and console
componentscomponents
Automatic failover of databaseAutomatic failover of database
servers and consoleservers and console
componentscomponents

16. 18
SkySQL Data Suite Provisioning
AMI
userdata.sh
User Account
Variable definition - theyVariable definition - they
will be used later whenwill be used later when
other scripts are calledother scripts are called
Variable definition - theyVariable definition - they
will be used later whenwill be used later when
other scripts are calledother scripts are called
On-the-fly configurationOn-the-fly configuration
files, such as my.cnffiles, such as my.cnf
On-the-fly configurationOn-the-fly configuration
files, such as my.cnffiles, such as my.cnf
Execute predefinedExecute predefined
scripts already in thescripts already in the
AMIAMI
Execute predefinedExecute predefined
scripts already in thescripts already in the
AMIAMI
Adaptive, distributedAdaptive, distributed
setupsetup
Adaptive, distributedAdaptive, distributed
setupsetup
Elastic IP onElastic IP on
the Master Serverthe Master Server
Elastic IP onElastic IP on
the Master Serverthe Master Server
Currently, the AdministrationCurrently, the Administration
Console is on the Master serverConsole is on the Master server
Currently, the AdministrationCurrently, the Administration
Console is on the Master serverConsole is on the Master server

17. 19
SkySQL Data Suite Deployment

18. 20
SkySQL Data Suite Deployment

19. 21
Automated Provisioning in AWS

20. 22
Starting Here!
You can try it forYou can try it for
FREE! Even withoutFREE! Even without
anan
AWS accountAWS account
You can try it forYou can try it for
FREE! Even withoutFREE! Even without
anan
AWS accountAWS account

21. 23
Simplified Interface
Immediate HelpImmediate HelpImmediate HelpImmediate Help
Real Time CheckReal Time CheckReal Time CheckReal Time Check

22. 24
Advanced Configuration

23. 25
Cost Estimation

24. 26
Delivery Mechanism

25. 27
Online Deployment
The keys will beThe keys will be
discarded immediatelydiscarded immediately
after the provisioningafter the provisioning
The keys will beThe keys will be
discarded immediatelydiscarded immediately
after the provisioningafter the provisioning

26. 28
Extra AWS & Contact Info

27. 29
Welcome Email
mysql command linemysql command linemysql command linemysql command line
Administration ConsoleAdministration ConsoleAdministration ConsoleAdministration Console
ssh connectionssh connectionssh connectionssh connection
How to remove SCDSHow to remove SCDSHow to remove SCDSHow to remove SCDS
An inventory of theAn inventory of the
AWS componentsAWS components
An inventory of theAn inventory of the
AWS componentsAWS components
IMPORTANT!IMPORTANT!
Your pem keyYour pem key
IMPORTANT!IMPORTANT!
Your pem keyYour pem key

28. 30
Job Done!

29. 31
SkySQL Cloud Data Suite in
Amazon EC2

30. 32
Security
Identity Access Manager Roles
• EIP Address
Associate/Disassociate, Describe
• Instances
Start, Stop, Describe
Attributes, Status, Tags, Net IF
Create, Describe
• S3 Buckets
List, Create, Delete
• Objects
Get, Put, List, Delete

31. 33
Security
Security Groups
• ssh
• http
• MySQL
• MONyog

32. 34
Storage

33. 35
Components for High
Availability in EC2

34. 36
Automatic Failover with Pacemaker
CRM Configuration
Replicator
Data StoreAdmin
Console

35. 37
Automatic Failover with Pacemaker
CRM Configuration
Replicator
Data StoreAdmin
Console
[root@node2 init.d]#[root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec============Last updated: Sat Dec
1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via
crm_attribute on node2Stack: openaisCurrent DC: node2 - partitioncrm_attribute on node2Stack: openaisCurrent DC: node2 - partition
with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-
148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4
expected votes14 Resources configured.============Online: [ node2expected votes14 Resources configured.============Online: [ node2
node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL]node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL]
Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP
(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-
myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4
] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):
Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): StartedStarted
node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs
(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor
(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7
(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2
[root@node2 init.d]#[root@node2 init.d]# crm statuscrm status============Last updated: Sat Dec============Last updated: Sat Dec
1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via
crm_attribute on node2Stack: openaisCurrent DC: node2 - partitioncrm_attribute on node2Stack: openaisCurrent DC: node2 - partition
with quorumVersion: 1.1.7-6.el6-with quorumVersion: 1.1.7-6.el6-
148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4
expected votes14 Resources configured.============Online: [ node2expected votes14 Resources configured.============Online: [ node2
node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL]node3 node1 node4 ] Master/Slave Set: ms_resMySQL [resMySQL]
Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIPMasters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP
(lsb:amazon_master_eip):(lsb:amazon_master_eip): Started node2 Clone Set: apache-php-Started node2 Clone Set: apache-php-
myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4
] Resource Group: monyog monyog-fs] Resource Group: monyog monyog-fs (ocf::custom:Filesystem):(ocf::custom:Filesystem):
Started node2 monyog-serviceStarted node2 monyog-service (lsb:MONyogd):(lsb:MONyogd): StartedStarted
node2 Resource Group: admin-console console-fsnode2 Resource Group: admin-console console-fs
(ocf::custom:Filesystem):(ocf::custom:Filesystem): Started node2 SkySQL-monitorStarted node2 SkySQL-monitor
(lsb:skysql_monitor):(lsb:skysql_monitor): Started node2 Tomcat7Started node2 Tomcat7
(ocf::heartbeat:tomcat):(ocf::heartbeat:tomcat): Started node2Started node2
Node 2Node 2Node 2Node 2
Elastic IPElastic IPElastic IPElastic IP
Public IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPsPublic IPs
admin-console andadmin-console and
MONyog groupMONyog group
admin-console andadmin-console and
MONyog groupMONyog group

36. 38
The Administration Console

37. 39
Admin Console Dec 2012

38. 40
Admin Console Dec 2012
Access toAccess to
MONyogMONyog
Access toAccess to
MONyogMONyog
MasterMaster
ServerServer
MasterMaster
ServerServer
ContextContext
SensitiveSensitive
ContextContext
SensitiveSensitive
ContextContext
SensitiveSensitive
ContextContext
SensitiveSensitive
CustomisableCustomisable
GraphsGraphs
CustomisableCustomisable
GraphsGraphs

39. 41
Admin Console Dec 2012
Access toAccess to
PhpMyAdminPhpMyAdmin
Access toAccess to
PhpMyAdminPhpMyAdmin
CustomisableCustomisable
CommandsCommands
CustomisableCustomisable
CommandsCommands
Backup/RestoreBackup/RestoreBackup/RestoreBackup/Restore

40. 42
Administration Console Dec 2012
Customizable Graphs
Name SQL Icon Delta Avg Chart
Monitors

41. 43
Administration Console Dec 2012
Customizable Commands
Desc Icon Order
Commands
Desc Icon Script
Steps
CommandStep

42. 44
Automated Backup/Restore
with S3

43. 45
Automated Backup
Using Xtrabackup from the Admin Console

44. 46
Automated Backup
Using Xtrabackup from the Admin Console

45. 47
Automated Backup
Using Xtrabackup from the Admin Console

46. 48
Why is the SkySQL Cloud Data Suite
important for you?
• You can test MariaDB/MySQL on AWS for FREE
• You can use and fully control a ready made
highly available solution AWS
• You can compare MySQL on EC2 and RDS
• You can study/review security and availability
aspects in AWS
• You can use some of the SCDS components in
your environment

47. 49
SkySQL Cloud Data Suite & OpenStack ?
• Nova, glance, swift, keystone, horizon,
quantum ceph
• Position for public/private/hybrid cloud
• Rackspace,HPcloud,VMware ...
CloudWatt, Numergy
• Our solution will be there

48. 50
Is not Amazon RDS enough for my needs ?
It is fully managed !
• Provisioning
• Patching
• Backup/ PointInTIme recovery
• Replicas
• HA / Failover
• monitoring

49. 51
Or maybe Google Cloud SQL ?
• New offer
• Similar to AWS RDS
• Rely on storage feature for
HA
• Fully managed

50. 52
Private/Hybrid cloud
Eucalyptus / VMware Vcloud
• They own a huge market share
• Eucalyptus is tied to Amazon / open core
• VMware is tied toVMware and fully closed
• Excellent solutions but ...

51. 53
For more information...
• Try the Configurator and How-ToVideo:
http://cloud.skysql.com
• The Admin Console:
http://www.skysql.com/downloads/community/skysql-console
• Forums:
https://www.skysql.com/forums/skysql-cloud-data-suite
• Web: https://www.skysql.com/cloud
• Bugs: http://bugs.skysql.com
• Try it! http://config.skysql.com
• Free SkySQL account available for testing
• Test on your account