1. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Tango Database
&
MySQL Cluster
Claudio Scafuri claudio.scafuri@elettra.trieste.it
Claudio Scafuri, Elettra

2. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Topics
● news & experiences
● considerations
● presentation of “deliverables”
Claudio Scafuri, Elettra

3. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Where we left ....
HostA HostB
Databaseds Databaseds
mysqld mysqld
API API
DB DB
MGM
Claudio Scafuri, Elettra

4. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
News
– MySQL max 4.1.12 . Clustering is “stable”
– improved documentation
● but refers to MySQL 5 ! They are pushing in that direction
● still some gray area ...
– forums & mailing list by MySQL with precious
informations
Claudio Scafuri, Elettra

5. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
“Crashes” explained
Conservative design:
save DB from corruption and inconsitencies
Not crashes but shut­down of storage nodes!
Claudio Scafuri, Elettra

6. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
“Crashes” explained
● lack of memory
● disk space
● network connectivity
Tests were carried on “low end” PCs !
Claudio Scafuri, Elettra

7. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
“Crashes” explained
● lack of memory:
98 devices
895 properties
176 attribute properties
– ndbd : 427 MB Virtual / 27 MB Resident
Claudio Scafuri, Elettra

8. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
“Crashes” explained
● disk space:
98 devices
895 properties
176 attribute properties
– 314 MB on Master node
– 236 MB on Slave node
– plus logs...
Claudio Scafuri, Elettra

9. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
“Crashes” explained
● network connectivity:
must avoid “split brain” syndrome: 2 different commits
on the same rows/tables.
­ majority of storage node must “see” each other
or
– storage nodes can see an “arbitrator”:
– MGM node
– myqld node
Claudio Scafuri, Elettra

10. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Cluster Configuration
­ MySQL strongly reccomends to have a single
management node in the cluster. It works as
arbitrator.
In principle it is possible to have multiple
mangement nodes by assigning an arbitration
rank. But :“the configuration has not been
completely tested.”
Claudio Scafuri, Elettra

11. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Cluster Configuration
HostA HostB
HostC
Databaseds Databaseds
MGM
mysqld mysqld
API API
DB DB
Switched Network
Claudio Scafuri, Elettra

12. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
HostA HostB
HostC
Databaseds Databaseds
MGM
mysqld mysqld
API API
DB DB
Switched Network
Claudio Scafuri, Elettra

13. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
HostA HostB
HostC
Databaseds Databaseds
MGM
mysqld mysqld
API API
DB DB
Switched Network
Claudio Scafuri, Elettra

14. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
HostA HostB
HostC
Databaseds Databaseds
MGM
mysqld mysqld
API API
DB DB
Switched Network
Claudio Scafuri, Elettra

15. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
HostA HostB
HostC
Databaseds Databaseds
MGM
mysqld mysqld
API API
DB DB
Switched Network
Claudio Scafuri, Elettra

16. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
HostA HostB
HostC
Databaseds MGM Databaseds
mysqld mysqld
API API
DB DB
Switched Network
According to MySQL this should trigger a shutdown of the
cluster. But it may keep working...
Claudio Scafuri, Elettra

17. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Failure scenarios
­ “failures” never led to data loss or corruption
­ lack of explicit notification
You have to check the logs and the status of the processes
Claudio Scafuri, Elettra

18. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Considerations
● excelent tool, does its job
– but keep in mind it design and goals
● redundancy does not come for free
– 3 or more servers , lots of memory, disk space
– non­trivial management (compared to base MySQL)
– network failure to be dealt with by othe means
Claudio Scafuri, Elettra

19. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Deliverables
­ configuration files
my.cnf : MySQL general configuration file with
cluster support enabled
config.ini: cluster configuartion file.
Describes the minimal cluster showed in the slides
­ database creation sql script
create-db.sql: creates the TANGO schema with
cluster enabled tables. Preloads 2 instances of
Database: sys/database/2 , sys/database/4
Claudio Scafuri, Elettra

20. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
Deliverables
extra bonus!
2 HOW­TOs retrieved from the web :
cluster-howto-1.pdf
by A. Davies. Official MySQL document
cluster-howto-2.pdf
courtesy of LOD Communications, Inc.
http://www.lod.com
Claudio Scafuri, Elettra

21. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
To be done:
● testing of deliverables by other institutes
● choose how (if) to distribute the deliverables
● decide whether to add support for multiple tango
host in Java:
TANGO_HOST=myhost:10000,otherhost:1000
now it is understood only by C++ thanks to omniORB.
Claudio Scafuri, Elettra

22. Tango Meeting
Tango database & MySQL cluster 26­27/05/2005
To be done:
● tuning of db parameters (lots of them!)
● testing of backup and recovery
● trying to break and then recover the database.
Claudio Scafuri, Elettra