2. Who am I ?
Frédéric Descamps
@lefred
http://about.me/lefred
Percona Consultant
Managing mysql since 3.23 (as far as I
remember)
devops believer
www.percona.com
3. Agenda
● What is Percona
Toolkit ?
● What tools are
included and what
can they do ?
● Quick intro to several
tools
● Resources
www.percona.com
4. Percona Toolkit
● Origin: Maatkit and Aspersa (both projects
started by Baron Schwartz with Daniel Nichter's
contribution)
● Indispensable tools for MySQL administrators
● Opensource GNU GPL v2
● https://launchpad.net/percona-toolkit
● http://www.percona.com/software/percona-toolkit/
www.percona.com
10. Know your environment
(Hardware & OS)
We need to find the answers to these questions:
● On what kind of machine runs my db ? Physical/Virtual ?
● How much memory is used/free ? Does it swap ? Can I add more ?
● How many CPUs and CPU cores ? Speed ?
● Is this a 64bit machine ?
● What kind of RAID controller ? Volumes ? Disks ?
● What about network interfaces ? Connections ? Processes ?
● What's the Operating System ? Kernel ? Release ?
● Which IO scheduler is in use ?
● On which filesystem is the data stored ?
● Do we have free space ?
www.percona.com 10
11. Summary !
Name: ptsummary
Function: summarize system information in
a nice way
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-summary.html
www.percona.com 11
14. Know your environment (MySQL)
We need to find the answers to these questions:
● Version ? Build ?
● How many database ? Where is the data directory ?
● What about replication ? How many replicas ?
● Do you use some Key XtraDB features ?
● What are the Key InnoDB settings ?
● Which storage engines are used ?
● What types of data fields are used ? Index types ?
● Do we use foreign keys ? Views ? Triggers ?
● Do we use Full-text indexes ? Partitioning ?
● What about Privileges ?
www.percona.com 14
15. Summary... again !
Name: ptmysqlsummary
Function: Summarize MySQL information in
a nice way
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-mysql-summary.html
www.percona.com 15
17. Master ? Slave ?... both ?
Name: ptslavefind
Function: find and print replication hierarchy
tree of MySQL slaves.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-slave-find.html
www.percona.com 17
19. Where's My Disk I/O Going ?
● Use ptdiskstats as an improved iostat
➢ Interactively controllable, slice-and-dice
➢ Doesn't lump reads and writes together
➢ Meaningful, correct columns
● Use ptioprofile to measure & time your I/O
➢ Which files is MySQL using ? How much times does it
spend reading, writing, syncing, etc? How many
reads/writes? How large are they ?
pt-ioprofile can be dangerous in production
www.percona.com 19
20. Disk I/O
Name: ptdiskstats
Function: Aggregate and summarize
/proc/diskstats
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-diskstats.html
www.percona.com 20
22. I/O Profile
Name: ptioprofile
Function: Watch process IO and print a table
of file and I/O activity.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-ioprofile.html
www.percona.com 22
25. Let's try to find the answer to these
indiscreet questions:
● How are the indexes used ?
● Are there duplicate keys ?
● Which queries are eating most of the
resources ?
www.percona.com 25
26. Analyze Schema & Queries
● Duplicate/redundant indexes or foreign keys?
➢ ptduplicatekeychecker can tell you that
● Which indexes are unused?
➢ ptindexusage can tell you that
➢ It can also show which queries have unstable
plans, which indexes have alternatives that are
never chosen, and lots more
● Bad SQL?
➢ ptqueryadvisor to the rescue
www.percona.com 26
27. Analyze & Profile Your Workload
● Use ptquerydigest to analyze the “slow
query log” and show a profile of the workload
● It does the Right Thing by default
● Mostly used with slow query logs & tcpdumps
www.percona.com 27
28. The Report
● Each class of query is ranked by execution time
● There is a report for each class of query
● It's easy to use
● It's a great way to start optimizing queries
www.percona.com 28
33. Log & Analyze Errors ?
● Use ptdeadlocklogger to capture
deadlocks
● Use ptfkerrorlogger for foreign key
errors
● The tools give you the possibility to log the
results to a database table for analysis
www.percona.com 33
34. Monitor dead locks
Name: ptdeadlocklogger
Function: extract and log MySQL deadlock
information
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-deadlock-logger.html
● SHOW ENGINE INNODB STATUS reports only the most
recent deadlock
● Information in status output lacks details about user, client
host, other locks in the deadlock cycle.
● ptdeadlocklogger formats the needed information
nicely and saves it
www.percona.com 34
35. Monitor Foreign Key Errors
Name: ptfkerrorlogger
Function: extract and log MySQL foreign key
errors.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-fk-error-logger.html
● Like ptdeadlocklogger, this tool collects more useful
information, formats it nicely and saves it.
www.percona.com 35
36. Non-Blocking ALTER TABLE
● Use ptonlineschemachange to alter
tables
➢ It makes a “shadow copy” and swaps them
➢ Extremely useful for large, long-running
ALTER
➢ Facebook uses the same technique
www.percona.com 36
37. Validate Upgrades
● Upgrades are a leading cause of downtime
➢ Are any queries using different indexes ? Is QEP
different ?
➢ Are there new errors or warnings ?
➢ Do queries return the same results ?
➢ Are there significant response time differences ?
● Validate the results before you upgrade!
➢ Capture a snapshot of the data
➢ Capture a log of queries
➢ Compare the queries on two test servers running the
old and the new version
www.percona.com 37
38. Validate updates
Name: ptupgrade
Function: Execute queries on multiple
servers and check for differences.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-upgrade.html
www.percona.com 38
41. Verify replication Integrity
Name: pttablechecksum
Function: perform an online replication
consistency check, or checksum MySQL
tables efficiently on one or many servers.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-table-checksum.html
● Use it routinely (mandatory for 95% of MySQL users)
● It used to be complicated; no more. Just run it!
● Put it in a weekly crontab
● Repair differences with pttablesync
www.percona.com 41
43. Find problems
● What do you advise me doctor ?
● Get the symptoms when it hurts !
www.percona.com 43
44. Get the symptoms when it hurts !
To perform this action, do a check-up
when you feel bad:
● ptstalk
● ptcollect
● ptsift
www.percona.com 44
45. Does it hurt ?
Name: ptstalk
Function: wait for a condition to occur then
begin collecting data.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-stalk.html
www.percona.com 45
46. We need your health parameters
Name: ptcollect
Function: collect information from a server
for some period of time.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-collect.html
www.percona.com 46
47. Let me check...
Name: ptsift
Function: browses files created by pt-collect.
Url: http://www.percona.com/doc/percona-toolkit/2.0/pt-collect.html
www.percona.com 47
48. Very useful
Name: ptmext
Function: Look at many samples of MySQL
SHOW GLOBAL STATUS side-by-side
Url: http://www.percona.com/doc/percona-toolkit/pt-mext.html
● Default STATUS shows counter since the MySQL
instances started.
● It is more helpful to see a delta of recent activity, to check
if a change you applied made a difference.
www.percona.com 48
49. pt-mext
ptmext r mysqladmin ext i10 c5
All values are interesting but you should pay even more attention to:
● Created_tmp_*
● Handler_read_*
● Innodb_buffer_pool_read*
● Innodb_os_log_written
● Key_read_requests
● Key_reads
● Qchache*
● Select_full_*join
● Select_range
● Select_scan
● Table_locks_waited
● Threads_running
www.percona.com 49
50. Stop Bad Queries
● Use ptkill to watch for long-running queries
and optionally kill them
● Prevent long-running idle transactions too.
www.percona.com 50
51. Percona Toolkit's Future
● Rapid development & improvement of tools
● Fully supported via Percona Support contract
● Upcoming:
➢ ptquerydigest “query reviews” improved
➢ ptstalk “magical fault detection algorithm”
➢ new ptdisktats using Perl
➢ Lots more goodies
•https://blueprints.launchpad.net/percona-toolkit
www.percona.com 51
52. Thank you for your participation to
FOSDEM MySQL & Friends devroom