SlideShare une entreprise Scribd logo
1  sur  61
Télécharger pour lire hors ligne
The MySQL SYS Schema 
Mark Leith 
Senior Software Development Manager 
MySQL Enterprise Tools, Oracle 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement 
The following is intended to outline our general product direction. It is intended 
for information purposes only, and may not be incorporated into any contract. It 
is not a commitment to deliver any material, code, or functionality, and should 
not be relied upon in making purchasing decisions. The development, release, 
and timing of any features or functionality described for Oracle’s products 
remains at the sole discretion of Oracle. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Program Agenda 
Introduction to the MySQL sys schema 
Installation 
Summary Views 
Procedures and Functions 
1 
2 
3 
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Program Agenda 
Introduction to the MySQL sys schema 
Installation 
Summary Views 
Procedures and Functions 
1 
2 
3 
4
Introduction to the MySQL sys schema 
• Originally called “ps_helper” 
• A collection of views, procedures and functions, designed to help 
DBA, Developer, and Ops teams 
• Implements many of the common day to day DBA and Developer 
debugging and tuning use cases 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Why the sys schema? 
• Performance Schema exposes a huge amount of data, that can be 
sliced and diced in many ways 
• Many of its tables can implement many different use cases when 
looking at the raw data 
• The sys schema layers on top the (sometimes complex) queries in 
to a packaged set of easy to use views 
• The sys schema also layers on a number of functions and 
procedures that help format, and record the raw data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Program Agenda 
Introduction to the MySQL sys schema 
Installation 
Summary Views 
Procedures and Functions 
1 
2 
3 
4
Installation from the command line 
• $ git clone https://github.com/MarkLeith/mysql-sys.git /tmp/sys 
• $ cd /tmp/sys 
• $ mysql -u user -p < sys_<version>.sql 
! 
! 
<version> can be 56 or 57, for 5.6 and 5.7 respectively 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Installation from MySQL Workbench 6.1+ 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
! 
! 
select * from sys.version;! 
+-------------+---------------+! 
| sys_version | mysql_version |! 
+-------------+---------------+! 
| 1.1.0 | 5.6.17 |! 
+-------------+---------------+ 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
11 
Checking sys is loaded 
select *! 
from sys.schema_object_overview! 
where db = 'sys';! 
+-----+-------------+-------+! 
| db | object_type | count |! 
+-----+-------------+-------+! 
| sys | VIEW | 78 |! 
| sys | PROCEDURE | 22 |! 
| sys | FUNCTION | 11 |! 
+-----+-------------+-------+
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Program Agenda 
Introduction to the MySQL sys schema 
Installation 
Summary Views 
Procedures and Functions 
1 
2 
3 
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
MySQL sys views 
• Reference set of views solving various administrator use cases 
• Simple views, create/copy your own, sys is not “locked down” 
• Build upon both performance_schema and INFORMATION_SCHEMA 
• Both formatted and raw views are available 
• All raw views are prefixed with x$ 
• Raw views are there for tools to poll 
• Formatted views are for humans and the command line
! 
• High level user/host overviews 
• Breakdowns by: 
• IO usage 
• Stages 
• Statement details 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Following show users only 
14 
User / Host Summary Views 
user_summary 
user_summary_by_file_io 
user_summary_by_file_io_type 
user_summary_by_stages 
user_summary_by_statement_latency 
user_summary_by_statement_type 
host_summary 
host_summary_by_file_io 
host_summary_by_file_io_type 
host_summary_by_stages 
host_summary_by_statement_latency 
host_summary_by_statement_type
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
user_summary view 
mysql> select * from user_summaryG! 
*************************** 1. row ***************************! 
user: mark! 
statements: 3072! 
statement_latency: 1.77 s! 
statement_avg_latency: 575.29 us! 
table_scans: 7! 
file_ios: 20043! 
file_io_latency: 346.79 ms! 
current_connections: 1! 
total_connections: 1! 
unique_hosts: 1! 
current_memory: 515.81 KiB! 
total_memory_allocated: 30.69 MiB 
MySQL 5.7.4+
user_summary_by_file_io_type view 
mysql> select * from user_summary_by_file_io_type;! 
+------+--------------------------------------+---------+-------------+-------------+! 
| user | event_name | total | latency | max_latency |! 
+------+--------------------------------------+---------+-------------+-------------+! 
| mark | wait/io/file/myisam/dfile | 19540 | 215.20 ms | 120.79 ms |! 
| mark | wait/io/file/myisam/kfile | 501 | 131.58 ms | 57.74 ms |! 
| mark | wait/io/file/sql/dbopt | 2 | 17.29 us | 9.17 us |! 
| root | wait/io/file/innodb/innodb_log_file | 2079432 | 00:28:27.62 | 1.30 s |! 
| root | wait/io/file/myisam/kfile | 5195927 | 00:17:07.33 | 1.17 s |! 
| root | wait/io/file/myisam/dfile | 1533813 | 00:07:27.59 | 1.18 s |! 
| root | wait/io/file/sql/binlog | 3517008 | 00:03:07.34 | 1.42 s |! 
| root | wait/io/file/innodb/innodb_data_file | 20720 | 00:02:37.81 | 492.23 ms |! 
| root | wait/io/file/sql/FRM | 41412 | 3.17 s | 80.43 ms |! 
| root | wait/io/file/sql/dbopt | 110993 | 1.23 s | 63.27 ms |! 
! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
user_summary_by_statement_latency view 
mysql> select * from user_summary_by_statement_latencyG! 
*************************** 1. row ***************************! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
user: mark! 
total: 176! 
total_latency: 1.26 s! 
max_latency: 583.33 ms! 
lock_latency: 216.95 ms! 
rows_sent: 311! 
rows_examined: 307! 
rows_affected: 0! 
full_scans: 7!
user_summary_by_statement_type view 
mysql> select * from user_summary_by_statement_typeG! 
*************************** 1. row ***************************! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
user: mark! 
statement: commit! 
total: 2463190! 
total_latency: 13.33h! 
max_latency: 12.18 s! 
lock_latency: 0 ps! 
rows_sent: 0! 
rows_examined: 0! 
rows_affected: 0! 
full_scans: 0!
! 
• IO by current threads 
• Global summaries by file and 
class, aggregating by bytes or 
latency 
• Stream of last file IO events in 
raw form 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
19 
IO Summary Views 
! 
io_by_thread_by_latency 
io_global_by_file_by_bytes 
io_global_by_file_by_latency 
io_global_by_wait_by_bytes 
io_global_by_wait_by_latency
io_by_thread_by_latency view 
mysql> select * from io_by_thread_by_latencyG! 
*************************** 1. row ***************************! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
user: main! 
total: 1193! 
total_latency: 136.45 ms! 
min_latency: 731.38 ns! 
avg_latency: 269.99 us! 
max_latency: 20.72 ms! 
thread_id: 1! 
processlist_id: NULL! 
*************************** 2. row ***************************! 
user: root@localhost! 
total: 533! 
total_latency: 53.81 ms! 
min_latency: 780.39 ns! 
avg_latency: 8.41 ms! 
max_latency: 25.06 ms! 
thread_id: 23! 
processlist_id: 4
io_global_by_file_by_latency view 
mysql> select * from io_global_by_file_by_latency limit 1G! 
*************************** 1. row ***************************! 
file: @@datadir/ibdata1! 
total: 1395714! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
total_latency: 00:28:53.06! 
count_read: 1006! 
read_latency: 4.51 s! 
count_write: 1326529! 
write_latency: 33.39 s! 
count_misc: 68179! 
misc_latency: 00:28:15.16
latest_file_io view 
mysql> select * from latest_file_io;! 
+-----------------------------+----------------------------------+-----------+-----------+-----------+! 
| thread | file | latency | operation | requested |! 
+-----------------------------+----------------------------------+-----------+-----------+-----------+! 
| root@localhost:63153:179371 | @@datadir/ib_logfile1 | 7.16 us | lock | NULL |! 
| root@localhost:63153:179371 | @@datadir/ib_logfile1 | 6.16 us | write | 1.50 KiB |! 
| root@localhost:63153:179371 | @@datadir/ib_logfile1 | 14.36 ms | sync | NULL |! 
| root@localhost:63153:179371 | @@datadir/cerberus-bin.000012 | 28.25 us | write | 905 bytes |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 15.25 us | write | 1.00 KiB |! 
| root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 32.12 us | read | 905 bytes |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 6.65 us | write | 1.00 KiB |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 213.32 us | sync | NULL |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 17.27 us | write | 1.00 KiB |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 286.01 us | sync | NULL |! 
| root@localhost:63177:179394 | @@datadir/cerberus-bin.000012 | 18.77 us | write | 601 bytes |! 
| root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 7.55 us | read | 601 bytes |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 10.67 us | write | 1.00 KiB |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 200.28 us | sync | NULL |! 
| root@localhost:63177:179394 | @@datadir/cerberus-bin.000012 | 12.91 us | write | 601 bytes |! 
| root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 6.47 us | read | 601 bytes |! 
| root@localhost:63177:179394 | @@datadir/ib_logfile1 | 9.22 us | write | 1.50 KiB |! 
…… 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Object overview 
• Table usage statistics 
• Index usage statistics 
• Following show users only 
23 
Schema Analysis Views 
schema_index_statistics 
schema_object_overview 
schema_table_statistics 
schema_table_statistics_with_buffer 
schema_tables_with_full_table_scans 
schema_unused_indexes
schema_table_statistics view 
mysql> select * from schema_table_statistics limit 1G! 
*************************** 1. row ***************************! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
table_schema: mem__quan! 
table_name: normalized_statements_by_server_by_schema_data! 
total_latency: 00:55:07.80! 
rows_fetched: 239685! 
fetch_latency: 17.10 s! 
rows_inserted: 1102895! 
insert_latency: 00:52:18.10! 
rows_updated: 55910! 
update_latency: 00:02:28.62! 
rows_deleted: 17899! 
delete_latency: 3.98 s 
io_read_requests: 20639! 
io_read: 324.94 MiB! 
io_read_latency: 00:03:04.10! 
io_write_requests: 1965973! 
io_write: 40.36 GiB! 
io_write_latency: 25.46 s! 
io_misc_requests: 18217! 
io_misc_latency: 00:14:06.02
schema_index_statistics view 
mysql> select * from schema_index_statistics limit 1G! 
*************************** 1. row ***************************! 
table_schema: mem__inventory! 
table_name: mysqlconnectionconfiguration! 
index_name: PRIMARY! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
rows_selected: 131399! 
select_latency: 00:07:05.03! 
rows_inserted: 0! 
insert_latency: 0 ps! 
rows_updated: 41608! 
update_latency: 21.96 s! 
rows_deleted: 0! 
delete_latency: 0 ps
schema_tables_with_full_table_scans view 
mysql> select * from schema_tables_with_full_table_scans limit 10;! 
+--------------------+--------------------------------+-------------------+-----------+! 
| object_schema | object_name | rows_full_scanned | latency |! 
+--------------------+--------------------------------+-------------------+-----------+! 
| mem30__instruments | fsstatistics | 10207042 | 13.10 s |! 
| mem30__instruments | preparedstatementapidata | 436428 | 973.27 ms |! 
| mem30__instruments | mysqlprocessactivity | 411702 | 282.07 ms |! 
| mem30__instruments | querycachequeriesincachedata | 374011 | 767.15 ms |! 
| mem30__instruments | rowaccessesdata | 322321 | 1.55 s |! 
| mem30__instruments | connectionsmaxdata | 301242 | 857.63 ms |! 
| mem30__instruments | innodbdatadictionarymemorydata | 292564 | 495.38 ms |! 
| mem30__instruments | innodbbufferpooldata | 277112 | 1.95 s |! 
| mem30__instruments | avgrowaccessesdata | 269236 | 1.25 s |! 
| mem30__instruments | innodbrowdetailsdata | 248208 | 653.25 ms |! 
+--------------------+--------------------------------+-------------------+-----------+ 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
schema_unused_indexes view 
mysql> select * from schema_unused_indexes limit 10;! 
+--------------------+-----------------------------------+--------------------+! 
| object_schema | object_name | index_name |! 
+--------------------+-----------------------------------+--------------------+! 
| mem30__bean_config | plists | path |! 
| mem30__config | group_selections | name |! 
| mem30__config | notification_groups | name |! 
| mem30__config | user_form_defaults | FKC1AEF1F9E7EE2CFB |! 
| mem30__enterprise | whats_new_entries | entryId |! 
| mem30__events | action_logs | policyId |! 
| mem30__events | action_logs | eventId |! 
| mem30__events | action_logs | ts |! 
| mem30__events | advisor_scopes_advisorcategoryids | FKF0CB0DB962CAB4 |! 
| mem30__events | advisor_scopes_advisorclassids | FK650AF1BF962CAB4 |! 
+--------------------+-----------------------------------+--------------------+ 
You should ensure you have representative time frame before taking any actions! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Wait summaries by class 
• Wait details per: 
• User 
• Host 
• Globally 
28 
Wait Analysis Views 
wait_classes_global_by_avg_latency 
wait_classes_global_by_latency 
waits_by_host_by_latency 
waits_by_user_by_latency 
waits_global_by_latency 
!
waits_global_by_latency view 
mysql> select * from waits_global_by_latency limit 10;! 
+--------------------------------------+----------+---------------+-------------+-------------+! 
| events | total | total_latency | avg_latency | max_latency |! 
+--------------------------------------+----------+---------------+-------------+-------------+! 
| wait/io/table/sql/handler | 24196659 | 1.75h | 259.72 us | 2.29 s |! 
| wait/io/file/innodb/innodb_data_file | 5964255 | 1.36h | 822.01 us | 1.49 s |! 
| wait/io/file/innodb/innodb_log_file | 2272647 | 00:36:41.37 | 968.64 us | 1.30 s |! 
| wait/io/file/myisam/kfile | 5463587 | 00:18:14.16 | 200.26 us | 1.17 s |! 
| wait/io/file/myisam/dfile | 1715187 | 00:08:00.18 | 279.96 us | 1.18 s |! 
| wait/io/file/sql/binlog | 3722079 | 00:03:23.71 | 54.73 us | 1.42 s |! 
| wait/lock/table/sql/handler | 10074274 | 00:01:56.63 | 11.58 us | 186.90 ms |! 
| wait/io/socket/sql/client_connection | 3879827 | 25.36 s | 6.54 us | 219.25 ms |! 
| wait/io/file/sql/FRM | 43994 | 3.27 s | 74.35 us | 80.43 ms |! 
| wait/io/file/sql/dbopt | 116765 | 1.29 s | 11.03 us | 63.27 ms |! 
+--------------------------------------+----------+---------------+-------------+-------------+ 
InnoDB only instance, this MyISAM IO is all temporary tables, focus tuning there! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
waits_by_user_by_latency view 
mysql> select * from waits_by_user_by_latency;! 
+------+--------------------------------------+----------+---------------+-------------+-------------+! 
| user | event | total | total_latency | avg_latency | max_latency |! 
+------+--------------------------------------+----------+---------------+-------------+-------------+! 
| mark | wait/io/file/myisam/dfile | 19540 | 215.20 ms | 11.01 us | 120.79 ms |! 
| mark | wait/io/file/myisam/kfile | 501 | 131.58 ms | 262.63 us | 57.74 ms |! 
| mark | wait/io/file/sql/dbopt | 2 | 17.29 us | 8.64 us | 9.17 us |! 
| root | wait/io/table/sql/handler | 24189319 | 1.74h | 258.77 us | 2.29 s |! 
| root | wait/io/file/innodb/innodb_log_file | 2162650 | 00:30:52.87 | 856.76 us | 1.30 s |! 
| root | wait/io/file/myisam/kfile | 5460992 | 00:18:12.83 | 200.11 us | 1.17 s |! 
| root | wait/io/file/myisam/dfile | 1694624 | 00:07:59.50 | 282.95 us | 1.18 s |! 
| root | wait/io/file/sql/binlog | 3711637 | 00:03:23.16 | 54.74 us | 1.42 s |! 
| root | wait/io/file/innodb/innodb_data_file | 22577 | 00:03:05.73 | 8.23 ms | 492.23 ms |! 
| root | wait/lock/table/sql/handler | 10069000 | 00:01:56.59 | 11.58 us | 186.90 ms |! 
…… 
root is doing the most temporary table IO 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Statement overview 
• Find statements by: 
• Those with errors 
• Those with full table scans 
• Those creating temp tables 
• Those that cause sorting 
• With latency in 95th %ile 
31 
Statement Analysis Views 
statement_analysis 
statements_with_errors_or_warnings 
statements_with_full_table_scans 
statements_with_runtimes_in_95th_p 
ercentile 
statements_with_sorting 
statements_with_temp_tables
statement_analysis view 
mysql> select * from statement_analysis limit 2G! 
*************************** 1. row ***************************! 
query: INSERT INTO `mem30__quan` . `n ... nDuration` = IF ( VALUES ( ...! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
db: mem! 
full_scan:! 
exec_count: 4657! 
err_count: 0! 
warn_count: 0! 
total_latency: 00:03:57.40! 
max_latency: 10.54 s! 
avg_latency: 50.98 ms! 
lock_latency: 00:03:00.93! 
rows_sent: 0! 
rows_sent_avg: 0! 
rows_examined: 0! 
rows_examined_avg: 0! 
rows_affected: 4794! 
rows_affected_avg: 1 
tmp_tables: 0! 
tmp_disk_tables: 0! 
rows_sorted: 0! 
sort_merge_passes: 0! 
digest: b98bfe333642e8e4e72c73515810d865! 
first_seen: 2014-09-24 11:27:01! 
last_seen: 2014-09-24 11:53:00
statements_with_errors_or_warnings view 
mysql> select * from statements_with_errors_or_warnings limit 1G! 
*************************** 1. row ***************************! 
query: CREATE TEMPORARY TABLE IF NOT ... _logs` ( `id` INT8 NOT NULL )! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
db: mem! 
exec_count: 1725! 
errors: 1725! 
error_pct: 100.0000! 
warnings: 0! 
warning_pct: 0.0000! 
first_seen: 2014-05-20 10:42:32! 
last_seen: 2014-05-21 18:39:22! 
digest: 51fb979dbc3910a6294c7cdabd7839c3
statements_with_temp_tables view 
mysql> select * from statements_with_temp_tables limit 1G! 
*************************** 1. row ***************************! 
query: SELECT * FROM ( SELECT `digest ... ` , 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
`sum_no_index_used` AS ...! 
db: mysql! 
exec_count: 68! 
total_latency: 2.31 s! 
memory_tmp_tables: 476! 
disk_tmp_tables: 204! 
avg_tmp_tables_per_query: 7! 
tmp_tables_to_disk_pct: 43! 
first_seen: 2014-09-24 11:26:18! 
last_seen: 2014-09-24 12:33:00! 
digest: 50761c6a1818824328745d8a136b9ed6
statements_with_full_table_scans view 
mysql> select * from statements_with_full_table_scans limit 1G! 
*************************** 1. row ***************************! 
query: SELECT `pojotimese0_` . `id` A ... 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
ty_timeseries` `pojotimese0_`! 
db: mem! 
exec_count: 1! 
total_latency: 940.89 us! 
no_index_used_count: 1! 
no_good_index_used_count: 0! 
no_index_used_pct: 100! 
rows_sent: 116! 
rows_examined: 116! 
rows_sent_avg: 116! 
rows_examined_avg: 116! 
first_seen: 2014-09-24 11:25:03! 
last_seen: 2014-09-24 11:25:03! 
digest: f81b5ce10baee9be754cf35fd245a516
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Better SHOW PROCESSLIST 
• No mutex contention 
• Live statistics for query 
36 
Miscellaneous Views 
processlist 
!
processlist view 
mysql> select * from processlistG! 
*************************** 1. row ***************************! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
thd_id: 7890! 
conn_id: 7870! 
user: root@localhost! 
db: mem! 
command: Query! 
state: update! 
time: 0! 
current_statement: /* mem dbpool.default */ inser ...<truncated>! 
lock_latency: 0 ps! 
rows_examined: 0! 
rows_sent: 0! 
rows_affected: 0! 
tmp_tables: 0! 
! 
! 
tmp_disk_tables: 0! 
full_scan: NO! 
last_statement: NULL! 
last_statement_latency: NULL! 
last_wait: wait/io/table/sql/handler! 
last_wait_latency: Still Waiting! 
source: handler.cc:7274
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
Program Agenda 
Introduction to the MySQL sys schema 
Installation 
Summary Views 
Procedures and Functions 
1 
2 
3 
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
! 
• Make output human readable 
• Format time appropriately 
• Format bytes appropriately 
• Truncate output for large 
width values for CLI 
• Extract object names 
• Dump a thread stack 
• Check instrumented actors 
39 
Functions 
format_time 
format_bytes 
format_path 
format_statement 
extract_table_from_file_name 
extract_schema_from_file_name 
ps_thread_stack 
ps_is_account_enabled 
!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
format_time() function 
mysql> select format_time(23849723429) as time! 
-> union select format_time(8327423749233)! 
-> union select format_time(83274237492335);! 
+-------------+! 
| time |! 
+-------------+! 
| 23.85 ms |! 
| 8.33 s |! 
| 00:01:23.27 |! 
+-------------+! 
! 
!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
format_bytes() function 
mysql> select format_bytes(23423) as bytes! 
-> union select format_bytes(23432423)! 
-> union select format_bytes(42839479283)! 
-> union select format_bytes(2293848203489);! 
+-----------+! 
| bytes |! 
+-----------+! 
| 22.87 KiB |! 
| 22.35 MiB |! 
| 39.90 GiB |! 
| 2.09 TiB |! 
+-----------+!
Performance Schema Config Helper Procedures 
• ps_setup_show_disabled() / ps_setup_show_enabled() 
• ps_setup_disable_thread() / ps_setup_enable_thread() 
• ps_setup_disable_background_threads() / 
ps_setup_enable_background_threads() 
• ps_setup_disable_instrument() / ps_setup_enable_instrument() 
• ps_setup_disable_consumer() / ps_setup_enable_consumer() 
• ps_setup_save() / ps_setup_reload_saved() 
• ps_setup_reset_to_default() / ps_truncate_all_tables() 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
! 
• Dump all P_S data to a dot 
graph file to trace a thread 
• Look in the statement history 
table to capture more info on 
a statement digest 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
43 
Statement Tracing Procedures 
ps_trace_thread 
ps_trace_statement_digest 
ps_statement_avg_latency_histogram 
!
ps_trace_statement_digest procedure 
• ps_trace_statement_digest() analyses live traffic looking for 
certain statement digest for a period of time 
• Captures statistics on each matching statement it finds 
• Returns a report of the captured stats 
• An overall summary 
• A break down for the longest running example 
• An EXPLAIN (if the statement is not truncated) 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_statement_digest procedure 
• ps_trace_statement_digest() parameters 
• in_digest The statement digest to analyse 
• in_runtime How long to run analysis for 
• in_interval How often to snapshot data 
• in_start_fresh Whether to truncate P_S tables first 
• in_auto_enable Whether to auto enable required config 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_statement_digest procedure example 
!!! 
mysql> call analyze_statement_digest('6134e9d6f25eb8e6cddf11f6938f202a', 60, 1, true, true); 
+--------------------+ 
| SUMMARY STATISTICS | 
+--------------------+ 
| SUMMARY STATISTICS | 
+--------------------+ 
1 row in set (59.93 sec) 
! 
+------------+-----------+-----------+-----------+---------------+------------+------------+ 
| executions | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scans | 
+------------+-----------+-----------+-----------+---------------+------------+------------+ 
| 360 | 1.41 s | 138.39 ms | 720 | 0 | 0 | 0 | 
+------------+-----------+-----------+-----------+---------------+------------+------------+ 
1 row in set (59.93 sec) 
! 
+--------------------------------+-------+-----------+ 
| event_name | count | latency | 
+--------------------------------+-------+-----------+ 
| stage/sql/Sending data | 14 | 1.10 s | 
| stage/sql/update | 198 | 667.34 ms | 
| stage/sql/removing tmp table | 9 | 240.48 ms | 
... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_statement_digest procedure example 
!!! 
+---------------------------+ 
| LONGEST RUNNING STATEMENT | 
+---------------------------+ 
| LONGEST RUNNING STATEMENT | 
+---------------------------+ 
! 
1 row in set (59.95 sec) +-----------+-----------+-----------+-----------+---------------+------------+-----------+ 
| thread_id | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scan | 
+-----------+-----------+-----------+-----------+---------------+------------+-----------+ 
| 23277 | 77.64 ms | 97.00 us | 2 | 0 | 0 | 0 | 
+-----------+-----------+-----------+-----------+---------------+------------+-----------+ 
! 
1 row in set (59.95 sec) +-------------------------------------------------------------------------------------------+ 
| sql_text | 
+-------------------------------------------------------------------------------------------+ 
| /* mem dbpool.default */ insert into `mem__quan`.`normalized_statements_by_server_by_schema` 
(firstSeen, lastSeen, normalized_statement_id, `schema`, server, id) values (1378983745000, 
1378998600000, x'808ACEB88FC4B45BC47C4FE9D86C7F26', 'mem', '753c939e-1b99-11e3-b6d4-bc761a1f4f2f', 
x'7CFE81F98B1E3FE0895D1AA2C39B326D') ON DUPLICATE KEY UPDATE ... 
+-------------------------------------------------------------------------------------------+ 
... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_statement_digest procedure example 
+--------------------------------+----------+ 
| event_name | latency | 
+--------------------------------+----------+ 
| stage/sql/init | 10.41 ms | 
| stage/sql/checking permissions | 2.35 us | 
| stage/sql/Opening tables | 10.32 us | 
| stage/sql/init | 13.53 us | 
| stage/sql/System lock | 3.66 us | 
| stage/sql/update | 61.74 ms | 
| stage/sql/end | 1.11 us | 
| stage/sql/query end | 1.31 us | 
| stage/sql/closing tables | 5.44 ms | 
| stage/sql/freeing items | 23.31 us | 
+--------------------------------+----------+ 
10 rows in set (59.99 sec) ! 
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+ 
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | 
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+ 
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used | 
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+ 
... !!! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_thread procedure 
• ps_trace_thread() monitors a specific thread for a period 
• Captures as much information on the thread activity as possible 
• Returns a “dot” formatted file, that can graph the event hierarchy 
• http://en.wikipedia.org/wiki/DOT_(graph_description_language) 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_statement_digest procedure 
• ps_trace_statement_digest() parameters 
• in_thread_id The thread to analyse 
• in_outfile The file to dump the data to (INTO OUTFILE) 
• in_max_runtime How long to run analysis for 
• in_interval How often to snapshot for data 
• in_start_fresh Whether to truncate P_S tables first 
• in_auto_setup Whether to auto enable required config 
• in_debug Whether to also print debug info (source info) 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_thread procedure example 
mysql> call ps_trace_thread(27768, '/tmp/stack_27768.dot', 60, 0.1, true, true, true); 
+------------------------------------------------+ 
| Info | 
+------------------------------------------------+ 
| Data collection starting for THREAD_ID = 27768 | 
+------------------------------------------------+ 
1 row in set (4.82 sec) ! 
+---------------------------------------------+ 
| Info | 
+---------------------------------------------+ 
| Stack trace written to /tmp/stack_27768.dot | 
+---------------------------------------------+ 
1 row in set (60.90 sec) ! 
+--------------------------------------------------------+ 
| Convert to PDF | 
+--------------------------------------------------------+ 
| dot -Tpdf -o /tmp/stack_27768.pdf /tmp/stack_27768.dot | 
+--------------------------------------------------------+ 
1 row in set (60.90 sec) ! 
+--------------------------------------------------------+ 
| Convert to PNG | 
+--------------------------------------------------------+ 
| dot -Tpng -o /tmp/stack_27768.png /tmp/stack_27768.dot | 
+--------------------------------------------------------+ 
1 row in set (60.90 sec)+ 
! 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ps_trace_thread procedure example 
cerberus:~ mark$ more Documents/statement_graphs/deletes.dot! 
digraph events {! 
graph [rankdir=LR, nodesep="0.10"];! 
node [label="N"];! 
graph [bb="0,0,1860,17476"];! 
1 [label="(0.20μ) mysys/THR_LOCK_threadsn", style=filled, color=lightskyblue, pos="209,18", width="3.19", height="0.50"];! 
2 [label="(0.18μ) sql/THD::LOCK_thd_datan", style=filled, color=lightskyblue, pos="209,61", width="3.06", height="0.50"];! 
3 [label="(0.24μ) sql/hash_filo::lockn", style=filled, color=lightskyblue, pos="209,104", width="2.42", height=“0.50"];! 
4 [label="(0.31μ) sql/LOGGER::LOCK_logger - read_lockn", style=filled, color=orchid, pos="209,147", width="4.22", 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 
height="0.50"];! 
5 [label="(0.08μ) sql/LOG::LOCK_logn", style=filled, color=lightskyblue, pos="209,190", width="2.64", height="0.50"];! 
6 [label="(12.97μ) wait/io/file/sql/query_log - write 49 bytesn/Users/mark/mysql/lp-mysql-trunk/mysql-test/var/mysqld.1/ 
mysqld.logn", style=filled, color=red, shape=box, pos="209,235", width="5.78", height="0.56"];! 
7 [label="(176.14μ) sql/selectnselect @@version_comment limit 1nerrors: 0nwarnings: 0nlock time: 0.00μnrows affected: 
0nrows sent:! 
209,18", width="3.19", height="0.50"];mp disk tables: 0nselect scan: 0nselect full join: :! 
2 [label="(0.18μ) sql/THD::LOCK_thd_datan", style=filled, color=lightskyblue, pos="209,61", width="3.06", height="0.50"];! 
3 [label="(0.24μ) sql/hash_filo::lockn", style=filled, color=lightskyblue, pos="209,104", width="2.42", height="0.50"];! 
4 [label="(0.31μ) sql/LOGGER::LOCK_logger - read_lockn", style=filled, color=orchid, pos="209,147", width="4.22", 
height="0.50"];! 
5 [label="(0.08μ) sql/LOG::LOCK_logn", style=filled, color=lightskyblue, pos="209,190", width="2.64", height="0.50"];! 
……! !
ps_trace_thread procedure example 
MySQL 5.7 
! 
Procedures 
Transactions 
Statements 
Stages 
Waits 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
The MySQL SYS Schema

Contenu connexe

Tendances

MySQL Performance Schema in 20 Minutes
 MySQL Performance Schema in 20 Minutes MySQL Performance Schema in 20 Minutes
MySQL Performance Schema in 20 MinutesSveta Smirnova
 
Monitoring Oracle Database Instances with Zabbix
Monitoring Oracle Database Instances with ZabbixMonitoring Oracle Database Instances with Zabbix
Monitoring Oracle Database Instances with ZabbixGerger
 
How Safe is Asynchronous Master-Master Setup?
 How Safe is Asynchronous Master-Master Setup? How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
 
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesMySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesKenny Gryp
 
Redefining tables online without surprises
Redefining tables online without surprisesRedefining tables online without surprises
Redefining tables online without surprisesNelson Calero
 
Ceph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideCeph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideKaran Singh
 
Performance tuning in BlueStore & RocksDB - Li Xiaoyan
Performance tuning in BlueStore & RocksDB - Li XiaoyanPerformance tuning in BlueStore & RocksDB - Li Xiaoyan
Performance tuning in BlueStore & RocksDB - Li XiaoyanCeph Community
 
EM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsEM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsMaaz Anjum
 
Oracle Clusterware Node Management and Voting Disks
Oracle Clusterware Node Management and Voting DisksOracle Clusterware Node Management and Voting Disks
Oracle Clusterware Node Management and Voting DisksMarkus Michalewicz
 
Optimizing MariaDB for maximum performance
Optimizing MariaDB for maximum performanceOptimizing MariaDB for maximum performance
Optimizing MariaDB for maximum performanceMariaDB plc
 
MariaDB Galera Cluster
MariaDB Galera ClusterMariaDB Galera Cluster
MariaDB Galera ClusterAbdul Manaf
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1Federico Campoli
 
Oracle database performance tuning
Oracle database performance tuningOracle database performance tuning
Oracle database performance tuningYogiji Creations
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovAltinity Ltd
 
What to Expect From Oracle database 19c
What to Expect From Oracle database 19cWhat to Expect From Oracle database 19c
What to Expect From Oracle database 19cMaria Colgan
 
How to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleHow to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleMariaDB plc
 
ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortNAVER D2
 
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfJesmar Cannao'
 

Tendances (20)

MySQL Performance Schema in 20 Minutes
 MySQL Performance Schema in 20 Minutes MySQL Performance Schema in 20 Minutes
MySQL Performance Schema in 20 Minutes
 
Monitoring Oracle Database Instances with Zabbix
Monitoring Oracle Database Instances with ZabbixMonitoring Oracle Database Instances with Zabbix
Monitoring Oracle Database Instances with Zabbix
 
How Safe is Asynchronous Master-Master Setup?
 How Safe is Asynchronous Master-Master Setup? How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?
 
Oracle ASM Training
Oracle ASM TrainingOracle ASM Training
Oracle ASM Training
 
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesMySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
 
Redefining tables online without surprises
Redefining tables online without surprisesRedefining tables online without surprises
Redefining tables online without surprises
 
Ceph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing GuideCeph Object Storage Reference Architecture Performance and Sizing Guide
Ceph Object Storage Reference Architecture Performance and Sizing Guide
 
Performance tuning in BlueStore & RocksDB - Li Xiaoyan
Performance tuning in BlueStore & RocksDB - Li XiaoyanPerformance tuning in BlueStore & RocksDB - Li Xiaoyan
Performance tuning in BlueStore & RocksDB - Li Xiaoyan
 
EM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM MetricsEM12c: Capacity Planning with OEM Metrics
EM12c: Capacity Planning with OEM Metrics
 
Oracle Clusterware Node Management and Voting Disks
Oracle Clusterware Node Management and Voting DisksOracle Clusterware Node Management and Voting Disks
Oracle Clusterware Node Management and Voting Disks
 
Optimizing MariaDB for maximum performance
Optimizing MariaDB for maximum performanceOptimizing MariaDB for maximum performance
Optimizing MariaDB for maximum performance
 
MariaDB Galera Cluster
MariaDB Galera ClusterMariaDB Galera Cluster
MariaDB Galera Cluster
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1
 
Oracle database performance tuning
Oracle database performance tuningOracle database performance tuning
Oracle database performance tuning
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
 
What to Expect From Oracle database 19c
What to Expect From Oracle database 19cWhat to Expect From Oracle database 19c
What to Expect From Oracle database 19c
 
How to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScaleHow to Manage Scale-Out Environments with MariaDB MaxScale
How to Manage Scale-Out Environments with MariaDB MaxScale
 
MySQL SQL Tutorial
MySQL SQL TutorialMySQL SQL Tutorial
MySQL SQL Tutorial
 
ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-short
 
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdf
 

En vedette

Performance schema and sys schema
Performance schema and sys schemaPerformance schema and sys schema
Performance schema and sys schemaMark Leith
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAsMark Leith
 
Developing Information Schema Plugins
Developing Information Schema PluginsDeveloping Information Schema Plugins
Developing Information Schema PluginsMark Leith
 
Extending MySQL Enterprise Monitor
Extending MySQL Enterprise MonitorExtending MySQL Enterprise Monitor
Extending MySQL Enterprise MonitorMark Leith
 
Performance Schema and Sys Schema in MySQL 5.7
Performance Schema and Sys Schema in MySQL 5.7Performance Schema and Sys Schema in MySQL 5.7
Performance Schema and Sys Schema in MySQL 5.7Mark Leith
 
Instrumenting plugins for Performance Schema
Instrumenting plugins for Performance SchemaInstrumenting plugins for Performance Schema
Instrumenting plugins for Performance SchemaMark Leith
 

En vedette (6)

Performance schema and sys schema
Performance schema and sys schemaPerformance schema and sys schema
Performance schema and sys schema
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAs
 
Developing Information Schema Plugins
Developing Information Schema PluginsDeveloping Information Schema Plugins
Developing Information Schema Plugins
 
Extending MySQL Enterprise Monitor
Extending MySQL Enterprise MonitorExtending MySQL Enterprise Monitor
Extending MySQL Enterprise Monitor
 
Performance Schema and Sys Schema in MySQL 5.7
Performance Schema and Sys Schema in MySQL 5.7Performance Schema and Sys Schema in MySQL 5.7
Performance Schema and Sys Schema in MySQL 5.7
 
Instrumenting plugins for Performance Schema
Instrumenting plugins for Performance SchemaInstrumenting plugins for Performance Schema
Instrumenting plugins for Performance Schema
 

Similaire à The MySQL SYS Schema

MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep diveMark Leith
 
MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep diveMark Leith
 
Basic MySQL Troubleshooting for Oracle DBAs
Basic MySQL Troubleshooting for Oracle DBAsBasic MySQL Troubleshooting for Oracle DBAs
Basic MySQL Troubleshooting for Oracle DBAsSveta Smirnova
 
MySQL Manchester TT - Performance Tuning
MySQL Manchester TT  - Performance TuningMySQL Manchester TT  - Performance Tuning
MySQL Manchester TT - Performance TuningMark Swarbrick
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAsBen Krug
 
Mysql nowwhat
Mysql nowwhatMysql nowwhat
Mysql nowwhatsqlhjalp
 
MySQL Tech Tour 2015 - Manage & Tune
MySQL Tech Tour 2015 - Manage & TuneMySQL Tech Tour 2015 - Manage & Tune
MySQL Tech Tour 2015 - Manage & TuneMark Swarbrick
 
제3회난공불락 오픈소스 인프라세미나 - MySQL
제3회난공불락 오픈소스 인프라세미나 - MySQL제3회난공불락 오픈소스 인프라세미나 - MySQL
제3회난공불락 오픈소스 인프라세미나 - MySQLTommy Lee
 
Mysql 57-upcoming-changes
Mysql 57-upcoming-changesMysql 57-upcoming-changes
Mysql 57-upcoming-changesMorgan Tocker
 
6 Tips to MySQL Performance Tuning
6 Tips to MySQL Performance Tuning6 Tips to MySQL Performance Tuning
6 Tips to MySQL Performance TuningOracleMySQL
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMorgan Tocker
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 Geir Høydalsvik
 
2_MySQL_Cluster_Introduction.pdf
2_MySQL_Cluster_Introduction.pdf2_MySQL_Cluster_Introduction.pdf
2_MySQL_Cluster_Introduction.pdfHaiping Li
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAsMario Beck
 
제3회난공불락 오픈소스 인프라세미나 - MySQL Performance
제3회난공불락 오픈소스 인프라세미나 - MySQL Performance제3회난공불락 오픈소스 인프라세미나 - MySQL Performance
제3회난공불락 오픈소스 인프라세미나 - MySQL PerformanceTommy Lee
 

Similaire à The MySQL SYS Schema (20)

MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep dive
 
MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep dive
 
Basic MySQL Troubleshooting for Oracle DBAs
Basic MySQL Troubleshooting for Oracle DBAsBasic MySQL Troubleshooting for Oracle DBAs
Basic MySQL Troubleshooting for Oracle DBAs
 
MySQL Manchester TT - Performance Tuning
MySQL Manchester TT  - Performance TuningMySQL Manchester TT  - Performance Tuning
MySQL Manchester TT - Performance Tuning
 
My sql 5.6&MySQL Cluster 7.3
My sql 5.6&MySQL Cluster 7.3My sql 5.6&MySQL Cluster 7.3
My sql 5.6&MySQL Cluster 7.3
 
MySQL NoSQL APIs
MySQL NoSQL APIsMySQL NoSQL APIs
MySQL NoSQL APIs
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAs
 
Curso de MySQL 5.7
Curso de MySQL 5.7Curso de MySQL 5.7
Curso de MySQL 5.7
 
Mysql nowwhat
Mysql nowwhatMysql nowwhat
Mysql nowwhat
 
MySQL Tech Tour 2015 - Manage & Tune
MySQL Tech Tour 2015 - Manage & TuneMySQL Tech Tour 2015 - Manage & Tune
MySQL Tech Tour 2015 - Manage & Tune
 
제3회난공불락 오픈소스 인프라세미나 - MySQL
제3회난공불락 오픈소스 인프라세미나 - MySQL제3회난공불락 오픈소스 인프라세미나 - MySQL
제3회난공불락 오픈소스 인프라세미나 - MySQL
 
Mysql 57-upcoming-changes
Mysql 57-upcoming-changesMysql 57-upcoming-changes
Mysql 57-upcoming-changes
 
6 Tips to MySQL Performance Tuning
6 Tips to MySQL Performance Tuning6 Tips to MySQL Performance Tuning
6 Tips to MySQL Performance Tuning
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
 
Mysql tracing
Mysql tracingMysql tracing
Mysql tracing
 
Mysql tracing
Mysql tracingMysql tracing
Mysql tracing
 
2_MySQL_Cluster_Introduction.pdf
2_MySQL_Cluster_Introduction.pdf2_MySQL_Cluster_Introduction.pdf
2_MySQL_Cluster_Introduction.pdf
 
MySQL for Oracle DBAs
MySQL for Oracle DBAsMySQL for Oracle DBAs
MySQL for Oracle DBAs
 
제3회난공불락 오픈소스 인프라세미나 - MySQL Performance
제3회난공불락 오픈소스 인프라세미나 - MySQL Performance제3회난공불락 오픈소스 인프라세미나 - MySQL Performance
제3회난공불락 오픈소스 인프라세미나 - MySQL Performance
 

Plus de Mark Leith

Mysql tech day_paris_ps_and_sys
Mysql tech day_paris_ps_and_sysMysql tech day_paris_ps_and_sys
Mysql tech day_paris_ps_and_sysMark Leith
 
MySQL 5.7: Performance Schema Improvements
MySQL 5.7: Performance Schema ImprovementsMySQL 5.7: Performance Schema Improvements
MySQL 5.7: Performance Schema ImprovementsMark Leith
 
Introduction to MySQL Enterprise Monitor
Introduction to MySQL Enterprise MonitorIntroduction to MySQL Enterprise Monitor
Introduction to MySQL Enterprise MonitorMark Leith
 
MySQL Administration and Monitoring
MySQL Administration and MonitoringMySQL Administration and Monitoring
MySQL Administration and MonitoringMark Leith
 
Performance schema and_ps_helper
Performance schema and_ps_helperPerformance schema and_ps_helper
Performance schema and_ps_helperMark Leith
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMark Leith
 
Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorMark Leith
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMark Leith
 

Plus de Mark Leith (8)

Mysql tech day_paris_ps_and_sys
Mysql tech day_paris_ps_and_sysMysql tech day_paris_ps_and_sys
Mysql tech day_paris_ps_and_sys
 
MySQL 5.7: Performance Schema Improvements
MySQL 5.7: Performance Schema ImprovementsMySQL 5.7: Performance Schema Improvements
MySQL 5.7: Performance Schema Improvements
 
Introduction to MySQL Enterprise Monitor
Introduction to MySQL Enterprise MonitorIntroduction to MySQL Enterprise Monitor
Introduction to MySQL Enterprise Monitor
 
MySQL Administration and Monitoring
MySQL Administration and MonitoringMySQL Administration and Monitoring
MySQL Administration and Monitoring
 
Performance schema and_ps_helper
Performance schema and_ps_helperPerformance schema and_ps_helper
Performance schema and_ps_helper
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring Mechanisms
 
Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise Monitor
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring Mechanisms
 

Dernier

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 

Dernier (20)

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 

The MySQL SYS Schema

  • 1.
  • 2. The MySQL SYS Schema Mark Leith Senior Software Development Manager MySQL Enterprise Tools, Oracle Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 3. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction to the MySQL sys schema Installation Summary Views Procedures and Functions 1 2 3 4
  • 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction to the MySQL sys schema Installation Summary Views Procedures and Functions 1 2 3 4
  • 6. Introduction to the MySQL sys schema • Originally called “ps_helper” • A collection of views, procedures and functions, designed to help DBA, Developer, and Ops teams • Implements many of the common day to day DBA and Developer debugging and tuning use cases Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Why the sys schema? • Performance Schema exposes a huge amount of data, that can be sliced and diced in many ways • Many of its tables can implement many different use cases when looking at the raw data • The sys schema layers on top the (sometimes complex) queries in to a packaged set of easy to use views • The sys schema also layers on a number of functions and procedures that help format, and record the raw data
  • 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction to the MySQL sys schema Installation Summary Views Procedures and Functions 1 2 3 4
  • 9. Installation from the command line • $ git clone https://github.com/MarkLeith/mysql-sys.git /tmp/sys • $ cd /tmp/sys • $ mysql -u user -p < sys_<version>.sql ! ! <version> can be 56 or 57, for 5.6 and 5.7 respectively Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 10. Installation from MySQL Workbench 6.1+ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 11. ! ! select * from sys.version;! +-------------+---------------+! | sys_version | mysql_version |! +-------------+---------------+! | 1.1.0 | 5.6.17 |! +-------------+---------------+ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11 Checking sys is loaded select *! from sys.schema_object_overview! where db = 'sys';! +-----+-------------+-------+! | db | object_type | count |! +-----+-------------+-------+! | sys | VIEW | 78 |! | sys | PROCEDURE | 22 |! | sys | FUNCTION | 11 |! +-----+-------------+-------+
  • 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction to the MySQL sys schema Installation Summary Views Procedures and Functions 1 2 3 4
  • 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL sys views • Reference set of views solving various administrator use cases • Simple views, create/copy your own, sys is not “locked down” • Build upon both performance_schema and INFORMATION_SCHEMA • Both formatted and raw views are available • All raw views are prefixed with x$ • Raw views are there for tools to poll • Formatted views are for humans and the command line
  • 14. ! • High level user/host overviews • Breakdowns by: • IO usage • Stages • Statement details Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Following show users only 14 User / Host Summary Views user_summary user_summary_by_file_io user_summary_by_file_io_type user_summary_by_stages user_summary_by_statement_latency user_summary_by_statement_type host_summary host_summary_by_file_io host_summary_by_file_io_type host_summary_by_stages host_summary_by_statement_latency host_summary_by_statement_type
  • 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | user_summary view mysql> select * from user_summaryG! *************************** 1. row ***************************! user: mark! statements: 3072! statement_latency: 1.77 s! statement_avg_latency: 575.29 us! table_scans: 7! file_ios: 20043! file_io_latency: 346.79 ms! current_connections: 1! total_connections: 1! unique_hosts: 1! current_memory: 515.81 KiB! total_memory_allocated: 30.69 MiB MySQL 5.7.4+
  • 16. user_summary_by_file_io_type view mysql> select * from user_summary_by_file_io_type;! +------+--------------------------------------+---------+-------------+-------------+! | user | event_name | total | latency | max_latency |! +------+--------------------------------------+---------+-------------+-------------+! | mark | wait/io/file/myisam/dfile | 19540 | 215.20 ms | 120.79 ms |! | mark | wait/io/file/myisam/kfile | 501 | 131.58 ms | 57.74 ms |! | mark | wait/io/file/sql/dbopt | 2 | 17.29 us | 9.17 us |! | root | wait/io/file/innodb/innodb_log_file | 2079432 | 00:28:27.62 | 1.30 s |! | root | wait/io/file/myisam/kfile | 5195927 | 00:17:07.33 | 1.17 s |! | root | wait/io/file/myisam/dfile | 1533813 | 00:07:27.59 | 1.18 s |! | root | wait/io/file/sql/binlog | 3517008 | 00:03:07.34 | 1.42 s |! | root | wait/io/file/innodb/innodb_data_file | 20720 | 00:02:37.81 | 492.23 ms |! | root | wait/io/file/sql/FRM | 41412 | 3.17 s | 80.43 ms |! | root | wait/io/file/sql/dbopt | 110993 | 1.23 s | 63.27 ms |! ! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 17. user_summary_by_statement_latency view mysql> select * from user_summary_by_statement_latencyG! *************************** 1. row ***************************! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | user: mark! total: 176! total_latency: 1.26 s! max_latency: 583.33 ms! lock_latency: 216.95 ms! rows_sent: 311! rows_examined: 307! rows_affected: 0! full_scans: 7!
  • 18. user_summary_by_statement_type view mysql> select * from user_summary_by_statement_typeG! *************************** 1. row ***************************! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | user: mark! statement: commit! total: 2463190! total_latency: 13.33h! max_latency: 12.18 s! lock_latency: 0 ps! rows_sent: 0! rows_examined: 0! rows_affected: 0! full_scans: 0!
  • 19. ! • IO by current threads • Global summaries by file and class, aggregating by bytes or latency • Stream of last file IO events in raw form Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 19 IO Summary Views ! io_by_thread_by_latency io_global_by_file_by_bytes io_global_by_file_by_latency io_global_by_wait_by_bytes io_global_by_wait_by_latency
  • 20. io_by_thread_by_latency view mysql> select * from io_by_thread_by_latencyG! *************************** 1. row ***************************! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | user: main! total: 1193! total_latency: 136.45 ms! min_latency: 731.38 ns! avg_latency: 269.99 us! max_latency: 20.72 ms! thread_id: 1! processlist_id: NULL! *************************** 2. row ***************************! user: root@localhost! total: 533! total_latency: 53.81 ms! min_latency: 780.39 ns! avg_latency: 8.41 ms! max_latency: 25.06 ms! thread_id: 23! processlist_id: 4
  • 21. io_global_by_file_by_latency view mysql> select * from io_global_by_file_by_latency limit 1G! *************************** 1. row ***************************! file: @@datadir/ibdata1! total: 1395714! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | total_latency: 00:28:53.06! count_read: 1006! read_latency: 4.51 s! count_write: 1326529! write_latency: 33.39 s! count_misc: 68179! misc_latency: 00:28:15.16
  • 22. latest_file_io view mysql> select * from latest_file_io;! +-----------------------------+----------------------------------+-----------+-----------+-----------+! | thread | file | latency | operation | requested |! +-----------------------------+----------------------------------+-----------+-----------+-----------+! | root@localhost:63153:179371 | @@datadir/ib_logfile1 | 7.16 us | lock | NULL |! | root@localhost:63153:179371 | @@datadir/ib_logfile1 | 6.16 us | write | 1.50 KiB |! | root@localhost:63153:179371 | @@datadir/ib_logfile1 | 14.36 ms | sync | NULL |! | root@localhost:63153:179371 | @@datadir/cerberus-bin.000012 | 28.25 us | write | 905 bytes |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 15.25 us | write | 1.00 KiB |! | root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 32.12 us | read | 905 bytes |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 6.65 us | write | 1.00 KiB |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 213.32 us | sync | NULL |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 17.27 us | write | 1.00 KiB |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 286.01 us | sync | NULL |! | root@localhost:63177:179394 | @@datadir/cerberus-bin.000012 | 18.77 us | write | 601 bytes |! | root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 7.55 us | read | 601 bytes |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 10.67 us | write | 1.00 KiB |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 200.28 us | sync | NULL |! | root@localhost:63177:179394 | @@datadir/cerberus-bin.000012 | 12.91 us | write | 601 bytes |! | root@localhost:57487:3 | @@datadir/cerberus-bin.000012 | 6.47 us | read | 601 bytes |! | root@localhost:63177:179394 | @@datadir/ib_logfile1 | 9.22 us | write | 1.50 KiB |! …… Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Object overview • Table usage statistics • Index usage statistics • Following show users only 23 Schema Analysis Views schema_index_statistics schema_object_overview schema_table_statistics schema_table_statistics_with_buffer schema_tables_with_full_table_scans schema_unused_indexes
  • 24. schema_table_statistics view mysql> select * from schema_table_statistics limit 1G! *************************** 1. row ***************************! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | table_schema: mem__quan! table_name: normalized_statements_by_server_by_schema_data! total_latency: 00:55:07.80! rows_fetched: 239685! fetch_latency: 17.10 s! rows_inserted: 1102895! insert_latency: 00:52:18.10! rows_updated: 55910! update_latency: 00:02:28.62! rows_deleted: 17899! delete_latency: 3.98 s io_read_requests: 20639! io_read: 324.94 MiB! io_read_latency: 00:03:04.10! io_write_requests: 1965973! io_write: 40.36 GiB! io_write_latency: 25.46 s! io_misc_requests: 18217! io_misc_latency: 00:14:06.02
  • 25. schema_index_statistics view mysql> select * from schema_index_statistics limit 1G! *************************** 1. row ***************************! table_schema: mem__inventory! table_name: mysqlconnectionconfiguration! index_name: PRIMARY! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | rows_selected: 131399! select_latency: 00:07:05.03! rows_inserted: 0! insert_latency: 0 ps! rows_updated: 41608! update_latency: 21.96 s! rows_deleted: 0! delete_latency: 0 ps
  • 26. schema_tables_with_full_table_scans view mysql> select * from schema_tables_with_full_table_scans limit 10;! +--------------------+--------------------------------+-------------------+-----------+! | object_schema | object_name | rows_full_scanned | latency |! +--------------------+--------------------------------+-------------------+-----------+! | mem30__instruments | fsstatistics | 10207042 | 13.10 s |! | mem30__instruments | preparedstatementapidata | 436428 | 973.27 ms |! | mem30__instruments | mysqlprocessactivity | 411702 | 282.07 ms |! | mem30__instruments | querycachequeriesincachedata | 374011 | 767.15 ms |! | mem30__instruments | rowaccessesdata | 322321 | 1.55 s |! | mem30__instruments | connectionsmaxdata | 301242 | 857.63 ms |! | mem30__instruments | innodbdatadictionarymemorydata | 292564 | 495.38 ms |! | mem30__instruments | innodbbufferpooldata | 277112 | 1.95 s |! | mem30__instruments | avgrowaccessesdata | 269236 | 1.25 s |! | mem30__instruments | innodbrowdetailsdata | 248208 | 653.25 ms |! +--------------------+--------------------------------+-------------------+-----------+ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 27. schema_unused_indexes view mysql> select * from schema_unused_indexes limit 10;! +--------------------+-----------------------------------+--------------------+! | object_schema | object_name | index_name |! +--------------------+-----------------------------------+--------------------+! | mem30__bean_config | plists | path |! | mem30__config | group_selections | name |! | mem30__config | notification_groups | name |! | mem30__config | user_form_defaults | FKC1AEF1F9E7EE2CFB |! | mem30__enterprise | whats_new_entries | entryId |! | mem30__events | action_logs | policyId |! | mem30__events | action_logs | eventId |! | mem30__events | action_logs | ts |! | mem30__events | advisor_scopes_advisorcategoryids | FKF0CB0DB962CAB4 |! | mem30__events | advisor_scopes_advisorclassids | FK650AF1BF962CAB4 |! +--------------------+-----------------------------------+--------------------+ You should ensure you have representative time frame before taking any actions! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Wait summaries by class • Wait details per: • User • Host • Globally 28 Wait Analysis Views wait_classes_global_by_avg_latency wait_classes_global_by_latency waits_by_host_by_latency waits_by_user_by_latency waits_global_by_latency !
  • 29. waits_global_by_latency view mysql> select * from waits_global_by_latency limit 10;! +--------------------------------------+----------+---------------+-------------+-------------+! | events | total | total_latency | avg_latency | max_latency |! +--------------------------------------+----------+---------------+-------------+-------------+! | wait/io/table/sql/handler | 24196659 | 1.75h | 259.72 us | 2.29 s |! | wait/io/file/innodb/innodb_data_file | 5964255 | 1.36h | 822.01 us | 1.49 s |! | wait/io/file/innodb/innodb_log_file | 2272647 | 00:36:41.37 | 968.64 us | 1.30 s |! | wait/io/file/myisam/kfile | 5463587 | 00:18:14.16 | 200.26 us | 1.17 s |! | wait/io/file/myisam/dfile | 1715187 | 00:08:00.18 | 279.96 us | 1.18 s |! | wait/io/file/sql/binlog | 3722079 | 00:03:23.71 | 54.73 us | 1.42 s |! | wait/lock/table/sql/handler | 10074274 | 00:01:56.63 | 11.58 us | 186.90 ms |! | wait/io/socket/sql/client_connection | 3879827 | 25.36 s | 6.54 us | 219.25 ms |! | wait/io/file/sql/FRM | 43994 | 3.27 s | 74.35 us | 80.43 ms |! | wait/io/file/sql/dbopt | 116765 | 1.29 s | 11.03 us | 63.27 ms |! +--------------------------------------+----------+---------------+-------------+-------------+ InnoDB only instance, this MyISAM IO is all temporary tables, focus tuning there! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 30. waits_by_user_by_latency view mysql> select * from waits_by_user_by_latency;! +------+--------------------------------------+----------+---------------+-------------+-------------+! | user | event | total | total_latency | avg_latency | max_latency |! +------+--------------------------------------+----------+---------------+-------------+-------------+! | mark | wait/io/file/myisam/dfile | 19540 | 215.20 ms | 11.01 us | 120.79 ms |! | mark | wait/io/file/myisam/kfile | 501 | 131.58 ms | 262.63 us | 57.74 ms |! | mark | wait/io/file/sql/dbopt | 2 | 17.29 us | 8.64 us | 9.17 us |! | root | wait/io/table/sql/handler | 24189319 | 1.74h | 258.77 us | 2.29 s |! | root | wait/io/file/innodb/innodb_log_file | 2162650 | 00:30:52.87 | 856.76 us | 1.30 s |! | root | wait/io/file/myisam/kfile | 5460992 | 00:18:12.83 | 200.11 us | 1.17 s |! | root | wait/io/file/myisam/dfile | 1694624 | 00:07:59.50 | 282.95 us | 1.18 s |! | root | wait/io/file/sql/binlog | 3711637 | 00:03:23.16 | 54.74 us | 1.42 s |! | root | wait/io/file/innodb/innodb_data_file | 22577 | 00:03:05.73 | 8.23 ms | 492.23 ms |! | root | wait/lock/table/sql/handler | 10069000 | 00:01:56.59 | 11.58 us | 186.90 ms |! …… root is doing the most temporary table IO Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Statement overview • Find statements by: • Those with errors • Those with full table scans • Those creating temp tables • Those that cause sorting • With latency in 95th %ile 31 Statement Analysis Views statement_analysis statements_with_errors_or_warnings statements_with_full_table_scans statements_with_runtimes_in_95th_p ercentile statements_with_sorting statements_with_temp_tables
  • 32. statement_analysis view mysql> select * from statement_analysis limit 2G! *************************** 1. row ***************************! query: INSERT INTO `mem30__quan` . `n ... nDuration` = IF ( VALUES ( ...! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | db: mem! full_scan:! exec_count: 4657! err_count: 0! warn_count: 0! total_latency: 00:03:57.40! max_latency: 10.54 s! avg_latency: 50.98 ms! lock_latency: 00:03:00.93! rows_sent: 0! rows_sent_avg: 0! rows_examined: 0! rows_examined_avg: 0! rows_affected: 4794! rows_affected_avg: 1 tmp_tables: 0! tmp_disk_tables: 0! rows_sorted: 0! sort_merge_passes: 0! digest: b98bfe333642e8e4e72c73515810d865! first_seen: 2014-09-24 11:27:01! last_seen: 2014-09-24 11:53:00
  • 33. statements_with_errors_or_warnings view mysql> select * from statements_with_errors_or_warnings limit 1G! *************************** 1. row ***************************! query: CREATE TEMPORARY TABLE IF NOT ... _logs` ( `id` INT8 NOT NULL )! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | db: mem! exec_count: 1725! errors: 1725! error_pct: 100.0000! warnings: 0! warning_pct: 0.0000! first_seen: 2014-05-20 10:42:32! last_seen: 2014-05-21 18:39:22! digest: 51fb979dbc3910a6294c7cdabd7839c3
  • 34. statements_with_temp_tables view mysql> select * from statements_with_temp_tables limit 1G! *************************** 1. row ***************************! query: SELECT * FROM ( SELECT `digest ... ` , Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | `sum_no_index_used` AS ...! db: mysql! exec_count: 68! total_latency: 2.31 s! memory_tmp_tables: 476! disk_tmp_tables: 204! avg_tmp_tables_per_query: 7! tmp_tables_to_disk_pct: 43! first_seen: 2014-09-24 11:26:18! last_seen: 2014-09-24 12:33:00! digest: 50761c6a1818824328745d8a136b9ed6
  • 35. statements_with_full_table_scans view mysql> select * from statements_with_full_table_scans limit 1G! *************************** 1. row ***************************! query: SELECT `pojotimese0_` . `id` A ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ty_timeseries` `pojotimese0_`! db: mem! exec_count: 1! total_latency: 940.89 us! no_index_used_count: 1! no_good_index_used_count: 0! no_index_used_pct: 100! rows_sent: 116! rows_examined: 116! rows_sent_avg: 116! rows_examined_avg: 116! first_seen: 2014-09-24 11:25:03! last_seen: 2014-09-24 11:25:03! digest: f81b5ce10baee9be754cf35fd245a516
  • 36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Better SHOW PROCESSLIST • No mutex contention • Live statistics for query 36 Miscellaneous Views processlist !
  • 37. processlist view mysql> select * from processlistG! *************************** 1. row ***************************! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | thd_id: 7890! conn_id: 7870! user: root@localhost! db: mem! command: Query! state: update! time: 0! current_statement: /* mem dbpool.default */ inser ...<truncated>! lock_latency: 0 ps! rows_examined: 0! rows_sent: 0! rows_affected: 0! tmp_tables: 0! ! ! tmp_disk_tables: 0! full_scan: NO! last_statement: NULL! last_statement_latency: NULL! last_wait: wait/io/table/sql/handler! last_wait_latency: Still Waiting! source: handler.cc:7274
  • 38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction to the MySQL sys schema Installation Summary Views Procedures and Functions 1 2 3 4
  • 39. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | ! • Make output human readable • Format time appropriately • Format bytes appropriately • Truncate output for large width values for CLI • Extract object names • Dump a thread stack • Check instrumented actors 39 Functions format_time format_bytes format_path format_statement extract_table_from_file_name extract_schema_from_file_name ps_thread_stack ps_is_account_enabled !
  • 40. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | format_time() function mysql> select format_time(23849723429) as time! -> union select format_time(8327423749233)! -> union select format_time(83274237492335);! +-------------+! | time |! +-------------+! | 23.85 ms |! | 8.33 s |! | 00:01:23.27 |! +-------------+! ! !
  • 41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | format_bytes() function mysql> select format_bytes(23423) as bytes! -> union select format_bytes(23432423)! -> union select format_bytes(42839479283)! -> union select format_bytes(2293848203489);! +-----------+! | bytes |! +-----------+! | 22.87 KiB |! | 22.35 MiB |! | 39.90 GiB |! | 2.09 TiB |! +-----------+!
  • 42. Performance Schema Config Helper Procedures • ps_setup_show_disabled() / ps_setup_show_enabled() • ps_setup_disable_thread() / ps_setup_enable_thread() • ps_setup_disable_background_threads() / ps_setup_enable_background_threads() • ps_setup_disable_instrument() / ps_setup_enable_instrument() • ps_setup_disable_consumer() / ps_setup_enable_consumer() • ps_setup_save() / ps_setup_reload_saved() • ps_setup_reset_to_default() / ps_truncate_all_tables() Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 43. ! • Dump all P_S data to a dot graph file to trace a thread • Look in the statement history table to capture more info on a statement digest Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 43 Statement Tracing Procedures ps_trace_thread ps_trace_statement_digest ps_statement_avg_latency_histogram !
  • 44. ps_trace_statement_digest procedure • ps_trace_statement_digest() analyses live traffic looking for certain statement digest for a period of time • Captures statistics on each matching statement it finds • Returns a report of the captured stats • An overall summary • A break down for the longest running example • An EXPLAIN (if the statement is not truncated) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 45. ps_trace_statement_digest procedure • ps_trace_statement_digest() parameters • in_digest The statement digest to analyse • in_runtime How long to run analysis for • in_interval How often to snapshot data • in_start_fresh Whether to truncate P_S tables first • in_auto_enable Whether to auto enable required config Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 46. ps_trace_statement_digest procedure example !!! mysql> call analyze_statement_digest('6134e9d6f25eb8e6cddf11f6938f202a', 60, 1, true, true); +--------------------+ | SUMMARY STATISTICS | +--------------------+ | SUMMARY STATISTICS | +--------------------+ 1 row in set (59.93 sec) ! +------------+-----------+-----------+-----------+---------------+------------+------------+ | executions | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scans | +------------+-----------+-----------+-----------+---------------+------------+------------+ | 360 | 1.41 s | 138.39 ms | 720 | 0 | 0 | 0 | +------------+-----------+-----------+-----------+---------------+------------+------------+ 1 row in set (59.93 sec) ! +--------------------------------+-------+-----------+ | event_name | count | latency | +--------------------------------+-------+-----------+ | stage/sql/Sending data | 14 | 1.10 s | | stage/sql/update | 198 | 667.34 ms | | stage/sql/removing tmp table | 9 | 240.48 ms | ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 47. ps_trace_statement_digest procedure example !!! +---------------------------+ | LONGEST RUNNING STATEMENT | +---------------------------+ | LONGEST RUNNING STATEMENT | +---------------------------+ ! 1 row in set (59.95 sec) +-----------+-----------+-----------+-----------+---------------+------------+-----------+ | thread_id | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scan | +-----------+-----------+-----------+-----------+---------------+------------+-----------+ | 23277 | 77.64 ms | 97.00 us | 2 | 0 | 0 | 0 | +-----------+-----------+-----------+-----------+---------------+------------+-----------+ ! 1 row in set (59.95 sec) +-------------------------------------------------------------------------------------------+ | sql_text | +-------------------------------------------------------------------------------------------+ | /* mem dbpool.default */ insert into `mem__quan`.`normalized_statements_by_server_by_schema` (firstSeen, lastSeen, normalized_statement_id, `schema`, server, id) values (1378983745000, 1378998600000, x'808ACEB88FC4B45BC47C4FE9D86C7F26', 'mem', '753c939e-1b99-11e3-b6d4-bc761a1f4f2f', x'7CFE81F98B1E3FE0895D1AA2C39B326D') ON DUPLICATE KEY UPDATE ... +-------------------------------------------------------------------------------------------+ ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 48. ps_trace_statement_digest procedure example +--------------------------------+----------+ | event_name | latency | +--------------------------------+----------+ | stage/sql/init | 10.41 ms | | stage/sql/checking permissions | 2.35 us | | stage/sql/Opening tables | 10.32 us | | stage/sql/init | 13.53 us | | stage/sql/System lock | 3.66 us | | stage/sql/update | 61.74 ms | | stage/sql/end | 1.11 us | | stage/sql/query end | 1.31 us | | stage/sql/closing tables | 5.44 ms | | stage/sql/freeing items | 23.31 us | +--------------------------------+----------+ 10 rows in set (59.99 sec) ! +----+-------------+-------+------+---------------+------+---------+------+------+----------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+------+------+----------------+ | 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used | +----+-------------+-------+------+---------------+------+---------+------+------+----------------+ ... !!! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 49. ps_trace_thread procedure • ps_trace_thread() monitors a specific thread for a period • Captures as much information on the thread activity as possible • Returns a “dot” formatted file, that can graph the event hierarchy • http://en.wikipedia.org/wiki/DOT_(graph_description_language) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 50. ps_trace_statement_digest procedure • ps_trace_statement_digest() parameters • in_thread_id The thread to analyse • in_outfile The file to dump the data to (INTO OUTFILE) • in_max_runtime How long to run analysis for • in_interval How often to snapshot for data • in_start_fresh Whether to truncate P_S tables first • in_auto_setup Whether to auto enable required config • in_debug Whether to also print debug info (source info) Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 51. ps_trace_thread procedure example mysql> call ps_trace_thread(27768, '/tmp/stack_27768.dot', 60, 0.1, true, true, true); +------------------------------------------------+ | Info | +------------------------------------------------+ | Data collection starting for THREAD_ID = 27768 | +------------------------------------------------+ 1 row in set (4.82 sec) ! +---------------------------------------------+ | Info | +---------------------------------------------+ | Stack trace written to /tmp/stack_27768.dot | +---------------------------------------------+ 1 row in set (60.90 sec) ! +--------------------------------------------------------+ | Convert to PDF | +--------------------------------------------------------+ | dot -Tpdf -o /tmp/stack_27768.pdf /tmp/stack_27768.dot | +--------------------------------------------------------+ 1 row in set (60.90 sec) ! +--------------------------------------------------------+ | Convert to PNG | +--------------------------------------------------------+ | dot -Tpng -o /tmp/stack_27768.png /tmp/stack_27768.dot | +--------------------------------------------------------+ 1 row in set (60.90 sec)+ ! Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 52. ps_trace_thread procedure example cerberus:~ mark$ more Documents/statement_graphs/deletes.dot! digraph events {! graph [rankdir=LR, nodesep="0.10"];! node [label="N"];! graph [bb="0,0,1860,17476"];! 1 [label="(0.20μ) mysys/THR_LOCK_threadsn", style=filled, color=lightskyblue, pos="209,18", width="3.19", height="0.50"];! 2 [label="(0.18μ) sql/THD::LOCK_thd_datan", style=filled, color=lightskyblue, pos="209,61", width="3.06", height="0.50"];! 3 [label="(0.24μ) sql/hash_filo::lockn", style=filled, color=lightskyblue, pos="209,104", width="2.42", height=“0.50"];! 4 [label="(0.31μ) sql/LOGGER::LOCK_logger - read_lockn", style=filled, color=orchid, pos="209,147", width="4.22", Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | height="0.50"];! 5 [label="(0.08μ) sql/LOG::LOCK_logn", style=filled, color=lightskyblue, pos="209,190", width="2.64", height="0.50"];! 6 [label="(12.97μ) wait/io/file/sql/query_log - write 49 bytesn/Users/mark/mysql/lp-mysql-trunk/mysql-test/var/mysqld.1/ mysqld.logn", style=filled, color=red, shape=box, pos="209,235", width="5.78", height="0.56"];! 7 [label="(176.14μ) sql/selectnselect @@version_comment limit 1nerrors: 0nwarnings: 0nlock time: 0.00μnrows affected: 0nrows sent:! 209,18", width="3.19", height="0.50"];mp disk tables: 0nselect scan: 0nselect full join: :! 2 [label="(0.18μ) sql/THD::LOCK_thd_datan", style=filled, color=lightskyblue, pos="209,61", width="3.06", height="0.50"];! 3 [label="(0.24μ) sql/hash_filo::lockn", style=filled, color=lightskyblue, pos="209,104", width="2.42", height="0.50"];! 4 [label="(0.31μ) sql/LOGGER::LOCK_logger - read_lockn", style=filled, color=orchid, pos="209,147", width="4.22", height="0.50"];! 5 [label="(0.08μ) sql/LOG::LOCK_logn", style=filled, color=lightskyblue, pos="209,190", width="2.64", height="0.50"];! ……! !
  • 53. ps_trace_thread procedure example MySQL 5.7 ! Procedures Transactions Statements Stages Waits Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 54. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 55. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 56. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 57. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 58. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 59. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 60. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |