New data dictionary an internal server api that matters

New Data Dictionary:
An Internal Server API
That Matters
Alexander Nozdrin, Principle Software Developer
Copyright Copyright © © 2014, 2014, Oracle Oracle and/and/or or its its affiliates. affiliates. All All rights rights reserved. reserved.
|

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
2

MySQL Community Reception @ Oracle OpenWorld
Mingle with the MySQL community and the MySQL team from Oracle for
a fun and informative evening!
• Time: September 30 (Tue) @ 7pm
• Jillian’s at Metreon
175 Fourth Street, San Francisco, CA
At the corner of Howard and 4th st.; only 2-min walk from Moscone Center

Agenda
What is a Data Dictionary?
The MySQL Traditional Data Dictionary
New Data Dictionary
Benefits for users
Q & A
1
2
3
4
5

Data Dictionary
What is it

Data Dictionary
Definition
• Metadata is information about user data
– User table structure
– Column definitions
– Index definitions
– Foreign key definitions
– Stored program definitions
...
• Data Dictionary collects all metadata in RDBMS

CREATE PROCEDURE p1(v INT)
SQL SECURITY INVOKER
BEGIN
...
END
Data Dictionary
Overview
Data Dictionary
Table Definitions SP Definitions
View Definitions Plugins
Time zones
Privileges
CREATE TABLE customers(
id INT AUTO_INCREMENT
...
PRIMARY KEY (id),
INDEX ...
FOREIGN KEY ...
)

SE InnoDB
MySQL Server
Data Dictionary Definition
8
The ecosystem
Query Executor
Optimizer
Performance
Schema
SQL
statement
Client
Parser
Result
Information
Schema
Data
Dictionary

MySQL 5.6 and earlier

• A mix of files and tables:
– File based
• Tables: FRM
• Triggers: TRN, TRG
...
– Table based
• mysql.time_zone
...
• InnoDB has a separate data dictionary

11
Data Dictionary
Files
FRM TRG OPT
System tables (mysql.)
user time_zone proc
InnoDB internal data dictionary
Archive
CSV
InnoDB
INFORMATION_SCHEMA
File
Scan
Table
Scan
Intern.
Access

• Poor INFORMATION_SCHEMA performance
• Makes crash-safe / transactional DDL impossible
• Inconsistencies between files and tables
• Inconsistencies between DD in InnoDB and the server
• File-system dependency (lower-case-table-names)
• Makes replication of DDL statements difficult
• Too difficult to extend
12
Problems

New Data Dictionary
A great leap forward

New Data Dictionary : Main Features
• Stored in InnoDB tables
• Reliable & crash-safe
• Single repository of metadata
– for the MySQL server
– for Storage Engines
– for Plugins
• Redundancy
• Data Dictionary API
• INFORMATION_SCHEMA SQL VIEWs
– Queries can be optimized
– Improved performance
• Metadata versioning
• Extendable
– Simplify metadata upgrades
– Designed with plugins in mind

New Data Dictionary
15
Transition
Data Dictionary
Files
FRM TRG OPT
System tables (mysql.)
user time_zone proc
InnoDB internal dictionary
Archive
CSV
InnoDB
INFORMATION_SCHEMA
File
Scan
SQL DD Table
VIEW
Table
Scan
Intern.
Access

New Data Dictionary
16
Overview
InnoDB
Data Dictionary
DD Table User Table
INFORMATION
SCHEMA
Views
Archive
User Table
CSV
User Table

Plugin Plugin Plugin
Tablespace User Table Storage
New Data Dictionary
InnoDB
17
Architecture
Query Executor
Parser Optimizer
Data
Dictionary
Tablespace
Data Dictionary Internal API
Data Dictionary External API
Plugin Plugin
Archive
Engine
Storage
Engine

New Data Dictionary
• WL#6379: Schema definitions for new DD
• InnoDB Data Dictionary Tablespace
• Designed with INFORMATION_SCHEMA in mind
• Ability to store SE-specific data
• Use PK / FK to ensure consistency
18
Data Dictionary tables

New Data Dictionary API
• The only way to access Data Dictionary
– For the server core
– For Storage Engines
– For plugins
• Hard to misuse
• Internal API (non-stable) and external API (stable)
• Provide a way to handle SE specific data
19
Design goals

New Data Dictionary
SDI : Serialized Dictionary Information
20
Redundancy
InnoDB
Single User TS
User Table
SDI
General TS
User Table
SDI
System TS
User Table
SDI
Data Dictionary
Privileges
User Table
Definition
Stored
Program

New Data Dictionary
21
FRM shipping for MySQL Cluster?
SELECT ... FROM t1
FSRDMI

• No FRM files
• New INFORMATION_SCHEMA
• Migrated to InnoDB:
– time zone tables
– help tables
–mysql.plugins
–mysql.servers
• Draft of Data Dictionary API
• http://labs.mysql.com
– Do NOT use it in production
– Install on a spare server
• MTR can be run
22
New Data Dictionary : Labs Release

New Data Dictionary
Why does it matter for YOU?

INFORMATION_SCHEMA performance improvements
Get per table size
Blog post by Shlomi Noach: http://tinyurl.com/y8cnj7o
SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE,
SUM(DATA_LENGTH+INDEX_LENGTH) AS size,
SUM(INDEX_LENGTH) AS index_size
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA') AND ENGINE IS NOT NULL
GROUP BY TABLE_SCHEMA, TABLE_NAME;
24
Version Time
5.7.5-m15 0.38 sec
Labs Release 0.08 sec

Case: server crash
• There are some data files
• No / outdated backup
• FRM files lost
• How to use those data files?
25
The problem

Traditional Data Dictionary
• “Move FRM files around”
• CREATE TABLE t1 (...)
• ALTER TABLE t1
DISCARD TABLESPACE
• ALTER TABLE t1
IMPORT TABLESPACE ...
• Easy to make mistakes
New Data Dictionary
• Self-descriptive tablespaces (SDI)
• Dedicated IMPORT statement
• Goal: error-proof procedure
26
Case: server crash

For Plugin Developers
• A way to access Data Dictionary
• Persistent Storage for plugins
– Store/restore custom data
• Plugins can extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA
– Add new tables
– Add new columns to existing tables
27

Data Dictionary
Summary

Data Dictionary : Takeaways
• Fundamental component in RDBMS
• Critical for performance
• Critical for reliability
• Critical for scalability
29

The MySQL Traditional
Data Dictionary
• Mix of files and tables
• Server DD and InnoDB DD
• Inefficient INFORMATION_SCHEMA
• Difficult to extend
New Data Dictionary
• Crash-safe InnoDB tables
• Single repository
• INFORMATION_SCHEMA as VIEWs
• Designed to be extendable
• Aims for backward compatibility
• Huge reengineering
Data Dictionary : Takeaways

Questions?

Oracle University MySQL Training Services
Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications
Top Courses for Administrators and Developers
Top Certifications
“Training and team skill have the most significant impact
on overall performance of technology and success of
technology projects.” - IDC, 2013
Premier Support customers eligible to
save 20% on learning credits.
Benefits
 Expert-led training to support your MySQL learning needs
 Flexibility to train in the classroom or online
 Hands-on experience to gain real world experience
 Key skills needed for database administrators and developers
• MySQL for Beginners
MySQL for Database Administrators
MySQL Performance Tuning
MySQL Cluster – NEW - Register Your Interest!
MySQL and PHP - Developing Dynamic Web Applications
MySQL for Developers
MySQL Developer Techniques
MySQL 5.6 Database Administrator
MySQL 5.6 Developer
To find out more about available MySQL Training & Certification
offerings, go to: education.oracle.com/mysql
RECENTLY RELEASED
ALL NEW! MySQL Cluster Training
To Register your interest to influence the
schedule on this newly released course – go to
education.oracle.com/mysql and click on the
MySQL Cluster Course

New data dictionary an internal server api that matters

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (8)

Similaire à New data dictionary an internal server api that matters

Similaire à New data dictionary an internal server api that matters (20)

Dernier

Dernier (20)

New data dictionary an internal server api that matters

Notes de l'éditeur