SlideShare une entreprise Scribd logo
1  sur  16
Télécharger pour lire hors ligne
Data Organization in InnoDB
— by royalwzy
Personal Profile
More than 5 years experience in database use and management
Proficient in synchronizing data between heterogeneous database
Skilled in designing and optimizing database
Install and configure MySQL cluster and high-availability cluster environment
Synchronize and merge data from Oracle to MySQL
mysqlexp
mysqlclone
Oracle 10g/11g OCM
MySQL 5.6 Database
Administrator
MCITP/MCTS DBA(MSSQL2008)
RHCE
Java Programmer
Oracle ACEA, Oracle Young Expert
ACOUG/SHOUG/OCMU core member
ITPUB Moderator and Community Expert
Senior lecturer in "ENMO" and “UPLOOKING"
Active in OTN & ITPUB and other forums
My blog:royalwzy.com
Table of Contents
❖ Various files that are created by InnoDB
❖ Logical data organization like
tablespaces, pages, segments and extents
❖ Explore each of them in some detail and
discuss about their relationship with
each other
❖ Data layout within the InnoDB storage
engine
The Files in InnoDB
❖ MySQL will store all data within the data
directory
❖ By default, when InnoDB is initialized, it
creates 3 important files
❖ As of MySQL 5.6, only the system tablespace
can contain more than 1 data file. All other
tablespaces can contain only one data file
❖ The data files and the redo log files are
represented in the memory by the C structure
fil_node_t
Logical data organization
❖ Tablespaces
❖ Pages
❖ Extents
❖ File Segments
“Our tablespace concept is similar to the one of Oracle”
Logical and Physical Database Structures
in Oracle
Database
Logical Physical
Tablespace Data file
Segment
Extent
Oracle data

block
Storage System
• SAN
• NAS
• Exadata
• File System
• NFS
• ASM
• RAW
OS block
Logical and Physical Database Structures
in InnoDB
Database
Logical Physical
Tablespace Data file
Segment
Extent
Pages
Storage System
• SAN
• NAS
• File System
• NFS
• RAW
OS block
Tablesapce
❖ innodb_file_per_table configuration
parameter
❖ What's the relationship between the
tablespace and data files
“A tablespace consists of a chain of files. The size of the files does not have to be
divisible by the database block size, because we may just leave the last incomplete
block unused. When a new file is appended to the tablespace, the maximum size of the
file is also specified. At the moment, we think that it is best to extend the file to
its maximum size already at the creation of the file, because then we can avoid
dynamically extending the file when more space is needed for the tablespace.”
Tablespace Header
❖ Each tablespace will have a header of
type fsp_header_t
❖ Current size of the table space in pages
❖ List of free extents
❖ List of full extents not belonging to any segment
❖ List of partially full/free extents not belonging to any segment
❖ List of pages containing segment headers, where all the segment inode
slots are reserved
❖ List of pages containing segment headers, where not all the segment
inode slots are reserved
❖ From the tablespace header, we can
access the list of segments available
in the tablespace
Pages
❖ A data file is logically divided into
equal sized pages
❖ How the pages from different data files
relate to one another
❖ Every page has a page header
“A block's position in the tablespace is specified with a 32-bit unsigned
integer. The files in the chain are thought to be catenated, and the block
corresponding to an address n is the nth block in the catenated file (where
the first block is named the 0th block, and the incomplete block fragments
at the end of files are not taken into account). A tablespace can be extended
by appending a new file at the end of the chain.”
Page Types
Extents
❖ An extent is 1MB of consecutive pages
❖ #define FSP_EXTENT_SIZE (1048576U /
UNIV_PAGE_SIZE)
File Segments
❖ A tablespace can contain many file
segments
❖ Information of segment header
❖ The space to which the inode belongs
❖ The page_no of the inode
❖ The byte offset of the inode
❖ The length of the file segment header (in bytes)
Table
❖ In InnoDB, when a table is created, a
clustered index (B-tree) is created
internally
❖ The root page of a B-tree will be
obtained from the data dictionary
“In the root node of a B-tree there are two file segment headers. The leaf pages of a
tree are allocated from one file segment, to make them consecutive on disk if
possible. From the other file segment we allocate pages for the non-leaf levels of the
tree.”
Q & A

Contenu connexe

Tendances

Tendances (20)

RocksDB detail
RocksDB detailRocksDB detail
RocksDB detail
 
MonetDB :column-store approach in database
MonetDB :column-store approach in databaseMonetDB :column-store approach in database
MonetDB :column-store approach in database
 
Innodb 和 XtraDB 结构和性能优化
Innodb 和 XtraDB 结构和性能优化Innodb 和 XtraDB 结构和性能优化
Innodb 和 XtraDB 结构和性能优化
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
 
MySQL vs MonetDB Bencharmarks
MySQL vs MonetDB BencharmarksMySQL vs MonetDB Bencharmarks
MySQL vs MonetDB Bencharmarks
 
RocksDB storage engine for MySQL and MongoDB
RocksDB storage engine for MySQL and MongoDBRocksDB storage engine for MySQL and MongoDB
RocksDB storage engine for MySQL and MongoDB
 
MySQL vs. MonetDB
MySQL vs. MonetDBMySQL vs. MonetDB
MySQL vs. MonetDB
 
Db2 Important questions to read
Db2 Important questions to readDb2 Important questions to read
Db2 Important questions to read
 
MyRocks in MariaDB | M18
MyRocks in MariaDB | M18MyRocks in MariaDB | M18
MyRocks in MariaDB | M18
 
Percona FT / TokuDB
Percona FT / TokuDBPercona FT / TokuDB
Percona FT / TokuDB
 
Log Structured Merge Tree
Log Structured Merge TreeLog Structured Merge Tree
Log Structured Merge Tree
 
MyRocks Deep Dive
MyRocks Deep DiveMyRocks Deep Dive
MyRocks Deep Dive
 
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDBThe Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDB
 
Get More Out of MongoDB with TokuMX
Get More Out of MongoDB with TokuMXGet More Out of MongoDB with TokuMX
Get More Out of MongoDB with TokuMX
 
TokuDB - What You Need to Know
TokuDB - What You Need to KnowTokuDB - What You Need to Know
TokuDB - What You Need to Know
 
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank:  Rocking the Database World with RocksDBThe Hive Think Tank:  Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDB
 
InfiniFlux Time Series DBMS FAQ
InfiniFlux Time Series DBMS FAQInfiniFlux Time Series DBMS FAQ
InfiniFlux Time Series DBMS FAQ
 
InfiniFlux vs_RDBMS
InfiniFlux vs_RDBMSInfiniFlux vs_RDBMS
InfiniFlux vs_RDBMS
 
RocksDB meetup
RocksDB meetupRocksDB meetup
RocksDB meetup
 
Exchange 2010 storage improvements
Exchange 2010 storage improvementsExchange 2010 storage improvements
Exchange 2010 storage improvements
 

Similaire à Data Organization in InnoDB

Text Collections and CONTENTdm
Text Collections and CONTENTdmText Collections and CONTENTdm
Text Collections and CONTENTdm
Gena Chattin
 

Similaire à Data Organization in InnoDB (20)

SQLServer Database Structures
SQLServer Database Structures SQLServer Database Structures
SQLServer Database Structures
 
Falando de MySQL
Falando de MySQLFalando de MySQL
Falando de MySQL
 
1650607.ppt
1650607.ppt1650607.ppt
1650607.ppt
 
Why you should care about data layout in the file system with Cheng Lian and ...
Why you should care about data layout in the file system with Cheng Lian and ...Why you should care about data layout in the file system with Cheng Lian and ...
Why you should care about data layout in the file system with Cheng Lian and ...
 
Data storage and indexing
Data storage and indexingData storage and indexing
Data storage and indexing
 
Chapter13
Chapter13Chapter13
Chapter13
 
002-Storage Basics and Application Environments V1.0.pptx
002-Storage Basics and Application Environments V1.0.pptx002-Storage Basics and Application Environments V1.0.pptx
002-Storage Basics and Application Environments V1.0.pptx
 
[Altibase] 4-1 tablespace concept
[Altibase] 4-1 tablespace concept[Altibase] 4-1 tablespace concept
[Altibase] 4-1 tablespace concept
 
Physical architecture of sql server
Physical architecture of sql serverPhysical architecture of sql server
Physical architecture of sql server
 
Microsoft SQL Server - Files and Filegroups
Microsoft SQL Server - Files and FilegroupsMicrosoft SQL Server - Files and Filegroups
Microsoft SQL Server - Files and Filegroups
 
Big Data Architecture Workshop - Vahid Amiri
Big Data Architecture Workshop -  Vahid AmiriBig Data Architecture Workshop -  Vahid Amiri
Big Data Architecture Workshop - Vahid Amiri
 
Text Collections and CONTENTdm
Text Collections and CONTENTdmText Collections and CONTENTdm
Text Collections and CONTENTdm
 
No sq lv1_0
No sq lv1_0No sq lv1_0
No sq lv1_0
 
3620121datastructures.ppt
3620121datastructures.ppt3620121datastructures.ppt
3620121datastructures.ppt
 
Allocation and free space management
Allocation and free space managementAllocation and free space management
Allocation and free space management
 
DBA 101 : Calling all New Database Administrators (WP)
DBA 101 : Calling all New Database Administrators (WP)DBA 101 : Calling all New Database Administrators (WP)
DBA 101 : Calling all New Database Administrators (WP)
 
NoSql
NoSqlNoSql
NoSql
 
oslectureset7.pdf
oslectureset7.pdfoslectureset7.pdf
oslectureset7.pdf
 
Sql Server Basics
Sql Server BasicsSql Server Basics
Sql Server Basics
 
What is Object storage ?
What is Object storage ?What is Object storage ?
What is Object storage ?
 

Plus de Zhaoyang Wang

Plus de Zhaoyang Wang (20)

海通证券金融云思考与实践(数据技术嘉年华2017)
海通证券金融云思考与实践(数据技术嘉年华2017)海通证券金融云思考与实践(数据技术嘉年华2017)
海通证券金融云思考与实践(数据技术嘉年华2017)
 
云管理平台助力海通金融云建设
云管理平台助力海通金融云建设云管理平台助力海通金融云建设
云管理平台助力海通金融云建设
 
海通证券数据库备份恢复云平台实践(OTN Tour Shanghai 2017)
海通证券数据库备份恢复云平台实践(OTN Tour Shanghai 2017)海通证券数据库备份恢复云平台实践(OTN Tour Shanghai 2017)
海通证券数据库备份恢复云平台实践(OTN Tour Shanghai 2017)
 
Oracle Compute Cloud Service快速实践
Oracle Compute Cloud Service快速实践Oracle Compute Cloud Service快速实践
Oracle Compute Cloud Service快速实践
 
Oracle Compute Cloud Service介绍
Oracle Compute Cloud Service介绍Oracle Compute Cloud Service介绍
Oracle Compute Cloud Service介绍
 
Oracle cloud 使用云市场快速搭建小型电商网站
Oracle cloud 使用云市场快速搭建小型电商网站Oracle cloud 使用云市场快速搭建小型电商网站
Oracle cloud 使用云市场快速搭建小型电商网站
 
Oracle cloud ravello介绍及测试账户申请
Oracle cloud ravello介绍及测试账户申请Oracle cloud ravello介绍及测试账户申请
Oracle cloud ravello介绍及测试账户申请
 
Oracle cloud 云介绍及测试账户申请
Oracle cloud 云介绍及测试账户申请Oracle cloud 云介绍及测试账户申请
Oracle cloud 云介绍及测试账户申请
 
New awesome features in MySQL 5.7
New awesome features in MySQL 5.7New awesome features in MySQL 5.7
New awesome features in MySQL 5.7
 
Performance Tuning Tool01-Statspack
Performance Tuning Tool01-StatspackPerformance Tuning Tool01-Statspack
Performance Tuning Tool01-Statspack
 
SQL Tuning02-Intorduction to the CBO Optimizer
SQL Tuning02-Intorduction to the CBO OptimizerSQL Tuning02-Intorduction to the CBO Optimizer
SQL Tuning02-Intorduction to the CBO Optimizer
 
SQL Tuning04-Interpreting Execution Plans
SQL Tuning04-Interpreting Execution PlansSQL Tuning04-Interpreting Execution Plans
SQL Tuning04-Interpreting Execution Plans
 
SQL Tuning01-Introduction to SQL Tuning
SQL Tuning01-Introduction to SQL TuningSQL Tuning01-Introduction to SQL Tuning
SQL Tuning01-Introduction to SQL Tuning
 
MySQL Fulltext Search Tutorial
MySQL Fulltext Search TutorialMySQL Fulltext Search Tutorial
MySQL Fulltext Search Tutorial
 
Oracle enterprise manager cloud control 12c release 5 installation on oracle ...
Oracle enterprise manager cloud control 12c release 5 installation on oracle ...Oracle enterprise manager cloud control 12c release 5 installation on oracle ...
Oracle enterprise manager cloud control 12c release 5 installation on oracle ...
 
Oracle enterprise manager cloud control 12c r5 agent installation
Oracle enterprise manager cloud control 12c r5 agent installationOracle enterprise manager cloud control 12c r5 agent installation
Oracle enterprise manager cloud control 12c r5 agent installation
 
Why use MySQL
Why use MySQLWhy use MySQL
Why use MySQL
 
MYSQLCLONE Introduction
MYSQLCLONE IntroductionMYSQLCLONE Introduction
MYSQLCLONE Introduction
 
Oracle security 08-oracle network security
Oracle security 08-oracle network securityOracle security 08-oracle network security
Oracle security 08-oracle network security
 
Oracle security 02-administering user security
Oracle security 02-administering user securityOracle security 02-administering user security
Oracle security 02-administering user security
 

Dernier

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Dernier (20)

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 

Data Organization in InnoDB

  • 1. Data Organization in InnoDB — by royalwzy
  • 2. Personal Profile More than 5 years experience in database use and management Proficient in synchronizing data between heterogeneous database Skilled in designing and optimizing database Install and configure MySQL cluster and high-availability cluster environment Synchronize and merge data from Oracle to MySQL mysqlexp mysqlclone Oracle 10g/11g OCM MySQL 5.6 Database Administrator MCITP/MCTS DBA(MSSQL2008) RHCE Java Programmer Oracle ACEA, Oracle Young Expert ACOUG/SHOUG/OCMU core member ITPUB Moderator and Community Expert Senior lecturer in "ENMO" and “UPLOOKING" Active in OTN & ITPUB and other forums My blog:royalwzy.com
  • 3. Table of Contents ❖ Various files that are created by InnoDB ❖ Logical data organization like tablespaces, pages, segments and extents ❖ Explore each of them in some detail and discuss about their relationship with each other ❖ Data layout within the InnoDB storage engine
  • 4. The Files in InnoDB ❖ MySQL will store all data within the data directory ❖ By default, when InnoDB is initialized, it creates 3 important files ❖ As of MySQL 5.6, only the system tablespace can contain more than 1 data file. All other tablespaces can contain only one data file ❖ The data files and the redo log files are represented in the memory by the C structure fil_node_t
  • 5. Logical data organization ❖ Tablespaces ❖ Pages ❖ Extents ❖ File Segments “Our tablespace concept is similar to the one of Oracle”
  • 6. Logical and Physical Database Structures in Oracle Database Logical Physical Tablespace Data file Segment Extent Oracle data
 block Storage System • SAN • NAS • Exadata • File System • NFS • ASM • RAW OS block
  • 7. Logical and Physical Database Structures in InnoDB Database Logical Physical Tablespace Data file Segment Extent Pages Storage System • SAN • NAS • File System • NFS • RAW OS block
  • 8. Tablesapce ❖ innodb_file_per_table configuration parameter ❖ What's the relationship between the tablespace and data files “A tablespace consists of a chain of files. The size of the files does not have to be divisible by the database block size, because we may just leave the last incomplete block unused. When a new file is appended to the tablespace, the maximum size of the file is also specified. At the moment, we think that it is best to extend the file to its maximum size already at the creation of the file, because then we can avoid dynamically extending the file when more space is needed for the tablespace.”
  • 9. Tablespace Header ❖ Each tablespace will have a header of type fsp_header_t ❖ Current size of the table space in pages ❖ List of free extents ❖ List of full extents not belonging to any segment ❖ List of partially full/free extents not belonging to any segment ❖ List of pages containing segment headers, where all the segment inode slots are reserved ❖ List of pages containing segment headers, where not all the segment inode slots are reserved ❖ From the tablespace header, we can access the list of segments available in the tablespace
  • 10.
  • 11. Pages ❖ A data file is logically divided into equal sized pages ❖ How the pages from different data files relate to one another ❖ Every page has a page header “A block's position in the tablespace is specified with a 32-bit unsigned integer. The files in the chain are thought to be catenated, and the block corresponding to an address n is the nth block in the catenated file (where the first block is named the 0th block, and the incomplete block fragments at the end of files are not taken into account). A tablespace can be extended by appending a new file at the end of the chain.”
  • 13. Extents ❖ An extent is 1MB of consecutive pages ❖ #define FSP_EXTENT_SIZE (1048576U / UNIV_PAGE_SIZE)
  • 14. File Segments ❖ A tablespace can contain many file segments ❖ Information of segment header ❖ The space to which the inode belongs ❖ The page_no of the inode ❖ The byte offset of the inode ❖ The length of the file segment header (in bytes)
  • 15. Table ❖ In InnoDB, when a table is created, a clustered index (B-tree) is created internally ❖ The root page of a B-tree will be obtained from the data dictionary “In the root node of a B-tree there are two file segment headers. The leaf pages of a tree are allocated from one file segment, to make them consecutive on disk if possible. From the other file segment we allocate pages for the non-leaf levels of the tree.”
  • 16. Q & A