OpenExpo Virtual Experience 2020: MySQL 8.0 Document Store - Discovery of a new world

Frédéric Descamps
Community Manager
MySQL
June 2020
MySQL 8.0 Document Store
discovery of a new world
2 /
146

Safe Harbor
The following is intended to outline our general product direction. It is intended for information purpose only, and
may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality,
and should not be relied up in making purchasing decisions. The development, release, timing and pricing of any
features or functionality described for Oracle´s product may change and remains at the sole discretion of Oracle
Corporation.
Statement in this presentation relating to Oracle´s future plans, expectations, beliefs, intentions and ptospects
are "forward-looking statements" and are subject to material risks and uncertainties. A detailed discussion of
these factors and other risks that a ect our business is contained in Oracle´s Securities and Exchange
Commission (SEC) lings, including our most recent reports on Form 10-K and Form 10-Q under the heading
"Risk Factors". These lings are available on the SEC´s website or on Oracle´s website at
h p://www.oracle.com/investor. All information in this presentation is current as of September 2019 and Oracle
undertakes no duty to update any statement in light of new information or future events.
Copyright @ 2020 Oracle and/or its affiliates.
3 /
146

about.me/lefred
Who am I ?
4 /
146

Frédéric Descamps
@lefred
MySQL Evangelist
Managing MySQL since 3.20 !
devops believer
living in Belgium 🇧🇪
h ps://lefred.be
5 /
146

#MySQl8isGreat
MySQL
6 /
146

DB-Engines 2020
Database Ranking
7 /
146

DB-Engines 2020
Database Ranking
MySQL is the most popular Open Source database
8 /
146

MySQL is the DBMS of the Year 2019 !
9 /
146

Happy 25th Anniversary MySQL
10 /
146

from LAMP stack to modern Web applications
Evolution...
11 /
146

Web Applications 2000's
Who ?
12 /
146

Who ?
13 /
146

How ?
14 /
146

How ?
15 /
146

Relational Databases
Data Integrity
16 /
146

Data Integrity
normalization
17 /
146

Data Integrity
normalization
constraints (foreign keys, ...)
18 /
146

Data Integrity
normalization
Atomicity, Consistency, Isolation, Durability
19 /
146

Data Integrity
normalization
ACID compliant
20 /
146

Data Integrity
normalization
ACID compliant
transactions
21 /
146

Data Integrity
normalization
ACID compliant
transactions
SQL
22 /
146

Data Integrity
normalization
ACID compliant
transactions
SQL
powerfull query language
23 /
146

Web Applications current days
Who ?
24 /
146

Who ?
25 /
146

How ?
26 /
146

Why ?
27 /
146

Why ?
Developers don't really like SQL anymore...

28 /
146

SQL can be complicated and slows down
the initial development
Why ?
29 /
146

Developers don't have time to learn SQL,
they need time to workout and have a nice
look ;-)
Why ?
30 /
146

easy operations
What do developers want ?
31 /
146

Use Objects / Documents
Developers want just to use objects
32 /
146

that's why they usually love what DBAs hate the most ORMs !
33 /
146

They want to deal with these objects easily (CRUD operations) and they don't want to think
about schema design (slows down the initial development process).
34 /
146

They want to deal with these objects easily (CRUD operations) and they don't want to think
about schema design (slows down the initial development process).
But they also want to keep their data safe and use transactions.
35 /
146

How ?
36 /
146

How ?
37 /
146

JSON Document Store
NoSQL Databases
38 /
146

NoSQL Document Store
Schemaless
39 /
146

Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
40 /
146

Schemaless
very quick initial development
41 /
146

Schemaless
Flexible data structure
42 /
146

Schemaless
embedded arrays or objects
43 /
146

Schemaless
valid solution when natural data can't be modelized optimaly into a relational model
44 /
146

Schemaless
objects persistance without the use of any ORM - mapping oobject-oriented
45 /
146

Schemaless
JSON
46 /
146

Schemaless
JSON
close to frontend
47 /
146

Schemaless
JSON
close to frontend
easy to learn
48 /
146

How DBAs see data
49 /
146

How DBAs see data How Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
50 /
146

And they still need to do Analytics
SQL
SQL
SQL
51 /
146

... mmm...how ?
?
SQL
SQL
SQL
52 /
146

Help needed !
Who ?
53 /
146

Help needed !
Who ?
54 /
146

How ?
SQL
SQL
SQL
55 /
146

What if there was a way to provide both SQL
and NoSQL on one stable platform that has
proven stability on well know technology with
a large Community and a diverse ecosystem ?
56 /
146

DBMS or NoSQL ?
57 /
146

DBMS or NoSQL ?
Why not both ?
58 /
146

The MySQL Document Store !
SQL is now optional ?!
59 /
146

SQL is now optional ?!
60 /
146

Using MySQL Document Store !
SQL
SQL
SQL
61 /
146

Developers:
[x] schemaless
[x] rapid prototying/simpler APIs
[x] document model
[x] transactions
Operations:
[x] performance
management/visibility
[x] robust replication, backup, restore
[x] comprehensive tooling ecosystem
[x] simpler application schema
upgrades
Business Owner:
[x] don't lose my data == ACID trx
[x] capture all my data = extensible/schemaless
[x] product on schedule/time to market = rapid
developement
A solution for all
62 /
146

MySQL Document Store
the Solution
63 /
146

Built on
the MySQL
JSON Data
type and
Proven
MySQL
Server
Technology
Provides a schema exible JSON Document Store
No SQL required
No need to de ne all possible a ributes, tables, etc.
Uses new X DevAPI
Can leverage generated column to extract JSON values into
materialized columns that can be indexed for fast SQL searches.
Document can be ~1GB
It's a column in a row of a table
It cannot exceed max_allowed_packet
Allows use of modern programming styles
No more embedded strings of SQL in your code
Easy to read
Also works with relational Tables
Proven MySQL Technology
64 /
146

X Protocol Connectors
65 /
146

X DevAPI We provide connectors for
C++, Java, .Net, Node.js, Python, PHP
working with Communities to help them supporting it too
New MySQL Shell
Command Completion
Python, JavaScrips & SQL modes
Admin functions
New Util object
A new high-level session concept that can scale from single
.mysql-blue[My].mysql-orange[SQL] Server to a multiple
server environment
Non-blocking, asynchronous calls follow common language
pa erns
Supports CRUD operations
66 /
146

Setup
67 /
146

Installing MySQL Document Store
install MySQL 8.0
68 /
146

install MySQL 8.0
install MySQL Shell
69 /
146

install MySQL 8.0
install MySQL Shell
install MySQL Connector for your programming language
70 /
146

install MySQL 8.0
install MySQL Shell
php-pecl-mysql-xdevapi for PHP
mysql-connector-python for Python
...
71 /
146

install MySQL 8.0
install MySQL Shell
php-pecl-mysql-xdevapi for PHP
mysql-connector-python for Python
...
And nothing else, no need to install anything else or load any plugin, just be sure your
rewall allows you to connect through port 33060 (X Protocol).
72 /
146

Migration from MongoDB to MySQL DS
For this example, I will use the well known restaurants collection:
73 /
146

Migration from MongoDB to MySQL DS
For this example, I will use the well known restaurants collection:
74 /
146

75 /
146

76 /
146

Let's make a query
77 /
146

Let's make a query
That's too much records to show in here... let's limit it
78 /
146

79 /
146

Some more examples
80 /
146

Let's add a selection criteria:
81 /
146

Syntax slightly different than MongoDB
82 /
146

Syntax slightly different than MongoDB
83 /
146

And for developers ?
84 /
146

$session = mysql_xdevapigetSession("mysqlx://fred:MyP@ssw0rd%@localhost");
$schema = $session->getSchema("docstore");
$collection = $schema->getCollection("restaurants");
$results = $collection-> nd($search)->execute()->fetchAll();
...
foreach ($results as $doc) {
echo "<tr><td><a href='?id=${doc[_id]}'>${doc[name]}</a></td>";
echo "<td>${doc[borough]}</td><td>${doc[cuisine]}</td></tr>";
}
85 /
146

...
}

Easy, using only CRUD operations !
86 /
146

...
}

Easy, using only CRUD operations !
Not a single SQL statement !
87 /
146

CRUD operations
88 /
146

CRUD operations for collections
Add a document
collection.add({ name: 'fred', age: 42 })
.add({ name: 'dave', age: 23 })
.execute()
collection.add([
{ name: 'dimo', age: 50 },
{ name: 'kenny', age: 25 }
]).execute()
89 /
146

Modify a document
collection.modify('name = :name')
.bind('name', 'fred')
.set('age', 43)
.sort('name ASC')
.limit(1)
.execute()
collection.modify('name = :name')
.patch({ age: 43, active: false })
.sort('name DESC')
.limit(1)
.execute()
90 /
146

Remove a document
collection.remove('name = :name')
.sort('age ASC')
.limit(1)
.execute()
91 /
146

MySQL Document Store Objects Summary
92 /
146

All you need to know is here:
h ps://dev.mysql.com/doc/x-devapi-userguide/en/crud-operations-overview.html
93 /
146

we do care about your data
MySQL Document Store is Full ACID Compliant
94 /
146

Document Store Full ACID !
It relies on the proven MySQL InnoDB's strength & robustness:
95 /
146

innodb_ ush_log_at_trx_commit = 1
96 /
146

innodb_doublewrite = ON
97 /
146

sync_binlog = 1
98 /
146

sync_binlog = 1
transaction_isolation = REPEATABLE-READ|READ-COMMITTED|...
99 /
146

sync_binlog = 1
transaction_isolation = REPEATABLE-READ|READ-COMMITTED|...
We do care about your data !
100 /
146

Full ACID - Transactions support
101 /
146

Full ACID - Transactions support
102 /
146

OK we have Document Store, CRUD and ACID
but what makes MySQL Document Store unique ?
103 /
146

Challenge: list the best restaurant of each type
of food and show the top 10, with the best one
first !

don't forget that all these restaurants are just JSON documents
104 /
146

NoSQL as SQL - aggregation
105 /
146

106 /
146

107 /
146

108 /
146

NoSQL or SQL
You have the possibility to write clean and neat code:
109 /
146

NoSQL or SQL
$results = $collection-> nd('cuisine like "italian"')->execute()->fetchAll();
110 /
146

NoSQL or SQL
Or use SQL only when it's really needed:
111 /
146

NoSQL or SQL
$results = $session->sql('WITH cte1 AS (SELECT doc->>"$.name" AS name,
doc->>"$.cuisine" AS cuisine,
(SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT
PATH "$.score")) AS r) AS avg_score FROM docstore.restaurants)
SELECT *, RANK()
OVER ( PARTITION BY cuisine ORDER BY avg_score) AS `rank`
FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10;')->execute();
112 /
146

NoSQL or SQL
$results = $session->sql('WITH cte1 AS (SELECT doc->>"$.name" AS name,
doc->>"$.cuisine" AS cuisine,
(SELECT AVG(score) FROM JSON_TABLE(doc, "$.grades[*]" COLUMNS (score INT
PATH "$.score")) AS r) AS avg_score FROM docstore.restaurants)
SELECT *, RANK()
OVER ( PARTITION BY cuisine ORDER BY avg_score) AS `rank`
FROM cte1 ORDER BY `rank`, avg_score DESC LIMIT 10;')->execute();
All in the same MySQL X Session !
113 /
146

You can mix NoSQL & SQL as you want
114 /
146

Best of Both Worlds: JSON_TABLE
What are the maximum 10 ratings ever given to a restaurant?
115 /
146

Best of Both Worlds: JSON_TABLE
What are the maximum 10 ratings ever given to a restaurant?
Cool... but my app only processes JSON !
116 /
146

Best of Both Worlds: JSON_TABLE (2)
With JSON output:
117 /
146

And the DBA ?
118 /
146

Role of the MySQL Document Store DBA ?
119 /
146

nd the non optimal lookups
create the necessary indexes based on
the results

Role of the MySQL Document Store DBA ?
120 /
146

Role of the MySQL Document Store DBA
Looking for full table scans:
SELECT schema_name, sum_rows_examined, (sum_rows_examined/exec_count) avg_rows_call,
format_time(total_latency) tot_lat, exec_count,
format_time(total_latency/exec_count) AS latency_per_call, query_sample_text
FROM sys.x$statements_with_full_table_scans AS t1
JOIN performance_schema.events_statements_summary_by_digest AS t2
ON t2.digest=t1.digest
WHERE schema_name = "docstore" AND query_sample_text LIKE '%select%'
ORDER BY (total_latency/exec_count) descG
121 /
146

Role of the MySQL Document Store DBA
Looking for full table scans:
SELECT schema_name, sum_rows_examined, (sum_rows_examined/exec_count) avg_rows_call,
format_time(total_latency) tot_lat, exec_count,
format_time(total_latency/exec_count) AS latency_per_call, query_sample_text
FROM sys.x$statements_with_full_table_scans AS t1
JOIN performance_schema.events_statements_summary_by_digest AS t2
ON t2.digest=t1.digest
WHERE schema_name = "docstore" AND query_sample_text LIKE '%select%'
ORDER BY (total_latency/exec_count) descG
More info at h ps://lefred.be/content/mysql-8-0-if-i-should-optimize-only-one-query-
on-my-application-which-one-should-it-be/
122 /
146

Role of the MySQL Document Store DBA (2)
123 /
146

Getting the Query Explain Plan:
124 /
146

Getting the Query Explain Plan:
It's indeed a full table scan !
125 /
146

MySQL supports also EXPLAIN ANALYZE !!
126 /
146

Create Indexes:
127 /
146

Create Indexes:
For the "old" experienced MySQL DBAs, you can instead use:
128 /
146

INFO: JSON Notation Shortcut
129 /
146

Result:
130 /
146

For the curious: a collection in SQL:
131 /
146

Since MySQL 8.0.17
JSON Array Indexes
JSON Schema
JSON_SCHEMA_VALID(<json schema>,<json doc>)
JSON_SCHEMA_VALIDATION_REPORT(<json schema>,<json doc>)
132 /
146

Index of a JSON array
A functional index over a JSON
expression
The expression evaluates to an array
Several index entries per row
One index entry per array element
General mechanism, currently used
for JSON arrays
Used to speed up array lookups
JSON_CONTAINS(...)
JSON_OVERLAPS(...)
MEMBER OF (...)
JSON Array Indexes
133 /
146

JSON Array Indexes
134 /
146

JSON Array Indexes (2)
135 /
146

CHECK CONSTRAINTS
MySQL 8.0 supports Check Constraints:
136 /
146

JSON Schema Validation
137 /
146

JSON Schema Validation (2)
138 /
146

And the best of both worlds:
139 /
146

And the result in action:
140 /
146

And the result in action:
141 /
146

what do I gain ?
Conclusion
142 /
146

schemaless
exible data structure
easy to start (CRUD)
Conclusion
This is the best of the two worlds in one product !
Data integrity
ACID Compliant
Transactions
SQL
143 /
146

8.0
http://lefred.be/content/top-10-reasons-for-
nosql-with-mysql/
144 /
146

Q & A
146 /
146

OpenExpo Virtual Experience 2020: MySQL 8.0 Document Store - Discovery of a new world

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à OpenExpo Virtual Experience 2020: MySQL 8.0 Document Store - Discovery of a new world

Similaire à OpenExpo Virtual Experience 2020: MySQL 8.0 Document Store - Discovery of a new world (20)

Plus de Frederic Descamps

Plus de Frederic Descamps (20)

Dernier

Dernier (20)

OpenExpo Virtual Experience 2020: MySQL 8.0 Document Store - Discovery of a new world