Contenu connexe
Similaire à MySQL NoSQL APIs
Similaire à MySQL NoSQL APIs (20)
Plus de Morgan Tocker (8)
MySQL NoSQL APIs
- 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!2
The following is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
- 3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!3
MySQL 5.5
MySQL Enterprise Monitor 2.3 & 3.0
MySQL Enterprise Backup
Security
Scalability
HA
Audit
MySQL 5.6
MySQL Workbench 6.1
MySQL Utilities
MySQL Applier for
Hadoop
MySQL Workbench 5.2 & 6.0
MySQL Enterprise Oracle
Certifications
MySQL@Oracle: 4 Years of MySQL Innovation
MySQL Cluster Manager
Windows installer & Tools
MySQL Cluster 7.2
MySQL Cluster 7.1
MySQL Migration
Wizard
MySQL 5.7
MySQL Cluster 7.4
MySQL Fabric
MySQL Cluster 7.3
- 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!4
Discussion Topics
▪ What is memcached?
▪ History of HandlerSocket
▪ NoSQL Interface in MySQL 5.6
▪ Future in MySQL 5.7
- 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!5
What is memcached?
- 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!6
What do we know about memcached?
▪ Simple in-memory caching system.
▪ APIs available for most programming languages.
▪ Stores data in memory.
▪ Excellent read performance.
▪ Great write performance.
▪ Supports distribution across multiple servers.
▪ No security.
Good Introduction:
https://code.google.com/p/memcached/wiki/NewProgrammingFAQ
- 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!7
Common Configuration
▪ As a look-aside cache:
▪ Also possible to rebuild cache from cron/inject new cached data.
value = get_from_memcache(key);
if (value === null) {
value = get_value_from_database(key);
set_memcache(key, value)
}
- 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!8
Scaling
▪ Hashing is performed by the clients.
▪ Mod-based hashing.
▪ Consistent hashing.
▪ No cross talk by memcached servers.
- 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!9
History of HandlerSocket
- 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!10
The beginning
▪ Yoshinori Matsunobu working at DeNA.
▪ Profiled MySQL and discovered a lot of time spent in:
▪ Parsing SQL statements
▪ Opening, locking tables
▪ Making SQL execution plans
▪ Unlocking, closing tables
http://yoshinorimatsunobu.blogspot.ca/2010/10/using-mysql-as-nosql-story-for.html
- 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!11
The Question
▪ Is SQL really good for fast PK lookups?
▪ MySQL could only perform 100,000+ queries per second
▪ Memcached 400,000+ gets per second.
▪ Is it possible to bypass some parts of MySQL and get higher
throughput?
- 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!12
Implementation
Credits: http://yoshinorimatsunobu.blogspot.ca/2010/10/using-mysql-as-nosql-story-for.html
- 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!13
The Result
!
approx qps server CPU util
MySQL via SQL 105,000 %us 60% %sy 28%
memcached 420,000 %us 8% %sy 88%
MySQL via HandlerSocket 750,000 %us 45% %sy 53%
- 14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!14
Outcome
▪ Benefits:
▪ No inconsistent data.
▪ Crash-safe storage
▪ SQL can still be used.
▪ All operational benefits of
MySQL (backup, etc.)
!
!
▪ Limitations:
▪ New API
▪ No Security
▪ No Benefit for HDD bound
workloads
▪ In the case of DeNA:
▪ Switched from using memcached as a row cache.
▪ Kept memcached as a page/larger object cache.
- 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!15
NoSQL Interface in MySQL 5.6
- 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!16
MySQL 5.6
▪ Builds on the spirit of HandlerSocket.
▪ New implementation based on the standard memcached protocol.
▪ From a technical detail actually embeds memcached, but your
data remains safe between restarts.
▪ i.e. not just in memory.
- 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!17
Architecture
- 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!18
Implementation Details
▪ Plugin ships included in MySQL packages.
▪ Not enabled by default (security).
▪ Tables have to “opt-in” to be exposed via memcached.
- 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!19
Setup (1/3)
# Add this to my.cnf file:
!
plugin-load = daemon_memcached=libmemcached.so
!
# Load included SQL file to initialize innodb_memcached
# schema with permitted mappings.
!
mysql> source /usr/share/mysql/innodb_memcached_config.sql
- 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!20
Setup (2/3)
mysql> show plugins;
+----------------------------+----------+--------------------+-----------------+---------+
| Name | Status | Type | Library | License |
+----------------------------+----------+--------------------+-----------------+---------+
| binlog | ACTIVE | STORAGE ENGINE | NULL | GPL |
| mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| mysql_old_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| sha256_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| CSV | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| FEDERATED | DISABLED | STORAGE ENGINE | NULL | GPL |
| InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL |
| INNODB_TRX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCKS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCK_WAITS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| ..
| INNODB_SYS_DATAFILES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| PERFORMANCE_SCHEMA | ACTIVE | STORAGE ENGINE | NULL | GPL |
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
| daemon_memcached | ACTIVE | DAEMON | libmemcached.so | GPL |
+----------------------------+----------+--------------------+-----------------+---------+
43 rows in set (0.01 sec)
- 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!21
Setup (Part 3/3)
# Pretend this is my hypothetical table:
!
mysql> create database myapp;
mysql> create table myapp.user_session_data (id_name
varchar(255), value varchar(255), expiretime int, primary
key (id_name));
!
# Insert your desired mapping into the containers table:
!
mysql> insert into innodb_memcache.containers values
("sessions", "myapp", "user_session_data", "id_name",
"value", 0, 0, "expiretime", "PRIMARY");
- 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!22
Setup - Complete
mysql> select * from containersG
..
*************************** 2. row ***************************
name: sessions
db_schema: myapp
db_table: user_session_data
key_columns: id_name
value_columns: value
flags: 0
cas_column: 0
expire_time_column: expiretime
unique_idx_name_on_key: PRIMARY
2 rows in set (0.00 sec)
- 23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!23
Testing
shell> telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
set @@sessions.1|color 0 600 5
black
STORED
set @@sessions.1|size 0 600 5
large
STORED
quit
- 24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!24
Testing (cont.)
mysql> select * from user_session_dataG
*************************** 1. row ***************************
id_name: 1|color
value: black
expiretime: 1398608066
*************************** 2. row ***************************
id_name: 1|size
value: large
expiretime: 1398608106
2 rows in set (0.00 sec)
- 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!25
Updating Multiple Columns At Once
▪ Requires Serialization using the “|” character:
DROP TABLE myapp.users;
CREATE TABLE myapp.users (
`user_name` varchar(60) NOT NULL PRIMARY KEY,
`first_name` varchar(60),
`last_name` varchar(60),
expire_time_column INT
) ENGINE=InnoDB;
!
!
insert into innodb_memcache.containers
(name,db_schema,db_table,key_columns,value_columns,unique_idx_nam
e_on_key) VALUES (‘users’,'myapp','users','user_name',
'first_name,last_name','PRIMARY');
- 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!26
Testing Multiple Columns:
set @@users.fred 0 0 10
Fred|Smith
STORED
!
mysql> select * from users;
+-----------+------------+-----------+--------------------+
| user_name | first_name | last_name | expire_time_column |
+-----------+------------+-----------+--------------------+
| fred | Fred | Smith | 0 |
+-----------+------------+-----------+--------------------+
1 row in set (0.00 sec)
- 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!27
Benchmarks
- 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!28
Future in MySQL 5.7
- 29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!29
What has changed?
▪ Improvements inside of InnoDB to scale better.
▪ Several contentions related to MVCC code reduced.
▪ memcached plugin tuned to keep several reads within a single
internal transaction.
- 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!30
Benchmarks
48 Core Machine / Read Only Memcached
http://dimitrik.free.fr/blog/archives/2013/11/mysql-performance-over-1m-qps-with-innodb-memcached-plugin-in-mysql-57.html
- 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.!31
Further Reading
▪ Article by Matt Lord and Andrew Morgan:
http://www.drdobbs.com/database/nosql-with-mysql/240167115
!
▪ MySQL Manual page:
https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-
setup.html
- 32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12!32
- 33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12!33