A presentation on MySQL's NoSQL interface via memcached that provides SQL AND NoSQL access to the same data.

1. Dave Stokes
David.Stokes@oracle.com @stoker
MySQL's NoSQL -MySQL's NoSQL -
Best of Both Worlds on One Set of DisksBest of Both Worlds on One Set of Disks
Slideshare.net/davestokesSlideshare.net/davestokes

2. SQL
● Structured Query Language
– Store data efficiently, no or minimal duplication
– Set theory & relational calculus
– Declarative Language
● Made up of DML & DDL
– Data Manipulation Language
– Data Description Language
● Does not always mesh with procedural or object oriented
languages
– Not well taught

3. NoSQL
● Not relational, usually schema-less
● Don't know how data will look, or care
● Not 'new'
– Berkeley Data Base
● Scaling problems
● Usually not a general purpose data store

4. SQL & NoSQL – different platforms
● Many companies end up with:
– Separate RDMS environment
● Many nodes
– Sharding
– Replication
– Separate NoSQL environment
● Thousands of nodes
– Tools (lots of 'em) to move between the two environments
– Minimal cross functionality

5. But what if you could...
● Use the same server/disks?
● Access data as SQL and key/value pair
– The same data?
– The same time?
● Easily convert RDMS schemas into NoSQL?
● Drink from the NoSQL 'fire hose', use SQL to query
● Keep ACID
● Not have to learn an entire ecosystems of new tools

6. Some history
● Yohinori Matsunobu, then DeNA now Facebook creates
Handler Socket in ~2010
– 750,000 qps on commodity hardware
– Used memcached as front end cache but not for for
storing rows
– http://yoshinorimatsunobu.blogspot.com/2010/10/using-
mysql-as-nosql-story-for.html

7. MySQL's InnoDB Memcached Plugin
● It uses a standard memcached protocol, with drivers available
for many programming languages.
● It does not automatically allow access to all tables in your
database, but instead tables 'opt-in' via a mapping containers
table.
● 1,100,000 qps –
http://dimitrik.free.fr/blog/archives/2013/11/mysql-performance-
over-1m-qps-with-innodb-memcached-plugin-in-mysql-57.html

8. Benefits of the InnoDB / memcached
Combination
● The combination of InnoDB tables and memcached offers
advantages over using either by themselves:
– Direct access to the InnoDB storage engine avoids the parsing
and planning overhead of SQL.
– Running memcached in the same process space as the MySQL
server avoids the network overhead of passing requests back
and forth.
– Data that is written using the memcached protocol is
transparently written to an InnoDB table, without going through
the MySQL SQL layer. You can control the frequency of writes
to achieve higher raw performance when updating non-critical
data.

9. Benefits of the InnoDB / memcached
Combination
● Subsequent requests for the same data is served from the InnoDB
buffer pool. The buffer pool handles the in-memory caching. You can
tune the performance of data-intensive operations using the familiar
InnoDB configuration options.
● Data can be unstructured or structured, depending on the type of
application. You can make an all-new table for the data, or map the
NoSQL-style processing to one or more existing tables.
● InnoDB can handle composing and decomposing multiple column
values into a single memcached item value, reducing the amount of
string parsing and concatenation required in your application. For
example, you might store a string value 2|4|6|8 in the memcached
cache, and InnoDB splits that value based on a separator character,
then stores the result into four numeric columns.

10. Benefits continued
● The transfer between memory and disk is handled automatically,
simplifying application logic.
● Data is stored in a MySQL database to protect against crashes,
outages, and corruption.
● You can still access the underlying table through SQL, for
reporting, analysis, ad hoc queries, bulk loading, multi-step
transactional computations, set operations such as union and
intersection, and other operations well suited to the
expressiveness and flexibility of SQL.
● You can ensure high availability of the NoSQL data by using this
feature on a master server in combination with MySQL replication.

11. More benefits
● The integration of memcached with MySQL provides a painless way to make
the in-memory data persistent, so you can use it for more significant kinds of
data. You can put more add, incr, and similar write operations into your
application, without worrying that the data could disappear at any moment.
You can stop and start the memcached server without losing updates made
to the cached data. To guard against unexpected outages, you can take
advantage of InnoDB crash recovery, replication, and backup procedures.
● The way InnoDB does fast primary key lookups is a natural fit for memcached
single-item queries. The direct, low-level database access path used by the
memcached plugin is much more efficient for key-value lookups than
equivalent SQL queries.
●
The serialization features of memcached, which can turn complex data
structures, binary files, or even code blocks into storeable strings, offer a
simple way to get such objects into a database.

12. Even more benfits
● Because you can access the underlying data through SQL, you can produce
reports, search or update across multiple keys, and call functions such as
AVG() and MAX() on the memcached data. All of these operations are
expensive or complicated with the standalone memcached.
● You do not need to manually load data into memcached at startup. As
particular keys are requested by an application, the values are retrieved from
the database automatically, and cached in memory using the InnoDB buffer
pool.
● Because memcached consumes relatively little CPU, and its memory
footprint is easy to control, it can run comfortably alongside a MySQL
instance on the same system.
● Because data consistency is enforced by the mechanisms used for regular
InnoDB tables, you do not have to worry about stale memcached data or
fallback logic to query the database in the case of a missing key.

13. The Architecture

14. Installation:
● Because the memcached library comes with the MySQL
server, installation and setup are straightforward. You run a
SQL script to set up a table for memcached to use, issue a
one-time install plugin statement to enable memcached,
and add to the MySQL configuration file or startup script
any desired memcached options, for example to use a
different port. You might still install the regular memcached
distribution to get the additional utilities such as memcp,
memcat, and memcapable.

15. Programming interfaces:
● You can access the MySQL server through the InnoDB
and memcached combination using the same language as
always: C and C++, Java, Perl, Python, PHP, and Ruby.
Specify the server hostname and port as with any other
memcached server. By default, the integrated memcached
server listens on the same port as usual, 11211. You can
use both the text and binary protocols.

16. Prerequisites for the InnoDB
memcached Plugin
● Currently, the memcached Daemon Plugin is only
supported on Linux, Solaris, and OS X platforms.
● You must have libevent installed, since it is required by
memcached

17. Setup
● To configure the memcached plugin so it can interact with
InnoDB tables, run the innodb_memcached_config.sql
configuration script to install the necessary tables used
behind the scenes:
mysql> source MYSQL_HOME/share/innodb_memcached_config.sql
● This is a one-time operation. The tables remain in place if
you later disable and re-enable the memcached support.

18. Setup creates three tables
● In database innodb_memcache
– cache_policies
– config_options
– Containers
● Plus test.demo_test
– (there is also one in innodb_memcache but unused)
–

19. containers
Key
Column used for K/V lookup
Data Store

20. cache_policies & config_options
Policies can be InnoDB, Memory, or both!

21. Load plugin
● To activate the daemon plugin, use the install plugin statement, just
as when installing any other MySQL plugin:
– mysql> install plugin daemon_memcached soname "libmemcached.so";
– Once the plugin is installed this way, it is automatically activated
each time the MySQL server is booted or restarted.
● Disabling the Daemon Plugin
– When making major changes to the plugin configuration, you
might need to turn off the plugin. To do so, issue the following
statement:
– mysql> uninstall plugin daemon_memcached;

22. Test from memcached
An example using telnet to send memcached commands
and receive results through the ASCII protocol:
telnet 127.0.0.1 11211
set a11 10 0 9 – set 'a11' 'flag' 'ttl' size
123456789
STORED
get a11
VALUE a11 0 9
123456789
END
quit

23. From SQL side
To prove that all the same data has been stored
in MySQL, connect to the MySQL server and
issue:
mysql> select * from test.demo_test;

24. So what else can it do?!?!?
● daemon_memcached_w_batch_size
– Use to commit batches, 1=every record
● innodb_api_disable_rowlock
– SET and GET operations w/o locks
● innodb_memcache.cache_policies
– Memory, disk, or both
– Set independently for SET, INCR, GET, DECR, DELETE,
and FLUSH
– Use to have DELETE/FLUSH work on memory, not disk

25. Remember
● Memcached may be flushing slowly compared to InnoDB
– Use READ_UNCOMMITED SQL mode
● Memcached data can be sent to replication slaves if binary logging
enabled
– innodb_api_enable_binlog
● More details in MySQL Manual
– 14.18 InnoDB Integration with memcached
http://dev.mysql.com/doc/refman/5.6/en/innodb-memcached.html

26. 5.6 good, 5.7 better

27. Another Approaches with MySQL 5.7
● MySQL's Http Plugin
– Lets lightweight applications use JSON to use database
without using intermediate server
– three APIs:
● key-document for nested JSON documents
● CRUD for JSON mapped SQL tables
● SQL with JSON replies
● JSON UDFs
– 0.3.3 January release

28. Q&A
● Slides: slideshare.net/davestokes
● Tweets: @stoker
● Email: David.Stokes@Oracle.com
● Blog: OpenSourceDBA.Wordpress.com