Learn about the MariaDB 10 features that exist for developers: microseconds, virtual columns, PCRE regular expressions, DELETE ... RETURNING, geospatial extensions (GIS), dynamic columns. Use cases, and a hint of storage engines

1. MariaDB for
Developers
Colin Charles,Team MariaDB, SkySQL Ab
colin@mariadb.org | byte@bytebot.net
http://mariadb.org/ | http://skysql.com/
http://bytebot.net/blog/ | @bytebot on Twitter
Percona Live MySQL Conference & Expo, Santa Clara,
California, USA - 4 April 2014

2. Agenda
• Microseconds
• Virtual columns
• Regular expressions
• DELETE … RETURNING
• GIS
• Dynamic columns
• Engine features: Sphinx, Cassandra, Connect

3. Microseconds
• TIME, DATETIME, TIMESTAMP, temporal functions,
CAST, dynamic columns
CREATE TABLE microsec(
column_microsec DATETIME(6),
column_millisec TIME(3)
);
SELECT CURTIME(6);
MariaDB 5.3+

4. Microseconds & 5.6
• TIME_TO_SEC(), UNIX_TIMESTAMP() preserve
microseconds of the argument
MariaDB 10.0 MySQL 5.6
SELECT TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610.12345 |
+-------------------------------+
1 row in set (0.01 sec)
SELECT TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610 |
+-------------------------------+
1 row in set (0.00 sec)

5. Virtual Columns
• A column in a table that has its value
automatically calculated either with a pre-
calculated/deterministic expression or values of
other fields in the table
• VIRTUAL - computed on the fly when data is
queried (like a VIEW)
• PERSISTENT - computed when data is inserted
and stored in a table
MariaDB 5.2+

6. Virtual Columns
CREATE TABLE table1 (
a INT NOT NULL,
b VARCHAR(32),
c INT AS (a mod 10) VIRTUAL,
d VARCHAR(5) AS (left(b,5))
PERSISTENT);

7. Virtual columns example
CREATE TABLE product (
-> productname VARCHAR(25),
-> price_eur DOUBLE,
-> xrate DOUBLE,
-> price_usd DOUBLE AS (price_eur*xrate) VIRTUAL);
INSERT INTO product VALUES ('toothpaste', 1.5, 1.39,
default);
INSERT into product VALUES ('shaving cream', 3.59,
1.39, default);

8. Virtual columns example II
select * from product;
+---------------+-----------+-------+-------------------+
| productname | price_eur | xrate | price_usd |
+---------------+-----------+-------+-------------------+
| toothpaste | 1.5 | 1.39 | 2.085 |
| shaving cream | 3.59 | 1.39 | 4.990099999999999 |
+---------------+-----------+-------+-------------------+
2 rows in set (0.00 sec)

9. Virtual column use cases
elsewhere
• http://openlife.cc/blogs/2010/october/what-would-you-use-virtual-columns
• http://openlife.cc/blogs/2010/october/mariadb-52-using-mariadb-column-
store-and-virtual-columns-indexing
• http://www.jonathanlevin.co.uk/2012/04/mariadbs-virtual-columns.html
• http://daniel-bartholomew.com/wordpress/2010/09/road-to-mariadb-5-2-
virtual-columns/
• http://falseisnotnull.wordpress.com/2012/11/29/observations-about-
mariadbs-virtual-columns/
• https://mariadb.com/kb/en/virtual-columns/
• MariaDB Cookbook (2014) has a chapter dedicated to it

10. PCRE Regular Expressions
• Powerful REGEXP/RLIKE operator
• New operators:
• REGEXP_REPLACE(sub,pattern,replace)
• REGEXP_INSTR(sub,pattern)
• REGEXP_SUBSTR(sub,pattern)
• Works with multi-byte character sets that MariaDB
supports, including East-Asian sets
MariaDB 10.0+

11. REGEXP_REPLACE()
SELECT REGEXP_REPLACE('ab12cd','[0-9]','') AS
remove_digits;
+---------------+
| remove_digits |
+---------------+
| abcd |
+---------------+
1 row in set (0.00 sec)

12. REGEXP_INSTR()
SELECT REGEXP_INSTR('abc','b');
+-------------------------+
| REGEXP_INSTR('abc','b') |
+-------------------------+
| 2 |
+-------------------------+
1 row in set (0.00 sec)

13. REGEXP_SUBSTR()
SELECT REGEXP_SUBSTR(
-> 'See https://mariadb.org/en/foundation/ for details',
-> 'https?://[^/]*');
+---------------------------------------------------------------------------------------
-----+
| REGEXP_SUBSTR(
'See https://mariadb.org/en/foundation/ for details',
'https?://[^/]*') |
+---------------------------------------------------------------------------------------
-----+
| https://mariadb.org
|
+---------------------------------------------------------------------------------------
-----+
1 row in set (0.00 sec)

14. PCRE features
• Can have character classes, unicode character types, check script names
SELECT 'abc' RLIKE '^[[:ascii:]]+$';
+------------------------------+
| 'abc' RLIKE '^[[:ascii:]]+$' |
+------------------------------+
| 1 |
+------------------------------+
1 row in set (0.00 sec)

15. PCRE script names
SELECT 'ΣΦΩ' RLIKE '^p{Greek}+$';
+--------------------------------+
| 'ΣΦΩ' RLIKE '^p{Greek}+$' |
+--------------------------------+
| 1 |
+--------------------------------+
MariaDB [developers]> SELECT 'ΣΦΩ' RLIKE '^p{Old_Persian}+$';
+--------------------------------------+
| 'ΣΦΩ' RLIKE '^p{Old_Persian}+$' |
+--------------------------------------+
| 0 |
+--------------------------------------+

16. DELETE … RETURNING
• Delete operations that return a result set of the
deleted rows to the client
DELETE post FROM blog INNER JOIN post
WHERE blog.id = post.blog_id;
DELETE from t1 WHERE a=2 RETURNING *;
MariaDB 10.0+

17. GIS
• MariaDB implements a subset of SQL with
Geometry Types
• No longer just minimum bounding rectangles
(MBR) - shapes considered
CREATE TABLE geom (g GEOMETRY NOT
NULL, SPATIAL INDEX(g)) ENGINE=MyISAM;
• ST_ prefix - as per OpenGIS requirements
MariaDB 5.3+

18. Sample use cases
• Import OpenStreetMap data into MariaDB: http://
www.slideshare.net/hholzgra/fosdem-2014mariadbgis
• Use the OpenStreetMap dataset: https://mariadb.com/kb/
en/openstreetmap-dataset/
• Screencast: https://blog.mariadb.org/screencast-mariadb-
gis-demo/
• node.js example use case for mapping GPX data: https://
blog.mariadb.org/node-js-mariadb-and-gis/ & jQuery
usage: https://blog.mariadb.org/jquery-and-gis-distance-
in-mariadb/

19. Dynamic columns
• Allows you to create virtual columns with dynamic content for each
row in table. Store different attributes for each item (like a web
store).
• Basically a BLOB with handling functions: COLUMN_CREATE,
COLUMN_ADD, COLUMN_GET, COLUMN_DELETE,
COLUMN_EXISTS, COLUMN_LIST, COLUMN_CHECK,
COLUMN_JSON
• In MariaDB 10.0: name support (instead of referring to columns by
numbers, name it), convert all dynamic column content to JSON
array, interface with Cassandra
INSERT INTO tbl SET
dyncol_blob=COLUMN_CREATE("column_name", "value");
MariaDB 5.3+

20. Full-text search via
SphinxSE
mysql> INSTALL PLUGIN sphinx SONAME
'ha_sphinx.so';
Query OK, 0 rows affected (0.01 sec)
MariaDB 5.2+

21. What is SphinxSE?
• SphinxSE is just the storage engine that still
depends on the Sphinx daemon
• It doesn’t store any data itself
• Its just a built-in client to allow MariaDB to talk to
Sphinx searchd, run queries, obtain results
• Indexing, searching is performed on Sphinx

22. Sphinx search table
CREATE TABLE t1
(
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,
group_id INTEGER,
INDEX(query)
) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test";
!
SELECT * FROM t1 WHERE query='test it;mode=any';

23. Sphinx search tables
• 1st column: INTEGER UNSIGNED or BIGINT
(document ID)
• 2nd column: match weight
• 3rd column: VARCHAR or TEXT (your query)
• Query column needs indexing, no other column
needs to be

24. Query Cassandra
• Data is mapped: rowkey, static columns,
dynamic columns
• super columns aren’t supported
• No 1-1 direct map for data types
• Write to Cassandra from SQL (SELECT, INSERT,
UPDATE, DELETE)
MariaDB 10.0+

25. Cassandra II
pk varchar(36) primary key,
data1 varchar(60),
data2 bigint
) engine=cassandra keyspace='ks1' column_family='cf1'
• Table must have a primary key
• name/type must match Cassandra’s rowkey
• Columns map to Cassandra’s static columns
• name must be same as in Cassandra, datatypes must match, can be
subset of CF’s columns

26. Mapping
• Datatype mapping - table at KB
• Data mapping is safe - engine will refuse
incorrect mappings
• Command mapping: INSERT overwrites rows,
UPDATE reads then writes, DELETE reads then
writes

27. Typical use cases
• Web page hits collection, streaming data
• Sensor data
• Reads served with a lookup
• Want an auto-replicated, fault-tolerant table?

28. CONNECT
• Target: ETL for BI or analytics
• Import data from CSV, XML, ODBC, MS Access,
etc.
• WHERE conditions pushed to ODBC source
• DROP TABLE just removes the stored definition,
not data itself
• “Virtual” tables cannot be indexed
MariaDB 10.0+

29. Connectors
• The MariaDB project provides LGPL connectors
(client libraries) for:
• C
• Java
• ODBC
• Embedding a connector? Makes sense to use
these LGPL licensed ones…

30. Optimizer
MariaDB 10 MySQL 5.6
index_merge=on
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
index_merge_sort_intersection=off
engine_condition_pushdown=off
index_condition_pushdown=on
derived_merge=on
derived_with_keys=on
firstmatch=on
loosescan=on
materialization=on
in_to_exists=on
semijoin=on
partial_match_rowid_merge=on
partial_match_table_scan=on
subquery_cache=on
mrr=off
mrr_cost_based=off
mrr_sort_keys=off
outer_join_with_cache=on
semijoin_with_cache=on
join_cache_incremental=on
join_cache_hashed=on
join_cache_bka=on
optimize_join_buffer_size=off
table_elimination=on
extended_keys=on
exists_to_in=off
index_merge=on
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
engine_condition_pushdown=on
index_condition_pushdown=on
mrr=on
mrr_cost_based=on
block_nested_loop=on
batched_key_access=off
materialization=on
semijoin=on
loosescan=on
firstmatch=on
subquery_materialization_cost_based=on
use_index_extensions=on
MariaDB 5.3+

31. Engines, etc
• Plan for backups - TokuDB can be cool for your
uses as an example
• Galera: study your workload patterns, your
application, etc.
• SPIDER: its not going to be straightforward to
“just start” - need to know right tables to
implement, etc.

32. https://mariadb.com/kb/en/

33. Q&A
colin@mariadb.org | byte@bytebot.net
http://skysql.com/ | http://mariadb.org/
twitter: @bytebot | url: http://bytebot.net/blog/