Joins in a distributed world Distributed Matters Barcelona 2015

Joins in a distributed world
@LucianPrecup
@dmconf Barcelona 2015 #dmconf15
2015-11-21

whoami
• CTO of Adelean (http://adelean.com/)
• Integrate
– search, nosql and big data technologies
• to support
– ETL, BI, data mining, data processing and data
visualization
• use cases
2015-11-21 2@LucianPrecup #dmconf15 Barcelona

Poll - How many of you …
• Use a NoSQL database within your current
application / information system?
• Which NoSQL database?
• Use a relational database within your current
application / information system?
• Which RDBMS?
• Store the data into more than one system?
• Distribute data on several servers?
• Use an ORM framework (hibernate, doctrine, …) ?

Distributed databases
• Issues
– CAP theorem
– Two phase commit
– Distributed Transactions

Objective
• Show what distributed databases are able or
not able to do about joins and why
• Give YOU the ability to implement joins in
your applications (your service layers)

Joins
• A join combines the output from two sources
• A join condition – the relationship between
the sources
• Join trees

The "Query Optimizer"
SELECT DISTINCT offer_status FROM offer;
SELECT offer_status FROM offer GROUP by offer_status;
≡
SELECT A.ID as ID1, A.X, B.ID as ID2, B.Y
FROM A LEFT OUTER JOIN B
ON A.ID = B.ID
WHERE ID2 IS NOT NULL
SELECT A.ID as ID1, A.X, B.ID as ID2, B.Y
FROM A INNER JOIN B
ON A.ID = B.ID
≡

The query optimizer - decisions
• Access paths
– The optimizer must choose an access path to retrieve data from each table in
the join statement. For example, choose between a full table scan or an index
scan.
• Join methods
– To join each pair of row sources, the database must decide how to do it:
nested loop, sort merge, hash joins. Each join method has specific situations in
which it is more suitable than the others.
• Join types
– The join condition determines the join type: an inner join retrieves only rows
that match the join condition, an outer join retrieves rows that do not match
the join condition.
• Join order
– To execute a statement that joins more than two tables, the database joins
two tables and then joins the resulting row source to the next table and so on.
• Calculating the cost of a query plan
– Based on metrics : I/Os (single block I/O, multiblock I/Os), estimated CPU,
functions and expressions

Join methods
• Nested Loops Joins

Join methods
• Hash Joins

Join methods
• Sort Merge Joins
1
3
7
9
1
2
2
7
8
9

Join types
• Inner Joins
– Equijoins
– nonequijoins
• Outer Joins
– Nested Loop Outer Joins
– Hash Join Outer Joins
– Sort Merge Outer Joins
– Full Outer Joins
– Multiple Tables on the Left of an Outer Join
• Semijoins
– ~ IN or EXISTS clause
• Antijoins
– ~NOT IN or NOT EXISTS

Optimizing tricks
• Push filters as low as possible in the execution tree
• Cache small tables into memory
• Calculate statistics on each datasources and use them
• Think use cases
• “Fortunately, many of the lessons learned over the
years for optimizing and executing join queries can be
directly translated to the distributed world.” (see Nicolas
Bruno et all. in the references below)

Push filters
Table 1 Table 2 Table 1 Table 2
Filter 1
Join
Join‘
Filter 1’ Filter 1’’

Take advantage of small tables
Table 1
Table 2
Join
Table 1
Filter
with data from
Table 2

Use statistics
Table 1 Table 2
Join 1
Table 3
Join 2
Table 3
Join 3
Table 2
Join 4
Table 1

Other join optimisations (technical)
• Join algorithms
– Different ways to implement a logical join operator : nested-loop joins,
sort-based joins, hash-based joins.
– Auxiliary data structures: secondary indexes, join indexes, bitmap
indexes, bloom filters (e.g., indexed loop join, indexed sort-merge join,
and distributed semi-join).
• Bloom Filters
– especially useful when the amount of memory needed to store the
filter is small relative to the amount of data in the data set, and when
most data is expected to fail the membership test.
• Partition-Wise Joins
– A partition-wise join is a join optimization that divides a large join of
two tables, one of which must be partitioned on the join key, into
several smaller joins.
• Full partition-wise join : Both tables must be equipartitioned on their join keys.
We can then divide a large join into smaller joins between two partitions.
• Partial partition-wise joins : Only one table is partitioned on the join key. The
other table may or may not be partitioned.

NoSQLSQL/RDBMS Power to the DBA

SQL/RDBMS Power to the DBA NoSQL Power to the developer
192015-11-21 @LucianPrecup #dmconf15 Barcelona

SQL/RDBMS NoSQL / Distributed / Multi-Model

Distributing data
• Clusters of nodes
• Partitioning the data (sharding)
• Partition keys (random or functional)
• Joining heterogeneous systems
Persons
(Oracle)
Contracts
(SQL Server)
Logs
(Elasticsearch)

Partitions (shards)
• "Random" partitioning
• "Functional" routing key  colocated joins
Data A Data B Data C
Customers by
customer_id
Customers by
customer_id
Contracts by
customer_id
Contracts by
customer_id
Contracts by
customer_id

The join graph topology
• Partitioning schemes: a partition function (e.g., hash
partitioning, range partitioning, random partitioning, and
custom partitioning), a partition key, and a partition count.
• Data exchange operators: initial partitioning,
repartitioning, full merge, partial repartitioning and partial
merge
• Merging schemes: random merge, sort merge, concat-
merge and sort concat-merge.
• Distribution policies: include distribution with duplication
and distribution without duplication

Joins from a functional perspective
• Think use cases : Model and distribute the data according
to the queries you are expecting
– Enrich a table with data from lookup tables, I need objects of
type A filtered by criteria on properties of type B
• Loosen the generality
– I only want star schemas, I only want to bulk load data at night
and query it all day, I only want to run a few really expensive
queries not millions of tiny ones
• Polyglot persistence : store your data in multiple ways
– Graph databases for relations, document store for business
objects, key value for lookup properties
• Services and micro-services : you access the data through
the services layer
– Then you can implement the joins in many ways

The service layer
• Seamless integration for applications
• Caches the data model and exposes a business
model (services fit for business needs)
Service Layer
Customers Contracts …
Application 1 Application 2 …

Implementing your own joins
• According to use cases and capacities of each module, joins
can be implemented at different levels
26
NoSQL Front End
Service Layer
Source 1
Source 2
Batch
(init or delta)
The datasource
executes the join
Real time data
injection
Join executed by the
NoSQL database
(~E.g. Parent Child in
Elasticsearch)
Join implemented by the
Service layer at query time
(~"My Own Custom Join")
JMS
Queue
"GET" calls to the second
data source
(~Nested Loops Join)
Parallel reading of the
two datasources and
join done by the Batch
(~Sort Merge Join)
Read one source, hash it
then stream the other one
(~HashJoin)
2015-11-21

Joins with NoSQL databases
Normalized database Document
{"film" : {
"id" : "183070",
"title" : "The Artist",
"published" : "2011-10-12",
"genre" : ["Romance", "Drama", "Comedy"],
"language" : ["English", "French"],
"persons" : [
{"person" : { "id" : "5079", "name" : "Michel
Hazanavicius", "role" : "director" }},
{"person" : { "id" : "84145", "name" : "Jean
Dujardin", "role" : "actor" }},
{"person" : { "id" : "24485", "name" : "Bérénice
Bejo", "role" : "actor" }},
{"person" : { "id" : "4204", "name" : "John
Goodman", "role" : "actor" }}
]
}}

SQL vs. NoSQL : the issue with joins :-)
• Let’s say you have two relational entities: Persons
and Contracts
– A Person has zero, one or more Contracts
– A Contract is attached to one or more Persons (eg. the
Subscriber, the Grantee, …)
• Need a search services :
– S1: getPersonsDetailsByContractProperties
– S2: getContractsDetailsByPersonProperties
• Simple solution with SQL:
SELECT P.* FROM P, C WHERE P.id = C.pid AND C.a = 'A‘
SELECT C.* FROM P, C WHERE P.id = C.pid AND P.a = 'A'

The issue with joins - solutions
• Solution 1
– Store Persons with Contracts together for S1
{"person" : { "details" : …, … , "contracts" : ["contract" :{"id" : 1, …}, …] }}
– Store Contracts with Persons together for S2
{"contract" : { "details" : …, …, "persons" : ["person" :{"id" : 1, "role" : "S", …}, …]}}
• Issues with solution 1:
– A lot of data duplication
– Have to get Contracts when indexing Persons and vice-versa
• Solution 2
– Use the joins provided by the NoSQL system (Eg. Elasticsearch’s Parent/Child)
• Issues with solution 2:
– Works in one way but not the other (only one parent for n children, a 1 to n relationship)
• Solution 3
– Store Persons and Contracts separately
– Launch two NoSQL queries and join the results into your application to get the final response
– For S1 : First get all Contract ids by Contract properties, then get Persons by Contract ids (terms
query or mget)
– For S2 : First get all Persons ids by Person properties, then get Contracts by Person ids (terms
query or mget)
– The response to the second query can be returned “as is” to the client (pagination, etc.)

Optimizing tricks. Distributed.
• Model the data according to use cases
• Duplicate the data if different modeling
schemas are needed
• Loosen the generality
• Choose a good partitioning key
• Colocate joins as much as possible. The less
redistribution the better.
• Implement your own joins :-)

References
• Back to the future : SQL 92 for Elasticsearch? - Lucian Precup - NoSQL
Matters Dublin 2014 (https://2014.nosql-matters.org/dub/wp-
content/uploads/2014/09/lucian_precup_back_to_the_future_sql_92_for
_elasticsearch.pdf)
• Oracle Database Online Documentation - Database SQL Tuning Guide
(https://docs.oracle.com/database/121/TGSQL/tgsql_join.htm#TGSQL242)
• Advanced Join Strategies for Large-Scale Distributed Computation - Nicolas
Bruno, YongChul Kwon, Ming-Chuan Wu - VLDB
(http://www.vldb.org/pvldb/vol7/p1484-bruno.pdf)
• “Distributed joins are hard to scale”- Interview with Dwight Merriman by
Roberto V. Zicari (http://www.odbms.org/blog/2011/02/distributed-joins-
are-hard-to-scale-interview-with-dwight-merriman/)
• Joins and aggregations in a distributed NoSQL DB - Max Neunhöffer –
NoSQL Matters Dublin 2014 (https://2014.nosql-matters.org/dub/wp-
content/uploads/2014/09/NeunhoefferDublin.pdf)

Joins in a distributed world Distributed Matters Barcelona 2015

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à Joins in a distributed world Distributed Matters Barcelona 2015

Similaire à Joins in a distributed world Distributed Matters Barcelona 2015 (20)

Plus de Lucian Precup

Plus de Lucian Precup (9)

Dernier

Dernier (20)

Joins in a distributed world Distributed Matters Barcelona 2015

Notes de l'éditeur