SlideShare une entreprise Scribd logo
1  sur  43
DEEP DIVE INTO N1QL:
INTERNALS AND POWER FEATURES IN COUCHBASE 4.0
Keshav Murthy
Couchbase Engineering
keshav@couchbase.com
@N1QL @rkeshavmurthy
©2015 Couchbase Inc. 2
Agenda
Query Service Overview
Query ServiceArchitecture
N1QL Power Features
Q&A
Query Service Overview
©2015 Couchbase Inc. 4
Couchbase Server Cluster Architecture
4
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Managed Cache
Storage
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 2
Managed
Cache
Cluster
ManagerCluster
Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Data Service
Index Service
Query Service
STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Data Service
Index Service
Query Service
Managed Cache
Storage
Managed Cache
Storage
Managed Cache
Storage
Managed Cache
Storage
Managed Cache
Storage
©2015 Couchbase Inc. 5
Couchbase Server Cluster Service Deployment
5
STORAGE
Couchbase Server 1
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Managed Cache
Storage
Data
Servic
e
STORAGE
Couchbase Server 2
Managed
Cache
Cluster
ManagerCluster
Manager
Data
Servic
e
STORAGE
Couchbase Server 3
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Data
Servic
e
STORAGE
Couchbase Server 4
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Query
Servic
e
STORAGE
Couchbase Server 5
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Query
Servic
e
STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Index
Servic
e
Managed Cache
Storage
Managed Cache
Storage Storage
STORAGE
Couchbase Server 6
SHARD
7
SHARD
9
SHARD
5
SHARDSHARDSHARD
Managed
Cache
Cluster
ManagerCluster
Manager
Index
Servic
e
Storage
Managed Cache Managed Cache
©2015 Couchbase Inc. 6
N1QL: Query Execution Flow
Clients
1. Submit the query over RESTAPI 8. Query result
2. Parse, Analyze, create Plan 7. Evaluate: Documents to results
3. Scan Request;
index filters
6. Fetch the documents
Index
Servic
e
Query
Service
Data
Servic
e
4. Get qualified doc keys
5. Fetch Request,
doc keys
SELECT c_id,
c_first,
c_last,
c_max
FROM CUSTOMER
WHERE c_id = 49165;
{
"c_first": "Joe",
"c_id": 49165,
"c_last": "Montana",
"c_max" : 50000
}
Query Service Architecture
©2015 Couchbase Inc. 8
Inside a Query Service
Client
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
©2015 Couchbase Inc. 10
Client to Query Service: REST API
 Communication protocol is REST on top of HTTP
 The database protocol structure is embedded
within the REST API.
 Query Service is stateless: All query information
is embedded within the REST request.
 REST is open. All REST clients work with N1QL
 All N1QL clients, JDBC, ODBC drivers use REST
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
import requests
import json
url = "http://localhost:8093/query"
s1=”SELECT * FROM CUSTOMER WHERE C_ID = 1284";
r = requests.post(url, data=s1, auth=('Administrator', 'abc'))
print r.json()
©2015 Couchbase Inc. 11
Query Execution: Parse & Semantic Check
 Analyzes the Query for syntax & grammar
 Only verifies for existence of referenced buckets
 Flexible schema means, you can refer to arbitrary
attribute names
 Use IS MISSING clause to check if the keyname is
present
 Full reference to JSON structure
 Nested reference: CUSTOMER.contact.address.state
 Array Reference: CUSTOMER.c_contact.phone_number[0]
 SQL is enhanced to access & manipulate Arrays
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 12
Query Execution: Parse & Semantic Check
SELECT c_zip,
COUNT(c_id),
AVG(c_balance)
FROM CUSTOMER
WHERE c_state = ‘CA’
AND c_year = 2014
ORDER BY
COUNT(c_id) DESC
LIMIT 100
Simple refererences to the
attribute name, just like columns
Use expressions, just like SQL
Table/keyspace/bucket references.
Filters on the JSON document work
just like SQL
Sorting of the result set
Top N clause.
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 14
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Each query can be executed in several ways
 Create the query execution plan
 Access path for each keyspace reference
 Decide on the filters to push down
 Determine Join order and join method
 Create the execution tree
 For each keyspace reference:
 Look at the available indices
 Match the filters in the query with index keys
 Choose one or more indices for each keyspace
©2015 Couchbase Inc. 15
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
EXPLAIN SELECT c_id,
c_first,
c_middle,
c_last,
c_balance
FROM CUSTOMER
WHERE c_w_id = 49
AND c_d_id = 16
AND c_last = ‘Montana’;
 Explain provides the JSON representation of
the query plan
 Focus on the index selection and the
predicates pushed down
©2015 Couchbase Inc. 16
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
{
"#operator": "IndexScan",
"index":
"CU_W_ID_D_ID_LAST",
"keyspace": "CUSTOMER",
…
"spans": [
{
"Range": {
"High": [
"49",
"16",
""Montana""
],
"Inclusion": 3,
"Low": [
"49",
"16",
©2015 Couchbase Inc. 17
Query Execution: Plan
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
"#operator": "Parallel",
"~child": {
"#operator": "Sequence",
"~children": [
{
"#operator": "Fetch",
"keyspace": "CUSTOMER",
"namespace": "default »
},
{
"#operator": "Filter”, "condition":
"((((`CUSTOMER`.`C_W_ID`) = 49) and ((`CUSTOMER`.`C_D_ID`)
= 16)) and ((`CUSTOMER`.`C_LAST`) = "Montana"))”
},
©2015 Couchbase Inc. 18
Query Execution: Project
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
"#operator": "Parallel",
"~child": {
"#operator": "Sequence",
"~children": [
{
"#operator": ”Project",
"keyspace": "CUSTOMER",
"namespace": "default »
},
{
©2015 Couchbase Inc. 21
Query Execution: Scan
Data Service
Global
Secondary
Index
View Indexes
Global
Secondary
Index
Global
Secondary
Index
KeyScan
IndexScan
IndexScan
Data FetchFetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
Query
Service
Cluster Map
©2015 Couchbase Inc. 22
Query Execution: Fetch
 List of qualified document-keys are grouped into
batches.
 List of the documents is obtained from the Index
or specified directly via USE KEYS clause.
 Fetch request is done in parallel.
 The join operation use the fetch operation to get
the matching document.
 Fetch results are streamed into next operators.
 For big queries, scan-fetch-join-filter-aggregation
will be executing in parallel.
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 23
Query Execution: Join
 You can join any two key spaces if one has
document-key of the other.
 You can store multiple entities within the same
bucket and join between distinct groups
 Uses Nested Loop JOIN now
 JOINs are done in the same order specified in the
query
 Index selection is important for the first keyspace
in the FROM clause.
 Qualified documents from that scan is joined with
the other Keyspace using the DOCUMENT KEYS
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 24
Query Execution: Join
"CUSTOMER": {
"C_D_ID": 10,
"C_ID": 1938,
"C_W_ID": 1,
"C_BALANCE": -10,
"C_CITY": ”San Jose",
"C_CREDIT": "GC”,
"C_DELIVERY_CNT": 0,
"C_DISCOUNT": 0.3866,
"C_FIRST": ”Jay",
"C_LAST": ”Smith",
"C_MIDDLE": "OE",
"C_PAYMENT_CNT": 1,
"C_PHONE": ”555-123-1234",
"C_SINCE": "2015-03-22 00:50:42.822518",
"C_STATE": ”CA",
"C_STREET_1": ”555, Tideway Drive",
"C_STREET_2": ”Alameda",
"C_YTD_PAYMENT": 10,
"C_ZIP": ”94501"
}
Document key: “1.10.1938” Document key: “1.10.143”
“ORDERS”: {
“O_CUSTOMER_KEY”: “1.10.1938”:
"O_D_ID": 10,
"O_ID": 1,
"O_ALL_LOCAL": 1,
"O_CARRIER_ID": 2,
"O_C_ID": 1938,
"O_ENTRY_D": "2015-05-19 16:22:08.544472",
"O_ID": 143,
"O_OL_CNT": 10,
"O_W_ID": 1
}x
“ORDERS”: {
“O_CUSTOMER_KEY”: “1.10.1938”:
"O_ALL_LOCAL": 1,
"O_CARRIER_ID": 2,
"O_C_ID": 1938,
"O_D_ID": 10,
"O_ENTRY_D": "2015-05-19 16:22:08.544472",
"O_ID": 1355,
"O_OL_CNT": 10,
"O_W_ID": 3
}
Document key: “1.10.1355”
©2015 Couchbase Inc. 25
Query Execution: Join
SELECT COUNT(o.O_ORDER_CNT ) AS CNT_O_OL_C
NT
FROM ORDERS o
INNER JOIN CUSTOMER c
ON KEYS (o.O_CUSTOMER_KEY)
WHERE o.O_CARRIER_NAME = ”Penske”
AND c.C_STATE = “CA”;
Two keyspace joins
ON Clause for the join
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 26
N1QL: Join
SELECT *
FROM ORDERS o INNER JOIN CUSTOMER c
ON KEYS (o.O_C_ID)
LEFT JOIN PREMIUM p
ON KEYS (c.C_PR_ID)
LEFT JOIN demographics d
ON KEYS (c.c_DEMO_ID)
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
 Support INNER and LEFT OUTER joins
 Join order follows the order in the FROM clause.
 N1QL supports the nested loop joins now.
 Join is always from a key of one document(outer
table) to the document key of the second
document (inner table)
©2015 Couchbase Inc. 27
Query Execution: Filter
 Filters not pushed to the index scan will have to be
applied.
 Since the indices are maintained asynchronously,
we apply the filters again to ensure integrity of the
result set.
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
©2015 Couchbase Inc. 28
Query Execution: Aggregate, Sort, Offset, Limit
 Each stream creates partial grouping & aggregates
 The result set is sorted to evaluated the ORDER BY
 The sort is done in parallel
 OFFSET and LIMIT is typically used in pagination
 Evaluated after the ORDER BY clause is evaluated.
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
AGG
Scan
©2015 Couchbase Inc. 29
Query Execution: Project
Fetch
Parse
Plan
Join
Filter
Offset
Limit
Project
Sort
Aggre
gate
Scan
SELECT C_ZIP, count(*) as NUMCUSTOMERS
FROM CUSTOMER
GROUP BY C_ZIP ORDER BY COUNT(*) DESC LIMIT 10;
{
"requestID": "ff49a6e6-35f0-4eac-8d74-aa8a0aab58e7",
"signature": {
"C_ZIP": "json",
"NUMCUSTOMERS": "number"
},
"results": [
{
"C_ZIP": "304811111",
"NUMCUSTOMERS": 12
},
...
{
"C_ZIP": "709811111",
"NUMCUSTOMERS": 10
}
],
"status": "success",
"metrics": {
"elapsedTime": "1.57600634s",
"executionTime": "1.575851088s",
"resultCount": 10,
"resultSize": 228
}
Projection
Signature of the resultset
Query execution & resultset
information
N1QL Power Features: USE KEYS
©2015 Couchbase Inc. 31
Power Features: USE KEYS
Data Service
Global
Secondary
Index
View Indexes
Global
Secondary
Index
Global
Secondary
Index
KeyScan
IndexScan
IndexScan
Data Fetch
Query
Service
Cluster Map
©2015 Couchbase Inc. 32
Power Features: USE KEYS
SELECT c_id,
c_first,
c_middle,
c_last,
(c_max - c_balance)
FROM CUSTOMER USE KEYS [‘1.10.1938’];
 KeyScan: Directly use the Couchbase cluster map to get the
document
 You can give one or more values in the array
 From N1QL, get keys via: META(CUSTOMER).id
©2015 Couchbase Inc. 33
Power Features: USE KEYS
EXPLAIN SELECT * FROM CUSTOMER USE KEYS ['1.1.1634', '1.1.1639'];
{
…[
{
"#operator": "Sequence",
"~children": [
{
"#operator": "KeyScan",
"keys": "["1.1.1634”, "1.1.1639”]"
},
{
"#operator": "Parallel",
"~child": {
"#operator": "Sequence",
"~children": [
{
"#operator": "Fetch",
"keyspace": "CUSTOMER",
"namespace": "default"
},
©2015 Couchbase Inc. 34
Power Features: USE KEYS
UPDATE customer USE KEYS ['1.20.981', '12.42.196']
SET c_balance = c_balance + 200;
DELETE customer USE KEYS ['1.20.198', '12.42.2848'];
 Even when you use the USE KEYS, the indexes are
automatically maintained.
N1QL Power Features: UNNEST
©2015 Couchbase Inc. 36
UNNEST: Denormalized CUSTOMER Document
{
"C_ZIP" : "828011111",
"C_STATE" : "vt",
"C_FIRST" : "ykfdbqku",
"C_CREDIT" : "GC",
"C_DELIVERY_CNT" : 0,
"C_W_ID" : 1,
"C_CITY" : "quhpismkzumehqhr",
"C_STREET_1" : "rmtxadlsxqefdcwf",
"C_D_ID" : 1,
"ORDERS" : [
{
"ORDER_LINE" : [
{
"OL_AMOUNT" : 0,
"OL_DELIVERY_D" : "2015-02-11T14:55:25.480Z",
"OL_DIST_INFO" : "yptiwgjdelfxmathbjzirvye",
"OL_I_ID" : 35828,
"OL_SUPPLY_W_ID" : 1,
"OL_QUANTITY" : 5
},
{
"OL_AMOUNT" : 0,
"OL_DELIVERY_D" : "2015-02-11T14:55:25.480Z",
"OL_DIST_INFO" : "dxhqulhcgksjgqsicujzqhdb",
"OL_I_ID" : 26024,
"OL_SUPPLY_W_ID" : 1,
"OL_QUANTITY" : 5
},
}
….
©2015 Couchbase Inc. 37
Power Features: UNNEST operation
SELECT COUNT(my_order_line) AS total_orders,
MAX(my_order_line.ol_delivery_d) AS max_delivery_date,
MAX(my_order_line.ol_quantity) AS max_order_quantity,
MAX(my_orders.o_entry_d) AS max_customer_entry,
MAX(my_orders.o_ol_cnt) AS max_orderline_entry,
COUNT(customer) AS total_customers
FROM CUSTOMER MY_CUSTOMER
UNNEST ORDERS AS my_orders
UNNEST my_orders.order_line AS my_order_line
;
N1QL Power Features:
Named Prepared Statement
©2015 Couchbase Inc. 39
Power Features: Named Prepare Statement
Client
FetchParse Plan Join Filter
Pre-Aggregate
Offset Limit ProjectSortAggregateScan
Query Service
Index
Servic
e
Data
Servic
e
©2015 Couchbase Inc. 41
Named Prepared Statement
url="http://localhost:8093/query"
s = requests.Session()
s.keep_alive = True
s.auth = ('Administrator','password')
query = {'statement':'prepare select * from `beer-sample` where name =
[$1]’}
r = s.post(url, data=query, stream=False)
prepared = str(r.json()['results'][0]['name'])
for i in range (0, 5):
query={'prepared': '"' + prepared + '"', 'args': '["old_hat_brewery"]' }
r = s.post(url, data=query, stream=False)
print i, r.json()['metrics']['executionTime']
BindValues
Many times
Prepare ONCE
Functional Indices
©2015 Couchbase Inc. 43
Functional Indices
"contacts": {
"age": 46,
"children": [
{
"age": 17,
"fname": "Aiden",
"gender": "m"
},
{
"age": 2,
"fname": "Bill",
"gender": "f"
}
],
"email": "dave@gmail.com",
"fname": "Dave",
"hobbies": [
"golf",
"surfing"
],
"lname": "Smith",
"relation": "friend",
"title": "Mr.",
"type": "contact"
CREATE INDEX idx_lname_lower ON
contacts(LOWER(lname)) using GSI;
SELECT count(*)
FROM contacts
WHERE lower(lname) = smith;
©2015 Couchbase Inc. 44
Functional Indices
"contacts": {
"age": 46,
"children": [
{
"age": 17,
"fname": "Aiden",
"gender": "m"
},
{
"age": 2,
"fname": "Bill",
"gender": "f"
}
],
"email": "dave@gmail.com",
"fname": "Dave",
"hobbies": [
"golf",
"surfing"
],
"lname": "Smith",
"relation": "friend",
"title": "Mr.",
"type": "contact"
 The value indexed is the result of the function or
expression.
 The query has to use the same expression in the
WHERE clause for the planner to consider using
the index.
 Use EXPLAIN to verify using the index.
N1QL Power Features:
Multi-Index Scans
©2015 Couchbase Inc. 46
Power Features: IntersectScan (Multi-Index Scan)
SELECT * FROM customer
WHERE c_last = ’Smith’ AND c_city = 'Santa Clara';
 Index scan using composite index:
– Needs first N keys to be used to choose the index
– Will multiple indexes on same set of columns to support filter push down
 IntersectScan using multiple indices:
– Multiple indices are scanned in parallel
– Provides more flexibility in using the indices for filters
– Requires less number of indexes defined on table.
• Can save on disk space and memory space as well.
©2015 Couchbase Inc. 47
Multi-Index Scan
SELECT * FROM customer
WHERE c_last = ’Smith’ AND c_city = 'Santa Clara';
"#operator": "IntersectScan",
"scans": [
{
"#operator": "IndexScan",
"index": "idx_cust_city",
"keyspace": "CUSTOMER",
"limit": 9.223372036854776e+18,
"namespace": "default",
"spans": [
{
"Range": {
"High": [
""Santa Clara""
],
"Inclusion": 3,
"Low": [
""Santa Clara""
]
},
"Seek": null
}
{
"#operator": "IndexScan",
"index": "idx_last_name",
"keyspace": "CUSTOMER",
"limit": 9.223372036854776e+18,
"namespace": "default",
"spans": [
{
"Range": {
"High": [
"”Smith""
],
"Inclusion": 3,
"Low": [
"”Smith""
]
},
"Seek": null
}
],
query.couchbase.com
@N1QL
Keshav Murthy
keshav@couchbase.com
@rkeshavmurthy

Contenu connexe

Tendances

MongoDB ClickStream and Visualization
MongoDB ClickStream and VisualizationMongoDB ClickStream and Visualization
MongoDB ClickStream and VisualizationCameron Sim
 
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto Fernandez
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto FernandezKafka Summit 2018: A Journey Building Kafka Connectors - Pegerto Fernandez
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto FernandezOpenCredo
 
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...Flink Forward
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaGuido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...Guido Schmutz
 
The Mechanics of Testing Large Data Pipelines (QCon London 2016)
The Mechanics of Testing Large Data Pipelines (QCon London 2016)The Mechanics of Testing Large Data Pipelines (QCon London 2016)
The Mechanics of Testing Large Data Pipelines (QCon London 2016)Mathieu Bastian
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Introduction no sql solutions with couchbase and .net core
Introduction no sql solutions with couchbase and .net coreIntroduction no sql solutions with couchbase and .net core
Introduction no sql solutions with couchbase and .net coreBaris Ceviz
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Guido Schmutz
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Guido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Building a GraphQL API in PHP
Building a GraphQL API in PHPBuilding a GraphQL API in PHP
Building a GraphQL API in PHPAndrew Rota
 
Oracle Goldengate for Big Data - LendingClub Implementation
Oracle Goldengate for Big Data - LendingClub ImplementationOracle Goldengate for Big Data - LendingClub Implementation
Oracle Goldengate for Big Data - LendingClub ImplementationVengata Guruswamy
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureGuido Schmutz
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming VisualisationGuido Schmutz
 
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Matt Stubbs
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Guido Schmutz
 
Interactive analytics at scale with druid
Interactive analytics at scale with druidInteractive analytics at scale with druid
Interactive analytics at scale with druidJulien Lavigne du Cadet
 

Tendances (20)

MongoDB ClickStream and Visualization
MongoDB ClickStream and VisualizationMongoDB ClickStream and Visualization
MongoDB ClickStream and Visualization
 
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto Fernandez
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto FernandezKafka Summit 2018: A Journey Building Kafka Connectors - Pegerto Fernandez
Kafka Summit 2018: A Journey Building Kafka Connectors - Pegerto Fernandez
 
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...
Flink Forward San Francisco 2019: Adventures in Scaling from Zero to 5 Billio...
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...
Spark (Structured) Streaming vs. Kafka Streams - two stream processing platfo...
 
The Mechanics of Testing Large Data Pipelines (QCon London 2016)
The Mechanics of Testing Large Data Pipelines (QCon London 2016)The Mechanics of Testing Large Data Pipelines (QCon London 2016)
The Mechanics of Testing Large Data Pipelines (QCon London 2016)
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache Kafka
 
Introduction no sql solutions with couchbase and .net core
Introduction no sql solutions with couchbase and .net coreIntroduction no sql solutions with couchbase and .net core
Introduction no sql solutions with couchbase and .net core
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Building a GraphQL API in PHP
Building a GraphQL API in PHPBuilding a GraphQL API in PHP
Building a GraphQL API in PHP
 
Oracle Goldengate for Big Data - LendingClub Implementation
Oracle Goldengate for Big Data - LendingClub ImplementationOracle Goldengate for Big Data - LendingClub Implementation
Oracle Goldengate for Big Data - LendingClub Implementation
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data Architecture
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming Visualisation
 
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka
 
Interactive analytics at scale with druid
Interactive analytics at scale with druidInteractive analytics at scale with druid
Interactive analytics at scale with druid
 

En vedette

Tim edwards new ways to deliver availability to add value for consumers
Tim edwards new ways to deliver availability to add value for consumersTim edwards new ways to deliver availability to add value for consumers
Tim edwards new ways to deliver availability to add value for consumersECR Community
 
Servic es marketing project report
Servic es marketing  project report Servic es marketing  project report
Servic es marketing project report Babasab Patil
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseCecile Le Pape
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingKeshav Murthy
 
Understanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesUnderstanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesKeshav Murthy
 
Query in Couchbase. N1QL: SQL for JSON
Query in Couchbase.  N1QL: SQL for JSONQuery in Couchbase.  N1QL: SQL for JSON
Query in Couchbase. N1QL: SQL for JSONKeshav Murthy
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLKeshav Murthy
 
N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.Keshav Murthy
 
Tuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesTuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesKeshav Murthy
 
Building Service 2 Presentation
Building Service 2 PresentationBuilding Service 2 Presentation
Building Service 2 PresentationKellyann Hiew
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Cecile Le Pape
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedKorea Sdec
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Keshav Murthy
 
Mechanical ventilation in neonates
Mechanical ventilation in neonatesMechanical ventilation in neonates
Mechanical ventilation in neonatespalpeds
 
Hospital housekeeping services
Hospital housekeeping servicesHospital housekeeping services
Hospital housekeeping servicesNc Das
 

En vedette (19)

Portfolio
PortfolioPortfolio
Portfolio
 
Tim edwards new ways to deliver availability to add value for consumers
Tim edwards new ways to deliver availability to add value for consumersTim edwards new ways to deliver availability to add value for consumers
Tim edwards new ways to deliver availability to add value for consumers
 
Servic es marketing project report
Servic es marketing  project report Servic es marketing  project report
Servic es marketing project report
 
Pipes in sanitation
Pipes in sanitationPipes in sanitation
Pipes in sanitation
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
 
Drilling on JSON
Drilling on JSONDrilling on JSON
Drilling on JSON
 
Utilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and IndexingUtilizing Arrays: Modeling, Querying and Indexing
Utilizing Arrays: Modeling, Querying and Indexing
 
Understanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune QueriesUnderstanding N1QL Optimizer to Tune Queries
Understanding N1QL Optimizer to Tune Queries
 
Query in Couchbase. N1QL: SQL for JSON
Query in Couchbase.  N1QL: SQL for JSONQuery in Couchbase.  N1QL: SQL for JSON
Query in Couchbase. N1QL: SQL for JSON
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQLBringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
 
N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.N1QL workshop: Indexing & Query turning.
N1QL workshop: Indexing & Query turning.
 
Tuning for Performance: indexes & Queries
Tuning for Performance: indexes & QueriesTuning for Performance: indexes & Queries
Tuning for Performance: indexes & Queries
 
Couchbase Day
Couchbase DayCouchbase Day
Couchbase Day
 
Building Service 2 Presentation
Building Service 2 PresentationBuilding Service 2 Presentation
Building Service 2 Presentation
 
Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016Couchbase @ Big Data France 2016
Couchbase @ Big Data France 2016
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speed
 
Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data. Accelerating analytics on the Sensor and IoT Data.
Accelerating analytics on the Sensor and IoT Data.
 
Mechanical ventilation in neonates
Mechanical ventilation in neonatesMechanical ventilation in neonates
Mechanical ventilation in neonates
 
Hospital housekeeping services
Hospital housekeeping servicesHospital housekeeping services
Hospital housekeeping services
 

Similaire à Deep dive into N1QL: SQL for JSON: Internals and power features.

MongoDB Stich Overview
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich OverviewMongoDB
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB
 
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBIntroducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBMongoDB
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5Keshav Murthy
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0Keshav Murthy
 
Consul: Service Mesh for Microservices
Consul: Service Mesh for MicroservicesConsul: Service Mesh for Microservices
Consul: Service Mesh for MicroservicesArmonDadgar
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONKeshav Murthy
 
NoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseNoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseBrant Burnett
 
APAC ksqlDB Workshop
APAC ksqlDB WorkshopAPAC ksqlDB Workshop
APAC ksqlDB Workshopconfluent
 
Day 4 - Cloud Migration - But How?
Day 4 - Cloud Migration - But How?Day 4 - Cloud Migration - But How?
Day 4 - Cloud Migration - But How?Amazon Web Services
 
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016 Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016 Lucas Jellema
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revisedMongoDB
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessMongoDB
 
All Streams Ahead! ksqlDB Workshop ANZ
All Streams Ahead! ksqlDB Workshop ANZAll Streams Ahead! ksqlDB Workshop ANZ
All Streams Ahead! ksqlDB Workshop ANZconfluent
 
Connecting the Dots: Kong for GraphQL Endpoints
Connecting the Dots: Kong for GraphQL EndpointsConnecting the Dots: Kong for GraphQL Endpoints
Connecting the Dots: Kong for GraphQL EndpointsJulien Bataillé
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications Keshav Murthy
 
Webinar: The Anatomy of the Cloudant Data Layer
Webinar: The Anatomy of the Cloudant Data LayerWebinar: The Anatomy of the Cloudant Data Layer
Webinar: The Anatomy of the Cloudant Data LayerIBM Cloud Data Services
 
SDKs, the good the bad the ugly - Japan
SDKs, the good the bad the ugly - JapanSDKs, the good the bad the ugly - Japan
SDKs, the good the bad the ugly - Japantristansokol
 
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...Noriaki Tatsumi
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLKeshav Murthy
 

Similaire à Deep dive into N1QL: SQL for JSON: Internals and power features. (20)

MongoDB Stich Overview
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich Overview
 
MongoDB Stitch Introduction
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch Introduction
 
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBIntroducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0
 
Consul: Service Mesh for Microservices
Consul: Service Mesh for MicroservicesConsul: Service Mesh for Microservices
Consul: Service Mesh for Microservices
 
Introducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSONIntroducing N1QL: New SQL Based Query Language for JSON
Introducing N1QL: New SQL Based Query Language for JSON
 
NoSQL Data Modeling using Couchbase
NoSQL Data Modeling using CouchbaseNoSQL Data Modeling using Couchbase
NoSQL Data Modeling using Couchbase
 
APAC ksqlDB Workshop
APAC ksqlDB WorkshopAPAC ksqlDB Workshop
APAC ksqlDB Workshop
 
Day 4 - Cloud Migration - But How?
Day 4 - Cloud Migration - But How?Day 4 - Cloud Migration - But How?
Day 4 - Cloud Migration - But How?
 
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016 Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016
Soaring through the Clouds - Oracle Fusion Middleware Partner Forum 2016
 
Eagle6 mongo dc revised
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revised
 
Eagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational Awareness
 
All Streams Ahead! ksqlDB Workshop ANZ
All Streams Ahead! ksqlDB Workshop ANZAll Streams Ahead! ksqlDB Workshop ANZ
All Streams Ahead! ksqlDB Workshop ANZ
 
Connecting the Dots: Kong for GraphQL Endpoints
Connecting the Dots: Kong for GraphQL EndpointsConnecting the Dots: Kong for GraphQL Endpoints
Connecting the Dots: Kong for GraphQL Endpoints
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
 
Webinar: The Anatomy of the Cloudant Data Layer
Webinar: The Anatomy of the Cloudant Data LayerWebinar: The Anatomy of the Cloudant Data Layer
Webinar: The Anatomy of the Cloudant Data Layer
 
SDKs, the good the bad the ugly - Japan
SDKs, the good the bad the ugly - JapanSDKs, the good the bad the ugly - Japan
SDKs, the good the bad the ugly - Japan
 
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...
GraphQL Summit 2019 - Configuration Driven Data as a Service Gateway with Gra...
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QLEnterprise Architect's view of Couchbase 4.0 with N1QL
Enterprise Architect's view of Couchbase 4.0 with N1QL
 

Plus de Keshav Murthy

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0Keshav Murthy
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Keshav Murthy
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...Keshav Murthy
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresKeshav Murthy
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliKeshav Murthy
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Keshav Murthy
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber Keshav Murthy
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersKeshav Murthy
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorKeshav Murthy
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONKeshav Murthy
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Keshav Murthy
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixKeshav Murthy
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherKeshav Murthy
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Keshav Murthy
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013Keshav Murthy
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveKeshav Murthy
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Keshav Murthy
 

Plus de Keshav Murthy (17)

N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0N1QL New Features in couchbase 7.0
N1QL New Features in couchbase 7.0
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
 
Couchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing featuresCouchbase 5.5: N1QL and Indexing features
Couchbase 5.5: N1QL and Indexing features
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram VemulapalliN1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
 
Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.Couchbase N1QL: Language & Architecture Overview.
Couchbase N1QL: Language & Architecture Overview.
 
Couchbase Query Workbench Enhancements By Eben Haber
Couchbase Query Workbench Enhancements  By Eben Haber Couchbase Query Workbench Enhancements  By Eben Haber
Couchbase Query Workbench Enhancements By Eben Haber
 
Mindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developersMindmap: Oracle to Couchbase for developers
Mindmap: Oracle to Couchbase for developers
 
Couchbase N1QL: Index Advisor
Couchbase N1QL: Index AdvisorCouchbase N1QL: Index Advisor
Couchbase N1QL: Index Advisor
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSON
 
Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5Extended JOIN in Couchbase Server 4.5
Extended JOIN in Couchbase Server 4.5
 
You know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on InformixYou know what iMEAN? Using MEAN stack for application dev on Informix
You know what iMEAN? Using MEAN stack for application dev on Informix
 
Informix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all togetherInformix SQL & NoSQL: Putting it all together
Informix SQL & NoSQL: Putting it all together
 
Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22Informix SQL & NoSQL -- for Chat with the labs on 4/22
Informix SQL & NoSQL -- for Chat with the labs on 4/22
 
NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013NoSQL Deepdive - with Informix NoSQL. IOD 2013
NoSQL Deepdive - with Informix NoSQL. IOD 2013
 
Informix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep diveInformix NoSQL & Hybrid SQL detailed deep dive
Informix NoSQL & Hybrid SQL detailed deep dive
 
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
Table for two? Hybrid approach to developing combined SQL, NoSQL applications...
 

Dernier

Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...MyFAA
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houstonjennysmithusa549
 
Einstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfEinstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfCloudMetic
 
Steps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic DevelopersSteps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic Developersmichealwillson701
 
MUT4SLX: Extensions for Mutation Testing of Stateflow Models
MUT4SLX: Extensions for Mutation Testing of Stateflow ModelsMUT4SLX: Extensions for Mutation Testing of Stateflow Models
MUT4SLX: Extensions for Mutation Testing of Stateflow ModelsUniversity of Antwerp
 
openEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleopenEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleShane Coughlan
 
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...jackiepotts6
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptxAGATSoftware
 
Boost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made EasyBoost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made Easymichealwillson701
 
Practical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfPractical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfICS
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsconfluent
 
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Inc
 
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurMinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurPriyadarshini T
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityRandy Shoup
 
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
Unlocking AI:Navigating Open Source vs. Commercial FrontiersUnlocking AI:Navigating Open Source vs. Commercial Frontiers
Unlocking AI: Navigating Open Source vs. Commercial FrontiersRaphaël Semeteys
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...Maxim Salnikov
 
BATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckNaval Singh
 
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...telebusocialmarketin
 
User Experience Designer | Kaylee Miller Resume
User Experience Designer | Kaylee Miller ResumeUser Experience Designer | Kaylee Miller Resume
User Experience Designer | Kaylee Miller ResumeKaylee Miller
 

Dernier (20)

Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houston
 
Einstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfEinstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdf
 
Steps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic DevelopersSteps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic Developers
 
MUT4SLX: Extensions for Mutation Testing of Stateflow Models
MUT4SLX: Extensions for Mutation Testing of Stateflow ModelsMUT4SLX: Extensions for Mutation Testing of Stateflow Models
MUT4SLX: Extensions for Mutation Testing of Stateflow Models
 
openEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleopenEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scale
 
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
 
Boost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made EasyBoost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made Easy
 
Practical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfPractical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdf
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
 
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
 
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurMinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
 
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
Unlocking AI:Navigating Open Source vs. Commercial FrontiersUnlocking AI:Navigating Open Source vs. Commercial Frontiers
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
 
BATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data Mesh
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deck
 
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...
Telebu Social -Whatsapp Business API : Mastering Omnichannel Business Communi...
 
User Experience Designer | Kaylee Miller Resume
User Experience Designer | Kaylee Miller ResumeUser Experience Designer | Kaylee Miller Resume
User Experience Designer | Kaylee Miller Resume
 

Deep dive into N1QL: SQL for JSON: Internals and power features.

  • 1. DEEP DIVE INTO N1QL: INTERNALS AND POWER FEATURES IN COUCHBASE 4.0 Keshav Murthy Couchbase Engineering keshav@couchbase.com @N1QL @rkeshavmurthy
  • 2. ©2015 Couchbase Inc. 2 Agenda Query Service Overview Query ServiceArchitecture N1QL Power Features Q&A
  • 4. ©2015 Couchbase Inc. 4 Couchbase Server Cluster Architecture 4 STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 Managed Cache Cluster ManagerCluster Manager Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Service Index Service Query Service Managed Cache Storage Managed Cache Storage Managed Cache Storage Managed Cache Storage Managed Cache Storage
  • 5. ©2015 Couchbase Inc. 5 Couchbase Server Cluster Service Deployment 5 STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Managed Cache Storage Data Servic e STORAGE Couchbase Server 2 Managed Cache Cluster ManagerCluster Manager Data Servic e STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Data Servic e STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Query Servic e STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Query Servic e STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Index Servic e Managed Cache Storage Managed Cache Storage Storage STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARDSHARDSHARD Managed Cache Cluster ManagerCluster Manager Index Servic e Storage Managed Cache Managed Cache
  • 6. ©2015 Couchbase Inc. 6 N1QL: Query Execution Flow Clients 1. Submit the query over RESTAPI 8. Query result 2. Parse, Analyze, create Plan 7. Evaluate: Documents to results 3. Scan Request; index filters 6. Fetch the documents Index Servic e Query Service Data Servic e 4. Get qualified doc keys 5. Fetch Request, doc keys SELECT c_id, c_first, c_last, c_max FROM CUSTOMER WHERE c_id = 49165; { "c_first": "Joe", "c_id": 49165, "c_last": "Montana", "c_max" : 50000 }
  • 8. ©2015 Couchbase Inc. 8 Inside a Query Service Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  • 9. ©2015 Couchbase Inc. 10 Client to Query Service: REST API  Communication protocol is REST on top of HTTP  The database protocol structure is embedded within the REST API.  Query Service is stateless: All query information is embedded within the REST request.  REST is open. All REST clients work with N1QL  All N1QL clients, JDBC, ODBC drivers use REST Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan import requests import json url = "http://localhost:8093/query" s1=”SELECT * FROM CUSTOMER WHERE C_ID = 1284"; r = requests.post(url, data=s1, auth=('Administrator', 'abc')) print r.json()
  • 10. ©2015 Couchbase Inc. 11 Query Execution: Parse & Semantic Check  Analyzes the Query for syntax & grammar  Only verifies for existence of referenced buckets  Flexible schema means, you can refer to arbitrary attribute names  Use IS MISSING clause to check if the keyname is present  Full reference to JSON structure  Nested reference: CUSTOMER.contact.address.state  Array Reference: CUSTOMER.c_contact.phone_number[0]  SQL is enhanced to access & manipulate Arrays Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 11. ©2015 Couchbase Inc. 12 Query Execution: Parse & Semantic Check SELECT c_zip, COUNT(c_id), AVG(c_balance) FROM CUSTOMER WHERE c_state = ‘CA’ AND c_year = 2014 ORDER BY COUNT(c_id) DESC LIMIT 100 Simple refererences to the attribute name, just like columns Use expressions, just like SQL Table/keyspace/bucket references. Filters on the JSON document work just like SQL Sorting of the result set Top N clause. Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 12. ©2015 Couchbase Inc. 14 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Each query can be executed in several ways  Create the query execution plan  Access path for each keyspace reference  Decide on the filters to push down  Determine Join order and join method  Create the execution tree  For each keyspace reference:  Look at the available indices  Match the filters in the query with index keys  Choose one or more indices for each keyspace
  • 13. ©2015 Couchbase Inc. 15 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan EXPLAIN SELECT c_id, c_first, c_middle, c_last, c_balance FROM CUSTOMER WHERE c_w_id = 49 AND c_d_id = 16 AND c_last = ‘Montana’;  Explain provides the JSON representation of the query plan  Focus on the index selection and the predicates pushed down
  • 14. ©2015 Couchbase Inc. 16 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan { "#operator": "IndexScan", "index": "CU_W_ID_D_ID_LAST", "keyspace": "CUSTOMER", … "spans": [ { "Range": { "High": [ "49", "16", ""Montana"" ], "Inclusion": 3, "Low": [ "49", "16",
  • 15. ©2015 Couchbase Inc. 17 Query Execution: Plan Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan "#operator": "Parallel", "~child": { "#operator": "Sequence", "~children": [ { "#operator": "Fetch", "keyspace": "CUSTOMER", "namespace": "default » }, { "#operator": "Filter”, "condition": "((((`CUSTOMER`.`C_W_ID`) = 49) and ((`CUSTOMER`.`C_D_ID`) = 16)) and ((`CUSTOMER`.`C_LAST`) = "Montana"))” },
  • 16. ©2015 Couchbase Inc. 18 Query Execution: Project Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan "#operator": "Parallel", "~child": { "#operator": "Sequence", "~children": [ { "#operator": ”Project", "keyspace": "CUSTOMER", "namespace": "default » }, {
  • 17. ©2015 Couchbase Inc. 21 Query Execution: Scan Data Service Global Secondary Index View Indexes Global Secondary Index Global Secondary Index KeyScan IndexScan IndexScan Data FetchFetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan Query Service Cluster Map
  • 18. ©2015 Couchbase Inc. 22 Query Execution: Fetch  List of qualified document-keys are grouped into batches.  List of the documents is obtained from the Index or specified directly via USE KEYS clause.  Fetch request is done in parallel.  The join operation use the fetch operation to get the matching document.  Fetch results are streamed into next operators.  For big queries, scan-fetch-join-filter-aggregation will be executing in parallel. Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 19. ©2015 Couchbase Inc. 23 Query Execution: Join  You can join any two key spaces if one has document-key of the other.  You can store multiple entities within the same bucket and join between distinct groups  Uses Nested Loop JOIN now  JOINs are done in the same order specified in the query  Index selection is important for the first keyspace in the FROM clause.  Qualified documents from that scan is joined with the other Keyspace using the DOCUMENT KEYS Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 20. ©2015 Couchbase Inc. 24 Query Execution: Join "CUSTOMER": { "C_D_ID": 10, "C_ID": 1938, "C_W_ID": 1, "C_BALANCE": -10, "C_CITY": ”San Jose", "C_CREDIT": "GC”, "C_DELIVERY_CNT": 0, "C_DISCOUNT": 0.3866, "C_FIRST": ”Jay", "C_LAST": ”Smith", "C_MIDDLE": "OE", "C_PAYMENT_CNT": 1, "C_PHONE": ”555-123-1234", "C_SINCE": "2015-03-22 00:50:42.822518", "C_STATE": ”CA", "C_STREET_1": ”555, Tideway Drive", "C_STREET_2": ”Alameda", "C_YTD_PAYMENT": 10, "C_ZIP": ”94501" } Document key: “1.10.1938” Document key: “1.10.143” “ORDERS”: { “O_CUSTOMER_KEY”: “1.10.1938”: "O_D_ID": 10, "O_ID": 1, "O_ALL_LOCAL": 1, "O_CARRIER_ID": 2, "O_C_ID": 1938, "O_ENTRY_D": "2015-05-19 16:22:08.544472", "O_ID": 143, "O_OL_CNT": 10, "O_W_ID": 1 }x “ORDERS”: { “O_CUSTOMER_KEY”: “1.10.1938”: "O_ALL_LOCAL": 1, "O_CARRIER_ID": 2, "O_C_ID": 1938, "O_D_ID": 10, "O_ENTRY_D": "2015-05-19 16:22:08.544472", "O_ID": 1355, "O_OL_CNT": 10, "O_W_ID": 3 } Document key: “1.10.1355”
  • 21. ©2015 Couchbase Inc. 25 Query Execution: Join SELECT COUNT(o.O_ORDER_CNT ) AS CNT_O_OL_C NT FROM ORDERS o INNER JOIN CUSTOMER c ON KEYS (o.O_CUSTOMER_KEY) WHERE o.O_CARRIER_NAME = ”Penske” AND c.C_STATE = “CA”; Two keyspace joins ON Clause for the join Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 22. ©2015 Couchbase Inc. 26 N1QL: Join SELECT * FROM ORDERS o INNER JOIN CUSTOMER c ON KEYS (o.O_C_ID) LEFT JOIN PREMIUM p ON KEYS (c.C_PR_ID) LEFT JOIN demographics d ON KEYS (c.c_DEMO_ID) Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan  Support INNER and LEFT OUTER joins  Join order follows the order in the FROM clause.  N1QL supports the nested loop joins now.  Join is always from a key of one document(outer table) to the document key of the second document (inner table)
  • 23. ©2015 Couchbase Inc. 27 Query Execution: Filter  Filters not pushed to the index scan will have to be applied.  Since the indices are maintained asynchronously, we apply the filters again to ensure integrity of the result set. Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan
  • 24. ©2015 Couchbase Inc. 28 Query Execution: Aggregate, Sort, Offset, Limit  Each stream creates partial grouping & aggregates  The result set is sorted to evaluated the ORDER BY  The sort is done in parallel  OFFSET and LIMIT is typically used in pagination  Evaluated after the ORDER BY clause is evaluated. Fetch Parse Plan Join Filter Offset Limit Project Sort AGG Scan
  • 25. ©2015 Couchbase Inc. 29 Query Execution: Project Fetch Parse Plan Join Filter Offset Limit Project Sort Aggre gate Scan SELECT C_ZIP, count(*) as NUMCUSTOMERS FROM CUSTOMER GROUP BY C_ZIP ORDER BY COUNT(*) DESC LIMIT 10; { "requestID": "ff49a6e6-35f0-4eac-8d74-aa8a0aab58e7", "signature": { "C_ZIP": "json", "NUMCUSTOMERS": "number" }, "results": [ { "C_ZIP": "304811111", "NUMCUSTOMERS": 12 }, ... { "C_ZIP": "709811111", "NUMCUSTOMERS": 10 } ], "status": "success", "metrics": { "elapsedTime": "1.57600634s", "executionTime": "1.575851088s", "resultCount": 10, "resultSize": 228 } Projection Signature of the resultset Query execution & resultset information
  • 27. ©2015 Couchbase Inc. 31 Power Features: USE KEYS Data Service Global Secondary Index View Indexes Global Secondary Index Global Secondary Index KeyScan IndexScan IndexScan Data Fetch Query Service Cluster Map
  • 28. ©2015 Couchbase Inc. 32 Power Features: USE KEYS SELECT c_id, c_first, c_middle, c_last, (c_max - c_balance) FROM CUSTOMER USE KEYS [‘1.10.1938’];  KeyScan: Directly use the Couchbase cluster map to get the document  You can give one or more values in the array  From N1QL, get keys via: META(CUSTOMER).id
  • 29. ©2015 Couchbase Inc. 33 Power Features: USE KEYS EXPLAIN SELECT * FROM CUSTOMER USE KEYS ['1.1.1634', '1.1.1639']; { …[ { "#operator": "Sequence", "~children": [ { "#operator": "KeyScan", "keys": "["1.1.1634”, "1.1.1639”]" }, { "#operator": "Parallel", "~child": { "#operator": "Sequence", "~children": [ { "#operator": "Fetch", "keyspace": "CUSTOMER", "namespace": "default" },
  • 30. ©2015 Couchbase Inc. 34 Power Features: USE KEYS UPDATE customer USE KEYS ['1.20.981', '12.42.196'] SET c_balance = c_balance + 200; DELETE customer USE KEYS ['1.20.198', '12.42.2848'];  Even when you use the USE KEYS, the indexes are automatically maintained.
  • 32. ©2015 Couchbase Inc. 36 UNNEST: Denormalized CUSTOMER Document { "C_ZIP" : "828011111", "C_STATE" : "vt", "C_FIRST" : "ykfdbqku", "C_CREDIT" : "GC", "C_DELIVERY_CNT" : 0, "C_W_ID" : 1, "C_CITY" : "quhpismkzumehqhr", "C_STREET_1" : "rmtxadlsxqefdcwf", "C_D_ID" : 1, "ORDERS" : [ { "ORDER_LINE" : [ { "OL_AMOUNT" : 0, "OL_DELIVERY_D" : "2015-02-11T14:55:25.480Z", "OL_DIST_INFO" : "yptiwgjdelfxmathbjzirvye", "OL_I_ID" : 35828, "OL_SUPPLY_W_ID" : 1, "OL_QUANTITY" : 5 }, { "OL_AMOUNT" : 0, "OL_DELIVERY_D" : "2015-02-11T14:55:25.480Z", "OL_DIST_INFO" : "dxhqulhcgksjgqsicujzqhdb", "OL_I_ID" : 26024, "OL_SUPPLY_W_ID" : 1, "OL_QUANTITY" : 5 }, } ….
  • 33. ©2015 Couchbase Inc. 37 Power Features: UNNEST operation SELECT COUNT(my_order_line) AS total_orders, MAX(my_order_line.ol_delivery_d) AS max_delivery_date, MAX(my_order_line.ol_quantity) AS max_order_quantity, MAX(my_orders.o_entry_d) AS max_customer_entry, MAX(my_orders.o_ol_cnt) AS max_orderline_entry, COUNT(customer) AS total_customers FROM CUSTOMER MY_CUSTOMER UNNEST ORDERS AS my_orders UNNEST my_orders.order_line AS my_order_line ;
  • 34. N1QL Power Features: Named Prepared Statement
  • 35. ©2015 Couchbase Inc. 39 Power Features: Named Prepare Statement Client FetchParse Plan Join Filter Pre-Aggregate Offset Limit ProjectSortAggregateScan Query Service Index Servic e Data Servic e
  • 36. ©2015 Couchbase Inc. 41 Named Prepared Statement url="http://localhost:8093/query" s = requests.Session() s.keep_alive = True s.auth = ('Administrator','password') query = {'statement':'prepare select * from `beer-sample` where name = [$1]’} r = s.post(url, data=query, stream=False) prepared = str(r.json()['results'][0]['name']) for i in range (0, 5): query={'prepared': '"' + prepared + '"', 'args': '["old_hat_brewery"]' } r = s.post(url, data=query, stream=False) print i, r.json()['metrics']['executionTime'] BindValues Many times Prepare ONCE
  • 38. ©2015 Couchbase Inc. 43 Functional Indices "contacts": { "age": 46, "children": [ { "age": 17, "fname": "Aiden", "gender": "m" }, { "age": 2, "fname": "Bill", "gender": "f" } ], "email": "dave@gmail.com", "fname": "Dave", "hobbies": [ "golf", "surfing" ], "lname": "Smith", "relation": "friend", "title": "Mr.", "type": "contact" CREATE INDEX idx_lname_lower ON contacts(LOWER(lname)) using GSI; SELECT count(*) FROM contacts WHERE lower(lname) = smith;
  • 39. ©2015 Couchbase Inc. 44 Functional Indices "contacts": { "age": 46, "children": [ { "age": 17, "fname": "Aiden", "gender": "m" }, { "age": 2, "fname": "Bill", "gender": "f" } ], "email": "dave@gmail.com", "fname": "Dave", "hobbies": [ "golf", "surfing" ], "lname": "Smith", "relation": "friend", "title": "Mr.", "type": "contact"  The value indexed is the result of the function or expression.  The query has to use the same expression in the WHERE clause for the planner to consider using the index.  Use EXPLAIN to verify using the index.
  • 41. ©2015 Couchbase Inc. 46 Power Features: IntersectScan (Multi-Index Scan) SELECT * FROM customer WHERE c_last = ’Smith’ AND c_city = 'Santa Clara';  Index scan using composite index: – Needs first N keys to be used to choose the index – Will multiple indexes on same set of columns to support filter push down  IntersectScan using multiple indices: – Multiple indices are scanned in parallel – Provides more flexibility in using the indices for filters – Requires less number of indexes defined on table. • Can save on disk space and memory space as well.
  • 42. ©2015 Couchbase Inc. 47 Multi-Index Scan SELECT * FROM customer WHERE c_last = ’Smith’ AND c_city = 'Santa Clara'; "#operator": "IntersectScan", "scans": [ { "#operator": "IndexScan", "index": "idx_cust_city", "keyspace": "CUSTOMER", "limit": 9.223372036854776e+18, "namespace": "default", "spans": [ { "Range": { "High": [ ""Santa Clara"" ], "Inclusion": 3, "Low": [ ""Santa Clara"" ] }, "Seek": null } { "#operator": "IndexScan", "index": "idx_last_name", "keyspace": "CUSTOMER", "limit": 9.223372036854776e+18, "namespace": "default", "spans": [ { "Range": { "High": [ "”Smith"" ], "Inclusion": 3, "Low": [ "”Smith"" ] }, "Seek": null } ],

Notes de l'éditeur

  1. Deep Dive into N1QL: Power Features and Internals in Couchbase Server 4.0 Abstract: N1QL is a rich query language for JSON data.   N1QL provides the following enhanced SQL statements: SELECT, INSERT, UPDATE, DELETE, MERGE. We’ll explain the advanced select-join-project-nest-unnest operations as well as data modification features in N1QL. We’ll also discuss basics of index selection and query planning in N1QL. ------------- Take aways Query engine designed for performance on scale-out/distributed Completeness of the N1QL Language – SQL and enhanced for JSON (flexible data model) --------------
  2. Simple One media installation of the cluster. 90 seconds to install!
  3. Data-parallel — Query latency scales up with cores Memory-bound
  4. PrimaryScan Equivalent of full table scan in RDBMS Uses the primary index to scan from start to finish Equivalent of full table scan in RDBMS
  5. Data-parallel — Query latency scales up with cores Memory-bound