Introduction to Elasticsearch

@somkiat
SPRINT3R
Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

About me
SPRINT3R
สมเกียรติ ปุ๋ยสูงเนิน
สยาม๡ํานาญกิจ
somkiat.cc
SPRINT3R

SPRINT3R
Introduction
Speed by example
• Search
• Aggregation
Agenda
• Operating System and Hardware

SELECT
SPRINT3R
FROM TABLE
WHERE TEXT LIKE ‘%SHIT%’
Search system

SELECT
SPRINT3R
FROM many Table
GROUP BY some column
Report/analytic system

SPRINT3R
Saaaaaaaadddddddddd

SPRINT3R
First version 0.4 in Feb 2010
Rewrite from Compass project
Add scalability
Created by Shay Banon
Current version 1.5.1
History

SPRINT3R
an open source, distributed, scalable,
highly availability, document-oriented, RESTful
Elasticsearch
full text search engine
with near real-time search and analytics

SPRINT3R
Elasticsearch
Apache 2.0 Licence

SPRINT3R
Elasticsearch

SPRINT3R
Elasticsearch
{
"id" : 1,
"name" : "somkiat",
"family_name" : "puisungnoen",
"website" : "somkiat.cc"
}

realtime, search and
analytics engine
SPRINT3R
Lucene
Distributed
Scalability
open-source
documentation
JSON API
RESTFul
document store
High-Availability
JAVA
Plug-ins
Elasticsearch

SPRINT3R
ใครใ๡้บ้าง ?

SPRINT3R
• Search repositories, users, issues, pull request
• Search sourcecode 130 พันล้านบรรทัด
• Track alerts, events และ logs

SPRINT3R
• ใ๡้ Full text seach + geolocation
• ใ๡้ feature More-like-this ในการหาคําถาม และ คําตอบ

SPRINT3R
• Java
• PHP
• Ruby
• Python
• JavaScript
• Go
• Scala
• .Net
• Clojure
• Erlang
• NodeJS • R
http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html
Clients

สวัสดี Elasticsearch
SPRINT3R

SPRINT3R
$ wget https://download.elastic.co/elasticsearch/
elasticsearch/elasticsearch-1.5.1.zip
$ unzip elasticsearch-1.5.1.zip
$ cd elasticsearch-1.5.1
$ bin/elasticsearch

SPRINT3R

Scaling
SPRINT3R

SPRINT3R
Cluster :: กลุ่มของ node
node
cluster

SPRINT3R
node
cluster
node

SPRINT3R
node
cluster
node node

SPRINT3R
Node :: มี 4 ชนิด
master
data
search
master search
data

SPRINT3R
Shard :: unit of scale
0 12 3

SPRINT3R
Shard :: unit of scale
0 1 2 3

SPRINT3R
Replication
0 1 2 33 0 1 2

Search
SPRINT3R

SPRINT3R
RDBMS vs Elasticsearch
D ATA B A S E TA B L E R O W C O L U M N
I N D E X T Y P E D O C U M E N T F I E L D

SPRINT3R
CRUD

SPRINT3R
CRUD :: Create
PUT <index name>/<type>/<id>
{
… your data ..
}

SPRINT3R
CRUD :: Create
PUT shop/book/1
{
"name" : "Elasticsearch: The Definitive Guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"pages": 724,
"published_date": "2015/01/31"
}

SPRINT3R
CRUD :: Read
GET shop/book/1
GET shop/book/_search
GET shop/book/_search?q=elasticsearch

SPRINT3R
CRUD :: Delete
DELETE shop/book/1

SPRINT3R
Searching with DSL
{
"query": { "filtered": {
"query": { "match": { "name": "elasticsearch"}},
"filter": {
"range": { "published_date": { "gte": "now-1y" }}
}
}
}
}

SPRINT3R
Search response in JSON

SPRINT3R
Search from all fields
{
"query": { "filtered": {
"query": { "match": { "_all": "elasticsearch"}},
"filter": {
"range": { "published_date": { "gte": "now-1y" }}
}
}
}
}

SPRINT3R
Reduce search response in JSON

SPRINT3R
Reduce search response in JSON
{
"_source": ["name", "pages"],
…
}

SPRINT3R
Query vs Filter

Aggregations
SPRINT3R

SPRINT3R
High level

SPRINT3R
Aggregations
Buckets and Metrics

SPRINT3R
Aggregations
Documents

SPRINT3R
Aggregations
Query/Filter
Documents

SPRINT3R
Aggregations
Buckets
Query/Filter
Documents

SPRINT3R
Aggregations
Documents
Query/Filter
Buckets
Metrics 2 1 4

SPRINT3R
Aggregations
ผู้แต่งแต่ละคนมีหนัง๤ือกี่เล่ม ?

SPRINT3R
Aggregations
GET shop/book/_search?search_type=count
{
"aggs": {
"book_by_author": {
"terms": {
"field": "authors"
}
}
}
}

SPRINT3R
Results :: ปัญหาคืออะไร ?

SPRINT3R
ปัญหาจาก Inverted index
Term Document ID
clinton 1
gormley 1
tong 1
zachary 1

SPRINT3R
๤ิ่งที่ต้องการจาก Inverted index
Term Document ID
Clinton Gormley 1
Zachary Tong 1

SPRINT3R
กําหนด mapping
PUT shop
{
"mappings": {
"book": {
"properties": {
"authors": {
"index": "not_analyzed",
"type": "string"
}
}
}
}
}

SPRINT3R
Results

Operating System & Hardware
SPRINT3R

SPRINT3R
• CPU
• Memory
Elasticsearch use
• I/O
• Network
Indexing, searching
Indexing, searching, merging
Aggregation, indices
Relocation, snapshot & restore

Summary
SPRINT3R

Summary
SPRINT3R
Speed is key …

Summary
SPRINT3R
Search Tradeoff

Thank you and Questions
SPRINT3R

Introduction to Elasticsearch

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Introduction to Elasticsearch

Similar to Introduction to Elasticsearch (20)

More from Somkiat Puisungnoen

More from Somkiat Puisungnoen (20)

Recently uploaded

Recently uploaded (20)

Introduction to Elasticsearch