PostgreSQL and Groonga Integration
•
6 likes•7,214 views
- PostgreSQL has many extension points including data types, functions, operators and more. This allows it to be extended with tools like Groonga. - There are two main ways to integrate Groonga with PostgreSQL: textsearch_groonga for full text search and groonga_fdw, which allows Groonga to be accessed via foreign tables. - textsearch_groonga adds Groonga's text search capabilities to PostgreSQL for features like morphological analysis and N-gram search. groonga_fdw treats a Groonga database as an external data source so it can be queried from PostgreSQL using standard SQL.
Recommended
More Related Content
Similar to PostgreSQL and Groonga Integration
Similar to PostgreSQL and Groonga Integration (20)
Recently uploaded
Recently uploaded (20)
PostgreSQL and Groonga Integration
- 1. groonga with PostgreSQL 2011/11/29 groonga #2 & (@choplin) 1
- 2. Agenda • Extensibility of PostgreSQL • How to work with groonga • textsearch_groonga • groonga_fdw 2
- 4. Extensibility of PostgreSQL dblink, adminpack, pgcrypto, ... Parser cube, ltree, citext, hstore, json, … Functions Data Types PL pl/R, pl/v8js, pl/Ruby, pl/Lua, … Optimizer pgadviser Executor Table auto_explain Index GiST/GIN, textsearch_senna, textsearch_groonga SQL/MED (9.1!) c.f. external tools: pgpool, slony, pg_bulkload… ©2011 FORCIA, Inc. http://www.slideshare.net/HitoshiHarada/extensions-on-postgresql 4
- 5. PGXN • PostgreSQL Extension Network • Central distribution system for PostgreSQL Extension • Inspired by CPAN for Perl library • Provides: • Manager, API, Search, Client • under development ... 5
- 6. PGXN 6
- 7. Only three steps to install an extension $ pgxn install tinyint $ psql # CREATE EXTENSION tinyint; --That’s all, yay! Coming soon 7
- 8. How to work with groonga 8
- 9. Extensibility of PostgreSQL dblink, adminpack, pgcrypto, ... Parser cube, ltree, citext, hstore, json, … Functions Data Types PL pl/R, pl/v8js, pl/Ruby, pl/Lua, … Optimizer pgadviser Executor Table auto_explain Index GiST/GIN, textsearch_senna, textsearch_groonga SQL/MED (9.1!) c.f. external tools: pgpool, slony, pg_bulkload… ©2011 FORCIA, Inc. http://www.slideshare.net/HitoshiHarada/extensions-on-postgresql 9
- 10. Extensibility of PostgreSQL dblink, adminpack, pgcrypto, ... Parser cube, ltree, citext, hstore, json, … Functions Data Types PL pl/R, pl/v8js, pl/Ruby, pl/Lua, … Optimizer pgadviser Executor auto_explain Table As Index Index GiST/GIN, textsearch_senna, textsearch_groonga SQL/MED (9.1!) c.f. external tools: pgpool, slony, pg_bulkload… ©2011 FORCIA, Inc. http://www.slideshare.net/HitoshiHarada/extensions-on-postgresql 9
- 11. Extensibility of PostgreSQL dblink, adminpack, pgcrypto, ... Parser cube, ltree, citext, hstore, json, … Functions Data Types PL pl/R, pl/v8js, pl/Ruby, pl/Lua, … Optimizer pgadviser Executor auto_explain Table As Index As FDW Index GiST/GIN, textsearch_senna, textsearch_groonga SQL/MED (9.1!) groonga_fdw c.f. external tools: pgpool, slony, pg_bulkload… ©2011 FORCIA, Inc. http://www.slideshare.net/HitoshiHarada/extensions-on-postgresql 9
- 12. textsearch_groonga 10
- 13. Full text search in PostgreSQL Builtin Extension Tsearch (8.3-) morphological + analysis textsearch_ja Ludia contrib/pg_trgm N-gram textsearch_senna (9.1-) textsearch_groonga https://docs.google.com/present/view?id=dhk2smpg_3h44cnrdv 11
- 14. Motivation textsearch_groonga • textsearch_ja ( ) • textsearch_senna (N-gram) PostgreSQL 8.3 HOT Senna … • textsearch_groonga ©2010 FORCIA, Inc. http://www.slideshare.net/ItagakiTakahiro/textsearch-groonga-v01 12
- 15. Indexing : • CREATE TABLE tbl(id serial, doc text); • COPY tbl(doc) FROM '...'; -- • CREATE INDEX idx ON tbl USING groonga (doc); CTID _key CTID _key (1, 0) foo 1.5 1 bar (1, 1) ar 1 (1, 1) bar 2.0 2 baz (2, 0) az 2 (2, 0) baz 4.5 3 foo (1,0) ba 1, 2 CTID = fo 3 oo 3 PostgreSQL groonga groonga ©2010 FORCIA, Inc. http://www.slideshare.net/ItagakiTakahiro/textsearch-groonga-v01 13
- 16. Querying : • SELECT * FROM tbl WHERE … – • doc = ' '– – <, <=, =, >=, >, <> • – • doc %% ' ' – N-gram • – grn_expr • doc @@ groonga.query(' ', ' ') • ©2010 FORCIA, Inc. http://www.slideshare.net/ItagakiTakahiro/textsearch-groonga-v01 14
- 17. groonga_fdw 15
- 18. SQL/MED • Management of External Data • SQL Standard (SQL:2003) • Consists: • Foreign Table • DATALINK 16
- 19. SQL/MED • Management of External Data • SQL Standard (SQL:2003) • Consists: • Foreign Table Implemented as FDW in Postgres • DATALINK 16
- 20. FDW • Foreign Data Wrapper • External data as Table of PostgreSQL • Available from 9.1 • Only “SELECT” statement is available now • Very simple API interface • Only 6 callbacks are required :) 17
- 21. FDW Internal Parser Planner Executer Storage 18
- 22. FDW Internal Parser Planner Executer Storage External Data FDW 18
- 23. FDW • Example: • Oracle • MySQL • Redis • Amazon S3 • Twitter • and more... (see http://pgxn.org/tag/fdw/ ) 19
- 24. groonga_fdw • Foreign Data Wrapper for groonga • You can query groonga via postgres with usual SQL • Under Development 20
- 25. How to use Install # CREATE EXTENTION groonga_fdw; # CREATE SERVER groonga_server FOREIGN DATA WRAPPER groonga_fdw Connect OPTIONS ( dbpath ‘ ); # CREATE FOREIGN TABLE groonga_fdw_tutorial ( _key text title text ) Table SERVER groonga_server OPTIONS ( ... ); 21
- 26. How to use Query # SELECT * FROM groonga_fdw_tutorial; 22
- 27. Compare • textsearch_groonga • <-> Tritton • groonga_fdw • <-> mroonga 23
- 28. Summary • PostgreSQL is quite extensible • Two ways to work with groonga • textsearch_groonga • groonga_fdw • PostgreSQL and groonga are good friend :) 24
- 29. Thanks! 25