How many times has a customer told you they want to search in their application “like Google“? How many times was the search engine in your application too slow? Hibernate Search brings full-text search capabilities to a persistent domain model, providing Google-like search capabilities while avoiding the traditional cost and difficulties to set up such solutions. In this session, you will learn what problems Hibernate Search can solve and you will follow the steps of adding it to a Hibernate based application. You will build your own application specific full-text search engine. We will also explore advance subjects such as clustering and the underlying of phonetic approximation. About the speaker - Emmanuel Bernard Emmanuel is a Lead developer at JBoss, a division of Red Hat. After graduating from Supelec (French “Grande Ecole”), Emmanuel has spent a few years in the retail industry where he started to be involved in the ORM space. He joined the Hibernate team 4 years ago. Emmanuel is the lead developer of Hibernate Annotations and Hibernate EntityManager, two key projects on top of Hibernate core implementing the Java Persistence(tm) specification, as well as Hibernate Search and Validator. Emmanuel is a member of the EJB 3.0 expert group and the spec lead of JSR 303: Bean Validation. He is a regular speaker at various conferences and JUGs, including JavaOne, JBoss World and JavaPolis and the co-author of Hibernate Search in Action from Manning.

1. Hibernate Search
Googling your persistence domain model
Emmanuel Bernard
Doer
JBoss, a division of Red Hat
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

2. Search: left over
of today’s applications
Add search dimension to the domain model
“Frankly, search sucks on this project”
-- Anonymous’ boss
Why? How to fix that? For cheap?
2
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

3. Integrate full-text search and
persistent domain
SQL Search vs Full-text Search
Object model / Full-text Search mismatches
Demo
Hibernate Search architecture
Configuration and Mapping
Full-text based object queries
Some more features
3
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

4. SQL search limits
Wildcard / word search
‘%hibernate%’
Approximation (or synonym)
‘hybernat’
Proximity
‘Java’ close to ‘Persistence’
Relevance or (result scoring)
multi-”column” search
4
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

5. Full Text Search
Search information
by word
inverted indices (word frequency, position)
In RDBMS engines
portability (proprietary add-on on top of SQL)
flexibility
Standalone engine
Apache Lucene(tm) http://lucene.apache.org
5
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

6. Mismatches with a domain model
Structural mismatch Appl
Fwk
full text index are text only
no reference/association between document Persistence
Synchronization mismatch
keeping index and database up to date
Retrieval mismatch
Domain
the index does not store objects Model
Search
certainly not managed objects
6
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

7. Demo
Let’s google our application!
7
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

8. Hibernate Search
Under the Hibernate platform
LGPL
Built on top of Hibernate Core
Use Apache Lucene(tm) under the hood
In top 10 downloaded at Apache JBoss Seam
Very powerful but low level
e
Appl
nc
te
Fwk
is
easy to use it the “wrong” way
rs
Pe
Solve the mismatches Domain
Search
Model
8
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

9. Architecture
Transparent indexing through event system (JPA)
PERSIST / UPDATE / DELETE
Convert the object structure into Index structure
Operation batching per transaction
better Lucene performance
“ACID”-ity
(pluggable scope)
Backend
synchronous / asynchronous mode
Lucene, JMS 9
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

10. Architecture (Backend)
Lucene Directory
standalone or symmetric cluster (limited scalability)
immediate visibility
can aect front end runtime
Hibernate
+
Hibernate Search
Search request
Index update Lucene
Directory
Database
(Index)
Search request
Index update
Hibernate
+
Hibernate Search
10
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

11. Architecture (Backend)
JMS (Cluster)
Search processed locally / Change sent to master
asynchronous indexing (delay)
No front end extra cost / good scalability
Slave
Hibernate Database
Lucene
+ Directory
Search request
Hibernate Search (Index)
Copy
Copy
Index update order
Master
Hibernate Lucene
+ Directory
Index update
Hibernate Search (Index)
Process
JMS
Master
queue
11
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

12. Configuration and Mapping
Configuration
event listener wiring
transparent in Hibernate Annotations
Backend configuration
Mapping: annotation based
@Indexed
@Field(store, index)
@IndexedEmbedded
@FieldBridge
12
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

13. Query
Retrieve objects, not documents
no boilerplate conversion code!
Objects from the Persistence Context
same semantic as a JPA-QL or Criteria query
Use org.hibernate.Query/javax.persistence.Query
common API for all your queries
Query on correlated objects
“JOIN”-like query quot;author.address.city:Atlantaquot;
13
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

14. Query possibilities
bring the “best” document first
recover from typos
recover from faulty orthography
find from words with the same meaning
find words from the same family
find an exact phrase
find similar documents
14
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

15. More query features
Fine grained fetching strategy
Sort by property rather than relevance
Projection (both field and metadata)
metadata: SCORE, ID, BOOST etc
no DB / Persistence Context access
Filters
security / temporal data / category
Total number of results
15
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

16. Some more features
Automatic index optimization
Index sharding
Manual indexing and purging
non event-based system
Shared Lucene resources
Native Lucene access
16
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

17. Full-Text search without the
hassle
Transparent index synchronization
Automatic Structural conversion through Mapping
No paradigm shift when retrieving data
Clustering capability out of the box
Easier / transparent optimized Lucene use
17
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

18. Road Map
Programmatic Mapping API Perf
Multi threaded mass index
Clustering
Query
Dictionary and spellchecker
Easier query building
MoreLikeThis?
Statistics
18
Monday, April 20, 2009

19. QA
19
Copyright Red Hat © 2007-2009
Monday, April 20, 2009

20. For More Information
Hibernate Search
http://search.hibernate.org
Hibernate Search in Action
Apache Lucene
http://lucene.apache.org
Lucene In Action
http://in.relation.to
http://blog.emmanuelbernard.com
20
Copyright Red Hat © 2007-2009
Monday, April 20, 2009