Mastering Solr will give an introduction to the more advanced use of Solr. It offers a starting point to go beyond the possibilies offered by the Solr integration modules. The following subjects will be covered: - Getting Solr running locally - Integration modules: Search API vs ApacheSolr - Configuring an index - Search relevancy: basic boosting of search results. - The anatomy of a request to Solr - Hooks for altering/adding parameters to the request to Solr - Search relevancy: advanced date boosting by boost functions - Search relevancy: advanced boosting by boost queries

1. Mastering solr
Jur de Vries

2. Who am I?
Developer/architect at Triquanta
Trainer at Wizzlern

3. Use case
Market place
Advertisements
Adjust relevancy
Paid boosting of add's
Of course we use Drupal and Apache Solr

4. Running Solr Locally
Download latest version (3.6)
Be sure to download distribution (not src)
Unpack solr
Go to example directory
Run
java -jar start.jar

5. Drupal: which contrib?
2 Possibilities
Apachesolr search
Search api with solr backend

6. Apache solr search
Streghts:
Supported by Acquia
Easy to set up
Mature
Weaknesses
Integration with views (still in dev)

7. Search Api
Strengths
Flexible
Indexes all entities
Excellent views integration
Related fields are easy to add to index
Weaknesses
Not supported (yet) by Acquia
Solr backend has some issues

8. Drupal: which contrib?
Apachesolr search integration
Quick setup
Acquia
Search API
Exportable configuaration
Views integration
Index all entities
Depends on your needs

9. Basic use of search api
Create server
Create index
Select fields to index
Define data alterations
Define processors
Start indexing

10. Field types
Integer, date, boolean
String or fulltext?
Fulltext will get processed!
Tokenize
Stopwords
Ignore case
String is as is

11. Demo
Run solr
Copy schema.xml and solrconfig.xml (!)
Create server
Create index
Create view
ads
Ad filter exposed: search

12. Advanced use of Search api
This talk is about Solr, not about search API
Understand Solr first!
Many resources on the web
Watch screencasts etc

13. Mastering Solr
Mastering solr is understanding solr
What happens after a Drupal module?
Let's have a look at the request

14. Solr request
Look at solr log
Parameters:
start
rows
q (query)
qf (query fields)
fl (fields)
fq (filter query)

15. Field names
item_id, id
t_.., ss_.., → why?
Solr has to know how to handle fields
Field api: field names differ
Dynamic field names: tell solr field type!

16. Schema.xml
Defines field types and fields
The real tweaking starts here!
Let's have a look!
dynamicField
field type
analyzers
Copyfield

17. What can you do in schema.xml?
Synonyms (is disabled by default)
Stopwords (and, or, etc)
Stemming
Proper multilingual handling

18. Browse the schema
Solr offers schema browsing
Go to: http://localhost:8983/solr/admin

19. Search relevancy
Types of boosting:
Field level boost
Boost function
Boost query
(QueryElevation)

20. Boost parameters
Field level boosting: qf
qf:t_body^20
score in field is multiplied by 20
Boost function: bf
bf:product(fieldname, 2)
result of function is added to score
Boost query: bq
boost (only for edismax) like bf but multiplication

21. Let's boost title
Field level boost is incorporated in Search API...
But, where are the numbers in the request???
Search api solr forgot to add them!
There is a patch :-)
But lets do it another way...

22. Debugging Solr
Lets add &echoParams=all to the request...
Where do all these parameters come from?
Solrconfig.xml!!!
Among other things: request handler
Let's look at the dismax request handler

23. Solrconfig.xml
(Default) Request handler:
Default parameters
Add Spellcheck
Tweak all kinds of search behavior!
Let's add default search fields with boost

24. Boost function
Mathematical functions on field values
Available functions:
sum(x,y): x + y
product(x,y): x * y
scale(x, minTarget, maxTarget)
recip(x, m, a, b): x / (m * a + b)
ms(): time → ms(NOW/DAY, created)
Many more!

25. Boost date
We need ms(): big values!
Linear? To much difference
Recip!
recip(x,1,1000,1000)
if x 1000: half
1 year: 3.1e10
recip(ms(NOW/YEAR?, created),1,3.1e10,3.1e10)
bf=recip(ms(NOW/YEAR?, created),1,3.1e10,3.1e10)^3
Use a graphing tool!

26. Boost queries
Do a query like fq:
Boost add's:
content_type:add
bq=content_type:add
bq=(content_type)^20

27. Debugging relevancy
We know how to boost
How can finetuning be done?
solr has the solutions:
add debugQuery=on

28. debugQuery=on
normal source

29. Relevancy
Choose your boosting methods
Try in your browser
Finetuning: debugQuery=on, source
Add parameters to solrconfig.xml
Or...

30. Add parameters in code
use
hook_search_api_solr_query_alter(array
&$call_args, SearchApiQueryInterface $query)
$call_args['params']['bq'] = '(t_title:foo)^20'
$call_args['params']['bf'][] = b_promote

31. Override solr service class
In Search API: define server class
extend solr service class
Only change key methods
It's all about passing parameters!

32. Conclusion
Tweak indexing in schema.xml
Stopwords
Multilingual
Tweak searching in solrconfig.xml
Tweak searching by passing variables
This is only an introduction!

33. Questions?

34. Feedback & follow-up:
http://drupalcampgent.be/feedback