Apache Solr lessons learned
•
3 j'aime•2,862 vues
Lessons learned while working with (a customized version of) Apache Solr for 3 years
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
Similaire à Apache Solr lessons learned
Similaire à Apache Solr lessons learned (20)
Plus de Jeroen Rosenberg
Dernier
Dernier (20)
Apache Solr lessons learned
- 3. Lucene xml/json api + field types + caching + faceting + grouping +
- 4. Indexing
- 5. Indexing
- 7. Efficient when many docs share the same value
- 8. Field types
- 9. <field name="id" type="string" indexed="true" stored="true" required=" true" multiValued="false"/> <field name="name" type="string" indexed="false" stored="true" required="true" multiValued="false"/> Field type definition
- 10. <field name="id" type="string" indexed="true" stored="true" required=" true" multiValued="false"/> <field name="name" type="string" indexed="false" stored="true" required="true" multiValued="false"/> Field type definition
- 11. ... <fieldtype name="pdate" class="solr.DateField" sortMissingLast="true"/> ... <field name="date" type="pdate" indexed="false" stored="true"/> <field name="range_date" type="pdate" indexed="true" stored="false"/> <copyField source="date" dest="range_date"/> Field type definition
- 12. ... <fieldtype name="pdate" class="solr.DateField" sortMissingLast="true"/> ... <field name="date" type="pdate" indexed="false" stored="true"/> <field name="range_date" type="pdate" indexed="true" stored="false"/> <copyField source="date" dest="range_date"/> Field type definition
- 13. <dynamicField name="*_s" type="string" indexed="true" stored="true"/> Schemaless
- 14. Segments
- 15. Tune the merge factor
- 16. Max. # of segments Faster search, but slower indexing Faster indexing, but slower search
- 19. Sharding
- 21. foo foo foo core1 core2 core3 Index distributor replication
- 24. <requestHandler name="distributedSearch" class="solr.SearchHandler" default="false"> <lst name="defaults"> <int name="rows">10</int> <str name="fl">*</str> <bool name="partialResults">true</bool> <str name="shards">solr2:7070/solr/foo,solr3:7070/solr/foo</str> </lst> </requestHandler> Distributed search config
- 25. <requestHandler name="distributedSearch" class="solr.SearchHandler" default="false"> <lst name="defaults"> <int name="rows">10</int> <str name="fl">*</str> <bool name="partialResults">true</bool> <str name="shards">solr2:7070/solr/foo,solr3:7070/solr/foo</str> </lst> </requestHandler> Distributed search config
- 26. <requestHandler name="distributedSearch" class="solr.SearchHandler" default false"> <lst name="defaults"> <int name="rows">10</int> <str name="fl">*</str> <bool name="partialResults">true</bool> <str name="shards">solr2:7070/solr/foo,solr3:7070/solr/foo</str> </lst> </requestHandler> Distributed search config
- 28. Caching
- 30. Document Field value Query result FilterDoc ids of results per filter query
- 31. Query result Document Filter Field value Field names (facets) mapped to mapping of doc ids to terms
- 32. Field value Filter Document Query result Ordered set of doc ids of top N results
- 33. Field value Filter Query result Document Stored fields for each doc
- 34. Autowarming
- 36. q=*:*&fq=country:AN&fq=duration:[1 TO *]& fq=date:[NOW TO 2013-07-01T00:00:00Z] Filter queries...
- 37. q=*:*&fq=country:AN&fq=duration:[1 TO *]& fq=date:[NOW TO 2013-07-01T00:00:00Z] Match all documents q=*:*
- 38. q=*:*&fq=country:AN&fq=duration:[1 TO *]& fq=date:[NOW TO 2013-07-01T00:00:00Z] Filter by field value fq=country:AN
- 39. q=*:*&fq=country:AN&fq=duration:[1 TO *]& fq=date:[NOW TO 2013-07-01T00:00:00Z] Range query with wildcard fq=duration:[1 TO *] range query using DateMath syntax fq=date:[NOW TO 2013-07-01T00:00:00Z]
- 41. Faceting
- 45. rows=0&facet=true&facet.field=departureairport& facet.field=touroperator&facet.limit=-1& facet.mincount=1&f.touroperator.facet.limit=2 facet.field=departureairport Specify a field name ...and another one facet.field=touroperator
- 46. rows=0&facet=true&facet.field=departureairport& facet.field=touroperator&facet.limit=-1& facet.mincount=1&f.touroperator.facet.limit=2 Unlimited field values (globally) facet.limit=-1
- 47. rows=0&facet=true&facet.field=departureairport& facet.field=touroperator&facet.limit=-1& facet.mincount=1&f.touroperator.facet.limit=2 Unlimited field values (globally) facet.limit=-1 Basically, always a good idea
- 48. rows=0&facet=true&facet.field=departureairport& facet.field=touroperator&facet.limit=-1& facet.mincount=1&f.touroperator.facet.limit=2 Override global limit for specific field names f.touroperator.facet.limit=2
- 49. rows=0&facet=true&facet.field=departureairport& facet.field=touroperator&facet.limit=-1& facet.mincount=1&f.touroperator.facet.limit=2 At least 1 document per field value facet.mincount=1
- 51. q=*:*&fq={!tag=country}country:AN&facet=true& facet.field={!ex=country}country&facet.limit=-1& facet.mincount=1 fq={!tag=country}country:AN Tag a filter query... ...and exclude it for a field value facet.field={!ex=country}country
- 52. FACET ALL THE THINGS! FACET ALL THE THINGS!
- 53. Grouping
- 54. group=true&group.field=accoid& group.sort=price asc&sort=popularity asc& group.facets=UNGROUPED A grouping query...
- 55. group=true&group.field=accoid& group.sort=price asc&sort=popularity asc& group.facets=UNGROUPED Enable grouping group=true
- 56. group=true&group.field=accoid& group.sort=price asc&sort=popularity asc& group.facets=UNGROUPED Specify the field name group.field=accoid
- 57. group=true&group.field=accoid& group.sort=price asc&sort=popularity asc& group.facets=UNGROUPED Determines group head group.sort=price asc Determine order of document results sort=popularity asc
- 58. group=true&group.field=accoid& group.sort=price asc&sort=popularity asc& group.facets=UNGROUPED Determines group head group.sort=price asc Determine order of document results sort=popularity asc Only group heads are returned!
- 59. ONE DOES NOT SIMPLY EXPLAIN SOLR QUERIES ONE DOES NOT SIMPLY EXPLAIN SOLR QUERIES
- 60. debugQuery=true
- 62. Queries?