How NOT to do searches using LIKE
•
12 j'aime•2,843 vues
The document discusses best practices for performing searches using LIKE in SQL and alternatives to LIKE for full-text search. It covers how LIKE with leading wildcards can result in inefficient table scans. Examples are provided of how frameworks like WordPress, Magento, and databases support more optimized full-text search options like full-text indexes, boolean mode search, and similarity functions. The benefits of search technologies like Elasticsearch and Solr over basic LIKE are also summarized.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
Similaire à How NOT to do searches using LIKE
Similaire à How NOT to do searches using LIKE (20)
Plus de Fabio Akita
Plus de Fabio Akita (20)
Dernier
Dernier (20)
How NOT to do searches using LIKE
- 1. Como NÃO fazer pesquisas usando LIKE Fabio Akita @akitaonrails
- 15. Search está em todos os lugares
- 16. SELECT * FROM PRODUCTS WHERE NAME LIKE '%Camisetas%' AND DESCRIPTION LIKE '%Camisetas%' AND NAME NOT LIKE '%Calças%' AND DESCRIPTION NOT LIKE '%Calças%'
- 20. Camisetas INDEX SEEK Rápido Camisetas% INDEX SCAN Quase Rápido %Camisetas% TABLE SCAN Indo pra trás
- 21. Índices não vão te ajudar
- 24. <?php function get_terms($taxonomies, $args = '') { ... if ( !empty($name__like) ) { $name__like = like_escape( $name__like ); $where .= $wpdb->prepare( " AND t.name LIKE %s", '%' . $name__like . '%' ); } if ( ! empty( $description__like ) ) { $description__like = like_escape( $description__like ); $where .= $wpdb->prepare( " AND tt.description LIKE %s", '%' . $description__like . '%' ); } ... if ( ! empty( $search ) ) { $search = like_escape( $search ); $where .= $wpdb->prepare( ' AND ((t.name LIKE %s) OR (t.slug LIKE %s))', '%' . $search . '%', '%' . $search . '%' ); } ... } ?>
- 26. <?php public function getCILike($field, $value, $options = array()) { $quotedField = $this->_getReadAdapter()->quoteIdentifier($field); return new Zend_Db_Expr($quotedField . ' LIKE ' . $this->addLikeEscape($value, $options)); } ?>
- 29. Rankeamento, Relevância Frases, Proximidade, Intervalos
- 30. Rankeamento, Relevância Frases, Proximidade, Intervalos Sinônimos, "Stemmer"
- 31. Rankeamento, Relevância Frases, Proximidade, Intervalos Sinônimos, "Stemmer" “More Like This"
- 32. Rankeamento, Relevância Frases, Proximidade, Intervalos Sinônimos, "Stemmer" “More Like This" “Did you mean …?"
- 33. Rankeamento, Relevância Frases, Proximidade, Intervalos Sinônimos, "Stemmer" “More Like This" “Did you mean …?" Faceting (Terms, Geolocation, etc)
- 41. SELECT * FROM PRODUCTS WHERE MATCH (NAME, DESCRIPTION) AGAINST ('+Camisetas -Calças' IN BOOLEAN MODE)
- 43. <?php public function chooseFulltext($table, $alias, $select) { $field = new Zend_Db_Expr( 'MATCH (' . $alias . '.data_index) AGAINST (:query IN BOOLEAN MODE)'); $select->columns(array('relevance' => $field)); return $field; } ?>
- 44. SELECT * FROM PRODUCTS WHERE CONTAINS( (NAME, DESCRIPTION), 'Camisetas AND NOT Calças')
- 45. SELECT * FROM PRODUCTS WHERE TO_TSVECTOR(NAME || '' || DESCRIPTION) @@ TO_TSQUERY('Camisetas &! Calças')
- 50. Cadeias de Markov Índices Invertidos
- 51. Cadeias de Markov Índices Invertidos Vector Space Model
- 52. Cadeias de Markov Índices Invertidos Vector Space Model Okapi BM25
- 54. Vector Space Model http://u.akita.ws/vsm_example (Exemplo Simplificado)
- 55. d1 “new york times" d2 “new york post" d3 “los angeles times"
- 59. angeles los q 0 0 new (2/2)*0.584= 0.584 post times york 0 (1/2)*0.584= 0.292 0 q = “new new times"
- 60. Distância d1 sqrt(0.584^2+0.584^2+0.584^2) 1.011 Distância d2 sqrt(0.584^2+1.584^2+0.584^2) 1.786 Distância d3 sqrt(1.584^2+1.584^2+0.584^2) 2.316 Distância q sqrt(0.584^2+0.292^2) 0.652
- 61. (0*0+0*0+0.584*0.584+0*0+0.584*0.292+0.584*0) / cosSim(d1,q) (1.011*0.652) 0.776 (0*0+0*0+0.584*0.584+1.584*0+0*0.292+0.584*0) / cosSim(d2,q) (1.786*0.652) 0.292 (1.584*0+1.584*0+0*0.584+0*0+0.584*0.292+0*0) / (2.316*0.652) 0.112 cosSim(d3,q)
- 63. 150GB/hora 20%-30% tamanho do índice Apache Lucene
- 66. HTML, XHTML, OOXML, ODF, XML, RSS, OLE2, iWorks (Pages, Numbers, Keynote), PDF, EPUB, RTF, Commons Compress (ar, cpio, Unix dump, tar, zip, gzip, XZ, Pack200, bzip2, 7z, arj e lzma), Audio (javax.sound, MIDI, Mp3), Image (javax.imageio, Tiff, Jpeg), Video (FLV, Flash), Mail (Mbox, RFC822), DWG, Font (TrueType), HDF, e plugins.
- 67. InputStream is = new BufferedInputStream( new FileInputStream( new File("sample.pdf"))); ! Parser parser = new AutoDetectParser(); ContentHandler handler = new BodyContentHandler( System.out); ! Metadata metadata = new Metadata(); ! parser.parse(is, handler, metadata, new ParseContext()); ! for (String name : metadata.names()) { String value = metadata.get(name); ! if (value != null) { System.out.println("Metadata Name: " + name); System.out.println("Metadata Value: " + value); } }
- 74. curl "http://localhost:8983/solr/update/extract? literal.id=doc5&defaultField=text” --data-binary @tutorial.html -H 'Content-type:text/html'
- 92. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim
- 93. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim Schema +/- Sim
- 94. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim Schema +/- Sim Nested Typing Não Sim
- 95. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim Schema +/- Sim Nested Typing Não Sim Queries Key / Value JSON
- 96. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim Schema +/- Sim Nested Typing Não Sim Queries Key / Value JSON Distributed Group By Sim Não
- 97. Solr ElasticSearch Coordination ZooKeeper Zen Discovery Shard Splitting Sim Não Automatic Shard Rebalancing Não Sim Schema +/- Sim Nested Typing Não Sim Queries Key / Value JSON Distributed Group By Sim Não Percolation Queries Não Sim
- 105. Setup cd ~ sudo apt-get update sudo apt-get install openjdk-7-jre-headless -y ### http://www.elasticsearch.org/download/ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/ elasticsearch-0.90.7.deb sudo dpkg -i elasticsearch-0.90.7.deb sudo service elasticsearch start
- 106. Setup # Bonsai heroku addons:add bonsai heroku config:add ELASTICSEARCH_URL=`heroku config:get BONSAI_URL` ! # Found heroku addons:add foundelasticsearch heroku config:add ELASTICSEARCH_URL=`heroku config:get FOUNDELASTICSEARCH_URL` ! # SearchBox heroku addons:add searchbox:starter heroku config:add ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL` ! # reindex heroku run rake searchkick:reindex CLASS=Product
- 107. Setup # Gemfile - bundle install gem "searchkick" ! # app/models/product.rb class Product < ActiveRecord::Base searchkick end ! # config/initializers/elasticsearch.rb ENV["ELASTICSEARCH_URL"] = "http://username:password@api.searchbox.io" ! # no shell rails r "Product.reindex"
- 108. # Search simples products = Product.search "Camisetas" products.each do |product| puts product.name end
- 109. # Search simples products = Product.search "Camisetas" products.each do |product| puts product.name end # Search com campos Product.search "Camisetas", fields: [:name, :description] where: { in_stock: true, expires_at: {gt: 1.week.from_now}, or: [ [{in_stock: true}, {backordered: true}] ] }, order: {_score: :desc}, # relevant first limit: 10, offset: 50 # , page: params[:page], per_page: 20
- 110. # Sinonimos class Product < ActiveRecord::Base searchkick synonyms: [ ["pc", "computador pessoal"], ["word", "microsoft office"] ] end
- 111. # Sinonimos class Product < ActiveRecord::Base searchkick synonyms: [ ["pc", "computador pessoal"], ["word", "microsoft office"] ] end # Sugestões class Product < ActiveRecord::Base searchkick suggest: ["name"] end ! products = Product.search "cold miner ", suggest: true products.suggestions # ["codeminer"]
- 113. class City < ActiveRecord::Base searchkick autocomplete: ["name"] end ! City.search "Sao P", autocomplete: true
- 114. # app/controllers/cities_controller.rb class CitiesController < ApplicationController def autocomplete render json: City.search(params[:query], autocomplete: true, limit: 10).map(&:name) end end
- 115. # app/controllers/cities_controller.rb class CitiesController < ApplicationController def autocomplete render json: City.search(params[:query], autocomplete: true, limit: 10).map(&:name) end end # partial <input type="text" id="query" name="query" /> ! <script src="jquery.js"></script> <script src="typeahead.js"></script> <script> $("#query").typeahead({ name: "city", remote: "/cities/autocomplete?query=%QUERY" }); </script>
- 117. products = Product.search "GPS", facets: [:type, :brand, :screen_size] puts products.facets
- 119. class City < ActiveRecord::Base searchkick locations: ["location"] ! def search_data attributes.merge location: [latitude, longitude] end end ! City.search "Codemi", where: { location: {near: [-23, -46], within: "10mi" } } # ou 16km
- 120. Próximos Capítulos
- 123. SELECT … LIKE ‘%'
- 124. SELECT … LIKE ‘%'