Elastic Stack roadmap deep dive
•
3 j'aime•1,153 vues
- Elastic provides a search and analytics platform called the Elastic Stack that includes the Elastic Stack, Beats data shippers, and Kibana analytics and visualization tools. - The presentation discussed updates to Elastic's products including performance improvements to search, new features for distributed search across data centers, and enhanced security options for authentication and authorization. - Elastic aims to provide customizable and extensible solutions for users to ingest, store, search, analyze and visualize large volumes of data from various sources.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Elastic Stack roadmap deep dive
Similaire à Elastic Stack roadmap deep dive (20)
Plus de Elasticsearch
Plus de Elasticsearch (20)
Dernier
Dernier (20)
Elastic Stack roadmap deep dive
- 1. Shay Banon | Founder & CEO Aaron Katz | CRO Janesh Moorjani | CFO Elastic Overview Elastic Stack Updates and Roadmap Alex Francoeur
- 5. Ingest
- 7. Ingest
- 8. Ingest
- 9. Ingest
- 10. Ingest
- 12. Ingest Node: Enrichment Processor source.ip => is_known_botnet? geo.location => city/region/country
- 13. Adding Data
- 15. • What technology? (eg. Nginx) • What to monitor? (eg. logs, metrics, packets) • Where is it? (eg. paths to logfiles) Integrations Manager
- 18. • Beats config • Ingest node config • Index template • First index • Index alias • Index lifecycle management policy • Snapshot lifecycle management policy • Index patterns • Kibana dashboards • Canvas workpads • Machine learning jobs • Alerts Automatically Setup
- 19. • Beats config • Ingest node config • Index template • First index • Index alias • Index lifecycle management policy • Snapshot lifecycle management policy • Index patterns • Kibana dashboards • Canvas workpads • Machine learning jobs • Alerts Automatically Setup
- 21. • Single config language • Installs required Beats • Upgrades Beats • Upgrades itself Beats Agent
- 22. Fleet • Centralized Config Deployment • Centralized Beats Monitoring • Centralized Upgrade Management
- 23. Data Management
- 24. Frozen Indices
- 25. Heap File system cache Disk
- 26. Heap File system cache Disk
- 28. Hot Nodes 1 2 3 Cold NodesWarm Nodes
- 29. 1 2 3 1 2 3 Hot Nodes Cold NodesWarm Nodes
- 30. 1 2 3 1 2 3 Hot Nodes Cold NodesWarm Nodes
- 31. 231 2 3 Hot Nodes Cold NodesWarm Nodes 1
- 32. 1 Hot Nodes Cold NodesWarm Nodes 1 2 3
- 33. 1 Hot Nodes Cold NodesWarm Nodes 1 2 3
- 34. Hot Nodes Cold NodesWarm Nodes 1 2 3
- 35. Hot Nodes Cold NodesWarm Nodes 1 2 3
- 36. (coming soon to X-Pack)
- 38. • Periodic scheduled backups • Retention polices for automatic deletion Snapshot Management
- 41. Data Transforms
- 42. Clickstream Data
- 43. Page views per minute? Clickstream Data
- 44. 99th percentile latency? Clickstream Data
- 45. Most frequent URLs? Clickstream Data
- 46. How long was session 1? Clickstream Data
- 47. How long was session 1? Clickstream Data
- 48. Average session length? Clickstream Data
- 49. Average session length? Session Data
- 50. Average number of pages per session? Session Data
- 51. Most frequent exit page per session? Session Data
- 52. Session Data
- 53. How frequently do users visit the site? Session Data
- 54. How frequently do users visit the site? User Data
- 55. • Pivot • Pattern Matching Data Transformation
- 58. • Outlier detection • Supervised model training for regression & classification • Ingest Prediction Processor Advanced ML Analytics
- 59. Search
- 61. Query Before After Improvement Fuzzy 46 qps Phrase 4 qps Bool AND 9.3 qps Bool OR 3.3 qps Term 33 qps
- 62. Query Before After Improvement Fuzzy 46 qps 59 qps 28% Phrase 4 qps 7 qps 87% Bool AND 9.3 qps 23.5 qps 247% Bool OR 3.3 qps 9.8 qps 292% Term 33 qps 1,160 qps 3,700%
- 63. Magic WAND
- 64. "query" : "elasticsearch and lucene" max_score(and) == 1 max_score(lucene) == 5 max_score(elasticsearch) == 3 Weak-AND
- 65. Min top-10 score and (1) elasticsearch (3) lucene (5) <=1 ✓ ✓ ✓ > 1 and <= 4 ✗ ✓ ✓ > 4 and <= 9 ✗ ✗ ✓ > 9 ✗ ✗ ✗ Weak-AND
- 66. Weak-AND
- 67. Weak-AND "aggs": { ... } "track_total_hits": true
- 68. "hits": { "total": 123456789, "hits": [ ... ] } "hits": { "total": { "value": 10000, "relation": "gte" }, "hits": [ ... ] } Weak-AND
- 69. Search as you type
- 70. index_prefixes: qu, qui, quic, quick br, bro, brow, brown fo, fox, foxe, foxes
- 71. index_phrases: the_quick quick_brown brown_fox fox_jumped jumped_over over_the the_lazy lazy_dog
- 73. Advanced Scoring
- 74. rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 75. score = BM25(Text) + PageRank rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 76. score = BM25(Text) + Saturation(PageRank) rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 77. distance_feature: rank_feature: Advanced Scoring • Date • Geopoint • Numeric • Star Ratings • PageRank • Popularity
- 78. script_score: • Custom scoring, including vectors distance_feature: • Date • Geopoint • Numeric rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity
- 79. Work with WAND script_score: • Custom scoring, including vectors distance_feature: rank_feature: Advanced Scoring • Star Ratings • PageRank • Popularity • Date • Geopoint • Numeric
- 80. Result Pinning
- 82. Geoshapes
- 83. • v2.3: 1 dim, for numbers and dates • v5.0: 2 dim, for geopoints • v5.2: 2 dim, for number & date ranges • v6.7: 7 dim, for geoshapes BKD Trees
- 84. BKD Geoshapes
- 85. • Accurate to 1cm, vs 50m • Index is 60% smaller • Indexing 60% faster • Queries 50% faster • Plus BKD GeoPoints 80% faster indexing BKD Geoshapes
- 87. Zen
- 94. New York London Tokyo
- 95. v5.6 v6.7 v7.x Three Major Versions
- 97. New York London Tokyo ldn_sales ldn_sales
- 98. New York London Tokyo tk_salesny_sales
- 99. New York London Tokyo tk_salesny_sales ldn_sales ldn_sales
- 101. Kibana
- 102. Security
- 106. Lens
- 108. New Platform
- 111. SIEM Stack Monitoring Machine Learning Observability
- 112. 112
- 113. Templated Alerts when [CPU] > [90%] then alert [alerts@me.com]
- 114. Chart-based Alerts
- 116. Guides
- 118. News Feed
- 120. Thank you