Log insight 3.3 customer presentation
•Download as PPTX, PDF•
1 like•1,194 views
Log insight 3.3 customer presentation
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Log insight 3.3 customer presentation
Similar to Log insight 3.3 customer presentation (20)
More from David Pasek
More from David Pasek (20)
Recently uploaded
Recently uploaded (20)
Log insight 3.3 customer presentation
- 1. © 2014 VMware Inc. All rights reserved. VMware® vRealize™ Log Insight™ Version 3.3 Overview 9/13/2016 David Pasek VMware TAM
- 2. Agenda 1 The Big Picture 2 Log Insight Overview 3 Technical Overview 4 Integration with vRealize Operations 5 Use Cases and Resources 2
- 3. Automation Service Catalog Governance Release Automation Operations Service Health Capacity Optimization Configuration Standards IT Business Cost Transparency Benchmarking Service Quality VMware Cloud Management 3 Compute Physical Hardware Private Clouds Public Clouds Hybrid Cloud VMware & vCloud Datacenter Partners Virtualized Infrastructure Abstract & Pool Compute Abstraction = Server Virtualization Storage Storage Abstraction = Software-Defined Storage Network Network Abstraction = Virtual Networking Applications Modern SaaSTraditional The Control Plane for the Software-Defined Data Center and the Hybrid Cloud
- 4. VMware Cloud Management Leadership All reports 2014 except 451 Research (Nov 2013) 1st in four of six Cloud Management and Automation categories: • Self-service Catalog • Unified Cloud Management Console • Cloud Governance • Metering and Billing - 451 Research 1st in Cloud Systems Management Software “Champion” in CMP Vendor Landscape - Info-Tech “... VMware is in a position to garner a significant portion of the CMP market to support enterprise hybrid clouds…” Market Guide for Cloud Management Platforms From Large Software and Emerging Vendors - Gartner 1st in Data Center Automation Software IDC 2013 Vendor Shares Reports
- 6. Powerful & Scalable Log Management 6 Solve problems faster, from days to hours vRealize™ Log Insight™ Find problems you didn’t know you had Get actionable insight into what logs mean Integrate log data with performance analytics
- 7. vRealize Log Insight Overview Intelligent Operations •Enterprise Scale •Predictive Analytics/Machine Learning for faster problem resolution Built for the Software Defined Data Center •Base version now included with vCenter •Insight into VMware products incl. NSX, vRealize Automation, Horizon View •Attractive pricing model for customers of all sizes – not based on log volume Unified Management • Integration with vRealize Operations Management Suite Inventory integration, 2-way alert visualization Extensible • Over 40 Third Party Content Packs Available The best real-time big data log management for SDDC Operating system vSphere System statistics Applications Security Other IT All Kinds of Logs Log Insight 2.0 Analyze Discover Search Visualize IT Operations Security Compliance 40B events 10 event types …by machine learning OverviewApp App
- 8. vRealize Log Insight for vCenter vRealize Log Insight Available with vCenter Server Standard Intelligent Log Analytics for vCenter • Free 25-OSI Log Insight pack per vCenter Benefits of vRealize Log Insight for vCenter • Powerful big data log management built for vSphere • Includes all VMware Content Packs • Extensive Log Management – Captures log data from physical servers, network and storage devices, OSs, applications, and more • Intuitive on-the-fly keyword filtering and custom dashboards • Integration with vRealize Operations – Inventory integration, 2-way Alert Visualization NEW The best real-time management for SDDC Security Ap p Operating System vSphere System Statistics Applications Other IT App Logs Upgrade to Full vRealize Log Insight for: • Extensibility – 3rd Party and Custom Content Packs such as Microsoft, Cisco, EMC, NetApp (29+ available) • Scalability – Cluster Support and Event Forwarding • High availability, Archiving and SSL
- 9. VMware vRealize Log Insight Extensibility Highly Extensible • Captures log data from physical servers, network and storage devices, OSs, applications, VMs, and hosts, and more Log Insight Content Packs • Encapsulate, pre-built dashboards and product-specific alerts from vRealize Log Insight • Provide vendor specific guidance and insight into which logs really matter Log Insight Marketplace • Built into the UI or available at www.solutionexchange.vmware.com Content Packs Overview • Operating System • Application • Network • Storage • SDDC • Security
- 10. Log Insight UI - Interactive Log Analytics Interactive Visualization of Query Results, Plus Easy Drop-Down Menu Options
- 11. Primary Use Cases Troubleshooting and Root Cause Analysis • Follow the trail from vRealize Operations Manager to logs to get to root cause to an observed problem • Identify the needle in the haystack in real time when troubleshooting a problem Monitoring •Monitor metrics and events (performance & change) that are visible only in logs •Identify problems proactively, ensure SLAs and comply to IT policies Unstructured Data Warehouse • Collect all the data in one place without the need for custom parsing, transformation of data • Get full visibility across all your IT environment from a single place 11
- 13. Log Insight Technical Overview Cloud / Data Center Log Management OS Logs VC Logs App Logs System Stats Security Logs API Syslog Analyze • Can analyze any unstructured time-series data, configuration etc. • Automatically identifies structures in the data Scale • Central, scale-out store (no-SQL) for all collected logs • Configurable retention and archiving • Maintenance free Best for SDDC • Queries, alerts, fields, charts in the vSphere Content Pack
- 14. Intelligent Operations Predictive Analytics • Machine Learning based Automatic Data Consolidation • Intelligent data summarization • Cluster similar messages together • Automatic Schema extraction • Automatically understand message structure • Intelligent automatic field extraction Technical Overview
- 15. Intelligent Operations Better Integration and Visualizations • Powerful Content Pack authoring capabilities • Dashboard wide filters • Link dashboard widgets • Visualizations (tables and chart types) for Interactive Analytics and Content Packs Technical Overview
- 16. It’s like ‘Rosetta Stone’ for logs Log Insight proactively learns: from: Then you can query it like a database!
- 18. Better Together: vRealize Operations and vRealize Log Insight Leverage all your IT data for comprehensive visibility in one place Structured Data Metrics Alerts Events VMware vRealize Operations Capacity, Performance and Configuration Management Events Launch in Context Unstructured Data Logs Messages VMware vRealize Log Insight Log analytics, aggregation, and search Public Cloud
- 19. Use Cases & Resources
- 20. Industry Headquarters Employees “vRealize Operations shows us what’s happening in our environment, precisely as it’s happening. This technology is able to marshal huge amounts of data to inform real-time metrics, all presented in a single pane of glass.” — Chris Nakagaki, Virtualization Architect, AutoTrader.com • VMware vRealize Operations Manager • VMware vRealize Log Insight Objectives • Managing far-flung data centers from a central location • Getting a clear, comprehensive view into key performance metrics • Correlate log events with performance metrics for proactive alerting VMware Solution With VMware vRealize Operations and Log Insight, AutoTrader.com can proactively manage system performance, automate the delivery of infrastructure and application services, and achieve a whole new level of business insight. Business Impact • Achieved comprehensive visibility into real-time performance • Identify and resolve issues before any system impact • Minimized system downtime with improved accuracy in capacity planning PROFILE E-commerce Atlanta, Georgia 3,300 VMWARE PRODUCTS & SERVICES AMERICAS
- 21. VMware IT - OneCloud Content Packs In Use • 18 node cluster • 500GB per day of logs.* • 51,459 Events Per Second • 42,607 Alert Queries run since last restart • 11,543,166,777 Total events ingested • 16 TB live storage • 15 TB offline archival storage *We are planning on doubling our ingestion rate over the next month to around 1 TB per day. *Numbers accurate as of Feb, 5 2016 On
- 22. Resources • Competitive inquiries – competition@vmware.com • Vault Site: – https://vault.vmware.com/group/vault-main-library/id- 2466389 • Log Insight Information email Alias: – LI-Info@vmware.com 22
- 23. Thank You!
- 24. Summary Log Insight 3.x Log Insight. Next vRealize Log Insight for Large Environments • Faster - Each node can ingest three times more data - up to 15,000 events per second. • Bigger - The number of nodes that can be included in a Log Insight cluster is doubled, from 6 to 12. • Better - A cluster of twelve nodes can process an astounding 3.8 TB of data per day. Improved Analytics Engine • Multi-Function Charts - Compare different aggregation functions within the same chart, such as MIN, MAX, and AVG. • Snapshots - Visualize your log browsing history and quickly create new dashboards based on your recent snapshots. • Event Types Highlighter – Quickly identify important events. • Event Trends Baselines – Set custom time periods to compare trends in event types. • • URL Shortener - Share shortened URLs with your colleagues to the Interactive Analytics page.
- 25. What’s New in vRealize Log Insight 3.3? New Intelligent Log Analytics Capabilities Key Features / Solution: • Simple Query API for easy integration to existing processes • Web Hooks support for 3rd party app integration (i.e., Slack) • Support for pure IPv6 environments The best real-time management for SDDC New: Free 25-OSIs of limited feature Log Insight included with vCenter Server Standard Customer Scenario Problem • Limited query flexibility • Limited alert extensibility • Pure IPv6 environment unable to benefit from log analysis within vRealize Log Insight Security A p pOperating System vSphere System Statistics Applications Other IT A p p Logs
Editor's Notes
- Log Management is just a small part of effectively managing your SDDC. Combining it with performance and capacity management, superior IT automation and cost visibility are key to VMware's management strategy.
- “Worldwide Cloud Systems Management Software 2013 Vendor Shares”, Mary Johnston Turner, IDC #249131, June 2014. “Worldwide Datacenter Automation Software 2013 Vendor Shares”, Mary Johnston Turner, IDC #248783, May 2014. “Market Guide for Cloud Management Platforms From Large Software and Emerging Vendors”, Ronni J. Colville, Donna Scott, Milind Govekar, Gartner G00247836, April 2014. “Cloud Management and Automation: Delivering the Enterprise Cloud Console”, 451 Research: Cloudscape, November 2013. “Vendor Landscape: Cloud Management Platforms”, Info-Tech Research Group, March 2014.
- The Basics Centralized log management for your entire stack Search & analyze log data for real-time troubleshooting Anyone in the organization can access log data without compromising production systems Best for VMware by VMware Optimized for vSphere logs, vSphere analytics are built in Automatic ESXi configuration for collection Integration with vRealize Operations Very easy to deploy, intuitive to use Simple & predictable pricing model Log Insight can digest any type of log data. Users do not need to think about it, they can just send their data to Log Insight. Automatically identifies structures in the data and create a high performance index for performing analytics Unlike databases there is no need to engage DB admins to Extract Transform and Load (ETL) the data…Just send it over It scales up and down very well. It can ingest TBs of data per node per day. It is much faster and much more efficient than the competition It has a configurable retention policy, i.e. you attach 500GB storage and if it runs out of space it rotates the old data out. Optionally they can be written to an archive, making it virtually maintenance free It ships with out of the box knowledge of vSphere and many other logs in the form of a Content Pack, a collection of queries, alerts, dashboards and fields. A vast library of VMware and 3rd party Content Packs are also available.
- VMware offers a comprehensive operations management solution for the cloud era. vRealize Operations Management Suite is a highly automated and integrated analytics platform and operations console that provides visibility into the health of your infrastructure and applications across hybrid clouds and heterogeneous environments. There are three key areas of focus for VMware’s approach to simplify and automate Operations Management. The first is Intelligent Operations, using patented analytics to provide better visibility into datacenter operations. vRealize Operations analyzes millions of metrics from vSphere and existing monitoring tools to learn the behavior of your infrastructure. It then sets dynamic thresholds that trigger smart alerts so you can proactively address building performance problems. The second area is Policy-based Automation, leveraging policies and thresholds to trigger orchestration workflows across a white variety of tasks, rather than manual intervention to kick off a script. These automated tasks include incident and problem remediation, policy enforcement for continuous compliance, and capacity analysis and planning to improve resource utilization. The third area is Unified Management, providing operations team with a unified view of what's happening in their highly virtualized and cloud environments. vRealize Operations delivers this unified view in three ways: first, through converged infrastructure management of network, storage and compute; second, through integration of the key disciplines of performance, capacity and configuration management; and third, through a consistent management approach across virtual, physical and private/public cloud domains. As shown here, customers are reaping the benefits of this approach.
- vRealize Log Insight for vCenter is a version of Log Insight that is limited to VMware Content Packs only (listed below). Customers that own vCenter Server get free 25-OSI pack for each vCenter license they own/purchase. This allows them to collect logs from vCenter, ESX/I and any other supported sources (up to 25 total OSI) from their vSphere environment Example vCenter logs: vpxd, profiler, alert, performance, agent, inventory, dump, endpoint, syslog, SSO, Web Client, etc. An OSI is essentially a Log stream Example scenario: Customer owns 2 vCenter Licenses and manage 10 hosts, 200 VMs Customer has 5 ESXi hosts per vCenter OSI License breakdown: **They’re existing vCenter license will be used to activate Log Insight for vCenter – Any additional OSI license needs will require a new LI key, which will include the original 25 plus whatever else they purchase. They’re entitled to 2 25-pack OSI licenses, in order to use both they must deploy 2 LI instances (1 25-pack license per LI) 1 can be used per vCenter (2 used) 1 can be used per ESXi host (10 used) 38 remain for VMs, network devices, other VMware management components like vRealize Automation, NSX, storage devices, etc. Customers can purchase additional licenses by 25 OSI packs or by CPU Available VMware Content Packs 01/2016: - Virtual SAN - vRealize Automation 6.1+ - vCenter Operations Manager - vRealize Operations Manager - vCloud Director - NSX - vCNS - Horizon View - vSphere - vCloud Automation Center (vCAC)
- There are roughly 40 content packs available for additional VMware and 3rd party solutions. Each come with their own set of built-in queries, dashboards and alerts specific to that solution and are extremely simple to add.
- On the fly dashboard creation based on search criteria. Create custom dashboards that align with your needs and saved for re-use .
- It can digest any type of log data. The users don’t need to think about it, they can just send their data to Log Insight. Log Insight automatically identified structures in the data and create a high performance index for performing analytics Unlike databases there is no need to engage db admins to Extract Transform and Load (ETL) the data. Just send it over It scales up and down very well. It can ingest TBs of data per node per day. It is much faster and much more efficient than the competition It has a configurable retention policy, i.e. you attach 500GB storage and if it runs out of space it rotates the old data out. Optionally they can be written to an archive, making it virtually maintenance free It ships with out of the box knowledge of vSphere logs in the form of a Content Pack, a collection of queries, alerts, dashboards and fields. We are also working with partners to include more Content Packs (e.g. for storage, applications, network devices) as well as more Vmware producst (Nicira/Networking, View, etc.)
- Proactive Analytics (Machine Learning) Automatic log consolidation - groups similar messages together with no runtime overhead (reverse engineer where each log message comes from in the code) Example: 1000 messages match a query but reports back that there are only 5 message types (in the 1000 matches) Schema discovery (Automatically understand the message structure) Automatic field extraction - discover fields in logs including their data types
- Easy and Flexible Easy to create Content Packs Dashboard widget without links Simple to create a limited set of Visualizations and Dashboards Content Packs Authoring Improvements Dynamic Filtering in Content Packs Enables Content Pack authors to provide templates (e.g. filter by ticket id) Widget Linking From a dashboard widget to a related dashboard page for guided troubleshooting Automatically discovers possible links
- Log Insight effectively translates cryptic and unstructured log details into meaningful, searchable results.
- Integration with vRealize Operations Manager means we’ve provided 2-way launch in context from each UI. For example, if you see an alert in vR Ops and it’s coming from a log entry you have the ability to launch the Log Insight UI and it will automatically render the specific log entries in that UI for further investigation.
- This customer profile is also available in a video - http://www.vmware.com/products/vrealize-operations-insight/#3886219553001
- VMware IT uses Log Insight. These are some metrics that speak to the scale of the solution and how impossible it would be to manage this type of log volume without automation.
- Detect transactions: The ability to automatically identify a series of events that lead to a failure. Detecting anomalous sources and events: Ability to find hosts that send offending messages. Creating fingerprints: Giving the ability to users to define a set of events that lead to failures. They would be able to share this ‘fingerprint’ with other groups to detect such a pattern in other systems.
- vRealize Log Insight, also part of vRealize Suite STD/ADV/ENT, introduced several new features in 3.3. Some of the features are backend and some operational, meaning some features like license support will be invisible to users, while some of the agent features and upgrade capabilities will drastically benefit customers. Federal and other IPv6 customers will benefit from the added support for pure IPv6. Usability and UI improvements will help new and existing Log Insight customers alike. The following is a list of features *projected* to ship with v3.3 – some may have slipped so please double check the product documentation: Additional agent-side parsers for LTSV and Syslog (potentially even JSON and RegEx) provide the ability to further parse events on the client side, which allows us to only send in pre-indexed content and provides greater efficiency. Web Hooks – Ability to send alerts to Socialcast, Slack or any other IM or app that supports web hooks, providing additional alerting extensibility Simple Query API. Support for simple keyword search, filters – “I.e., all events that contain host name SrvEast*”, regular expressions, math equations, grouping, search by, etc. Allows customers to run complex queries, integrate with CMDBs, perform their own UI analysis, etc. Support for pure IPV6 environment – both server and agent – pure ipv6 only. Especially benefits Federal customers. (This could be considered ‘Tech Preview’ initially.) Agent configuration builder – we can now build an Agent configuration using a UI with error handling and corrective syntax examples, etc. UI Improvements: Screen real estate feature – allows you to hide chart visualizations at the top of the Interactive Analytics screen Added a hover over capability to expand dashboard legend details Added Tables field to Events, which gives a different perspective of data – example: Failed logins can now be tallied and viewed per host Multi VIP (Virtual IP) and Tag – Integrated Load Balancer – Supports single VIP per cluster. Users can now define more than 1 VIP for each cluster and can tag these VIPs and add meta-data (I.e., Datacenter = Boston) Support for ‘CopyTruncate’ xNIX CMD – Used to manage agent log files on the agent side after they are sent to LI server Hybrid Licenses support for inclusion in vCenter 25 OSI vSphere Content Pack update – mainly bug fixes