SlideShare une entreprise Scribd logo
1  sur  18
Kyligence Introduction
MicroStrategy Partnership
Saswata Sengupta
© Kyligence Inc. 2019, Confidential.
Apache Kylin
Top Level Apache Project
 The only open-source OLAP on big data
platform
Best Open-source Big Data Tool
 InfoWorld’s Bossies (Best of Open
Source Software Awards) in 2015 &
2016
Sub-Second Interactive Query
 Large scale, high concurrency, sub
second query latency, multi-dimension
1000+ Organizations
 Adopted by thousands of organizations
globally
© Kyligence Inc. 2019, Confidential.
Kyligence = Kylin + Intelligence
• Founded in 2016 by the creators of Apache Kylin
• Built around Kylin, with augmented AI and enhanced to
deliver unprecedented enterprise analytic performance
• CRN Top-10 big data startups in 2018
• Global Presence: San Jose, Seattle, New York, Shanghai,
Beijing
• VCs: Fidelity International, Shunwei Capital, Broadband
Capital, Redpoint, Cisco, Coatue
Accelerate Critical Business Decisions with AI-Augmented Data Management and Analytics
2016
Founded Pre-
A
Redpoint
Cisco
2017
Series A
CBC
SHUNWEI
2018
Series B
8Roads
2019
Series C
Coatue
© Kyligence Inc. 2019, Confidential.
Trusted by Global Fortune 500
BFSI
Telecom
Technology
Manufacturing,
Retail, Etc.
© Kyligence Inc. 2019, Confidential.
Pains in Collaboration
Data Analyst
Data Engineer
• Manage data source
• Design data model to
keep one source of truth
• ETL and load data
• Develop dashboard/reporting
• Self-service analysis to
answer business questions
Low efficiency in development to fulfill business
requirements
Limited dimension and measures in a model to serve
complex calculations
Difficulty if analytics requirements or source
data change
Time to insight is slow
© Kyligence Inc. 2019, Confidential.
Kyligence Ecosystem
Global Partners
• Fully enabled on leading cloud and data
platforms (Azure, AWS, Google Cloud,
Cloudera)
• Integrated with popular BI and
virtualization (Tableau, Power BI, Qlik,
MicroStrategy)
• Certified on main Hadoop distributions
(CDP)
© Kyligence Inc. 2019, Confidential.
Kyligence Enterprise Accelerate Mission-critical Analytics Intelligently
• Unified Query Entrance
ODBC/JDBC API/SDK
Finance Marketing Sales Customer Checkout
Cube Index
10%4% 80%
RDBM
s Hive
SQL/MDX
Semantic Services
6%
Distributed
Query Engine
AI-Augmented
Engine
Smart
Pushdown
Metadata
Management
Enterprise
Security
• Business Semantic Layer
• Query Pattern for all data
• High Performance Engine
© Kyligence Inc. 2019, Confidential.
Kyligence Cloud
FinanceMarketingSales
Index
more…
Landing &
Transformation
Semantic & Augmentation ApplicationsSource
Azure Blob Storage
Azure Synapse
© Kyligence Inc. 2019, Confidential.
AI Augmented Engine: Intelligent Data Development
© Kyligence Inc. 2019, Confidential.
AI Augmented Engine: One-click Acceleration
• Self-maintaining
• Dynamic auto-modeling
• Self-learning engine
• One-click acceleration
• Adaptive model
© Kyligence Inc. 2019, Confidential.
AI-Augmented Engine — Learn From Your Analytics History
© Kyligence Inc. 2019, Confidential.
Advanced Tuning Features – Push Down and Aggregate Index
© Kyligence Inc. 2019, Confidential.
Under the hood : Smart Cuboids
• Each Model consists of N-Dimension Cuboids which is a
combination of several dimension in different permutations and
combinations.
• Apache Spark is used to build the cuboids making query results
extremely fast.
• When the user sends a query the model intelligently looks for
the Cuboids/segment returns the results extremely fast.
© Kyligence Inc. 2019, Confidential.
Unified Semantic Layer
BI Integration Access Control
Enterprise Security
Query Engine Model
Query Platform
Data Sources
Excel MicroStrategy Other BI Tools
Semantic Layer
Cloud DW Parquet ORC
Blob
Storage
CSVSnowflake
• Translate technical details into
business terminology
• Synchronize semantics across major
BI tools
• Unified business definitions
• Flexible business calculations
© Kyligence Inc. 2019, Confidential.
Elastic Scaling — Handle Peak Time Automatically
 Fewer compute and storage resources
utilized
 Dynamic on-demand cluster resizing
 Uses spot instances
 Efficient planning for data growth
© Kyligence Inc. 2019, Confidential.
TPC-H 22 Queries
SF=50
Query Response Time | 0.5 Billion
SF=500
Query Response Time | 5 Billion
• No warm up
• Lower is better
• Run each query 3 times
• Record the average time
For each Dataset:
© Kyligence Inc. 2019, Confidential.
Financial Risk Management - replacing the large SSAS cube
Challenges Kyligence’s Solution
modernization
same data source
same front-end BI
similar OLAP concepts
comparable semantic layer
finer granular access control
Scalability
Performance
Low Cost
• 5TB SSAS cube with 5 Billion rows daily
incremental data
• 14 Lookup tables, half over 20M
cardinalities (largest 200M)
• 600+ dimensions
• 30+ analysis users
• Analysts’ work locked by incremental
loading workload, system crashes
happen frequently
• Poor performance on data loading and
queries (especially on UHC, Count
Distinct, Correlation)
• Limited concurrent users
• Single cube easy management
• Analysts’ work no longer interrupted
• Transparent to business users, same
• analysis tool Excel
• Improved query and loading performance
• Support 1000+ concurrent users
• Meet future requirement - prediction of 40% data
volume growth, migration to cloud, Realtime
THANK YOU

Contenu connexe

Tendances

SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic
 
Importance of global certifications
Importance of global certificationsImportance of global certifications
Importance of global certificationsAnjani Phuyal
 
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...Tyler Wishnoff
 
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...VMware Tanzu
 
Qlik sense- Technical Seminar
Qlik sense- Technical SeminarQlik sense- Technical Seminar
Qlik sense- Technical SeminarSanjana Gondane
 
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...DataStax
 
Webinar: BI in the Sky - The New Rules of Cloud Analytics
Webinar: BI in the Sky - The New Rules of Cloud AnalyticsWebinar: BI in the Sky - The New Rules of Cloud Analytics
Webinar: BI in the Sky - The New Rules of Cloud AnalyticsSnapLogic
 
Mining Information from Data on Cloud
Mining Information from Data on CloudMining Information from Data on Cloud
Mining Information from Data on CloudAmazon Web Services
 
Augmented OLAP for Big Data
Augmented OLAP for Big DataAugmented OLAP for Big Data
Augmented OLAP for Big DataLuke Han
 
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...Alluxio, Inc.
 
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in HealthcareMoving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in HealthcarePerficient, Inc.
 
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...Amazon Web Services
 
Making Better Decisions Using BigData and Analytics
Making Better Decisions Using BigData and AnalyticsMaking Better Decisions Using BigData and Analytics
Making Better Decisions Using BigData and AnalyticsBoaz Ziniman
 
SnapLogic Live: Big Data Integration
SnapLogic Live: Big Data IntegrationSnapLogic Live: Big Data Integration
SnapLogic Live: Big Data IntegrationSnapLogic
 
Altis Webinar: Use Cases For The Modern Data Platform
Altis Webinar: Use Cases For The Modern Data PlatformAltis Webinar: Use Cases For The Modern Data Platform
Altis Webinar: Use Cases For The Modern Data PlatformAltis Consulting
 
Event Sponsor NetApp - CSO- Jon Kissane
Event Sponsor NetApp - CSO- Jon Kissane  Event Sponsor NetApp - CSO- Jon Kissane
Event Sponsor NetApp - CSO- Jon Kissane Hostway|HOSTING
 
Snaplogic Live: Big Data in Motion
Snaplogic Live: Big Data in MotionSnaplogic Live: Big Data in Motion
Snaplogic Live: Big Data in MotionSnapLogic
 
Cloud-Con: Integration & Web APIs
Cloud-Con: Integration & Web APIsCloud-Con: Integration & Web APIs
Cloud-Con: Integration & Web APIsSnapLogic
 

Tendances (20)

SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018SnapLogic Technology Open House – January 2018
SnapLogic Technology Open House – January 2018
 
Importance of global certifications
Importance of global certificationsImportance of global certifications
Importance of global certifications
 
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
AI-Powered Analytics: What It Is and How It’s Powering the Next Generation of...
 
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...
Pivotal Digital Transformation Forum: Requirements to Become a Data-Driven En...
 
Qlik sense- Technical Seminar
Qlik sense- Technical SeminarQlik sense- Technical Seminar
Qlik sense- Technical Seminar
 
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
Aeris + Cassandra: An IOT Solution Helping Automakers Make the Connected Car ...
 
Webinar: BI in the Sky - The New Rules of Cloud Analytics
Webinar: BI in the Sky - The New Rules of Cloud AnalyticsWebinar: BI in the Sky - The New Rules of Cloud Analytics
Webinar: BI in the Sky - The New Rules of Cloud Analytics
 
Mining Information from Data on Cloud
Mining Information from Data on CloudMining Information from Data on Cloud
Mining Information from Data on Cloud
 
Augmented OLAP for Big Data
Augmented OLAP for Big DataAugmented OLAP for Big Data
Augmented OLAP for Big Data
 
AWS Webcast - Tibco Jaspersoft
AWS Webcast - Tibco JaspersoftAWS Webcast - Tibco Jaspersoft
AWS Webcast - Tibco Jaspersoft
 
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
 
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in HealthcareMoving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in Healthcare
 
The API Lie
The API LieThe API Lie
The API Lie
 
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...
How Companies are Using Cloud-Based Data Visualization & Analytics to Transfo...
 
Making Better Decisions Using BigData and Analytics
Making Better Decisions Using BigData and AnalyticsMaking Better Decisions Using BigData and Analytics
Making Better Decisions Using BigData and Analytics
 
SnapLogic Live: Big Data Integration
SnapLogic Live: Big Data IntegrationSnapLogic Live: Big Data Integration
SnapLogic Live: Big Data Integration
 
Altis Webinar: Use Cases For The Modern Data Platform
Altis Webinar: Use Cases For The Modern Data PlatformAltis Webinar: Use Cases For The Modern Data Platform
Altis Webinar: Use Cases For The Modern Data Platform
 
Event Sponsor NetApp - CSO- Jon Kissane
Event Sponsor NetApp - CSO- Jon Kissane  Event Sponsor NetApp - CSO- Jon Kissane
Event Sponsor NetApp - CSO- Jon Kissane
 
Snaplogic Live: Big Data in Motion
Snaplogic Live: Big Data in MotionSnaplogic Live: Big Data in Motion
Snaplogic Live: Big Data in Motion
 
Cloud-Con: Integration & Web APIs
Cloud-Con: Integration & Web APIsCloud-Con: Integration & Web APIs
Cloud-Con: Integration & Web APIs
 

Similaire à Lightning-Fast, Interactive Business Intelligence Performance with MicroStrategy and Kyligence

Architecting Snowflake for High Concurrency and High Performance
Architecting Snowflake for High Concurrency and High PerformanceArchitecting Snowflake for High Concurrency and High Performance
Architecting Snowflake for High Concurrency and High PerformanceSamanthaBerlant
 
How Analytics Teams Using SSAS Can Embrace Big Data and the Cloud
How Analytics Teams Using SSAS Can Embrace Big Data and the CloudHow Analytics Teams Using SSAS Can Embrace Big Data and the Cloud
How Analytics Teams Using SSAS Can Embrace Big Data and the CloudTyler Wishnoff
 
Addressing the systemic shortcomings of cloud analytics
Addressing the systemic shortcomings of cloud analyticsAddressing the systemic shortcomings of cloud analytics
Addressing the systemic shortcomings of cloud analyticsSamanthaBerlant
 
Enhance Data Governance with Kyligence Unified Semantic Layer
Enhance Data Governance with Kyligence Unified Semantic LayerEnhance Data Governance with Kyligence Unified Semantic Layer
Enhance Data Governance with Kyligence Unified Semantic LayerSamanthaBerlant
 
Kyligence Cloud 4 - An Overview
Kyligence Cloud 4 - An OverviewKyligence Cloud 4 - An Overview
Kyligence Cloud 4 - An OverviewSamanthaBerlant
 
ICP for Data- Enterprise platform for AI, ML and Data Science
ICP for Data- Enterprise platform for AI, ML and Data ScienceICP for Data- Enterprise platform for AI, ML and Data Science
ICP for Data- Enterprise platform for AI, ML and Data ScienceKaran Sachdeva
 
Building Enterprise OLAP on Hadoop for FSI
Building Enterprise OLAP on Hadoop for FSIBuilding Enterprise OLAP on Hadoop for FSI
Building Enterprise OLAP on Hadoop for FSILuke Han
 
Augmented OLAP Analytics for Big Data
Augmented OLAP Analytics for Big DataAugmented OLAP Analytics for Big Data
Augmented OLAP Analytics for Big DataTyler Wishnoff
 
Accelerating Innovation with Hybrid Cloud
Accelerating Innovation with Hybrid CloudAccelerating Innovation with Hybrid Cloud
Accelerating Innovation with Hybrid CloudJeff Jakubiak
 
Building a Modern Analytic Database with Cloudera 5.8
Building a Modern Analytic Database with Cloudera 5.8Building a Modern Analytic Database with Cloudera 5.8
Building a Modern Analytic Database with Cloudera 5.8Cloudera, Inc.
 
The Cloud - What's different
The Cloud - What's differentThe Cloud - What's different
The Cloud - What's differentChen-Tien Tsai
 
Apache Kylin and Use Cases - 2018 Big Data Spain
Apache Kylin and Use Cases - 2018 Big Data SpainApache Kylin and Use Cases - 2018 Big Data Spain
Apache Kylin and Use Cases - 2018 Big Data SpainLuke Han
 
Building a hybrid, dynamic cloud on an open architecture
Building a hybrid, dynamic cloud on an open architectureBuilding a hybrid, dynamic cloud on an open architecture
Building a hybrid, dynamic cloud on an open architectureDaniel Krook
 
Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnectaDigital
 
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...Cloudera, Inc.
 
SQL + Hadoop: The High Performance Advantage�
SQL + Hadoop:  The High Performance Advantage�SQL + Hadoop:  The High Performance Advantage�
SQL + Hadoop: The High Performance Advantage�Actian Corporation
 
Move Cloud to the Core of your Business Strategy
Move Cloud to the Core of your Business StrategyMove Cloud to the Core of your Business Strategy
Move Cloud to the Core of your Business StrategyZNetLive
 

Similaire à Lightning-Fast, Interactive Business Intelligence Performance with MicroStrategy and Kyligence (20)

Architecting Snowflake for High Concurrency and High Performance
Architecting Snowflake for High Concurrency and High PerformanceArchitecting Snowflake for High Concurrency and High Performance
Architecting Snowflake for High Concurrency and High Performance
 
How Analytics Teams Using SSAS Can Embrace Big Data and the Cloud
How Analytics Teams Using SSAS Can Embrace Big Data and the CloudHow Analytics Teams Using SSAS Can Embrace Big Data and the Cloud
How Analytics Teams Using SSAS Can Embrace Big Data and the Cloud
 
Addressing the systemic shortcomings of cloud analytics
Addressing the systemic shortcomings of cloud analyticsAddressing the systemic shortcomings of cloud analytics
Addressing the systemic shortcomings of cloud analytics
 
Enhance Data Governance with Kyligence Unified Semantic Layer
Enhance Data Governance with Kyligence Unified Semantic LayerEnhance Data Governance with Kyligence Unified Semantic Layer
Enhance Data Governance with Kyligence Unified Semantic Layer
 
Kyligence Cloud 4 - An Overview
Kyligence Cloud 4 - An OverviewKyligence Cloud 4 - An Overview
Kyligence Cloud 4 - An Overview
 
ICP for Data- Enterprise platform for AI, ML and Data Science
ICP for Data- Enterprise platform for AI, ML and Data ScienceICP for Data- Enterprise platform for AI, ML and Data Science
ICP for Data- Enterprise platform for AI, ML and Data Science
 
Accelerating Data Warehouse Modernization
Accelerating Data Warehouse ModernizationAccelerating Data Warehouse Modernization
Accelerating Data Warehouse Modernization
 
Building Enterprise OLAP on Hadoop for FSI
Building Enterprise OLAP on Hadoop for FSIBuilding Enterprise OLAP on Hadoop for FSI
Building Enterprise OLAP on Hadoop for FSI
 
Augmented OLAP Analytics for Big Data
Augmented OLAP Analytics for Big DataAugmented OLAP Analytics for Big Data
Augmented OLAP Analytics for Big Data
 
Accelerating Innovation with Hybrid Cloud
Accelerating Innovation with Hybrid CloudAccelerating Innovation with Hybrid Cloud
Accelerating Innovation with Hybrid Cloud
 
Building a Modern Analytic Database with Cloudera 5.8
Building a Modern Analytic Database with Cloudera 5.8Building a Modern Analytic Database with Cloudera 5.8
Building a Modern Analytic Database with Cloudera 5.8
 
The Cloud - What's different
The Cloud - What's differentThe Cloud - What's different
The Cloud - What's different
 
Apache Kylin and Use Cases - 2018 Big Data Spain
Apache Kylin and Use Cases - 2018 Big Data SpainApache Kylin and Use Cases - 2018 Big Data Spain
Apache Kylin and Use Cases - 2018 Big Data Spain
 
Building a hybrid, dynamic cloud on an open architecture
Building a hybrid, dynamic cloud on an open architectureBuilding a hybrid, dynamic cloud on an open architecture
Building a hybrid, dynamic cloud on an open architecture
 
Cloud the current future v6
Cloud   the current future v6Cloud   the current future v6
Cloud the current future v6
 
Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud Platform
 
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...
Increase your ROI with Hadoop in Six Months - Presented by Dell, Cloudera and...
 
SQL + Hadoop: The High Performance Advantage�
SQL + Hadoop:  The High Performance Advantage�SQL + Hadoop:  The High Performance Advantage�
SQL + Hadoop: The High Performance Advantage�
 
Move Cloud to the Core of your Business Strategy
Move Cloud to the Core of your Business StrategyMove Cloud to the Core of your Business Strategy
Move Cloud to the Core of your Business Strategy
 
Hadoop in the Cloud
Hadoop in the CloudHadoop in the Cloud
Hadoop in the Cloud
 

Plus de Tyler Wishnoff

Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglyTyler Wishnoff
 
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...Tyler Wishnoff
 
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsApache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsTyler Wishnoff
 
Analysis of the Pressure Placed on Medical Systems during the COVID-19 Pandemic
Analysis of the Pressure Placed on Medical Systems during the COVID-19 PandemicAnalysis of the Pressure Placed on Medical Systems during the COVID-19 Pandemic
Analysis of the Pressure Placed on Medical Systems during the COVID-19 PandemicTyler Wishnoff
 
Apache Kylin Meetup: Berlin - With OLX Group
Apache Kylin Meetup: Berlin - With OLX GroupApache Kylin Meetup: Berlin - With OLX Group
Apache Kylin Meetup: Berlin - With OLX GroupTyler Wishnoff
 
Apache Kylin Data Summit 2019: Kyligence Presentation
Apache Kylin Data Summit 2019: Kyligence PresentationApache Kylin Data Summit 2019: Kyligence Presentation
Apache Kylin Data Summit 2019: Kyligence PresentationTyler Wishnoff
 
Augmented OLAP for Big Data Analytics
Augmented OLAP for Big Data AnalyticsAugmented OLAP for Big Data Analytics
Augmented OLAP for Big Data AnalyticsTyler Wishnoff
 
Accelerating Big Data Analytics with Apache Kylin
Accelerating Big Data Analytics with Apache KylinAccelerating Big Data Analytics with Apache Kylin
Accelerating Big Data Analytics with Apache KylinTyler Wishnoff
 

Plus de Tyler Wishnoff (8)

Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the Ugly
 
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...
How to Guarantee Exact COUNT DISTINCT Queries with Sub-Second Latency on Mass...
 
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive DatasetsApache kylin 101 - Get Sub-Second Analytics on Massive Datasets
Apache kylin 101 - Get Sub-Second Analytics on Massive Datasets
 
Analysis of the Pressure Placed on Medical Systems during the COVID-19 Pandemic
Analysis of the Pressure Placed on Medical Systems during the COVID-19 PandemicAnalysis of the Pressure Placed on Medical Systems during the COVID-19 Pandemic
Analysis of the Pressure Placed on Medical Systems during the COVID-19 Pandemic
 
Apache Kylin Meetup: Berlin - With OLX Group
Apache Kylin Meetup: Berlin - With OLX GroupApache Kylin Meetup: Berlin - With OLX Group
Apache Kylin Meetup: Berlin - With OLX Group
 
Apache Kylin Data Summit 2019: Kyligence Presentation
Apache Kylin Data Summit 2019: Kyligence PresentationApache Kylin Data Summit 2019: Kyligence Presentation
Apache Kylin Data Summit 2019: Kyligence Presentation
 
Augmented OLAP for Big Data Analytics
Augmented OLAP for Big Data AnalyticsAugmented OLAP for Big Data Analytics
Augmented OLAP for Big Data Analytics
 
Accelerating Big Data Analytics with Apache Kylin
Accelerating Big Data Analytics with Apache KylinAccelerating Big Data Analytics with Apache Kylin
Accelerating Big Data Analytics with Apache Kylin
 

Dernier

怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制vexqp
 
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制vexqp
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Valters Lauzums
 
Harnessing the Power of GenAI for BI and Reporting.pptx
Harnessing the Power of GenAI for BI and Reporting.pptxHarnessing the Power of GenAI for BI and Reporting.pptx
Harnessing the Power of GenAI for BI and Reporting.pptxParas Gupta
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...ZurliaSoop
 
Dubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls DubaiDubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls Dubaikojalkojal131
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxchadhar227
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...nirzagarg
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraGovindSinghDasila
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...gajnagarg
 
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowVadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowgargpaaro
 
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...gajnagarg
 
Data Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdfData Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdftheeltifs
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格q6pzkpark
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schscnajjemba
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangeThinkInnovation
 
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...nirzagarg
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNKTimothy Spann
 
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制vexqp
 

Dernier (20)

怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
怎样办理圣路易斯大学毕业证(SLU毕业证书)成绩单学校原版复制
 
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制
怎样办理纽约州立大学宾汉姆顿分校毕业证(SUNY-Bin毕业证书)成绩单学校原版复制
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Harnessing the Power of GenAI for BI and Reporting.pptx
Harnessing the Power of GenAI for BI and Reporting.pptxHarnessing the Power of GenAI for BI and Reporting.pptx
Harnessing the Power of GenAI for BI and Reporting.pptx
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Dubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls DubaiDubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls Dubai
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowVadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
 
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
 
Data Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdfData Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdf
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
一比一原版(曼大毕业证书)曼尼托巴大学毕业证成绩单留信学历认证一手价格
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schs
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Satna [ 7014168258 ] Call Me For Genuine Models We ...
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
Cytotec in Jeddah+966572737505) get unwanted pregnancy kit Riyadh
Cytotec in Jeddah+966572737505) get unwanted pregnancy kit RiyadhCytotec in Jeddah+966572737505) get unwanted pregnancy kit Riyadh
Cytotec in Jeddah+966572737505) get unwanted pregnancy kit Riyadh
 
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证(CITY毕业证书)成绩单学校原版复制
 

Lightning-Fast, Interactive Business Intelligence Performance with MicroStrategy and Kyligence

  • 2. © Kyligence Inc. 2019, Confidential. Apache Kylin Top Level Apache Project  The only open-source OLAP on big data platform Best Open-source Big Data Tool  InfoWorld’s Bossies (Best of Open Source Software Awards) in 2015 & 2016 Sub-Second Interactive Query  Large scale, high concurrency, sub second query latency, multi-dimension 1000+ Organizations  Adopted by thousands of organizations globally
  • 3. © Kyligence Inc. 2019, Confidential. Kyligence = Kylin + Intelligence • Founded in 2016 by the creators of Apache Kylin • Built around Kylin, with augmented AI and enhanced to deliver unprecedented enterprise analytic performance • CRN Top-10 big data startups in 2018 • Global Presence: San Jose, Seattle, New York, Shanghai, Beijing • VCs: Fidelity International, Shunwei Capital, Broadband Capital, Redpoint, Cisco, Coatue Accelerate Critical Business Decisions with AI-Augmented Data Management and Analytics 2016 Founded Pre- A Redpoint Cisco 2017 Series A CBC SHUNWEI 2018 Series B 8Roads 2019 Series C Coatue
  • 4. © Kyligence Inc. 2019, Confidential. Trusted by Global Fortune 500 BFSI Telecom Technology Manufacturing, Retail, Etc.
  • 5. © Kyligence Inc. 2019, Confidential. Pains in Collaboration Data Analyst Data Engineer • Manage data source • Design data model to keep one source of truth • ETL and load data • Develop dashboard/reporting • Self-service analysis to answer business questions Low efficiency in development to fulfill business requirements Limited dimension and measures in a model to serve complex calculations Difficulty if analytics requirements or source data change Time to insight is slow
  • 6. © Kyligence Inc. 2019, Confidential. Kyligence Ecosystem Global Partners • Fully enabled on leading cloud and data platforms (Azure, AWS, Google Cloud, Cloudera) • Integrated with popular BI and virtualization (Tableau, Power BI, Qlik, MicroStrategy) • Certified on main Hadoop distributions (CDP)
  • 7. © Kyligence Inc. 2019, Confidential. Kyligence Enterprise Accelerate Mission-critical Analytics Intelligently • Unified Query Entrance ODBC/JDBC API/SDK Finance Marketing Sales Customer Checkout Cube Index 10%4% 80% RDBM s Hive SQL/MDX Semantic Services 6% Distributed Query Engine AI-Augmented Engine Smart Pushdown Metadata Management Enterprise Security • Business Semantic Layer • Query Pattern for all data • High Performance Engine
  • 8. © Kyligence Inc. 2019, Confidential. Kyligence Cloud FinanceMarketingSales Index more… Landing & Transformation Semantic & Augmentation ApplicationsSource Azure Blob Storage Azure Synapse
  • 9. © Kyligence Inc. 2019, Confidential. AI Augmented Engine: Intelligent Data Development
  • 10. © Kyligence Inc. 2019, Confidential. AI Augmented Engine: One-click Acceleration • Self-maintaining • Dynamic auto-modeling • Self-learning engine • One-click acceleration • Adaptive model
  • 11. © Kyligence Inc. 2019, Confidential. AI-Augmented Engine — Learn From Your Analytics History
  • 12. © Kyligence Inc. 2019, Confidential. Advanced Tuning Features – Push Down and Aggregate Index
  • 13. © Kyligence Inc. 2019, Confidential. Under the hood : Smart Cuboids • Each Model consists of N-Dimension Cuboids which is a combination of several dimension in different permutations and combinations. • Apache Spark is used to build the cuboids making query results extremely fast. • When the user sends a query the model intelligently looks for the Cuboids/segment returns the results extremely fast.
  • 14. © Kyligence Inc. 2019, Confidential. Unified Semantic Layer BI Integration Access Control Enterprise Security Query Engine Model Query Platform Data Sources Excel MicroStrategy Other BI Tools Semantic Layer Cloud DW Parquet ORC Blob Storage CSVSnowflake • Translate technical details into business terminology • Synchronize semantics across major BI tools • Unified business definitions • Flexible business calculations
  • 15. © Kyligence Inc. 2019, Confidential. Elastic Scaling — Handle Peak Time Automatically  Fewer compute and storage resources utilized  Dynamic on-demand cluster resizing  Uses spot instances  Efficient planning for data growth
  • 16. © Kyligence Inc. 2019, Confidential. TPC-H 22 Queries SF=50 Query Response Time | 0.5 Billion SF=500 Query Response Time | 5 Billion • No warm up • Lower is better • Run each query 3 times • Record the average time For each Dataset:
  • 17. © Kyligence Inc. 2019, Confidential. Financial Risk Management - replacing the large SSAS cube Challenges Kyligence’s Solution modernization same data source same front-end BI similar OLAP concepts comparable semantic layer finer granular access control Scalability Performance Low Cost • 5TB SSAS cube with 5 Billion rows daily incremental data • 14 Lookup tables, half over 20M cardinalities (largest 200M) • 600+ dimensions • 30+ analysis users • Analysts’ work locked by incremental loading workload, system crashes happen frequently • Poor performance on data loading and queries (especially on UHC, Count Distinct, Correlation) • Limited concurrent users • Single cube easy management • Analysts’ work no longer interrupted • Transparent to business users, same • analysis tool Excel • Improved query and loading performance • Support 1000+ concurrent users • Meet future requirement - prediction of 40% data volume growth, migration to cloud, Realtime

Notes de l'éditeur

  1. UBS case uses databricks
  2. UBS case uses databricks
  3. Azure storage to be generic, replace Alibaba with Hadoop
  4. 灵活的多维建模 模型的变化只影响有关的索引; 模型定义的变化与数据加载互不影响; -------------------- Flexible multidimensional modeling Changes in the model affect only the relevant indexes Changes in model definitions and data loading do not affect each other
  5. 灵活的多维建模 模型的变化只影响有关的索引; 模型定义的变化与数据加载互不影响; -------------------- Flexible multidimensional modeling Changes in the model affect only the relevant indexes Changes in model definitions and data loading do not affect each other
  6. Industry-recognized data analysis test data sets Analysis of key business decisions Practical business significance 0.5 billion dataset, test TPC-H 22 queries. Test method: 3 times to average, no query engine to warm up. TPC-H Benchmark Examine large volumes of data High complexity queries Answers critical business questions 22 decision making queries E.g. The Shipping Priority Query retrieves the shipping priority and potential revenue of the orders having the largest revenue among those that had not been shipped as of a given date. Top 10 orders are listed in decreasing order of revenue. HARDWARE CONFIGURATION Same 4 physical nodes Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz * 2 Totally 86 vCores, 188 GB mem Same Spark configuration for both KE 4 Beta and SparkSQL 2.4 spark.driver.memory=16g spark.executor.memory=8g spark.yarn.executor.memoryOverhead=2g spark.yarn.am.memory=1024m spark.executor.cores=5 spark.executor.instances=17 Query Response Time | 5 Billion Same 4 physical nodes Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz * 2 Totally 86 vCores, 188 GB mem Same Spark configuration for both KE 4 Beta and SparkSQL 2.4 spark.driver.memory=16g spark.executor.memory=20g spark.yarn.executor.memoryOverhead=2g spark.yarn.am.memory=1024m spark.executor.cores=5 spark.executor.instances=30
  7. Benefits: Unlimited scale-out solution to fit future data volume growth 1 hour non-blocking incremental loading Single cube easy maintenance Low infrastructure cost with auto scaling support 100 concurrent users Transparent to business users, same analysis tool Excel Architecture Kyligence Enterprise 4.0 Azure HDInsight 3.6 Azure Data Lake gen2 Cluster size: 30 D3 V2 worker nodes (potentially) ingest data from Oracle Query performance 90% SQL queries within 5s 90% MDX queries within 60s 80% MDX queries within 20s 50 QPS per query node