Mike Boyarski gave a presentation on MemSQL, an operational data warehouse that provides real-time analytics capabilities. He discussed challenges with traditional databases around slow data loading, lengthy query times, and low concurrency. MemSQL addresses these issues with fast data ingestion, low latency queries, and high scalability. It can ingest streaming data, run on a variety of platforms, and provides security, SQL support, and integration with common data tools. MemSQL was shown augmenting an existing IoT architecture to enable real-time analytics through fast data loading, consolidated data storage, and high query performance.
2. Mike Boyarski
Sr. Director Product Marketing
2
Today’s Speaker & Agenda
Questions?
Submit via the Q&A or
chat function in the lower
right hand side of the
console or tweet us using
#memsqlwebcast
▪ Company Introduction
▪ Database Challenges
▪ Intro to MemSQL
▪ Demo
▪ Customer Stories
▪ Q&A
3. 3
MISSION
Growth of digital business impacting data architectures
We make every company a real-time enterprise
PRODUCT
Top Ranked Operational Data Warehouse
MemSQL provides you the ability to learn and react in real time
ABOUT
Founders are former Facebook, SQL Server database engineers
Funding from Top Tier investors; Enterprise Customers:
MemSQL at a Glance
6. Latency Holding Back the Enterprise
6
SLOW
Data Loading
Batched loading
Hours to load
Sampled data views
LENGTHY
Query Execution
Slow query responses
Slow reports
No real-time response
Single threaded operations
Challenge with mixed workloads
Overall poor performance
LOW
Concurrency
#memsqlwebcast
7. The Enterprise Requires Performance
7
FAST
Data Loading
Stream data
Real-time loading
Full data access
LOW
Query Latency
Vectorized queries
Real-time dashboards
Live data access
Multi-threaded processing
Transactions and Analytics
Scalable performance
HIGH
Concurrency
#memsqlwebcast
8. The Real-Time Database and Data Warehouse
for the Front Lines of Your Business
8
Real-Time
Fast data ingest
Low latency queries
High concurrency
Run Anywhere
On-premises to managed service
Multi-cloud
Enterprise grade security
Scalable SQL
Petabyte scale
Distributed
Industry standard hardware
#memsqlwebcast
9. MemSQL Integrated Architecture
9
Streaming Ingest
Real-time
data pipelines with
exactly-once semantics
Live Data
Memory optimized
tables for transacting
and analyzing
real-time events
Historical Data
Disk optimized tables
with compression for
fast analytic queries
#memsqlwebcast
10. Ecosystem Overview
10
Streaming Ingest Live Data Historical Data
Real-Time Data
Pipelines
Memory Optimized
Tables
Disk Optimized
Tables
Real-Time Data
Messaging and
Transforms
Historical Data
Real-Time
Application
Analytics
Business Intelligence
Dashboards
Bare Metal, Virtual Machines, Containers On-Premises, Cloud, As a Service
Kafka Spark
Relational Hadoop Amazon S3
#memsqlwebcast
11. 11
• Relational ANSI SQL
• JSON, Geospatial, Key Value formats
• ACID Transactions
• Vectorized and compiled queries
• User defined functions
Real-Time Analytics
#memsqlwebcast
12. Deliver Real-Time ETL
12
Extract
Ingest from Apache Kafka,
Spark, Amazon S3, or
HDFS
Load
Guarantee message
delivery with exactly-once
semantics
Transform
Map and enrich data with
user defined or Apache
Spark transformations
#memsqlwebcast
14. Simple Streaming Setup with
CREATE PIPELINE
14
memsql> CREATE PIPELINE twitter_pipeline AS
-> LOAD DATA KAFKA "public-kafka.memcompute.com:9092/tweets-json"
-> INTO TABLE tweets
-> (id, tweet);
Query OK, (0.89 sec)
memsql> START PIPELINE twitter_pipeline;
Query OK, (0.01 sec)
memsql> SELECT text FROM tweets ORDER BY id DESC LIMIT 5G
#memsqlwebcast
15. Durable Distributed Storage
15
Distributed and Durable
Store and process on clusters
of machines for performance
and persistence
Highly Available
Online replication ensures
data consistency and
protects against outages
Big Data Capacity
Petabyte scale with up to
10x compression and
instant query retrieval
#memsqlwebcast
17. MemSQL Solutions
17
Real-Time Analytics Internet of Things
Personalization and
Recommendations
Portfolio and
Risk Management
Telemetry and
Monitoring
Customer 360
Applications
#memsqlwebcast
18. Sample IoT Application: Traditional Architecture
Data Sources Dashboards
PI Historian Tableau, Looker, Microstrategy
Messaging Data Marts
Data
Lake
DW
ODS
Sensors
Spark, Kafka,
ESB, ETL
#memsqlwebcast
19. Data Sources Dashboards
PI Historian Tableau, Looker, Microstrategy
Messaging Data Marts
Data
Lake
Challenge:
Slow data ingestion
Several data marts
DW
ODS
Sensors
Spark, Kafka,
ESB, ETL
Sample IoT Application: Traditional Architecture
#memsqlwebcast
20. Data Sources
PI Historian Tableau, Looker, Microstrategy
Messaging Data Marts
Data
Lake
Challenge:
Slow running queries
DW
ODS
Sensors
Dashboards
Spark, Kafka,
ESB, ETL
Sample IoT Application: Traditional Architecture
#memsqlwebcast
21. Data Sources Dashboards
PI Historian
Messaging
Spark, Kafka,
ESB, ETL
Data Marts
Data
Lake
Challenge:
Limited scalability
prevents high user
concurrency
DW
ODS
Sensors
Tableau, Looker, Microstrategy
Sample IoT Application: Traditional Architecture
#memsqlwebcast
22. MemSQL Augments Existing Infrastructure for Real-Time
Tableau, Looker, Microstrategy
Kafka, Spark,
CDC
ESB, ETL,
Files
Data Integration
with Spark, HDFS,
S3
PI Historian
Sensors
Data Integration
#memsqlwebcast
23. MemSQL Augments Existing Infrastructure for Real-Time
Tableau, Looker, Microstrategy
Kafka, Spark,
CDC
ESB, ETL,
Files
PI Historian
Sensors
Why MemSQL?
- Fast real-time data ingestion
- Scalable SQL consolidates data marts
Data Integration
with Spark, HDFS,
S3
#memsqlwebcast
24. MemSQL Augments Existing Infrastructure for Real-Time
Kafka, Spark,
CDC
ESB, ETL,
Files
PI Historian
Sensors
Why MemSQL?
- Low latency queries for
faster dashboards
- Scale-out architecture enables
high user concurrency
Data Integration Tableau, Looker, Microstrategy
Data Integration
with Spark, HDFS,
S3
#memsqlwebcast