The Perfect Storm: The Impact of Analytics, Big Data and Analytics

Eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr The Briefing Room

!   Reveal the essential characteristics of enterprise
software, good and bad

!   Provide a forum for detailed analysis of today s
innovative technologies

!   Give vendors a chance to explain their product to
savvy analysts

!   Allow audience members to pose serious questions...
and get answers!


November: Cloud

December: Innovators

January: Big Data

February: Performance

March: Integration


!  Historically, databases have been built around SQL, a
declarative query language targeted at organizing data in two-
dimensional tables

!  The ever increasing variety, volume and velocity of data has
taxed traditional relational databases and created performance
bottlenecks, particularly around CPU, memory, disk I/O and
network saturation

!  Alternatives like NoSQL and NewSQL have emerged to better
support extreme and diverse workloads without suffering hits in
performance


Dr. Barry Devlin is a founder of the data
warehousing industry and among the foremost
authorities worldwide on business intelligence
(BI). He is a widely respected consultant,
lecturer and author of “Data Warehouse—from
Architecture to Implementation.” Barry has 30
years of experience in the IT industry, previously
with IBM, as an architect, consultant, manager
and software evangelist.
As founder and principal and 9sight Consulting
(www.9sight.com), Barry provides strategic
consulting and thought leadership to buyers and
vendors of BI solutions. He is currently
developing a new architectural model for fully
consistent business support—from informational
to operational and collaborative—Business
Integrated Insight (BI2). He is based in Cape
Town, South Africa.


! NuoDB is an ACID-compliant NewSQL relational
database management system

!  It is architected to scale elastically on the cloud

!  It leverages a peer-to-peer, distributed architecture

! NuoDB currently has 1000+ users in beta


Barry is an accomplished software CEO with
over 25 years of industry experience in running
private and public companies around industry-
changing paradigm shifts in technology. He had
leadership roles at IONA Technologies, which
helped lay the groundwork for modern SOA-
based systems, and StreamBase Systems, a
pioneer of complex event processing. Barry’s
early career included technical, management
and business development roles. Barry does a
great deal of consulting and has served on a
variety of boards for startup companies in
Boston, Ireland and South Africa.
He earned his Degree in Engineering from New
College Oxford University and holds an
Honorary Doctorate in Business Administration
from the IMCA.


The Elastically Scalable Database™

Copyright © NuoDB 2012 1

NuoDB
The Database for the 21st Century
NuoDB is a revolutionary database system based on a patented
Emergent Architecture.

NuoDB is designed for modern datacenters, workloads and business
models.

NuoDB delivers all of the capabilities and services of the 20th Century
RDBMS.

NuoDB has a SQL personality but it could just as easily be a Document
Database, an Object Database, a Graph Database or something else.

NuoDB Inc is building next generation capabilities that will redeﬁne the
role of databases in next generation applications.


20th Century Database

9% Powerful Query
3%
4% Language

19% 44%
Industry Standards

Data Guarantees
ORACLE
21%
IBM
Microsoft Employee Skills
Sybase
Teradata
Others
Existing Data
Tools


21st Century Problem
Commodity Datacenters ✗
Big Data ✗
Powerful Query
Modern Workloads ✗ Language

24x7 Operation ✗ Industry Standards

Geo-distribution ✗ Data Guarantees

Developer Employee Skills
Empowerment
✗ 3%
4%
9%

Existing Data
19% 44%

21%

Tools ORACLE
IBM
Microsoft
Sybase
Teradata
Others


Database Crisis

Wikipedia Flickr Facebook

Amazon Google

Source: Marc Bojoly


Jim Starkey
“Elastically Scalable Transactions represent the biggest
breakthrough in database technology in 25 years”

‣ DEC RDB/ELN
‣ InterBase
‣ Firebird
‣ Falcon
‣ BLOBS
‣ MVCC
‣ etc


Emergent Database
Architecture
“An emergent
behavior can
appear when a
number of simple
entities operate in
an environment,
forming more
complex behaviors
as a collective.”

- Wikipedia


Poleposition - Single Node
Notes

MySQL 5.1
NuoDB Beta 3 - Single Node
http://www.polepos.org

In early tests NuoDB on a
single node was 2x to 20x
faster than MySQL 5.1
running the industry
standard Poleposition
Benchmarks.

Your mileage may vary.

‣ Time taken for given benchmark, normalized to NuoDB = 1
‣ Less is Better

Adding a Second Machine
• Second machine typically
doubles TPS

• Second machine is added to
live database while it is
running at 1,000’s of TPS

• Performance increase is
immediate

• BTW - you can take either
machine away and the
database keeps running
without data loss Second Machine
Instant Performance Increase


Adding a Third Machine
• Third machine typically
triples single machine TPS

• Third machine is added to
live database while it is
running at 1,000’s of TPS

• Performance increase is
immediate
Second & Third Machine
• BTW - you can take any Instant Performance Increase
machine away and the
database keeps running
without data loss


More Machines? Bring ‘em On

Nodes TPS &!!!!"

MySQL 1 3,000 %#!!!"

NuoDB 1 4,500 %!!!!"
NuoDB 9 27,000
$#!!!"

Technical Details: TPS $!!!!"

‣ 2-9 Tx engines #!!!"
‣ 1 storage manager
‣ Best sustained TPS and !"
# clients combination $" %" &" '" #" (" )" *" +"

‣ 50% updates
Number of Nodes

NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node.


Or Scale-out on IAAS
'#!!!"

‣ Nuodb scales linearly on
'!!!!"
EC2
‣ Per-node performance on &!!!"

m1.large nodes approx 50%
of our commodity servers %!!!"
TPS
‣ Just started on optimizing
$!!!"
‣ RDS runs on 1 node, and
gets overloaded with 10+
#!!!"
connections
!"
'" #" (" $" )" %" *" &" +"

Number of EC2 Nodes


Standard SQL - Favorite Tools

MS Excel (and other MS tools)

Squirrel SQL

You already know how to use NuoDB

DBVisualizer

NuoDB

Applications Brokers Transaction Engines Storage Managers


NuoDB Architecture


The 21st Century Database
OldSQL NoSQL NuoDB
Powerful Query Language (SQL) ✓ ✓
Industry Standards (SQL, JDBC,
ODBC etc) ✓ ✓
20th C. Data Guarantees (ACID
Database Transactions) ✓ ✓
Employee Skills
✓ ✓
Existing Data
✓ ✓
On-demand Capacity
✓ ✓
Commodity Datacenters /
Virtualization / Cloud ✓ ✓
Modern Workloads (Concurrency,
TPS, Latency) ½ ✓
Big Data ½ ✓ ✓
21st C.
100% Uptime
✓ ✓
Database Online Maintenance, Admin and
Schema Evolution ✓ ✓
Geo-distribution
✓ ✓
Developer Empowerment
✓ ✓
Zero Touch Backup
✓ ✓
“Zero” Admin
✓ ✓


The Perfect Storm: The Impact of
Analytics, Big Data and Cloud

The Briefing Room, 23 October 2012

Dr Barry Devlin
Founder & Principal
9sight Consulting

Copyright © 2012 9sight Consulting, All Rights Reserved

Three key trends in business are driving rapid change.
1.  Closed-loop business – strategy to execution
–  Merge operational, informational & collaborative
–  Extreme flexibility in adapting to change
2.  Massive information volumes for use
–  Volumes, sources, types
3.  Collaborate to innovate
–  Millennials move into power
–  Mobile users and applications
Faster
Bigger
Distributed
More flexible
More personal

11
Copyright © 2010-12 9sight Consulting

Recent technology advances offer new ways to address
emerging business needs.
1.  Closed-loop business – strategy to execution
2.  Massive information volumes for use
3.  Collaborate to innovate

4.  SOA, Mobile Apps and Analytics
–  Adaptive IT and design flexibility
5.  Advances in “Data Processing”
–  RDBMS advances, Big Data and Cloud
6.  Web / Enterprise 2.0 and beyond
–  Collaborative tools, semantic web and more
12
Copyright © 2010-12 9sight Consulting

Big data is really all data
Three domains
Business Analytics

Flexibility
§  Process-mediated data
–  “Traditional” operational
& informational data
–  Via data entry & cleansing Human-sourced information

§  Machine-generated data
–  Output of machines & sensors
–  High-speed, high-volume (Traditional)
–  The Internet of Things Business
Processes
§  Human-sourced information Machine-
–  Subjectively interpreted record generated
of personal experiences
Process-
data
–  Model unknown before usage mediated
–  From Tweets to Videos data

§  See: bit.ly/Big_Data_Zoo
Timeliness
[In the context of these domains, “data” signifies well-structured and/or
modeled and “information” is more loosely structured and human-centric.]
13
Copyright © 2012 9sight Consulting

Technology drives and dictates progress

§  Vast improvements in price-performance for memory
–  Critical data for most businesses can fit in main memory
–  Traditional database design is disk-centric
– Commit means on disk
– Disk I/O bottleneck is a key design point

§  Single processors cannot go any faster; the move to multi-
core / multi-processing has been ongoing for over 5 years
–  Traditional programming is single-CPU-centric
–  MPP – from specialized / high-cost to wide-spread / low-cost

§  Physical data representation back at the forefront
–  Row store vs column store vs key-value store
–  Compression ratios
–  Are column stores slow for update?
14

Database – Innovation and evolution
§  “Post-relational”
Features / –  Flexibility
Performance Cumulative progress –  Scalability
§  Relational
–  A logical model of data’s
relationships to “reality”
–  Predefined model Next wave?

§  Hierarchical & Network
–  Speed of record update and
access
–  Physical storage optimization Relational

Niche?
Disruptive
change
Hierarchical
& Network Sustaining change
“Post-relational”

1960 1970 1980 1990 2000 2010 2020
Clayton M. Christensen, “The Innovator’s Dilemma”, 1997
15

The emerging biz-tech ecosystem
§  Fully symbiotic existence of business and IT

1.  Interdependence
–  New technology enables new
business possibilities;
new business opportunities
drive technology advances
2.  Reintegration
–  Silos in business and IT deter Web-savvy customers;
coherence becomes mandatory
3.  Cross-over
–  Business people need IT skills to see how to recreate the business
with new technology;
IT people need business acumen to see how to satisfy business
needs in new ways with emerging technology
16

Questions (1)
1.  You emphasize the object-oriented / distributed / message-oriented
nature of NuoDB as well as in-memory operation. With improving
memory price-performance and the possibility that many businesses
will be able to fit all business-critical data in memory, why do you need
both?
2.  It seems that disk storage is replaced first by distributed computer
storage, and then “failback” to disk. Are you replacing disk I/O latencies
with network latencies? How is this an advantage?
3.  As an in-memory database, how do you position NuoDB vs. SAP
HANA?
4.  With advances in memory, MPP, columnar stores, etc., I see the
possible end of the old operational vs informational split. What is your
view? Where does NuoDB fit in that scenario?
5.  Big data – what do you mean by the term? On which aspects of big
data does NuoDB focus?

17

Questions (2)
6.  “NoSQL” databases emphasize flexibility to changing data structures
mainly by exposing a key-value store to applications. Is that why you
use a KV store? How do you benefit from the KV store as it is “locked
behind” the relational model?
7.  The query optimizer is perhaps the key to database performance. For
most new DBs, it has proven to be a long road to build an optimized
optimizer – how will NuoDB address this?
8.  In your white paper you say “database designers don’t need to
compromise on schema design by de-normalizing tables, removing
joins” for performance… sounds like magic. Why not?
9.  You support indexing. Why do you need it / use it in an in-memory
database?
10.  You put Multiversion Concurrency Control (MVCC) forward as the
solution to ACID requirements. Do you always insert rather than
update?

18

Barry Devlin Founder and Principal
9sight Consulting, www.9sight.com

Dr. Barry Devlin is a founder of the data warehousing industry
and among the foremost authorities worldwide on business
intelligence (BI) and beyond. He is a widely respected
consultant, lecturer and author of “Data Warehouse—from
Architecture to Implementation”. Barry has 30 years of
experience in the IT industry, previously with IBM, as an
architect, consultant, manager and software evangelist.

As founder and principal of 9sight Consulting (www.
9sight.com), Barry provides strategic consulting and thought-
leadership to buyers and vendors of BI solutions. He is
currently developing a new architectural model for fully
consistent business support—from informational to
operational and collaborative—Business Integrated Insight
(BI2). Based in Cape Town, South Africa, Barry’s knowledge
and expertise are in demand both locally and internationally.

Email: barry@9sight.com
Twitter: @BarryDevlin

20

This Month: Database

November: Cloud

December: Innovators

January: Big Data

2013 Editorial Calendar
(www.insideanalysis.com)


The Perfect Storm: The Impact of Analytics, Big Data and Analytics

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à The Perfect Storm: The Impact of Analytics, Big Data and Analytics

Similaire à The Perfect Storm: The Impact of Analytics, Big Data and Analytics (20)

Plus de Inside Analysis

Plus de Inside Analysis (20)

Dernier

Dernier (20)

The Perfect Storm: The Impact of Analytics, Big Data and Analytics