The New Possible: Very Big Data for Serious Business Value

Eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr

!   Reveal the essential characteristics of enterprise
software, good and bad

!   Provide a forum for detailed analysis of today s
innovative technologies

!   Give vendors a chance to explain their product to
savvy analysts

!   Allow audience members to pose serious questions...
and get answers!


!  November: Cloud
!  December: Innovators
!  January: Big Data
!  February: Performance
!  March: Integration


!  Databases were designed primarily to store information for
retrieval at a later time.

!  Big Data requires big databases.
!  The convergence of multi-structured data and the need to
perform both transactional and operational analytics has led to
substantial innovations in database technologies.

!  Today some of the biggest databases blend the best of both
worlds, transforming the way organizations store and analyze
enterprise data.


Robin Bloor is
Chief Analyst at
The Bloor Group.

Robin.Bloor@Bloorgroup.com


!  German-founded SAP is one of the largest software companies in
the world. Its best-known products are SAP ERP, SAP Business
Warehouse, SAP Business Objects, SAP Sybase IQ and SAP HANA.

!  SAP offers a comprehensive set of database management
solutions that spans the needs of the enterprise, leveraging in-
memory, cloud and mobile technologies.

!  Recent innovations include a Big Data analytics platform that
loads, processes and delivers massive amounts of multi-
structured data and is accessible on demand enterprise-wide.


Courtney Claussen is a product manager
at Sybase, Inc., concentrating on Sybase's
data warehousing and analytics products.
She has enjoyed a 30 year career in
software development, technical support
and product marketing in the areas of
computer aided design, computer aided
software engineering, database
management systems, middleware, and
analytics.


The New Possible: Very Big Data for Serious Business Value
The Briefing Room with Dr. Robin Bloor and SAP

October 9, 2012

CON
FIDE
N TIAL

AGENDA

•  Big Data Analytics: A Reality
•  SAP Sybase IQ: Built for Big Data Analytics
•  SAP Sybase IQ: Continuing Innovation

©  2012 SAP AG. All rights reserved. 10

Big Data Analytics
A Reality

THE NEW DYNAMICS OF BUSINESS
COMPETING ON BIG DATA DRIVEN ANALYTICS

New Strategies &
Business Models

Business
Value*

Operational Revenue
Efficiencies Growth

*A McKinsey study titled “Big Data: Next frontier for innovation, competition, and productivity”, May 2011, has found huge potential for
Big Data Analytics with metrics as impressive as 60% improvements in Retail operating margins, 8% reduction in (US) national healthcare
expenditures, and $150M savings in operational efficiencies in European economies


Getting Value from Big Data

Find supply chain
inefficiencies

Predict financial Uncover insurance fraud
performance

Applied
Optimize stocking of Big Data Dispense correct health
care
products
Analytics

Maintain customer loyalty


EDW AND BIG DATA PLATFORMS
CONTRASTS

Big Data
Clickstreams,
EDW sensors, log data,
unstructured social
media
Large -> ENORMOUS

Pre-processed data -> Raw data
Business
Schema -> No schemaVale*

SQL -> Programmatic
Enterprise data,
relational, OLAP -> Batch processing
structured,
indexed text Scale up -> Scale out


EDW AND BIG DATA PLATFORMS
PARTNERSHIP

Big Data
Clickstreams, sensors,
EDW log data, unstructured
social media

•  Combine all relevant data
for better insights
Business
•  Real-time BI Vale*
•  SQL declarative processing
•  Big Data pre-processing
with EDW deep analytics
Enterprise data,
relational, structured,
indexed text


Big-data analytics plus data warehousing
Deserves a new platform

Data loading
Mobile OLAP*
Integrated Web Operational
workflow reporting
Specialized Data
apps mining

Ÿ  Platform accessible
to all business
Support
massive
numbers

processes and all
Ÿ  Volume
of
users
and
workloads
business users
Ÿ  Velocity
Ÿ  Requirement for
Ÿ  Variety Ÿ  MapReduce data and algorithms
Ÿ  Costs Ÿ  RDBMS+
Ÿ  EDW together in the
Ÿ  Skills In-DB analytics
platform
Analyze
massive
volumes
Ÿ  Ability to distribute
of
complex
data
from
many
sources
interactions
throughout the
enterprise
*Online analytical processing
HDFS +Relationaldatabase management
system


SAP SYBASE IQ
BUILT FOR BIG DATA ANALYTICS

Grid architecture
System scale out

Full Mesh Interconnect

Storage Fabric

Multi-dimensional scale out
•  Multiple resources can scale out independently
–  Storage, server (CPU, memory), SAN switches, interconnect can scale on their own
•  Scale out is incremental and linear
–  No need to add large units of monolithic CPU/storage pairs


Deployed use case
comScore Networks measures the digital world

Ÿ  comScore provides solutions for online audience measurement, e-commerce, advertising,
search, video and mobile to analysts with digital marketing and vertical-specific industry
expertise
Ÿ  Large SAP Sybase IQ Multiplex Grid on v15.x with 10s of servers and hundreds of CPU
cores
Ÿ  Manages more than 150TB of data with trillions of rows and 10s of thousands of tables
Ÿ  More than 200+ concurrent users with highly parallel and distributed workload
Ÿ  Incrementally scalable on commodity hardware

……………

Storage Fabric


Community platform
Elastic virtual data marts

VDM1
VDM2
Shared

Full Mesh High Speed Interconnect

Virtual Shared CPU,
Memory
Logical Server 1 Logical Server 2

Storage Fabric

Virtual Shared
Storage

Virtual data marts
•  VDM is logical binding of mutually exclusive nodes, memory, storage
–  Logical Server (LS) is a mutually exclusive logical binding of nodes, memory
–  Logical Server (LS) is a subset of VDM
–  Bindings are elastic i.e. they can dynamically grow/shrink


Robust load engine

Loading can be from multiple Extraction, transformation, and load (ETL) in SAP software
modes:
Scale out Scale out
Ÿ  Parallel bulk load processing: ETL project 1 ETL project 1 ETL project 1

–  Load rates in excess of 250 GB/hr are
common even with modest-size Full-‐mesh
interconnect

hardware nodes

Ÿ  Continuous and trickle feed via
microbatching (change data capture)

Page-level snapshot versioning:
Storage fabric
Ÿ  Allows non-blocking concurrent loads
and queries

Load from client machines


Query engine
Distributed query processing

Query 1 Query 2
5 node DQP 4 node DQP

Storage Fabric

Massively parallel processing
•  Leader node: Receives and initiates queries
–  Any node can be a leader
–  Leader node may satisfy query within itself
•  Worker node: Nodes pick up work units from leader
–  Many worker nodes per query
–  Same worker node can serve multiple queries


Text search and analysis

Table in SAP Text index Full text
Text File ingestion into
load blob or clob Sybase IQ queries

?
TextCol ID Term Pos
Info
Text Filtering to plain abc
filtering 0 a 1,3,4
text and formatting feed
dad 1 b 1,5

dead 2 c 1
Schema Hierarchical to Visualization
transform 3 d 2,3,4
relational beef
… 4 e 2,4,5

… 5 f 2,5
Entity Categorization
extraction tokenization

Full-text queries:
SELECT * FROM myTable WHERE CONTAINS (TextCol, ‘d’); – returns rows
SELECT * FROM myTable CONTAINS (TextCol, ‘d’); – returns rows and scoring
SELECT * FROM myTable WHERE CONTAINS (TextCol, ‘a AND NOT b’); – Boolean
SELECT * FROM myTable WHERE CONTAINS (TextCol, ‘a NEAR b’); – proximity


In-database analytics

No compromise for complex analytics:
Ÿ  Basic to advanced analytical functions available to SQL
Ÿ  Data never leaves the database until results are materialized
Ÿ  Analytics code and models are shareable
Ÿ  Analytics code and models are applicable to the latest data set
Ÿ  Average developer can build in database analytical models

Process in SAP Sybase IQ Database =
logic and filtering

Built-in functions External DLL “A” applied in database

External DLL “A” Analytics simplified: Logic to data = fast and efficient


Federation
With external file systems (Hadoop distributed file system)

1.
Client-side federation: Join data from SAP Sybase IQ and
Hadoop at a client-application level

Load Hadoop data into column store of
2. ETL
SAP Sybase IQ: Extract, transform, and load data from Hadoop
distributed file system (HDFS) into schemas of SAP Sybase IQ

Join HDFS data with data of SAP Sybase IQ on the fly: Fetch
and join subsets of HDFS data on demand, using SQL queries
3. from SAP Sybase IQ (data federation technique)

Combine results of Hadoop MapReduce (MR) jobs with SAP
Sybase IQ data on the fly: Initiate and join results of MR jobs
on demand using SQL queries from data in SAP Sybase IQ
4. (query federation technique)


Native MapReduce
Highly distributed processing without Hadoop

SELECT (Reducer… (Mapper… OVER PARTITION BY…) OVER PARTITION BY…)

… …
Parallel mapper TPFs Parallel reducer TPFs

Storage Fabric

•  TPFs (Table Parameterized Functions) consume/produce data sets in bulk
•  TPFs run in parallel
•  TPFs are fed with disjoint data sets
•  TPFs can be arbitrarily nested to multiple levels via sub-queries
•  TPFs currently available in popular, performance efficient C++


SAP Sybase IQ: Continuing Innovation

IMPORTANT LEGAL DISCLAIMER CONCERNING PROGRAM DATES, RELEASE-RELATED INFORMATION & CONTENT

All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

SAP Sybase IQ: Next Wave
Innovations for extremely large databases (XLDB)

Storage Architecture Loading Engine
•  New generation column store •  Fully parallel bulk loading
•  New partitioning and compression •  Real-time loading into delta store

Petabytes SAP Real-time
Sybase IQ:
Next Wave

System Reliability Query Processing
•  Grid resiliency •  Data affinity
•  Data availability •  Aggressively parallel and distributed


SAP SYBASE IQ
A COMPREHENSIVE PLATFORM FOR BIG DATA ANALYTICS

Sybase
PowerDesigner,
Bradmark,
SAS,
SPSS,
KXEN,

Sybase
Replica9on
Server,
Symantec,
Fuzzy
Logix,
BMMSoZ,

SAP
BusinessObjects
Whitesands,
Zemen9s,
Visual
SOLIX,
PBS

ISYS,
Panop9con
Quest,
ZEND
Numerics

Eco-System Op9mized
BI,EIM,

Dev
and
admin
tools
Predic9ve
Analy9cs

Packaged
ILM
apps

Model,
Replicate

App. Services
Hadoop,

DBMS R

Comprehensive
Built-‐in
Full
InDB
Analy9cs
w/
Big
Data

Web
2.0
APIs

ANSI
SQL
w/OLAP
Text
Search
MapReduce
+
simulator
OpnSrc

APIs

Most
mature
Comprehensive
MPP
queries
+
Virtual
High
Speed
Structured
+

column
store
lifecycle
9ering
Marts
+
User
scaling
loads
Unstructured
Store


© 2012 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects
purpose without the express permission of SAP AG. The information contained Explorer, StreamWork, SAP HANA, and other SAP products and services
herein may be changed without prior notice. mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP AG in Germany and other countries.
Some software products marketed by SAP AG and its distributors contain
proprietary software components of other software vendors. Business Objects and the Business Objects logo, BusinessObjects, Crystal
Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Objects products and services mentioned herein as well as their respective logos
Microsoft Corporation. are trademarks or registered trademarks of Business Objects Software Ltd.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, Business Objects is an
System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, SAP company.
zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other
Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, Sybase products and services mentioned herein as well as their respective logos
POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,
are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP
BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF,
company.
Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere,
Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM All other product and service names mentioned are the trademarks of their
Corporation. respective companies. Data contained in this document serves informational
purposes only. National product specifications may vary.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
The information in this document is proprietary to SAP. No part of this document
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or
may be reproduced, copied, or transmitted in any form or for any purpose without
registered trademarks of Adobe Systems Incorporated in the United States and/or
the express prior written permission of SAP AG.
other countries.
Oracle and Java are registered trademarks of Oracle.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and
MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®,
World Wide Web Consortium, Massachusetts Institute of Technology.


The Universe
of Big Data

The
Bloor
Group

In marketing terms
BIG DATA
is as big a trend as
cloud computing

(if you measure the trend in terms of column inches)

The
Bloor
Group

The Big Data Trend
q  Corporate data volumes
grow at about 55% per
annum
q  VLDB volumes grow at
about 55% per annum
q  This is exponential
q  Data has been growing
at this rate for at least
20 years
q  As such there is nothing
new about big data
other than the current
data volumes which
follow a well established
trend

Twitter Tag: #briefr The
Bloor
Group

So What s New?

q  Volume, velocity, variety, verifiability and other
words beginning with V - but not all at once
q  Hadoop is new
q  Big Data in the cloud is new
q  And there’s a new dynamic in data analytics
q  Volume (and velocity) is now mostly about events,
not transactions, and the world of embedded
processors is going to expand the number of
events worth processing

The
Bloor
Group

The Analytics Two-Step

The
Bloor
Group

The Future?

q  The data growth trend is likely to continue
q  More and more companies will be drawn into using
Big Data technologies

q  Will the two-step become a one-step? Not sure. If
you gather Big Data, you also need to be able to
throw it away
q  RDBMS (column store) will remain as the analytics
engine

The
Bloor
Group

!  Please explain why you believe that the Sybase IQ
shared some things architecture is equal to or
better than a shared nothing architecture?

!  Are you seeing the same trend that I seem to be
noticing with Big Data in respect to analytics?

!  Roughly how many of your customers are using
Hadoop?

!  If I were a Sybase customer would you recommend
Hadoop as an ETL mechanism or is it your view that
Sybase IQ can do it all?

The
Bloor
Group

!  Please describe the most extensive use of Sybase
IQ (in respect of data volumes, daily ingest,
instances, etc.).

!  How difficult is it to use (in other words, what are
the labor/DBA overheads compared to a traditional
RDBMS)?

!  Are your competitors always the usual
suspects (i.e. other column store products)? Do
you ever compete with the NoSQL crowd?

!  Explain how you usually fit with HANA in sites
where both products are in use. Is HANA promoting
sales of Sybase IQ?
The
Bloor
Group

!  This Month: Database
!  November: Cloud
!  December: Innovators
!  January: Big Data
!  2013 Editorial Calendar
(www.insideanalysis.com)


The New Possible: Very Big Data for Serious Business Value

Recommandé

Recommandé

Contenu connexe

Plus de Inside Analysis

Plus de Inside Analysis (20)

Dernier

Dernier (20)

The New Possible: Very Big Data for Serious Business Value