Contenu connexe Similaire à The SAP Startup Focus Program – Tackling Big Data With the Power of Small by Soenke Moosmann (20) The SAP Startup Focus Program – Tackling Big Data With the Power of Small by Soenke Moosmann1. The SAP Startup Focus Program –
Tackling Big Data With the Power of Small
Marcus Krug and Sönke Moosmann, SAP Innovation Center
2. © 2013 SAP AG. All rights reserved. 2Public
Agenda
SAP Innovation Center
Why Startups Matter To Us?
SAP Startups Focus Program (SFP)
4. © 2013 SAP AG. All rights reserved. 4Public
Who We Are
Established in February 2011
First of its kind for SAP AG
Now: 35 FTEs
Soon: 100 FTEs plus 200
students
5. © 2013 SAP AG. All rights reserved. 5Public
Looking Beyond ERP
Personalized
Medicine
Online
Gaming
Film
Production
Smart
Energy
Supply-
Chain
Innovation
Technologies
In-Memory-
Technology
Cloud
Mobility
7. © 2013 SAP AG. All rights reserved. 7Public
Some Might See Us as…
Grow old
Slow
BUT TURTLES ARE
ALSO…
Quite „fundamental“,
according to ancient
Asian mythology
8. © 2013 SAP AG. All rights reserved. 8Public
Commonalities - Innovation….
Business
(viable)
Technology
(feasible)
Innovation
Human Values
(desirable,usable)
9. © 2013 SAP AG. All rights reserved. 9Public
BIG Data
Video
Audio
DemandContent
GPS
Customer Data
ServiceCalls
Emails
Virtual Goods
Social MediaMobile
InstantMessages
SAP HANA
10. © 2013 SAP AG. All rights reserved. 10Public
BIG Ambitions
SAP Strategic Goals 2015
20 Bn in Revenue
35% Margin
1,000,000,000 Users
→ Dramatically extend SAP‘s ecosystem
And if we join forces…
11. © 2013 SAP AG. All rights reserved. 11Public
This is What Will Happen…
13. © 2013 SAP AG. All rights reserved. 13Public
Access…
1. Technology
Many startups face „big data“ and „real-time“ challenges → SAP
HANA
2. Customers
SAP Install Base of close to 200,000 customers across 25 industries
3. Financing
SAP Ventures
14. © 2013 SAP AG. All rights reserved. 14Public
SFP – All About Access
Technology
SAP HANA One Developer Edition (free) → 1 year per default (flexible)
Physical and virtual HANA bootcamps, HANA virtual learning platform
technical advisor
Joint Go-to-Market
Appearances at SAP and non-SAP events
Solution showcases on HANA marketplace
Dedicated GTM advisor → GTM plan
Pipeline creation to drive SFP startups‘ revenue
Access to SAP Ventures and other VCs
SAP Ventures and other VCs (155 Mio $ HANA Real-time Fund)
15. © 2013 SAP AG. All rights reserved. 15Public
SFP – From Idea to (Market) Impact
Attend Startup
Forum
Development
Accelerator GTM
Boot Camp
SFP ≈ 1 year ≥1 year: commercial state
Market-ready
solution
SFP selection
Pitch at Forum
16. © 2013 SAP AG. All rights reserved. 16Public
SFP – Then, Now and Beyond
March - May 2012
- The first 10
startups
- Recruited from
friends + family
- High touch
March – Sept 2012
- From 10 -100
- startup forums
held globally
- HANA boot camps
- HANA developer
edition on AWS
Sapphire EMEA
2012
- ≈150 startups
- 50 with Proof-of-
Concept
2013 – Transition
to Scale!
As of now:
- 200+ startups in
SFP
- 60+ productive
solutions
Goals 2013
- 1000+ startups in
SFP
- 200+ productive
solutions
17. © 2013 SAP AG. All rights reserved. 17Public
Some References
(Israel)(US) (US)
(France)
(UK)(UK)(Israel)
(US)
(Canada)
(US) (US) (Germany)
etc….
19. © 2013 SAP AG. All rights reserved. 19Public
The Microscope :: A Tool for Biological Exploration
Before the invention of the microscope
Difficult to study tiny structures
Only models, hypotheses about
Cells
Micro-organisms
Difficult to verify / falsify hypotheses
After the invention of the microscope
Tiny structures are plain to see
Can be studied in real time
21. © 2013 SAP AG. All rights reserved. 21Public
Data Challenge
CRM* data
GPS
Demand
Speed
Velocity
Transactions
Opportunities
Servicecalls
Customer
Sales orders
Inventory
E-mails
Tweets
Planning
Things
Mobile
Instantmessages
VELOCITY
VOLUME VARIETY
22. © 2013 SAP AG. All rights reserved. 22Public
Algorithmic Challenge
Challenges
Forecasting
Key
Influencers
Trends
Anomalies
Relationships
23. © 2013 SAP AG. All rights reserved. 23Public
Presentation Challenge
24. © 2013 SAP AG. All rights reserved. 24Public
Application
Server
SAP HANA Overview
Predictive
analytics
Scripting
Data
Modeling
R Integration
Math
Libraries
Column and
row store
+
Multi-core/
parallelization
In-memory
Compression
SQL interface on
columns & rows
SQL
T
Text
Engine
26. © 2013 SAP AG. All rights reserved. 27Public
Optimizing Data Access Patterns
Challenge: Data locality!
Yes, DRAM is 100,000 times faster than disk…
But DRAM access is still 4-60 times slower than on-chip caches
27. © 2013 SAP AG. All rights reserved. 28Public
SAP HANA supports rows, but is optimized for column-order data organization
Order Country Product Sales
456 France corn 1000
457 Italy wheat 900
458 Italy corn 600
459 Spain rice 800
Column and Row Store
456 France corn 1000
457 Italy wheat 900
458 Italy corn 600
459 Spain rice 800
456
457
458
459
France
Italy
Italy
Spain
corn
wheat
corn
rice
1000
900
600
800
Row order organization
Column order organization
Single-record access:
SELECT * FROM SalesOrders
WHERE Order = ‘457’
SQL
Single-scan aggregation:
SELECT Country, SUM(sales) FROM
SalesOrders WHERE Product=‘corn’ GROUP BY
Country
28. © 2013 SAP AG. All rights reserved. 29Public
Combining OLTP and OLAP
Write operations are accumulated
in a dedicated data structure (delta
store)
Write operations are insert-only!
Integration of differential data in async.
merge process.
MVCC enables processing of
OLTP workloads
Insert only approach favors implementation of
MVCC
Main Memory
at Blade i
Log
SnapshotsPassive Data (History)
Non-Volatile
Memory
RecoveryLogging
Time
travel
Data
aging
Query Execution Metadata TA Manager
Interface Services and Session Management
Distribution Layer
at Blade i
Main Store Differential
Store
Active Data
Merge
Column
Column
Combined
Column
Column
Column
Combined
Column
Indexes
Inverted
Object
Data Guide
29. © 2013 SAP AG. All rights reserved. 30Public
Order Country Product Sales
456 France corn 1000
457 Italy wheat 900
458 Spain rice 600
459 Italy rice 800
460 Denmark corn 500
461 Denmark rice 600
462 Belgium rice 600
463 Italy rice 1100
… … … …
Columnar Dictionary Compression
Dictionary per column
Uses data-driven fixed-length bit encodings
Operations directly on compressed data, using integers
More in cache, less main memory access
1 Belgium
2 Denmark
3 France
4 Italy
5 Spain
1 3
2 4
3 5
4 4
5 2
6 2
7 1
8 4
… …
1 7
2 5,6
3 1
4 2,4,8
5 3
Logical Table
Dictionary
5 entries, so
need 3 bits to
encode!
Compressed
column
(bit fields)
Inverted
indexDictionary
Where was
order 460?
Which orders
in Italy?
30. © 2013 SAP AG. All rights reserved. 31Public
More Columnar Compression Techniques
31. © 2013 SAP AG. All rights reserved. 32Public
Concurrent users
Concurrent operations within a query
Data partitioning, on one host
or
distributed to multiple hosts
Horizontal and vertical
parallelization of a
single query
operation, using
multiple
cores / threads
Transparent to developer
Parallelization
Inter Transaction Intra Transaction
Inter Query Intra Query
Inter Operation Intra Operation
Pipeline
Parallelism
Data Parallelism
Pipeline
Parallelism
Data Parallelism
Parallelism
33. © 2013 SAP AG. All rights reserved. 34Public
Extending your analytics capabilities
ANALYTICS MATURITY
LEVELOFINSIGHT
Sense & Respond Predict & Act
Raw
Data
Cleaned
Data
Standard
Reports
Ad Hoc
Reports &
OLAP
Generic
Predictive
Analytics
Predictive
Modeling
Optimization
What happened?
Why did it happen?
What will happen?
What is the best
that could
happen?
34. © 2013 SAP AG. All rights reserved. 35Public
SAP HANA Modeling
Analytical View Attribute View Column Table
Calculation View
35. © 2013 SAP AG. All rights reserved. 36Public
Implementing Predictive Analytics
SQLScript
set of SQL extensions to push data-intensive logic into the database and leverage parallel
execution strategies of the database
PAL (Predictive Analysis Library)
Built-in C++ statistical and data mining algorithms
K-means, k-nearest neighbor, decision trees, multiple linear regression, classification, and many
more
R integration
Leverage R’s 3000+ external packages to perform wide-range data mining and statistical
analysis.
36. © 2013 SAP AG. All rights reserved. 37Public
Intuitively Design Predictive Models using Predictive Analysis
37. © 2013 SAP AG. All rights reserved. 38Public
SAP Predictive Analysis
39. © 2013 SAP AG. All rights reserved. 40Public
SAP HANA XS Engine
Rationale: Enable application development
and deployment – minimize layers
HTTP-based UI (browser, mobile apps)
Runs directly on HANA, minimizes TCO
Leverages built-in strengths of SAP HANA
for the best possible performance
Scope
From lightweight environment for small web-
based applications
To robust environment for complex high-
speed business applications
Control flow logic
Calculation logic
Data
Clients
Presentation logic
HANA
XS
40. © 2013 SAP AG. All rights reserved. 41Public
Implement UIs with SAPUI5
SAPUI5 is an extensible JavaScript-based HTML5
browser rendering library for Business
Applications.
Uses the jQuery library as a foundation
Open AJAX compliant and can be used together
with/uses other standard JS libs
Supports RIA like client-side features based on
JavaScript
Supports an extensibility concept regarding
custom controls
Allows usage of own JavaScript and HTML
Internet
Explorer
Version 9
Version 8
Chrome
Latest
version
Firefox
Version 3.6
and latest
version
Safari
Latest
version
41. © 2013 SAP AG. All rights reserved. 42Public
SAPUI5 Templates
42. © 2013 SAP AG. All rights reserved. 43Public
Demo :: In-Game Promotion Management
Bigpoint
Europe‘s largest browser game provider with 300 Mio
users
Revenue through selling „virtual goods“
1-3% of users are buying virtual offers.
Goals
Perform real-time massive amount of event stream
analytics
Filter and monitor online players
Enabling in-game promotion offers and track results in
real-time
43. © 2013 SAP AG. All rights reserved. 44Public
In Summary
In-memory data management
Column-oriented data layout
Compression
Parallelization
Optimized for big data
Transparent to developer
HANA Applications
XS Engine
Application Server
Control logic
SAPUI5
Reduced TCO
Predictive Analytics
SQLScript
R integration
Predictive Analysis
Library
Unlock new insights
44. © 2013 SAP AG. All rights reserved. 45Public
Take Your Chance!
The SAP Startup Forum is coming to
Berlin again!
When: June 19th
Where: SAP Office Berlin
(Rosenthaler Str. 30)
For more info, visit our event website:
http://www.saphana.com/community/learn/startups/forums/berlin
Or turn to us directly
Notes de l'éditeur Data ComplexityBig dataStructured and unstructuredNoiseHidden patternsAlgorithmic ComplexityMore complex information needs: Predictive, simulationsComplex workflows: provisioning data, cleaning, denoising, pattern miningPresentation ComplexityHow to present complex results in an intellible way Critical slide!!!Developing a database to solve these two critical challenges requires a careful design and development from the ground up of every aspect of the database. Relabeling an existing DB “in-memory” doesn’t do it. Carful optimizing for optimal cache utilization and for hundreds of parallel threads is what makes the difference, and allows HNA to reach the speeds I just discussed. I can’t over-emphasize hwo important solving these two challenges is to the performance of SAP HANA. By accessing data in column-store order, you benefit immensely from simplified table-scan and data pre-caching. This can make all the difference in performance. Intuitively design complex predictive modelsRead and write from data stored in SAP HANA, Universes, IQ, and other sourcesDrag-and-drop visual interface for data selection, preparation, and processing Visualize, discover, and share hidden insightsAdvanced visualization designed where you’d expect it – natively from within the modelling toolShare insights via PMML and with other BI client tools Predictive Analytics is a Goldmine for Startups (Forbes, March 2013)http://www.forbes.com/sites/martinzwilling/2013/03/11/predictive-analytics-is-a-goldmine-for-startups/ Predictive Analytics is a Goldmine for Startups (Forbes, March 2013)http://www.forbes.com/sites/martinzwilling/2013/03/11/predictive-analytics-is-a-goldmine-for-startups/