Agilität, Cloud, Offenheit sind nur einige wichtige Anforderungen, die an moderne Data-Warehouse-Systeme gestellt werden. Lange Zeit stand SAP mit ihrer Lösung nicht für diese Art, ein Data Warehouse zu bauen. Aber gilt das noch?
Hier zeigen wir Ihnen, wie ein SQL Data Warehouse auf der HANA-Plattform aufgebaut wird, wie es im Kern funktioniert und welche Entwicklungswerkzeuge genutzt werden, um Ihre Anforderungen umzusetzen. Dabei lernen Sie die Stärken des SAP-Ansatzes anhand eines realen Kunden-Beispiels kennen.
1. Warum SAP HANA SQL Data Warehousing?
KOSTENLOSES LIVE-WEBINAR
2. INHALTE Warum HANA SQL Data
Warehousing?
HANA als Plattform nutzen
SAP HANA im DWH-Kontext
Werkzeuge des HANA SQL DWH
Praxisbeispiel HANA SQL DWH
ISR Information Products AG | isr.de2
4. ISR Information Products AG | isr.de4
Model-driven Ansatz über alle Ebenen
Conceptual Model
Business Model
Source
Physical Model
Virtual Analytical Model
Physical Model
Core Data Vault
Physical Model
Data Aquisition
Physical Model
Sources
mapping
generate
reverse
engineer
5. ISR Information Products AG | isr.de5
Agilität durch DevOps und Kontinuitätsprozesse
Continuous Feedback
ReleasePlan
Model
Develop Operate
Deploy
DESIGNTIME RUNTIME
6. ISR Information Products AG | isr.de6
Plattformübergreifend durch Cloud Readiness
Containerisierung der
Transporteinheiten
Infrastruktur
unabhängige MTAR
Transporteinheit
MTA Applikationen
aus skalierbaren
Mikro-Services
7. SAP HANA PLATFORM
Source Code Management Build ServerIDE
Modelling OperationETL
DB XSA HDI
>_
SAP Web IDE
Eclipse
VS Code
Bitbucket
GitLab
GitHub
Jenkins
Bamboo
Travis
SAP SDI
Talend
Informatica
SAP PD
Visio
Erwin
SAP DWF
Automic
Munin
ISR Information Products AG | isr.de7
Offenheit für die Wahl der Entwicklungstools
8. 8
HANA Plattform
Data Warehouse
• Green Field
• Hybrid Scenario
• Migration und Konvertierung
Customer-Applikation
• SOA-Architecture
• XS Service Migration
• End-User Applikations
Company -Services
• Operative Systems
• Reporting Apps
• SAP Analytics Cloud
SAP HANA XSA: SAP HANA Extended Applikation
Services Advanced
• Applikationsserver innerhalb der HANA DB
• Unterstützt unterschiedliche Entwicklungssprachen,
wie NodeJS, Java and SQL
• Managet die individuellen
Entwicklungsumgebungen in Sandboxen
• Stellt die Applikationsplattform dar
• Entkoppelung der Infrastruktur und Entwicklung
• Open Source Basis, Public Version ist Cloud Foundry
ISR Information Products AG | isr.de
ENTFESSELUNG DER HANA DATENBANK MIT XSA APPLIKATIONSSERVER
9. ISR Information Products AG | isr.de9
HANA Plattform
Zeit und
Ortsunabhängige
Zusammenarbeit
Geteiltes Code
Repository
Continuous
integration / delivery
DIE ZENTRALEN VORTEILE VON XSA IM ENTWICKLUNGSPROZESS
10. Aktueller Data-Warehouse-Kontext
ANFORDERUNGEN UND KNACKPUNKTE EINES DATA WAREHOUSE
Analysis Planning Reporting Predictive
ESTABLISHED BUSINESS INTELLIGENCE
Economically oriented Production oriented
Data MartData Mart
Virtual Data
Mart
DATA WAREHOUSE - CONSISTENT TECHNOLOGY AND METHODS
Purchasing Logistics Production Sales & Marketing
Value Creation
Big Data
Machine Data Sensor DataSD
FI / CO
PP
MM
Manufacturing Execution
System
CUSTOMERS
11. ISR Information Products AG | isr.de11
Entwicklungswerkzeuge für ein HANA SQL DWH
ENTWICKLUNGSWERKZEUGE IN AKTION
In WebIDE:
• Create SDI Flowgraphs
• Create Calculation Views
In PowerDesigner:
• Create Data Models
(Business Model, Data Vault Model)
• Generate Objects
2
3
HANA
HANA Database
Git Jenkins (CI/CD)
SAP Analytics
Cloud
XSA / Cloud Foundry
CREATE
MODELS
PROVIDE
SOURCE CODE
DEPLOYMENT
INSTRUCTIONS
DEPLOYMENT
TRACK
DEVELOPMENT
TRACK
DEPLOYMENT
SAP WebIDE
Source Systems
SAP Smart Data
Integration
CONNECT
SOURCES
EXPORT
MODELS
WORK WITH
SOURCE CODE
FRONTEND
ACCESS
JSON Flatfile
DATA
IMPORT
Further formats
3rd Party Frontend
1
4
Issue Management Tool
SAP PowerDesigner
12. BERICHT AUS DER PRAXIS
Start Punkt
Quellsysteme SAP BW OSCARE, SAP ERP, SAP CRM welche in
einem DB2-DWH integriert wurden, soll durch eine neue
moderne und agile Architektur für die Zukunft ersetzt
werden
ISR Lösung
• Bewertung der DWH- Varianten HANA SQL DW, BW/4
HANA und DB2 Blu anhand unterschiedlicher Use Cases
• Architektur – Empfehlung HANA SQL DWH
• Detailliertes Konzept für eine Zielarchitektur auf der
Hana Plattform
• Durchgängige Modelgetriebene & Agile
Entwicklungsmethodik nach DevOps und Scrum
• DWH Automatisierung & Test Automatisierung
• Aktuelle in der Entwicklung/ Projektstart 2017
Technologien
• HANA 2.0, HANA XSA, Jenkins, Git, Java, Node.js
1 PHASE: HANA SQL INTEGARTION ARCHITEKTUR+ IMPLEMENTIERUNG
ISR Information Products AG | isr.de12
AUSZUG
ONLY
VIRTUAL
CRM / ERP / OSCARE BW / Flatfiles
SINGLE
PERSISTENCE
Data Vault
DV4HANA
Data Marts
DB DB DB …
Micro
Strategy
SAPHANAPlattformSourceSystems
Harmonization
Business
Apps
13. BERICHT AUS DER PRAXIS
Start Punkt
• Cloudbasierte mobile Anwendung welche Kundenspezifische und
Businessdaten bereitstellt
ISR Lösung
• Lösungsdesign und Implementierung
• Business and Technische Konzeption
Microbasierte Service Anforderungen
• SAP HANA Platform mit XSA Services (Odata, ActiveMQ, etc.)
• Synchrone und asynchrone Kommunikation
• Realtime Replication und Transformation von SAP Data
• Implementierung eines Node.js Producer welcher JSON-Nachrichten
asynchron zu Apache Kafka sendet
• Stabilität, Generisch, Ausfallsicher, Lastverteilung
Technologien
• HANA 2.0, HANA XSA, Jenkins, Docker, Java, Node.js, Apache Kafka
2 PHASE: MICROSERVICE ARCHITEKTUR (1 MIO KUNDEN) HANA SQL PLATTFORM, XSA APACHE KAFKA
ISR Information Products AG | isr.de13
CRM / SQL DWH / BW
XSA
SAP HANA
Persistence
REST API Producer
Integration
DB DB DB …
REST APIREST API
Consumer / App.
DMZ–24/7SAPDomain
Pull (https) Push (JSON)
Replication
Odata
AUSZUG
14. BERICHT AUS DER PRAXIS
ZAHLEN, DATEN, FAKTEN
ISR Information Products AG | isr.de14
AUSZUG
+ 35 Stories + 3 Applications + 5 Sprint teams + 3 Week Sprints
+ 10 Repositories < 4700 Merges < 280000 Commits
+ 40 Developers + 55 Business + 6 Scrum
+ 5800 Builds + 4900 Objects + 150 Releases
+ 10 Sourcesystems + 22 Services > 2 B Datasets
+ 5 Systems + 12 Tennants + 15 XSA Spaces
SEIT2017
16. ISR Information Products AG | isr.de16
WIE GEHT ES WEITER?
SAP Schulung - SAP HANA SQL Data Warehousing HDW410
zur Schulung
17. Ich freue mich
auf Ihren Anruf.
ISR Information Products AG
Am Mittelhafen 14 | 48155 Münster
ISR Information Products AG | isr.de17
DOMINIK FISCHERSenior Consultant | SAP Information Management
Agile & Flexible:
Quick Time to market
Short Time to value / Shorten innovation cycle with faster development and deployment process
Reduced complexity
Increased development performance
High Quality over time as well
Adaptability
Using automation to harness advantages of automated testing
Fokus the work of developers on the critical tasks
Cloud Ready:
Flexible deployment options (onPrem / Cloud)
Future proven architecture
Independency of technical basis
Model Driven:
Communication basis with the business
Modelling the Business*
Transparency
Industry standardized notations
High Quality
Consistent meta data
Industrialize data warehouse development
Openess:
War of talents
Sharded work repository for development objects
Parallel development
Openess of development tools
High Quality over time as well
1. Aggregated CDM: Conventionally, modelling starts with the aggregated CDM that gives an overview of the content that is supposed to get integrated into the DWH (big picture). Only few information is necessary:
Which entities are in the scope of the project?
What is the business-related definition of the entity?
Which connections exist between the entities?
What is the Business Key?
Does the subsequent modelling of an object take place top down or bottom up?
A Conceptual Data Model (CDM, also called Business Model) is created as an Entity Relationship Model (ER Model). It contains the entities, the attributes of the entities and the relationships between the entities. As it is an abstract, business-oriented model the CDM also contains the definitions for the business relevance of the entities and attributes. Therefore it also serves as the basis for the Business Glossary. For the reference Data Warehouse architecture, we start with an aggregated CDM. This is a CDM that only contains the entities, the respective business key attributes and the relationships between the entities.
2. Utilizing top down modelling, the detailed CDM is derived from the aggregated CDM. It is less abstract and includes all attributes and all data domains (data types and lengths). However, it does not yet contain database-specific objects (e.g. views, indexes, etc.).
3. Based on the detailed CDM, a physical data model is created by the business intelligence architect. In our approach it is based on the Data Vault modelling method which will be discussed in more detail later. When the PDM is fully specified, it contains all database objects that are necessary for the implementation (e.g. tables, indexes, constraints, triggers, etc.)
4. Utilizing bottom up modelling, instead, the structure of the considered entity is adopted in the EAD from the source system via reverse engineering. This constitutes the basis for subsequent physical modelling.
5. Analogous to step 3, a physical data model is created by the business intelligence architect.
6. In the raw data warehouse (RAW) and in the business integrated data warehouse (BID), information is technically modelled and not supposed to be accessed by end users. Only when information reaches the analytical layer, the technical view gets transformed into a business view. This is achieved through Calculation Views which are modelled in a physical model in SAP HANA EAD. Technologically, these Calculation Views represent the analytical objects from a business view. For every entity modelled in the CDM a Calculation Views is built in the analytical layer. Consistency between CDM and analytical model is assured.
7. A number of analytical objects can be modelled to Cubes (multidimensional objects) using transactional data. These cubes are represented on SAP HANA by a star join. Cubes are also modelled in a physical data model in SAP HANA EAD.
It is important that different data models know each other and that the objects inside the models are related. This prevents the loss of transparency in large data warehouse applications and ensures a good overview at all times. Also, this constitutes the basis for Data Lineage.