Data Warehouse migration to Azure, PoV for the Retail industry, and what's next, it's Customer Analytics

1. October 2018
PoV – EDW Migration to Azure

2. Migration to Azure Cloud
Introduction
Azure SQL Data Warehouse is a cloud-based Enterprise Data Warehouse (EDW) that leverages Massively Parallel Processing (MPP) to quickly run
complex queries across petabytes of data. Its columnar storage in relational tables significantly reduces data storage cost and improves query performance.
It can run analytics near real time at massive scale using Azure Databricks streaming Dataframes.
SQL Datawarehouse uses PolyBase (T-SQL compliance) to query data from big data sources. Azure SQL DW migration provide utilities/services like Azure
Data Factory, Data Migration Assistant, SSIS and data migration service to make migration more streamlined.
Enablers in Retail
Azure is preferred by many Retailers as it isn’t viewed as a competitor but an enabler for digital transformation, providing more regions than other Cloud
providers, one of the best platforms with TCO, easy to use IoT ecosystem, strategic partnerships for data lifecycle such as Snowflake, strong AI / ML
capabilities, and greater control to build custom applications. TBD (i.e. advantages such as TCO, accelerators (ML frameworks, data pipeline frameworks,
data visualization)
• 10x - increase in the number of data sets that can be effectively handled
• 1 day to 15 minutes – develop granular data analytics reports
• $800 k-cost efficiencies from data analytics resulted in significant annual savings
• 158% - Average ROI for customers who modernized with Azure SQL DW
• $533K – In annual savings from enhanced IT team productivity
• $1 M + - Less per year thanks to simplified DW deployment and management
• $120K - In saved data replication costs by moving failover to Azure SQL DW
• $100K – Cost of backup DR data warehouse that was avoided with Azure SQL DW
• Fewer vulnerabilities - With Always Encrypted and standard endpoints
*Customers who modernized with Azure SQL DW…
*Reference: Forrester TEI Commissioned By Microsoft December 2017; https://pdfs.semanticscholar.org/presentation/318e/d2faf8df5c441637a3000cfa74f50cbb57cd.pdf

3. Azure DW Reference Architecture
Storage
Process Orchestration Data Governance
Job Scheduler
Workflow ADF
Azure Data
Catalogue
Data Profiling
Data Lifecycle
Metadata
Management
Audit
User roles and
Security
PII information
Compliance
ADF
Logic Apps
Event Hub
Data Producers / Source Systems
Structured Data
Merchandise
Planning
Sales
Master Data
Store Profile
• Supply Chain
• POS
• Sales
• Marketing
• Customer
Experience
Data
Acquisition
Batch
Integration tools
Native connectors
Real
time/Near
Real time
Unstructured Data
Logs
• Sensor Data
• Social Media
• Emails
• Clickstreams
External
Data
Source EDW Systems
Data Processing
Batch Realtime Advanced Analytics Layer
Join
Calculate
Aggregate
Azure Data Factory
Logic Apps
Stream analytics
Parse Validate
Cleanse
Transform
Semantic Analytics Layer
Pattern Mining
ML workflows
Classify
Analyze
Predict
Prepare
Train
Correlate
Data Storage (Target EDW)
Azure SQL DWH
Staging
Dynamic Layer
Azure HD Insight
Aggregated
Data Store
Data
Distribution
ContentDeliveryDataAbstractionAPIGateway
Data Consumption
(BI Tool)
Big Data
Connections
Data
Federation
Self Service Reports
and visualizations
Customer
360
Operational
Reporting
Next Best
Action
Churn
Propensity
Event Hub
Azure Blob
Storage
Azure Databricks
Azure ML
Deep Learning
Cognitive AI
Azure Analysis
Services
Azure AI
services
PolyBase
Azure AD
ADF
Event Hub
ETL
Logic Apps
Retail Analytics

4. EDW to Azure – Migration Strategies & Decision tree
Current
State
Lift and
Shift
Review &
Refine
Rearchitect
Data Models &
Taxonomy
MOM maturity
Valuation
Data Characteristics
(Quality, Volumes)
Sharding
Workload
Characteristics
Optimized
data
Needs
Optimization
Flawed
Data
By User Groups
By Pain Points
Batch
Real Time /
Near Real
time
Structured
Semi Structured
Unstructured
Current State Methodology Criteria Migration
Strategy
Mode Data Types
Schema Data / Tables
Logic Apps
Event Hub
Azure Data
Factory
Data Transfer
Data
Movement
ETL Tool
CDC
Metadata
Remodel
Aggregate
Join
Transform
Replicate
Azure
Functions
Stream
Analytics
Blob
Storage
Azure
SQL
DWH
PolyBase
Test
Validate
Operationalize
Migration Paths
Migrate

5. Retail eCommerce Value Chain
Advanced retail analytics
solutions spanning descriptive,
predictive, & prescriptive
modeling accelerating ROI

6. www.ness.com
Sanjay.Bhakta@ness.com
Head of Solutions Architecture, North America
www.ness.com