By Mark Gschwind, VP Business Intelligence > A recent survey by Information Week found that data quality is the greatest barrier to BI adoption in enterprises. MDS addresses this challenge with modeling, validation, alerting and security capabilities.
In this presentation, you will learn how to use MDS to model your data to ensure correctness, update it with changes from your ERP, and alert users to data conditions that require attention. Next you will learn the capabilities of DQS and see how it addresses data standardization, completeness and uniqueness challenges.
You will then see how to use them together to enable Enterprise Information Management. BI professionals will come away with knowledge on how to use tools that address the greatest risk to success for BI projects
Mark Gschwind is an expert on Data Warehousing, OLAP, and ERP migration. He is VP, Business Intelligence at DesignMind in San Francisco. Prior to that he was with AMB Property Corporation. He has authored three successful enterprise data warehouses and over 80 OLAP cubes for 46 clients in a wide range of industries. Certified in SQL Server and Oracle Essbase. MBA in Finance from Duke University, BBA from the University of Notre Dame.
Microsoft SQL Server Master Data Services DesignMind
1. Master Data and Data Quality
Management in SQL Server 2012
Mark Gschwind
VP, Business Intelligence
DesignMind
2. Mark Gschwind
VP of Business Intelligence at DesignMind
PASS member for over 10 years
BI Consultant since 1995
BI implementations for over 50 clients
Data Warehousing/Cubing/Reporting/Data Mining/EIM
MCP, certified in Oracle Essbase
Working with clients on EIM since 2008
mgschwind@designmind.com
find me on
3. DesignMind
Microsoft Gold Certified Partner
San Francisco based, 25 people, 3 MVPs
Capabilities include .NET Development, SharePoint, SQL
Server, and Business Intelligence
Data Warehouses, Reporting, Analytics, Dashboards,
Mobile, EIM
Focus on delivering value from BI using Agile
Methodology
www.designmind.com
4. Agenda
Enterprise Information Management (EIM)
What is it and why do we need it?
Microsoft EIM, 3 technologies working together
DQS
• Capabilities
• Demo
SSIS
MDS
• Capabilities
• Demo
EIM=DQS+MDS+SSIS
Wrap up
Questions
9. DQS: What is Data Quality?
Data Quality represents the degree to which the
data is suitable for business usages
Data Quality is built through People + Processes +
Technology
Bad Data Bad Business
“Poor data quality can cost companies 15%
to 25% (or more) of their operating budget”
- Larry English (International Data Quality Expert)
10. Common Data Quality Issues
Data Issue Sample Data Problem
Quality
Standard Are data elements consistently Gender code = M, F, U in one system and
defined and understood? Gender code = 0, 1, 2 in another system
Complete Is all necessary data present? 20% of customers’ last name is blank,
50% of zip-codes are 99999
Accurate Does the data accurately A Supplier is listed as ‘Active’ but went out of
represent reality or a verifiable business six years ago
source?
Valid Do data values fall within Salary values should be between
acceptable ranges? 60,000-120,000
Unique Data appears several times Both John Ryan and Jack Ryan appear in
the system – are they the same person?
11. Common DQ Issues Illustrated
Name Gender Street House # Zip code City State D.O.B
Before John Doe Male 60th street 45 New York New York 08/12/64
Jane Doe Male Jonathan ln 36 10023 Poughkeepsy NY 21-dec-1954
Name Gender Street House # Zip City State D.O.B
code
John Doe Male E 60th St 45W 10022 New York NY 08/12/64
After
Jane Doe Female Jonathan 36 10023 Poughkeepsie NY 12/21/54
Lane
Completeness Accuracy Conformity Consistency Uniqueness
Name Address Postal Code City State
Before John Smith 545 S Valley View Drive # 136 34563 Anytown New York
Margaret & John smith 545 Valley View ave unit 136 34563-2341 Anytown New York
Maggie Smith 545 S Valley View Dr Anytown New York
John Smith 545 Valley Drive St. 34253 NY NY
Name Address Zip Code City State Cluster
After John Smith 545 S Valley View Drive # 136 34563 Anytown New York 1
Margaret & John smith 545 Valley View ave unit 136 34563-2341 Anytown New York 1
Maggie Smith 545 S Valley View Dr Anytown New York 1
John Smith 545 Valley Drive St. 34253 NY NY 2
12. DQ Use Cases
• One-Time cleanups
o Merge/Migrate multiple divisional CRMs into one
• Continuous Process with Steward Intervention
o Vendor master with continuous trickle of data
o Customer master with incomplete data
• Continuous Process with Minimal Intervention
o Database marketing mailing list
13. DQS Process
Knowledge
Management
Build
Knowledge
Base
Use
16. MDS: What is Master Data?
Continuous quality management
Ease of use for business users (not just IT)
Effective sharing (producing and consuming)
Centralized maintenance, by different departments
Changes that keep pace with the business
Master Data contains different attributes for
different departments (marketing, finance,
operations, business groups…)
The challenge: To make a trusted single source
of business data used across multiple systems,
applications, and processes
17. MDS Use Cases
Regulatory Data Warehouse / Operational Data
Data Marts Mgmt Management
Enable security Enable business users to Central data records
management and auditing manage the dimensions mgmt and consumption
of data used for and hierarchies of DW / sourced by other
regulatory reporting Data Marts operational systems
The IT department has built a A company has adopted 6 “best
There are 3 G/L systems of breed” systems from
data warehouse and reporting
whose G/L accounts need to
platform, but business users different vendors. They need
be consolidated and rolled up
complain about the to be able to propagate the
to create financial statements
correctness of the dimensions correct customer information to
for regulatory reporting to
and lack of agility in making each system in a consistent
several countries
updates. way.
MDS enables an approval MDS provides a platform for
MDS empowers the
process for changes with
business users to manage central schema, integration
role-based security and
dimensions themselves points and validation for
transactional auditing of all
while IT can govern the SI/ISV/Internal IT to develop a
changes custom solution
changes
19. MDS Capabilities
Modeling Validation
Authoring business rules
Entities, Attributes, to ensure data
Hierarchies correctness
Master Data
Data Matching
Role-based Security and Stewardship
(DQS Integrated)
Transaction Annotation
Excel Add-In Web UI
Versioning
Enabling Integration & Sharing
Loading batched Registering to Consuming data Workflow /
data through changes through through Views Notifications
Staging Tables APIs
External
Excel DWH
(CRM, ..)
20. MDS Architecture
WEB-UI Excel Add-In
Silverlight
WCF
BizTalk / Others
Workflow /
MDS Service
Notifications
CRM/ERP
IIS Service
DWH SSIS
BI
OLAP
SSIS
Excel Subscription Entity Based Cleansing and
PW Views Staging Tables Matching
Pivot MDS Database SSIS
(DQS)
External System
External
System
22. Business Rules
Business Rules are expressions and actions that
can govern the conduct of business processes*
Enable data governance by:
-- Enforcing data standards
-- Alerting users to data quality issues
-- Creating simple workflows
Have limitations, but can be extended
*EIM = DQS+MDS+SSIS+People+Process
23. Security
Functional area permissions
Model/Entity level permissions provide column-
level security
Hierarchy permissions allow row-level security
Use AD groups, not individual users
Only use Hierarchy permissions if row-level
security is required
24. DATA QUALITY MASTER DATA
SERVICES SERVICES
INTEGRATION
SERVICES
25. Key Takeaways
SQL Server has tools to address EIM, the biggest
impediment to BI success
EIM is People + Processes + Technology