SlideShare une entreprise Scribd logo
1  sur  34
Best Practices:  Data Administration and Quality Daniel Linstedt, all rights reserved, http://LearnDataVault.com
Introduction and Expectations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Agenda ,[object Object],[object Object],[object Object],[object Object]
Defining Data Administration Issues
What is Data Administration? “ What do we mean by that in the case of data administration? We mean that DA must get out of the design review committee mentality and substitute something more value-added and flexible. It must recognize that systems tend to grow organically, and be a part of that process, rather than an instiller of order upon it.”  Eric Rawlins, 1995 Originally Published by: Database Research Group, Inc http://www.well.com/user/woodman/organic.html
The Role of Data Administration ,[object Object],[object Object],[object Object]
Cross-Organization Roles and Responsibilities Business ( Owner View) Data Steward Discipline Authority Business Process  Manager Data Usage Contact Data Manager Data Modeler DA is a  ROLE  and typically involves more than one person in order to achieve success. Logical (Designer  View) Data Administrator Physical ( Builder View) Database Administrator
Data Administrator Responsibilities ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Top 10 Data Administration Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Defining Data Administration Issues Top 4 Examples
Defining Master Metadata ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Defining Master Data Management ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Assessing Logical Model Viability ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Defining Business Process Models ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Applying Best Practices
Revealing the DA Best Practices ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
DA: MDM and Master Metadata ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Many times we see a cross-role responsibility of data management and data administration.  The cross-role is responsible for the following:
Work Breakdown Structure ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Organizational Breakdown Structure ,[object Object],[object Object],[object Object],[object Object]
DA: Architecting Data Governance Business Rules & IQ EDW Source Systems Non Compliant Data Marts Business Rules & IQ EDW Source Systems Data Marts Compliant Hard Business Rules Soft Business Rules & IQ  Shift  to process AFTER  the EDW Hard Business Rules Still process  Before the EDW
Establishing Auditable Sources Sync  Routines Data 2 nd  Source System Staging EDW Data Warehouse Source System Data Export Sync  Routines OLTP Oper Reports DW Exports ,[object Object],[object Object],[object Object]
DA – Defining Data Errors and Models ,[object Object],[object Object],[object Object],[object Object],B.I. Tool Database Wrtr xform Rdr ETL Load Process Source System Staging Area Data Warehouse Data Marts **Error Stage **Error Warehouse Error Marts ** Not usually implemented
DA Example –  Classifications of Errors ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Business Owns the Error I.T. Owns the Error
DA: Tracking Errors – KPIs at Work
Metadata and Data Administration ,[object Object],[object Object],[object Object],[object Object],[object Object]
Metadata Administration Lifecycle Identify New  Metadata Integrate With Master Metadata  Repository Edit and Manage Master Metadata (Provide Business Users  with Web Interface) Stitch  Master Metadata Together Compare Master Metadata With Business Process And Objectives Export Master Metadata or Deploy via SOA With Master Data Set Derived from Meta Integration Metadata Lifecycle
Monitoring DA Efforts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Establish KPIs for Each of the Following Areas
Case Study for DA Results ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],After Implementing DA Best Practices
Conclusions and Q&A
Revealing the DA Best Practices (Recap) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The Experts Say… “ The Data Vault is the optimal choice for modeling the EDW in the DW 2.0 framework.”  Bill Inmon ,[object Object],Stephen Brobst “ The Data Vault is a technique which some industry experts have predicted may spark a revolution as the next big thing in data modeling for enterprise warehousing....”  Doug Laney
More Notables… ,[object Object],Scott Ambler
Where To Learn More ,[object Object],[object Object],[object Object],[object Object]
Thank you Contact us today: Dan Linstedt [email_address] http://LearnDataVault.com

Contenu connexe

Tendances

Oracle 12c Multitenant architecture
Oracle 12c Multitenant architectureOracle 12c Multitenant architecture
Oracle 12c Multitenant architecturenaderattia
 
Data Privacy in the DMBOK - No Need to Reinvent the Wheel
Data Privacy in the DMBOK - No Need to Reinvent the WheelData Privacy in the DMBOK - No Need to Reinvent the Wheel
Data Privacy in the DMBOK - No Need to Reinvent the WheelDATAVERSITY
 
Snowflake Data Governance
Snowflake Data GovernanceSnowflake Data Governance
Snowflake Data Governancessuser538b022
 
Database Administration
Database AdministrationDatabase Administration
Database AdministrationBilal Arshad
 
Free Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseFree Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseDatabricks
 
Oracle RAC 19c and Later - Best Practices #OOWLON
Oracle RAC 19c and Later - Best Practices #OOWLONOracle RAC 19c and Later - Best Practices #OOWLON
Oracle RAC 19c and Later - Best Practices #OOWLONMarkus Michalewicz
 
Oracle data guard for beginners
Oracle data guard for beginnersOracle data guard for beginners
Oracle data guard for beginnersPini Dibask
 
Data Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookData Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookJames Serra
 
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best PracticesOracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best PracticesMarkus Michalewicz
 
Data Modeling for Big Data
Data Modeling for Big DataData Modeling for Big Data
Data Modeling for Big DataDATAVERSITY
 
Oracle Sql Developer Data Modeler 3 3 new features
Oracle Sql Developer Data Modeler 3 3 new featuresOracle Sql Developer Data Modeler 3 3 new features
Oracle Sql Developer Data Modeler 3 3 new featuresPhilip Stoyanov
 
Microsoft SQL Server internals & architecture
Microsoft SQL Server internals & architectureMicrosoft SQL Server internals & architecture
Microsoft SQL Server internals & architectureKevin Kline
 
Introduction To Data Vault - DAMA Oregon 2012
Introduction To Data Vault - DAMA Oregon 2012Introduction To Data Vault - DAMA Oregon 2012
Introduction To Data Vault - DAMA Oregon 2012Empowered Holdings, LLC
 
MySQL Data Encryption at Rest
MySQL Data Encryption at RestMySQL Data Encryption at Rest
MySQL Data Encryption at RestMydbops
 
Data Modeling & Metadata for Graph Databases
Data Modeling & Metadata for Graph DatabasesData Modeling & Metadata for Graph Databases
Data Modeling & Metadata for Graph DatabasesDATAVERSITY
 
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdf
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdfOracle_Multitenant_19c_-_All_About_Pluggable_D.pdf
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdfSrirakshaSrinivasan2
 
Oracle MAA (Maximum Availability Architecture) 18c - An Overview
Oracle MAA (Maximum Availability Architecture) 18c - An OverviewOracle MAA (Maximum Availability Architecture) 18c - An Overview
Oracle MAA (Maximum Availability Architecture) 18c - An OverviewMarkus Michalewicz
 

Tendances (20)

Oracle 12c Multitenant architecture
Oracle 12c Multitenant architectureOracle 12c Multitenant architecture
Oracle 12c Multitenant architecture
 
Data Privacy in the DMBOK - No Need to Reinvent the Wheel
Data Privacy in the DMBOK - No Need to Reinvent the WheelData Privacy in the DMBOK - No Need to Reinvent the Wheel
Data Privacy in the DMBOK - No Need to Reinvent the Wheel
 
Snowflake Data Governance
Snowflake Data GovernanceSnowflake Data Governance
Snowflake Data Governance
 
Data administration
Data administrationData administration
Data administration
 
Database Administration
Database AdministrationDatabase Administration
Database Administration
 
Free Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseFree Training: How to Build a Lakehouse
Free Training: How to Build a Lakehouse
 
Oracle RAC 19c and Later - Best Practices #OOWLON
Oracle RAC 19c and Later - Best Practices #OOWLONOracle RAC 19c and Later - Best Practices #OOWLON
Oracle RAC 19c and Later - Best Practices #OOWLON
 
Data modeling for the business
Data modeling for the businessData modeling for the business
Data modeling for the business
 
Oracle data guard for beginners
Oracle data guard for beginnersOracle data guard for beginners
Oracle data guard for beginners
 
Data Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookData Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future Outlook
 
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best PracticesOracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
 
Data Modeling for Big Data
Data Modeling for Big DataData Modeling for Big Data
Data Modeling for Big Data
 
Filing System
Filing SystemFiling System
Filing System
 
Oracle Sql Developer Data Modeler 3 3 new features
Oracle Sql Developer Data Modeler 3 3 new featuresOracle Sql Developer Data Modeler 3 3 new features
Oracle Sql Developer Data Modeler 3 3 new features
 
Microsoft SQL Server internals & architecture
Microsoft SQL Server internals & architectureMicrosoft SQL Server internals & architecture
Microsoft SQL Server internals & architecture
 
Introduction To Data Vault - DAMA Oregon 2012
Introduction To Data Vault - DAMA Oregon 2012Introduction To Data Vault - DAMA Oregon 2012
Introduction To Data Vault - DAMA Oregon 2012
 
MySQL Data Encryption at Rest
MySQL Data Encryption at RestMySQL Data Encryption at Rest
MySQL Data Encryption at Rest
 
Data Modeling & Metadata for Graph Databases
Data Modeling & Metadata for Graph DatabasesData Modeling & Metadata for Graph Databases
Data Modeling & Metadata for Graph Databases
 
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdf
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdfOracle_Multitenant_19c_-_All_About_Pluggable_D.pdf
Oracle_Multitenant_19c_-_All_About_Pluggable_D.pdf
 
Oracle MAA (Maximum Availability Architecture) 18c - An Overview
Oracle MAA (Maximum Availability Architecture) 18c - An OverviewOracle MAA (Maximum Availability Architecture) 18c - An Overview
Oracle MAA (Maximum Availability Architecture) 18c - An Overview
 

En vedette

NLP Data Cleansing Based on Linguistic Ontology Constraints
NLP Data Cleansing Based on Linguistic Ontology ConstraintsNLP Data Cleansing Based on Linguistic Ontology Constraints
NLP Data Cleansing Based on Linguistic Ontology ConstraintsDimitris Kontokostas
 
Data Cleansing introduction (for BigClean Prague 2011)
Data Cleansing introduction (for BigClean Prague 2011)Data Cleansing introduction (for BigClean Prague 2011)
Data Cleansing introduction (for BigClean Prague 2011)Stefan Urbanek
 
Scaling Big Data Cleansing
Scaling Big Data CleansingScaling Big Data Cleansing
Scaling Big Data CleansingZuhair khayyat
 
Data Quality Best Practices Nbk Auto May 06 2010
Data Quality Best Practices  Nbk Auto May 06 2010Data Quality Best Practices  Nbk Auto May 06 2010
Data Quality Best Practices Nbk Auto May 06 2010Rami Mansour
 
Data-Ed: Best Practices with the Data Management Maturity Model
Data-Ed: Best Practices with the Data Management Maturity ModelData-Ed: Best Practices with the Data Management Maturity Model
Data-Ed: Best Practices with the Data Management Maturity ModelData Blueprint
 
Applying Data Quality Best Practices at Big Data Scale
Applying Data Quality Best Practices at Big Data ScaleApplying Data Quality Best Practices at Big Data Scale
Applying Data Quality Best Practices at Big Data ScalePrecisely
 

En vedette (8)

NLP Data Cleansing Based on Linguistic Ontology Constraints
NLP Data Cleansing Based on Linguistic Ontology ConstraintsNLP Data Cleansing Based on Linguistic Ontology Constraints
NLP Data Cleansing Based on Linguistic Ontology Constraints
 
Data Cleansing introduction (for BigClean Prague 2011)
Data Cleansing introduction (for BigClean Prague 2011)Data Cleansing introduction (for BigClean Prague 2011)
Data Cleansing introduction (for BigClean Prague 2011)
 
Scaling Big Data Cleansing
Scaling Big Data CleansingScaling Big Data Cleansing
Scaling Big Data Cleansing
 
Data Cleaning Process
Data Cleaning ProcessData Cleaning Process
Data Cleaning Process
 
Data Quality Best Practices Nbk Auto May 06 2010
Data Quality Best Practices  Nbk Auto May 06 2010Data Quality Best Practices  Nbk Auto May 06 2010
Data Quality Best Practices Nbk Auto May 06 2010
 
Data-Ed: Best Practices with the Data Management Maturity Model
Data-Ed: Best Practices with the Data Management Maturity ModelData-Ed: Best Practices with the Data Management Maturity Model
Data-Ed: Best Practices with the Data Management Maturity Model
 
Applying Data Quality Best Practices at Big Data Scale
Applying Data Quality Best Practices at Big Data ScaleApplying Data Quality Best Practices at Big Data Scale
Applying Data Quality Best Practices at Big Data Scale
 
Data cleansing
Data cleansingData cleansing
Data cleansing
 

Similaire à Best Practices: Data Admin & Data Management

Data Governance challenges in a major Energy Company
Data Governance challenges in a major Energy CompanyData Governance challenges in a major Energy Company
Data Governance challenges in a major Energy CompanyChristopher Bradley
 
Enterprise Data Governance for Financial Institutions
Enterprise Data Governance for Financial InstitutionsEnterprise Data Governance for Financial Institutions
Enterprise Data Governance for Financial InstitutionsSheldon McCarthy
 
MDM & BI Strategy For Large Enterprises
MDM & BI Strategy For Large EnterprisesMDM & BI Strategy For Large Enterprises
MDM & BI Strategy For Large EnterprisesMark Schoeppel
 
Metadata Strategies
Metadata StrategiesMetadata Strategies
Metadata StrategiesDATAVERSITY
 
How JCI Prepared a Data Governance Program for Big Data & MDG on HANA
How JCI Prepared a Data Governance Program for Big Data & MDG on HANAHow JCI Prepared a Data Governance Program for Big Data & MDG on HANA
How JCI Prepared a Data Governance Program for Big Data & MDG on HANADATUM LLC
 
Implementing Agile Data Governance
Implementing Agile Data GovernanceImplementing Agile Data Governance
Implementing Agile Data GovernanceTami Flowers
 
11626 Bitt I 2008 Lec 2
11626 Bitt I 2008 Lec 211626 Bitt I 2008 Lec 2
11626 Bitt I 2008 Lec 2ambujm
 
A Step-by-Step Guide to Metadata Management
A Step-by-Step Guide to Metadata ManagementA Step-by-Step Guide to Metadata Management
A Step-by-Step Guide to Metadata ManagementSaachiShankar
 
Adopting a Process-Driven Approach to Master Data Management
Adopting a Process-Driven Approach to Master Data ManagementAdopting a Process-Driven Approach to Master Data Management
Adopting a Process-Driven Approach to Master Data ManagementSoftware AG
 
Master Data Management's Place in the Data Governance Landscape
Master Data Management's Place in the Data Governance Landscape Master Data Management's Place in the Data Governance Landscape
Master Data Management's Place in the Data Governance Landscape CCG
 
AnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS
 
3._DWH_Architecture__Components.ppt
3._DWH_Architecture__Components.ppt3._DWH_Architecture__Components.ppt
3._DWH_Architecture__Components.pptBsMath3rdsem
 
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdf
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdfEDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdf
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdfAbhinav195887
 
The Key Reason Why Your DG Program is Failing
The Key Reason Why Your DG Program is FailingThe Key Reason Why Your DG Program is Failing
The Key Reason Why Your DG Program is FailingCCG
 
Introduction to Master Data Services in SQL Server 2012
Introduction to Master Data Services in SQL Server 2012Introduction to Master Data Services in SQL Server 2012
Introduction to Master Data Services in SQL Server 2012Stéphane Fréchette
 
Metadata Repositories in Health Care - Master Data Management Approach to Met...
Metadata Repositories in Health Care - Master Data Management Approach to Met...Metadata Repositories in Health Care - Master Data Management Approach to Met...
Metadata Repositories in Health Care - Master Data Management Approach to Met...Health Informatics New Zealand
 

Similaire à Best Practices: Data Admin & Data Management (20)

Planning Data Warehouse
Planning Data WarehousePlanning Data Warehouse
Planning Data Warehouse
 
Data Governance challenges in a major Energy Company
Data Governance challenges in a major Energy CompanyData Governance challenges in a major Energy Company
Data Governance challenges in a major Energy Company
 
Enterprise Data Governance for Financial Institutions
Enterprise Data Governance for Financial InstitutionsEnterprise Data Governance for Financial Institutions
Enterprise Data Governance for Financial Institutions
 
MDM & BI Strategy For Large Enterprises
MDM & BI Strategy For Large EnterprisesMDM & BI Strategy For Large Enterprises
MDM & BI Strategy For Large Enterprises
 
Metadata Strategies
Metadata StrategiesMetadata Strategies
Metadata Strategies
 
These Are The Data You Are Looking For
These Are The Data You Are Looking ForThese Are The Data You Are Looking For
These Are The Data You Are Looking For
 
How JCI Prepared a Data Governance Program for Big Data & MDG on HANA
How JCI Prepared a Data Governance Program for Big Data & MDG on HANAHow JCI Prepared a Data Governance Program for Big Data & MDG on HANA
How JCI Prepared a Data Governance Program for Big Data & MDG on HANA
 
Implementing Agile Data Governance
Implementing Agile Data GovernanceImplementing Agile Data Governance
Implementing Agile Data Governance
 
Database 2 External Schema
Database 2   External SchemaDatabase 2   External Schema
Database 2 External Schema
 
Business intelligence
Business intelligenceBusiness intelligence
Business intelligence
 
11626 Bitt I 2008 Lec 2
11626 Bitt I 2008 Lec 211626 Bitt I 2008 Lec 2
11626 Bitt I 2008 Lec 2
 
A Step-by-Step Guide to Metadata Management
A Step-by-Step Guide to Metadata ManagementA Step-by-Step Guide to Metadata Management
A Step-by-Step Guide to Metadata Management
 
Adopting a Process-Driven Approach to Master Data Management
Adopting a Process-Driven Approach to Master Data ManagementAdopting a Process-Driven Approach to Master Data Management
Adopting a Process-Driven Approach to Master Data Management
 
Master Data Management's Place in the Data Governance Landscape
Master Data Management's Place in the Data Governance Landscape Master Data Management's Place in the Data Governance Landscape
Master Data Management's Place in the Data Governance Landscape
 
AnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS - Master Deck
AnalytiX DS - Master Deck
 
3._DWH_Architecture__Components.ppt
3._DWH_Architecture__Components.ppt3._DWH_Architecture__Components.ppt
3._DWH_Architecture__Components.ppt
 
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdf
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdfEDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdf
EDMC_DCAM_-_WORKING_DRAFT_VERSION_0.7.pdf
 
The Key Reason Why Your DG Program is Failing
The Key Reason Why Your DG Program is FailingThe Key Reason Why Your DG Program is Failing
The Key Reason Why Your DG Program is Failing
 
Introduction to Master Data Services in SQL Server 2012
Introduction to Master Data Services in SQL Server 2012Introduction to Master Data Services in SQL Server 2012
Introduction to Master Data Services in SQL Server 2012
 
Metadata Repositories in Health Care - Master Data Management Approach to Met...
Metadata Repositories in Health Care - Master Data Management Approach to Met...Metadata Repositories in Health Care - Master Data Management Approach to Met...
Metadata Repositories in Health Care - Master Data Management Approach to Met...
 

Plus de Empowered Holdings, LLC

Plus de Empowered Holdings, LLC (7)

Présentation data vault et bi v20120508
Présentation data vault et bi v20120508Présentation data vault et bi v20120508
Présentation data vault et bi v20120508
 
IRM UK - 2009: DV Modeling And Methodology
IRM UK - 2009: DV Modeling And MethodologyIRM UK - 2009: DV Modeling And Methodology
IRM UK - 2009: DV Modeling And Methodology
 
Data Vault and DW2.0
Data Vault and DW2.0Data Vault and DW2.0
Data Vault and DW2.0
 
Data vault what's Next: Part 2
Data vault what's Next: Part 2Data vault what's Next: Part 2
Data vault what's Next: Part 2
 
Data vault: What's Next
Data vault: What's NextData vault: What's Next
Data vault: What's Next
 
Operational Data Vault
Operational Data VaultOperational Data Vault
Operational Data Vault
 
Data Vault Overview
Data Vault OverviewData Vault Overview
Data Vault Overview
 

Dernier

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 

Dernier (20)

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 

Best Practices: Data Admin & Data Management

  • 1. Best Practices: Data Administration and Quality Daniel Linstedt, all rights reserved, http://LearnDataVault.com
  • 2.
  • 3.
  • 5. What is Data Administration? “ What do we mean by that in the case of data administration? We mean that DA must get out of the design review committee mentality and substitute something more value-added and flexible. It must recognize that systems tend to grow organically, and be a part of that process, rather than an instiller of order upon it.”  Eric Rawlins, 1995 Originally Published by: Database Research Group, Inc http://www.well.com/user/woodman/organic.html
  • 6.
  • 7. Cross-Organization Roles and Responsibilities Business ( Owner View) Data Steward Discipline Authority Business Process Manager Data Usage Contact Data Manager Data Modeler DA is a ROLE and typically involves more than one person in order to achieve success. Logical (Designer View) Data Administrator Physical ( Builder View) Database Administrator
  • 8.
  • 9.
  • 10. Defining Data Administration Issues Top 4 Examples
  • 11.
  • 12.
  • 13.
  • 14.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20. DA: Architecting Data Governance Business Rules & IQ EDW Source Systems Non Compliant Data Marts Business Rules & IQ EDW Source Systems Data Marts Compliant Hard Business Rules Soft Business Rules & IQ Shift to process AFTER the EDW Hard Business Rules Still process Before the EDW
  • 21.
  • 22.
  • 23.
  • 24. DA: Tracking Errors – KPIs at Work
  • 25.
  • 26. Metadata Administration Lifecycle Identify New Metadata Integrate With Master Metadata Repository Edit and Manage Master Metadata (Provide Business Users with Web Interface) Stitch Master Metadata Together Compare Master Metadata With Business Process And Objectives Export Master Metadata or Deploy via SOA With Master Data Set Derived from Meta Integration Metadata Lifecycle
  • 27.
  • 28.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34. Thank you Contact us today: Dan Linstedt [email_address] http://LearnDataVault.com

Notes de l'éditeur

  1. The purpose of this slide show is to present and discuss the role of data administration in the data integration world. Here we define some of the business and technical problems that DA’s face on a daily basis, then we move on to discuss the types of activities that a DA will under-take in an enterprise level initiative. Please bear in mind, that the DA is a role, and may not end-up being just a single individual, but rather a group of individuals, some of whom are directly responsible for Data Management as well.
  2. In this section we define different DA roles, issues, and conceptual notions. We discuss the DA role from a 20,000 foot level where the enterprise “see’s” data administrators, and begins to understand what they do. The role of the DA ranges from monitoring business user meetings to over-seeing the design of data flow through business processes. Business Process flow has a large impact on the world of the DA and what they need to be capable of achieving. They need to work across multiple groups in order to achieve an enterprise vision of the data assets and models that will serve the enterprise.
  3. http://www.cio.gov.bc.ca/other/daf/DMRolesRespV1.pdf
  4. http://www.cio.gov.bc.ca/other/daf/DMRolesRespV1.pdf
  5. http://www.educause.edu/ir/library/text/CEM9047.txt
  6. Data Must Be: Auditable, Traceable, Stored in the granular format it arrived in, A “statement-of-fact” Business Rules must move to the output side of the equation. Data can be integrated by the same semantic grain, but cannot be altered.
  7. The Data Administrator is responsible for identifying auditable or audited sources of data. The DA will be responsible for ensuring which data sets can and should be utilized to load enterprise data warehouses. The DA will set policies and procedures for measuring, auditing, and assessing the quality of information flowing to and from the source systems.
  8. The Data Administrator is responsible for assigning or classifying different groups of errors, what will make the data set or break the data set. They are also responsible for the integrity of the data set, and ensuring that the data set matches the requirements set forth by the business users.
  9. The Data Administrator might use a live chart like this one to examine the errors and the occurrences of errors over time. The DA will be responsible for the quality of the data, as it relates to the business metrics put forward. The DA will be responsible for maintaining the logical models, and the business processes – and if the error count is too high for a specific area of expertise, then the Data Manager must be notified, and corrective action must be taken.
  10. Organic Data Administration, http://www.well.com/user/woodman/organic.html