Best Practices in Data Integration

1. MY STORY
Designed By
Arvind Murali

2. I AM A DATA INTEGRATION EXPERT
I WORK FOR HIGHPOINT SOLUTIONS

3. ABOUT HIGHPOINT

4. My Evolution in Data Integration

5. My Humble Beginning
Started as a Datawarehouse Analyst/Designer
Sorting through
100’s of entities
To analyze and
report data in excel

6. FFWD Three Years
DATA INTEGRATION
CONSULTING
Time to move on to bigger and
better things!! Marriage.
&

7. FFWD Two More Years and ever since…
Moving ON to even better
things. My Son!!
INTEGRATIONS SOLUTIONS LEAD
(Why we are doing what we are
doing?)
&

8. CHALLENGES

9. Challenge 1 – Relevance
Storing all irrelevant information only to
find out increased Storage and
performance slowness.
STICK TO REQUIREMENTS. DO NOT
OVERDO.

10. Challenge 2 – Heterogeneous Data
Operational Databases
Data
Integration
Hub
CRM
Social Media
Rules
Repository
Ontology
Synchronizing huge quantities and formats of data with internal and external systems.

11. Challenge 3 – Data Quality
What is
Data Quality?
Data Quality & Data Audit are 2 of the most critical components of any data integration effort.
Developers and Users should work together to determine the quality control metrics and
measures.

12. Challenge 4 – Unanticipated Cost
•
•
•
•
Because of the involvement of hardware and software to accomplish data integration, it is
common to see some unexpected costs in the process.
Labor costs for initial planning, evaluation, programming and additional data acquisition.
Software and hardware purchases.
Unanticipated technology changes/advances.
Both labor and the direct costs of data storage and maintenance.

13. Challenge 5 – Expertise
Data Integration Lead, Data Stewards, Analysts, Designers, and Data Managers are a MUST
with practice experience in data management arena. In addition, setting incremental goal
helps understand, deliver, and monetize mission.

14. BEST PRACTICES

15. Lesson 1 – End Goal
Understand and Clarify the end goal (BIG PICTURE) with
your Stakeholders.

16. Lesson 2 -- Requirements
Gather Requirements and Specifications Thoroughly and as
much detailed as possible.

17. Lesson 3 –Standards
Understand and Design Industry and Organizational Standards customized to process.
ONCE DEFINED, STICK TO THOSE STANDARDS.

18. Lesson 4 – Data Analysis
A thorough analysis of characteristics and usage of data to mitigate bad and
heterogeneous data challenges.

19. Lesson 5 – Testing
Go GAGA over testing. Most critical phase of integration.

20. THE PROCESS!!

21. REFERENCES
Information Management Magazine
Dilbert Cartoons
The Datawarehouse Institute (TDWI)
Cartoons for Data
Data Integration Slideshares