SlideShare une entreprise Scribd logo
1  sur  20
Télécharger pour lire hors ligne
WHITE PAPER




A ROADMAP TO DATA MIGRATION SUCCESS


CONTENTS                                  Data migration is the one-time movement of data from a legacy source, or multiple
 2 Planning and Scoping the Migration
                                          sources, to a new target database. This simple concept and requirement can
   2 Allowing for Dependency Management
   3 Establishing Data Governance and     drive a scope that is much larger than expected. A data migration requirement
     Stewardship                          can be driven by a range of initiatives, such as an application replacement or
   5 Building the Migration Team          upgrade, the need to consolidate data within a data warehouse, or a requirement to
 6 Project Planning Considerations        create a single view of product within an organization. In cases where migrated
   6 Mapping Business Needs
                                          data is transformed for new uses, your project team encounters some very
   6 Assessing Infrastructure
   9 Understanding Data                   specific management and technical challenges. For example, a team that has
                                          been writing extraction, transformation, and load (ETL) code for a data warehouse
                                          faces a new set of challenges when migrating data to a live, operational system.
                                          Although a 2% error rate may be acceptable for aggregate reporting, it is not
                                          acceptable for customer contact data—in this example, we would fail to recognize
                                          one out of 50 customers when they call!
                                          Many significant business initiatives and large IT projects depend upon a
                                          successful data migration. Your goal is to minimize as much of your risk as
                                          possible through effective planning and scoping. The objective of this paper
                                          from Business Objects, an SAP company, is to provide insight into what
                                          issues are unique to data migration projects and to offer advice on how to
                                          best approach them.
PLANNING AND SCOPING THE MIGRATION


            As with any project, success depends upon a good plan. Data migration projects
            have a distinct methodology and project approach. Too often, project managers
            make the costly mistake of thinking that migrating data is a simple task within a
            larger project and hand it off to a development team. Table 1 lists the migration
            tasks that must be considered.

             Table 1. Migration Tasks

             Migration Element                          Activities
             Business needs mapping                     Ensure that your migration plan is driven by the expectations
                                                        and needs of the users
             Infrastructure                             There are unique infrastructure requirements that should
                                                        be sorted as early as possible in the project timeline
             Identifying data sources and targets       Validate the system of record for your source data
                                                        and identify the data needs of your target database
             Evaluating the data quality                Assess the quality of your data to meet the target
                                                        application requirements and business needs
             Gap analysis between source and target Identification, and plan for mitigation, of any gaps between
                                                    available source data and target application data requirements
             Impact of multiple sources of data         Estimate the challenges of consolidating similar data
                                                        from several sources, or integrating dissimilar data
             Mapping assessment                         Understand the effort required to accurately identify source
                                                        data at column level detail, including transformation specifications
             Migration assessment                       Understand the effort required to design, code, test and
                                                        implement the data migration



            In addition to these tasks, the following sections detail two additional project
            management considerations that are essential to any data migration – dependency
            management and data governance planning.

            ALLOWING FOR DEPENDENCY MANAGEMENT
            Don’t forget that most risk is outside of your control or scope. Numerous other
            projects, initiatives, and issues may influence a data migration project. Your project
            manager or team leader must be a good communicator and be aware of the decisions
            and changes happening in the larger environment. The following scenarios can
            change your plan and affect the scope of your data migration.
            Scenario: Larger Projects
            Data migration plans and schedules typically are driven by larger projects—for
            example a master data management (MDM) initiative, a new business process,
            or a data consolidation effort that supports a new reporting environment.




            Business Objects. A Roadmap To Data Migration Success
Any changes to the parent project schedule affect the data migration schedule,
including changes to production deployment, and the changes to test and beta
deployment schedules.
The parent project may change its approach to implementation, such as moving
from a regional phased approach to a product phased approach. For example, in
a cell phone company, rather than convert customers by region, the team converts
all customers internationally that have the newest service plan, followed by a family
service plan, and so on. What may be a straightforward change for the larger project
could exponentially increase the scope of the data migration effort.
It is not uncommon for resources to shift roles during a project. For example, the
priorities of the larger project could drive key players away from the migration tasks,
which are deemed less important than other development. Thus, your schedule is
delayed and your risks increase.
Scenario: Know Your Data
The success of your migration project largely depends upon how well you know the
data content and business rules. Your data experts help with identifying business
needs, analyzing source systems, and testing, covering the full life cycle of your
project. Experts likely will be from the business and tasked to the migration project
on an as-needed or part-time basis.
Scenario: Changing Source Systems and Business Rules
The source data and the business rules that govern it may change during the
course of your project. There could be other initiatives in the organization, new
product offerings, or business process changes that impact the data.
The source system itself could change, due to application or server upgrades.
Application upgrades are particularly risky, since the application features and data
model of the source may be altered.

ESTABLISHING DATA GOVERNANCE AND STEWARDSHIP
Involving business users and data experts in the migration project is the single biggest
factor driving your success. Have you ever seen a data intensive project implemented
successfully without organized, committed business user involvement? In order to
fulfill your data migration needs, you must include a data governance strategy that
provides leadership and direction for user input. In your organization, you may not
refer to the strategy as data governance or stewardship, but regardless of what you
call it, the key is that the data experts—often business users—are involved in all data
migration phases and tasks.




Business Objects. A Roadmap To Data Migration Success
Your data migration project team must plan for the participation of data experts, set
expectations for resource commitments, and initiate the identification of measurable
success factors. Project planning must consider the two key roles for this team:
1) ensuring that the project team is aware of the business needs being met by the
target application, and 2) providing detailed data expertise and ongoing monitoring
(ownership) demanded by the new application. Team responsibilities include:
• Participating in the identification of “best” sources or “system of record” sources

• Actively reviewing, creating, and correcting data to support cleansing and reference
  (master) data requirements

• Monitoring data quality and data auditing status

• Sharing and capturing content detail about source system data and business processes

• Testing and confirming application functionality that is supported by migrated data

The approach to establishing a data governance team is specific to your organization.
If possible, leverage the talents of an existing team or committee. Often a data
warehouse user’s group is an excellent source of data knowledge, or you may be
able to leverage a team that supported a previous project.
Consider a metadata tool for the project team. The data migration project is a
unique opportunity to capture metadata about your source systems and to initiate
a business process for managing data in the target application. What can you
leverage that you have in place now? What are the requirements for managing
the metadata and the business rules for the data migration project? What are the
requirements going forward? Consider:
• The need to share content knowledge and business rules for multiple team members

• Testing against identified business rules and mapping/transformation specifications

• Historical documentation (audit trails) of the decisions made, and logic behind
  them, for the data migration

• Historical documentation about the source of data, especially useful when merging
  data from multiple sources

• Identifying data owners and experts for the migration and future data
  management requirements




Business Objects. A Roadmap To Data Migration Success
BUILDING THE MIGRATION TEAM
Table 2 lists roles and responsibilities for the planning and scope phase of the data
migration project.


 Table 2: Roles and Responsibilities for Planning and Scoping

 Team Member                                Responsibilities

 Project manager or team leader             - Coordinating and communicating task status
                                            - Managing issues
                                            - Creating and maintaining project plan
                                            - Managing dependencies to larger project, target, and source
                                              application schedules
                                            - Coordinating the creation of a data governance strategy and team
 Business users (data experts)              - Identifying and validating business needs
                                            - Analyzing target and source systems
                                            - Assessing data quality
                                            - Testing
                                            - Assessing data mapping and business rules for
                                              data transformation
 Business or data analysts                  - Infrastructure support, including database sizing, optimization,
                                              and life cycle management. Works closely with the
                                              database analyst.
                                            - Data support, including data analysis, data quality assessment,
                                              mapping, and migration assessment.
 Database administrator                     - Initial sizing for staging database(s)
                                            - Planning for life cycle management of database objects
                                            - Management planning for schema changes and user security
 Data architect                             - Architecture and methodology
                                            - Source and target application data analysis
                                            - Data quality assessment, mapping assessment
                                            - Data governance planning
                                            - Configuration management
                                            - Configuration standards and quality assurance
 Migration lead (for planning and scoping) - Architecture and programming standards
                                           - Data quality, data mapping, and data migration assessment
                                           - Source to target gap analysis and multiple systems
                                             impact assessment



Consider using consultants for key roles on a migration project, since it is a one-time,
custom effort and your team may not have the specific planning, strategy, or development
experience. Focus your project plan on growing the skills needed in your organization,
which may include future data migration, data integration, or data quality projects.




Business Objects. A Roadmap To Data Migration Success
PROJECT PLANNING CONSIDERATIONS


            The following sections address planning considerations for your data
            migration project.

            MAPPING BUSINESS NEEDS
            Business needs represent why you are migrating data. How many new applications
            go live but do not deliver the business value expected? You must articulate the
            business needs that are driving your project. In addition to ensuring that you
            are meeting the needs of the consumer, you are also controlling the scope of a
            data migration project, especially source systems analysis and mapping. At first
            glance, this may seem obvious, but consider as an example the full functionality
            of a customer relationship management (CRM) application. Organizations pick
            and choose from the menu of functions and features, and the implementation
            may be phased over time. Rather than migrating all of the data subject areas of
            a full customer database, the team can focus on only those tables and columns
            that support the business needs of the CRM application. A clear understanding
            of business needs also provides focus for the sizing estimates, data quality
            assessment, and test planning.
            Additionally, as unexpected data quality and gap analysis issues arise (and
            they will), knowing the business needs aids in identifying priorities for the
            project team. For example, if the customer age is not fully populated, or has
            inaccurate data (230 years old), the team can reference the known uses of that
            data to determine the extent of data cleansing needed. In another example, the
            medical history for an automated patient chart must be of the highest qual-
            ity when migrated. The business need, which is driven by legal requirements,
            must be clearly understood by the entire migration team. Ideally, the project
            team will include data experts and business users as part of a data governance
            strategy to ensure that this process is as straightforward as possible.

            ASSESSING INFRASTRUCTURE
            A data migration project demands a dedicated infrastructure. Don’t make the
            mistake of using your existing development and test environments. For example,
            do not underestimate the planning needed for a dedicated staging area or for
            load testing (yes, load testing!) You might be converting many years worth of
            data in a single weekend. Far too many organizations wait until the beta or production
            conversion to discover performance issues that can set the schedule back by
            weeks or months.




            Business Objects. A Roadmap To Data Migration Success
Current Architecture
What is your current architecture? Do you have data quality and data integration
tools needed to accomplish a data conversion? Is there a database or server available to
host your data and code? When and how will you test the full conversion to ensure
that your performance will meet the schedule needs? The team should evaluate
what servers, databases, and software tools are available to host a migration project.
Leverage what you can and do not reinvent the wheel. Specifically, review:
• Requirements documentation and capture

• Metadata and data integration mapping repositories

• Data integration tools

• Data quality tools

• Code management tools and processes

• Test tools and processes

Given that the data migration is a one-time project, you should evaluate what code
management processes can be bypassed or customized for this effort. This may
sound like a surprising recommendation, but consider the environment surrounding
the data migration project. Your schedule is not your own; rather, it is dictated by
the larger project. Also the priorities and phased approach can change at any time.
The data migration team needs to be very agile. The approach to analysis, coding,
and testing should be lightweight and mutable.
Target Architecture
The dedicated infrastructure for the project should be planned as part of your
scope phase. Do not make the error of waiting until the development phase of the
project to discover that you need additional resources for integration testing, such as
servers or new database instances. The planning for this infrastructure should include:
• A staging database for analysis and development. Impact on production services is
  minimized by the use of a separate migration server and by the use of configured
  staging and process areas within the service to manage multiple data versions,
  sources, and status values. Your team can also use the staging area to created
  needed reference data for the target system.

• A sizing estimate for the staging database, which includes any needed replication
  required to support both migration code development and application




Business Objects. A Roadmap To Data Migration Success
• A code life-cycle management plan

• Performance test requirements

• Backup and recovery planning

• Scalable connectivity requirements and approaches (for example, how will you
  extract and load data to the source and target systems?)

Planning for Testing
The test requirements for a data migration project are unique, as you may have
multiple conversions during parallel and beta testing of an application. Adding to
the complexity is the fact that the migration team is supporting the application test
requirements of the larger project and also testing the conversion code itself.
Your migration team needs to first consider the basics of audit and reconciliation.
This is missed more often than it should be. Are you looking at the right set of
data? Can you trace back to your source system at a record level? Include basic
record counts and validation of ETL processes. Many of these processes will be
accomplished multiple times, as testing and re-loads occur. The audit statistics
need to be reported and evaluated each time. Also, if the project is a phased
implementation, you may be running similar data across parallel databases. Audits and
reconciliation are needed to ensure that the data is consistent across these applications.
You should also plan for performance testing. Many projects are delayed because
the development team has successfully converted a test set of data but waited until
the last minute to run the conversion of the full set. A customer consolidation project
for one of the largest global computer manufacturer’s was delayed by months when
the development team attempted to convert (at go-live!) a year’s worth of customer
data – using code that had been developed using a day’s worth of test data.
For your conversion, there may be millions of records to process, with complex
referential integrity. Performance issues for data processing at this level of magnitude
can require significant code changes. What to do? Estimate your load and include
this consideration when unit and integration testing. Participate in peer reviews, or contract
with experienced consultants, to address code optimization, data query tuning, and
database tuning challenges. And include these options in your earliest planning.
The primary reason for capturing the metadata around your business rules is to test
the mapping and transformations. All phases of testing, unit, integration, and target
application, should include a validation of the business rules.




Business Objects. A Roadmap To Data Migration Success
As early as possible in your migration schedule, allow the business users to test
the data in the target application. Does it support the features and functionality as
expected? Data migration mapping is very abstract, especially if the users have
never been part of a similar project. They may not “get it” until the converted data
is actually supporting the application features and processes that they are
implementing. Allow time in your planning for changes to the mapping, business
rules, and re-loads.
It is not uncommon for there to be confusion during beta testing and implementa-
tion as to where errors originate. It’s all too easy for “bad data” to be blamed for
every test error. What can you do as part of your planning to avoid this? Implement
the migration with as much metadata and reconciliation as possible. Not only are
you ensuring the integrity of your data, you are creating the lineage necessary to
support the integration test process.

UNDERSTANDING DATA
Data migration is, in fact, all about the data. Data related tasks, especially those
around quality, will be the bulk of your work. Plan enough time to evaluate the
source data, to compare the available data to the needed data, and to drill down to
the detail needed for source to target mapping. Your goal is to ensure that the data
migration development is the shortest task on your project plan. You must understand
the source data and target needs before you start coding. You don’t want to experience
the very costly surprise of learning that the source data is not “fit to use” at integration
testing or implementation.
Evaluate Data Sources and Targets
As part of the planning process, your team will take an initial look at the source and
target systems. The objective is to know enough about each system to make an
estimate of the effort to complete mapping and migration. At a high level, the team
will identify subject areas, data entities, and supported functionality. For example,
customer address data will come from the billing system, customer phone contact
data will come from the data warehouse, and so on.
The planning team needs to look for the following risks and complexities:
• The commonly agreed to, or popular, system of record may not be the best source
  of data. Consult as many data experts as possible and look for reconciliation statistics,
  compliance numbers, and other forms of audit to identify a trusted source.




Business Objects. A Roadmap To Data Migration Success
• Multiple sources increase your schedule exponentially. Never assume that a
  second source of data will make your project twice the effort. If you have one
  source of data you may have to look for duplicate data, but if you have two
  sources you have to look for duplicate data, harmonize reference data, and run
  record-matching algorithms to determine correct mapping. (A requirement for
  sharing master data across multiple applications is a project in itself. The need
  to harmonize reference data is complex enough to warrant its own white paper.)

• Certain types of projects introduce their own challenges. For example, the
  approach to managing a data migration that is part of an MDM project will be
  distinct from other kinds of migrations, with a focus on dependency management,
  data quality, and data governance. An MDM implementation requires initial
  data migrations, but also ongoing integration and reconciliation. The business
  rules, matching criteria, and governance processes that are established for
  migration will be propagated to support the live application.

 Also, mergers and acquisitions require data consolidation, with a focus on
 reference data and business processes (that are represented in the data
 structures and content). Even if the two organizations have the same software
 the mapping may not necessarily be straightforward.

 Another challenge is that a data warehouse migration can seem less
 complicated because the data is relatively static in the target database;
 however, the expectations of the business users can be extensive. In a well-
 designed warehouse, one data mart will support multiple business units. Also,
 the data migration process is likely to be the initial load of an ongoing ETL
 process and there should be reuse of many components.

• Conduct a gap analysis. The features that the business expects from the
  target system may not have a data source at all, and the migration team may
  be the designated messenger to carry this bad news. For example, in an MDM
  customer data project, one region out of five has never captured customer
  contact history and, when consolidating the customer data, a subset will not
  have a full history. The business requirement to “establish customer segmentation
  based on contact history” is seriously impacted.

Your planning should include the time needed and the approach for resolving
gaps in the source to target data mapping (and a cost benefit of the effort).
There are several options, and each will increase the scope of your project:




Business Objects. A Roadmap To Data Migration Success                                 10
• The business users and data experts could manually create data. This is not uncommon
  when creating reference data, such as product types or customer status. The
  project plan will include the time needed to work with the governance team and
  facilitate the data entry, updates, and testing.

• You may have the option to source data externally—for example, using Dun  Bradstreet
  data to create more consistent customer data.

• You could default the data to an unknown value and plan to populate it over time,
  as part of the target application. Using the customer segmentation example above,
  if it requires a year’s worth of customer contact history, the business will accept
  the 12-month delay before having a 100% correct segmentation calculation.




Figure 1: A Sample Data Profiling Report that Shows the Frequency Distribution of Values


Perform a Data Quality Assessment
The biggest factor in your success—and your biggest risk—is the quality of the
source data. Many projects fail, or are significantly delayed, upon discovering that
the data is not “fit to use.” The most important phase in a data migration project
involves the tasks needed to understand the data content. You would not put dirty petrol
in your new car—why would you map bad data into your new software application?




Business Objects. A Roadmap To Data Migration Success                                      11
It’s unlikely that anyone on your team really understands the current state of the
data at the level of detail needed for your project. And even if the source data is
pristine, that doesn’t mean that it is fit for the requirements of the new application.
Experience shows that unexpected data conditions encountered in load programs
can significantly impact the timelines of data migration projects. Early assessment
of data quality reduces project risk.
Your team will do an initial data quality assessment as part of the planning process.
The sooner that you look at the data, the more likely your success. For this initial
pass, you are looking for the big-ticket items – missing data, data that is not fully
populated, columns names that are inconsistent with content, and so on. Your goal
is to know enough to estimate the level of effort to get the migration job done.
You have done a review of business needs and this will drive the type of data you
review. For example, if the objective is to create a consolidated view of product
data across six regional SAP® applications, you may pull a subset of product
data from each and attempt to match them. How good is the data? What are the
challenges? This should give you a pretty clear idea of the project requirements for
mapping and migration. Another example is pulling customer data from two systems,
such as billing and CRM. You want to run some high-level reconciliations and
audits between the two systems, looking for deltas. What if you have customers who
have products but they’re not getting billed? It has happened! What if you have
historical data for billing that isn’t represented in your CRM application?
When capturing metadata, a tool is a good option – but when assessing data quality,
a profiling tool is a must. Use data profiling technology to systematically scan the
tables of interest to quantify any data quality and data integrity issues, such as:
• Missing values

• Pattern analysis

• Frequency analysis

• Ranges and outliers

• Redundant data analysis

• Foreign key integrity analysis

• Statistics relevant to numeric fields

• Parsing requirements to enable mapping of source columns to the target columns




Business Objects. A Roadmap To Data Migration Success                                     1
Perform a Data Mapping Assessment
Hand-in-hand with data quality is data mapping. What are we moving and what are
the rules? How much time will it take the team to complete data mapping? Given
your planning, you should have a general idea of what it will take to create the final
design and implementation. The captured business needs, the source and target
system metadata, and the data profiling results from the data quality assessment all
create the information need to understand the mapping effort.
Much of the mapping will be data analysis work, if done right. It is not uncommon
for teams to accomplish this mapping based on guessing which column names
match in a spreadsheet, or worse, jotting a few notes down on a hardcopy of the
DDL listing before coding. Be aware that there are development teams capable of
this behavior.
Where possible, identify the data challenges. One example illustrates how complex
your mapping can become. An insurance premium discount for a new membership
system is stored in a single column on a single record. However, in the custom
source systems, it is stored in multiple locations and averaged according to complex
business rules. These rules are captured in the source system code. There are no
data experts who understand fully how the logic works. The data migration team
must recreate this logic in the data integration code in order to populate the target
system value. To make it even more complicated, the data conversion is an ongoing
process that happens at policy renewal, and getting the calculation wrong will cost
the company money. If the discount is too large, the organization loses money; if
it’s too small, the organization may lose customers. Finally, there is the importance
of capturing this logic in the metadata or design of the data migration for auditors,
actuaries, and the membership support teams.
Some of the things to consider when estimating the time it will take to accomplish
data mapping are:
• Know that some of your target requirements will require significant transformations,
  or the creation of new data content. For example, the mapping scope should include
  the effort to create reference data for the target application, such as product types,
  customer status, and vendor segmentation types. It’s not uncommon for the team,
  working with the data experts, to create these and set up some cross-reference
  tables to support the conversion of the data. This should be planned early in the project,
  and data quality validation should be included, such that all records are converted.




Business Objects. A Roadmap To Data Migration Success                                          1
Also, matching and consolidating is required when sourcing from multiple systems. It
 could be that you need to make a match decision on the best record from multiple
 source systems, rather than having a single system of record. Or that your reference
 data is so in your target system that you need to combine multiple records to make
 a “best of” instance.

 Data quality matching could also be used to eliminate duplicate data within a
 single source. This logic could be implemented to select a best of record that will
 later be evaluated by a data expert. The logic and the process to identify the ‘good
 record’ should be part of your mapping specification.




Figure 2: Requirements Gathering and Mapping Tool Within a Multi-Developer Environment



• Allow time to look at the data. Don’t allow your team to fall into the trap of mapping
  based on column names, copybooks (remember those?), or DDL. For each subject
  area, you should allow at least a week of time for the analyst to gather information
  about the meaning of the data, how it is populated (data profiling), and how it supports
  the target application.




Business Objects. A Roadmap To Data Migration Success                                        1
• For high priority data content, the team should learn as much as possible about the
  business processes that create the data and the target applications business processes.
  Data experts and business users are an invaluable source for this information.
  For example, a patient record in a hospital radiology application has some very
  basic requirements for radiologist review. There may be scheduling and workflow
  included in the application. If that data is mapped to an “automated patient chart,”
  it will be used by physicians other than radiologists to make broader diagnoses
  and evaluations. What are the differences in these processes—and does the data
  support the new needs represented by the automated patient chart?

• The planning should include the evaluation of a mapping tool, or mapping functionality
  within a data integration tool. Keys things to look for are the ability to easily connect
  to and analyze the source data as part of the mapping process, the ability to share
  analysis results across a larger team, and how well the mapping drives the
  development of code.

• Include validation and reconciliation steps in your mapping.

• For any matching that you do, or any consolidation, verify that each source record
  is present in the source systems as designed. Each subject area mapped must
  have a reconciliation base to the source, validating that all records extracted were
  processed. For example, if I have 2,000 customers on my billing system, then I
  should have 2,000 customers on my new CRM system.

 External to the migration process, you should also have a process that is designed
 for a checks-and-balance approach.

 Another approach is to create a high-level mapping to a database that is wholly
 external to the migration process, thus creating a further audit validation. For example,
 you may map your in-patient data in an automated chart application to hospital
 admissions data that is captured by a financial system. Is this overkill? That entirely
 depends on your business user community and the extent to which you want to
 guarantee trusted data.




Business Objects. A Roadmap To Data Migration Success                                         1
Figure 3: Business Objects Provides the Ability to Standardize, Cleanse, and Match Data Regardless
          of Type, Source, or Target


Perform a Data Migration Assessment
Effective planning will allow you to mitigate the risks associated with your project.
If you have done the work of scoping the infrastructure, the mapping, and the data
quality assessment, then the migration itself will be your shortest task. When it
comes time to migrate your data, the people and the processes will make it work.
It’s more a team effort than another other type of IT project. Throughout the
migration process, you will have checks and balances in place, including a team of
data experts available to address the expected data content decisions and to help
resolve the unexpected. This can include team members who are clearly responsible
for tracking set of audit and reconciliation metrics to evaluate:
• Do the record counts match from source to target through each phase of your processing?

• Can you reconcile counts and reports to other systems in the organization that
  host the same data, for example numbers of active customers, or product sales.
  This is especially important if you are running systems in parallel. Can you track
  the deltas between the parallel systems and can you understand them?




Business Objects. A Roadmap To Data Migration Success                                                1
• Do the data experts have access to review the data for quality and consistency?

• Can you create an audit back to your source system that would satisfy financial
  compliance requirements?

• An approach to a phased project and how to keep parallel systems in synch as
  you migrate forward. This can include ongoing integration or replication between
  applications for the duration of the project.

Agility is key. This is a custom (bespoke) development project and you should
apply the rigor that you would to any other development project. However, keep the
implementation approach as lightweight as possible. There are many dependencies
to other projects and applications within the organization. Plan for the inevitable
need to change scope, schedules, and roadmaps as quickly as possible.
It‘s not unusual for a new application to source from multiple databases, flat files,
or external feeds. Additional sources will increase your complexity exponentially.
Consider your approach to rationalizing the data across these sources. Also what
is your cutover strategy? And how will these sources impact your schedule? If you
are de-duping or consolidating data across sources, consider what your record-
matching requirements are—and be certain to evaluate your data integration soft-
ware with this in mind.
Finally, map all of your data integration requirements and data quality needs against
an off-the-shelf toolset. Simple things, such as populating a gender column based
on customer name, come out of the package, and save significant amounts of time.
The more complicated needs, such as matching and consolidation, are infinitely
easer using dedicated tools. Don’t make the mistake of thinking you are saving
money by hand coding—you are risking your schedule and the quality of your product.




Business Objects. A Roadmap To Data Migration Success                                   1
Figure 4: A Data Integration Tool Provides the Ability to Extract, Clean, Transform, and Deliver the Data
          Between the Source(s) and Target Applications


Perform a Post-Migration Needs Assessment
Finally, as part of your planning process, do not forget the needs of your team
going forward. These include:
• The need for archiving data that is not converted, or needs to be reported on in its
  historical structure.

• The ongoing reconciliation and balancing between the new application and parallel
  applications. For example, if you implement a new customer data mart, it should
  include a frequent reconciliation back to its source (lineage) and to another
  unrelated customer application (trust).

• It’s likely that the data governance team has determined resolution of a set of data
  quality issues can be deferred post-production. Plan to capture as much detail as
  possible over the course of your project.




Business Objects. A Roadmap To Data Migration Success                                                       1
DATA MIGRATION SUCCESS


            If you have successfully implemented your data migration, you should have data
            governance, standards, integration, and quality processes in place. Leverage these
            on your next project. Use the migration effort as a route to growing your overall
            information management skills, which will make subsequent data projects easier,
            faster, and cheaper.




            Business Objects. A Roadmap To Data Migration Success                                1
businessobjects.com



© 2008 Business Objects. All rights reserved. Business Objects owns the following U.S. patents, which may cover products that are offered and licensed by Business Objects: 5,555,403; 5,857,205;
6,289,352; 6,247,008; 6,490,593; 6,578,027; 6,831,668; 6,768,986; 6,772,409; 6,882,998; 7,139,766; 7,299,419; 7,194,465; 7,222,130; 7,181,440 and 7,181,435. Business Objects and the Business
Objects logo, BusinessObjects, Business Objects Crystal Vision, Business Process On Demand, BusinessQuery, Crystal Analysis, Crystal Applications, Crystal Decisions, Crystal Enterprise, Crystal Insider,
Crystal Reports, Desktop Intelligence, Inxight, the Inxight Logo, LinguistX, Star Tree, Table Lens, ThingFinder, Timewall, Let there be light, Metify, NSite, Rapid Marts, RapidMarts, the Spectrum Design, Web
Intelligence, Workmail and Xcelsius are trademarks or registered trademarks in the United States and/or other countries of Business Objects and/or affiliated companies. All other names mentioned herein
may be trademarks of their respective owners.                                                                                                                                          February 2008 WP3127-A

Contenu connexe

Tendances

Agile Methodology - Data Migration v1.0
Agile Methodology - Data Migration v1.0Agile Methodology - Data Migration v1.0
Agile Methodology - Data Migration v1.0Julian Samuels
 
Strategic Approach To Data Migration Project Plan
Strategic Approach To Data Migration Project PlanStrategic Approach To Data Migration Project Plan
Strategic Approach To Data Migration Project PlanSlideTeam
 
Data Migration Strategies PowerPoint Presentation Slides
Data Migration Strategies PowerPoint Presentation SlidesData Migration Strategies PowerPoint Presentation Slides
Data Migration Strategies PowerPoint Presentation SlidesSlideTeam
 
ERP Data Migration Methodologies
ERP Data Migration MethodologiesERP Data Migration Methodologies
ERP Data Migration MethodologiesAhmed M. Rafik
 
L1_RISE_with_SAP_NNN_V3.4.pptx
L1_RISE_with_SAP_NNN_V3.4.pptxL1_RISE_with_SAP_NNN_V3.4.pptx
L1_RISE_with_SAP_NNN_V3.4.pptxGuruprasad Bellary
 
Building Data Quality Audit Framework using Delta Lake at Cerner
Building Data Quality Audit Framework using Delta Lake at CernerBuilding Data Quality Audit Framework using Delta Lake at Cerner
Building Data Quality Audit Framework using Delta Lake at CernerDatabricks
 
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...Hortonworks
 
Oracle Hyperion overview
Oracle Hyperion overviewOracle Hyperion overview
Oracle Hyperion overviewClick4learning
 
Microsoft Power BI Technical Overview
Microsoft Power BI Technical OverviewMicrosoft Power BI Technical Overview
Microsoft Power BI Technical OverviewDavid J Rosenthal
 
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...SreeGe1
 
SAP Integration Suite L1
SAP Integration Suite L1SAP Integration Suite L1
SAP Integration Suite L1SAP Technology
 
Introduction to power apps
Introduction to power appsIntroduction to power apps
Introduction to power appsRezaDorrani1
 
“Migration to Suite of HANA”
“Migration to Suite of HANA”“Migration to Suite of HANA”
“Migration to Suite of HANA”Wise Men
 
Azure data platform overview
Azure data platform overviewAzure data platform overview
Azure data platform overviewJames Serra
 
Wp sap data_migration
Wp sap data_migrationWp sap data_migration
Wp sap data_migrationBiswajit Kar
 
Benefit SAP S4HANA.pptx
Benefit SAP S4HANA.pptxBenefit SAP S4HANA.pptx
Benefit SAP S4HANA.pptxAlexYuniarto1
 
Gartner: Master Data Management Functionality
Gartner: Master Data Management FunctionalityGartner: Master Data Management Functionality
Gartner: Master Data Management FunctionalityGartner
 

Tendances (20)

Agile Methodology - Data Migration v1.0
Agile Methodology - Data Migration v1.0Agile Methodology - Data Migration v1.0
Agile Methodology - Data Migration v1.0
 
Strategic Approach To Data Migration Project Plan
Strategic Approach To Data Migration Project PlanStrategic Approach To Data Migration Project Plan
Strategic Approach To Data Migration Project Plan
 
Data Migration Strategies PowerPoint Presentation Slides
Data Migration Strategies PowerPoint Presentation SlidesData Migration Strategies PowerPoint Presentation Slides
Data Migration Strategies PowerPoint Presentation Slides
 
ERP Data Migration Methodologies
ERP Data Migration MethodologiesERP Data Migration Methodologies
ERP Data Migration Methodologies
 
Microsoft PowerApps
Microsoft PowerAppsMicrosoft PowerApps
Microsoft PowerApps
 
L1_RISE_with_SAP_NNN_V3.4.pptx
L1_RISE_with_SAP_NNN_V3.4.pptxL1_RISE_with_SAP_NNN_V3.4.pptx
L1_RISE_with_SAP_NNN_V3.4.pptx
 
Building Data Quality Audit Framework using Delta Lake at Cerner
Building Data Quality Audit Framework using Delta Lake at CernerBuilding Data Quality Audit Framework using Delta Lake at Cerner
Building Data Quality Audit Framework using Delta Lake at Cerner
 
SharePoint as a Document Management System (DMS)
SharePoint as a Document Management System (DMS)SharePoint as a Document Management System (DMS)
SharePoint as a Document Management System (DMS)
 
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...
Modern Data Architecture for a Data Lake with Informatica and Hortonworks Dat...
 
Oracle Hyperion overview
Oracle Hyperion overviewOracle Hyperion overview
Oracle Hyperion overview
 
Migrating to SAP S/4HANA
Migrating to SAP S/4HANAMigrating to SAP S/4HANA
Migrating to SAP S/4HANA
 
Microsoft Power BI Technical Overview
Microsoft Power BI Technical OverviewMicrosoft Power BI Technical Overview
Microsoft Power BI Technical Overview
 
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...
Data Migration Tools for the MOVE to SAP S_4HANA - Comparison_ MC _ RDM _ LSM...
 
SAP Integration Suite L1
SAP Integration Suite L1SAP Integration Suite L1
SAP Integration Suite L1
 
Introduction to power apps
Introduction to power appsIntroduction to power apps
Introduction to power apps
 
“Migration to Suite of HANA”
“Migration to Suite of HANA”“Migration to Suite of HANA”
“Migration to Suite of HANA”
 
Azure data platform overview
Azure data platform overviewAzure data platform overview
Azure data platform overview
 
Wp sap data_migration
Wp sap data_migrationWp sap data_migration
Wp sap data_migration
 
Benefit SAP S4HANA.pptx
Benefit SAP S4HANA.pptxBenefit SAP S4HANA.pptx
Benefit SAP S4HANA.pptx
 
Gartner: Master Data Management Functionality
Gartner: Master Data Management FunctionalityGartner: Master Data Management Functionality
Gartner: Master Data Management Functionality
 

En vedette

Inventory Management
Inventory ManagementInventory Management
Inventory Managementmbababs
 
Live migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchasLive migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchasDocker, Inc.
 
Systems Migration
Systems MigrationSystems Migration
Systems Migrationrichchihlee
 
T44u 2015, content migration
T44u 2015, content migrationT44u 2015, content migration
T44u 2015, content migrationTerminalfour
 
Large Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBossLarge Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBossC2B2 Consulting
 
Modular Enterprise Systems - An Introduction
Modular Enterprise Systems - An IntroductionModular Enterprise Systems - An Introduction
Modular Enterprise Systems - An IntroductionAndreas Weidinger
 

En vedette (8)

Inventory Management
Inventory ManagementInventory Management
Inventory Management
 
Inventory planning
Inventory planningInventory planning
Inventory planning
 
Live migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchasLive migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchas
 
Systems Migration
Systems MigrationSystems Migration
Systems Migration
 
T44u 2015, content migration
T44u 2015, content migrationT44u 2015, content migration
T44u 2015, content migration
 
Large Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBossLarge Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBoss
 
Seminar - JBoss Migration
Seminar - JBoss MigrationSeminar - JBoss Migration
Seminar - JBoss Migration
 
Modular Enterprise Systems - An Introduction
Modular Enterprise Systems - An IntroductionModular Enterprise Systems - An Introduction
Modular Enterprise Systems - An Introduction
 

Similaire à A Roadmap to Data Migration Success

Sricharan_Sana_11yrs_MDM_DM_CRM
Sricharan_Sana_11yrs_MDM_DM_CRMSricharan_Sana_11yrs_MDM_DM_CRM
Sricharan_Sana_11yrs_MDM_DM_CRMsricharan sana
 
Scope of Data Integration
Scope of Data IntegrationScope of Data Integration
Scope of Data IntegrationHEXANIKA
 
Accelerating Time to Success for Your Big Data Initiatives
Accelerating Time to Success for Your Big Data InitiativesAccelerating Time to Success for Your Big Data Initiatives
Accelerating Time to Success for Your Big Data Initiatives☁Jake Weaver ☁
 
The #1 Success Factor for Data Migration Projects
The #1 Success Factor for Data Migration ProjectsThe #1 Success Factor for Data Migration Projects
The #1 Success Factor for Data Migration ProjectsRoque Daudt, BCompSc
 
Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Nathan Bijnens
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsJane Roberts
 
Resume_Padmashree _Business Systems Analyst
Resume_Padmashree _Business Systems AnalystResume_Padmashree _Business Systems Analyst
Resume_Padmashree _Business Systems AnalystPadmashree Muralidharan
 
The Xoriant Whitepaper: Last Mile Soa Implementation
The Xoriant Whitepaper: Last Mile Soa ImplementationThe Xoriant Whitepaper: Last Mile Soa Implementation
The Xoriant Whitepaper: Last Mile Soa ImplementationXoriant Corporation
 
Real Time Analytics
Real Time AnalyticsReal Time Analytics
Real Time AnalyticsMohsin Hakim
 
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...Denodo
 
Neoaug 2013 critical success factors for data quality management-chain-sys-co...
Neoaug 2013 critical success factors for data quality management-chain-sys-co...Neoaug 2013 critical success factors for data quality management-chain-sys-co...
Neoaug 2013 critical success factors for data quality management-chain-sys-co...Chain Sys Corporation
 
Data Governance: Description, Design, Delivery
Data Governance: Description, Design, DeliveryData Governance: Description, Design, Delivery
Data Governance: Description, Design, DeliveryInnoTech
 
Getting Started with Advanced Network Operations
Getting Started with Advanced Network OperationsGetting Started with Advanced Network Operations
Getting Started with Advanced Network OperationsSchneider Electric
 
The Digital Evolution Simplifying Data Migration for Business Excellence.pdf
The Digital Evolution Simplifying Data Migration for Business Excellence.pdfThe Digital Evolution Simplifying Data Migration for Business Excellence.pdf
The Digital Evolution Simplifying Data Migration for Business Excellence.pdfComplere Infosystem
 
GouriShankar_Informatica
GouriShankar_InformaticaGouriShankar_Informatica
GouriShankar_InformaticaGouri Shankar M
 

Similaire à A Roadmap to Data Migration Success (20)

Sricharan_Sana_11yrs_MDM_DM_CRM
Sricharan_Sana_11yrs_MDM_DM_CRMSricharan_Sana_11yrs_MDM_DM_CRM
Sricharan_Sana_11yrs_MDM_DM_CRM
 
Scope of Data Integration
Scope of Data IntegrationScope of Data Integration
Scope of Data Integration
 
Accelerating Time to Success for Your Big Data Initiatives
Accelerating Time to Success for Your Big Data InitiativesAccelerating Time to Success for Your Big Data Initiatives
Accelerating Time to Success for Your Big Data Initiatives
 
The #1 Success Factor for Data Migration Projects
The #1 Success Factor for Data Migration ProjectsThe #1 Success Factor for Data Migration Projects
The #1 Success Factor for Data Migration Projects
 
Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
 
Resume_Padmashree _Business Systems Analyst
Resume_Padmashree _Business Systems AnalystResume_Padmashree _Business Systems Analyst
Resume_Padmashree _Business Systems Analyst
 
The Xoriant Whitepaper: Last Mile Soa Implementation
The Xoriant Whitepaper: Last Mile Soa ImplementationThe Xoriant Whitepaper: Last Mile Soa Implementation
The Xoriant Whitepaper: Last Mile Soa Implementation
 
Souvik_Das_CV
Souvik_Das_CVSouvik_Das_CV
Souvik_Das_CV
 
Planning Data Warehouse
Planning Data WarehousePlanning Data Warehouse
Planning Data Warehouse
 
Souvik das cv
Souvik das cvSouvik das cv
Souvik das cv
 
Real Time Analytics
Real Time AnalyticsReal Time Analytics
Real Time Analytics
 
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...
Quicker Insights and Sustainable Business Agility Powered By Data Virtualizat...
 
Neoaug 2013 critical success factors for data quality management-chain-sys-co...
Neoaug 2013 critical success factors for data quality management-chain-sys-co...Neoaug 2013 critical success factors for data quality management-chain-sys-co...
Neoaug 2013 critical success factors for data quality management-chain-sys-co...
 
مدیریت کیفیت داده
مدیریت کیفیت دادهمدیریت کیفیت داده
مدیریت کیفیت داده
 
Data Governance: Description, Design, Delivery
Data Governance: Description, Design, DeliveryData Governance: Description, Design, Delivery
Data Governance: Description, Design, Delivery
 
Getting Started with Advanced Network Operations
Getting Started with Advanced Network OperationsGetting Started with Advanced Network Operations
Getting Started with Advanced Network Operations
 
The Digital Evolution Simplifying Data Migration for Business Excellence.pdf
The Digital Evolution Simplifying Data Migration for Business Excellence.pdfThe Digital Evolution Simplifying Data Migration for Business Excellence.pdf
The Digital Evolution Simplifying Data Migration for Business Excellence.pdf
 
Koteswararao_Resume
Koteswararao_ResumeKoteswararao_Resume
Koteswararao_Resume
 
GouriShankar_Informatica
GouriShankar_InformaticaGouriShankar_Informatica
GouriShankar_Informatica
 

Plus de FindWhitePapers

The state of privacy and data security compliance
The state of privacy and data security complianceThe state of privacy and data security compliance
The state of privacy and data security complianceFindWhitePapers
 
Is your data at risk? Why physical security is insufficient for laptop computers
Is your data at risk? Why physical security is insufficient for laptop computersIs your data at risk? Why physical security is insufficient for laptop computers
Is your data at risk? Why physical security is insufficient for laptop computersFindWhitePapers
 
Closing the gaps in enterprise data security: A model for 360 degrees protection
Closing the gaps in enterprise data security: A model for 360 degrees protectionClosing the gaps in enterprise data security: A model for 360 degrees protection
Closing the gaps in enterprise data security: A model for 360 degrees protectionFindWhitePapers
 
Buyers Guide to Endpoint Protection Platforms
Buyers Guide to Endpoint Protection PlatformsBuyers Guide to Endpoint Protection Platforms
Buyers Guide to Endpoint Protection PlatformsFindWhitePapers
 
VMware DRS: Why You Still Need Assured Application Delivery and Application D...
VMware DRS: Why You Still Need Assured Application Delivery and Application D...VMware DRS: Why You Still Need Assured Application Delivery and Application D...
VMware DRS: Why You Still Need Assured Application Delivery and Application D...FindWhitePapers
 
The ROI of Application Delivery Controllers in Traditional and Virtualized En...
The ROI of Application Delivery Controllers in Traditional and Virtualized En...The ROI of Application Delivery Controllers in Traditional and Virtualized En...
The ROI of Application Delivery Controllers in Traditional and Virtualized En...FindWhitePapers
 
The Economic Impact of File Virtualization
The Economic Impact of File VirtualizationThe Economic Impact of File Virtualization
The Economic Impact of File VirtualizationFindWhitePapers
 
Geolocation and Application Delivery
Geolocation and Application DeliveryGeolocation and Application Delivery
Geolocation and Application DeliveryFindWhitePapers
 
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS Attacks
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS AttacksDNSSEC: The Antidote to DNS Cache Poisoning and Other DNS Attacks
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS AttacksFindWhitePapers
 
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...FindWhitePapers
 
Inventory Optimization: A Technique for Improving Operational-Inventory Targets
Inventory Optimization: A Technique for Improving Operational-Inventory TargetsInventory Optimization: A Technique for Improving Operational-Inventory Targets
Inventory Optimization: A Technique for Improving Operational-Inventory TargetsFindWhitePapers
 
Improving Organizational Performance Through Pervasive Business Intelligence
Improving Organizational Performance Through Pervasive Business IntelligenceImproving Organizational Performance Through Pervasive Business Intelligence
Improving Organizational Performance Through Pervasive Business IntelligenceFindWhitePapers
 
IDC Energy Insights - Enterprise Risk Management
IDC Energy Insights - Enterprise Risk ManagementIDC Energy Insights - Enterprise Risk Management
IDC Energy Insights - Enterprise Risk ManagementFindWhitePapers
 
How to Use Technology to Support the Lean Enterprise
How to Use Technology to Support the Lean EnterpriseHow to Use Technology to Support the Lean Enterprise
How to Use Technology to Support the Lean EnterpriseFindWhitePapers
 
High Efficiency in Manufacturing Operations
High Efficiency in Manufacturing OperationsHigh Efficiency in Manufacturing Operations
High Efficiency in Manufacturing OperationsFindWhitePapers
 
Enterprise Knowledge Workers: Understanding Risks and Opportunities
Enterprise Knowledge Workers: Understanding Risks and OpportunitiesEnterprise Knowledge Workers: Understanding Risks and Opportunities
Enterprise Knowledge Workers: Understanding Risks and OpportunitiesFindWhitePapers
 
Enterprise Information Management: In Support of Operational, Analytic, and G...
Enterprise Information Management: In Support of Operational, Analytic, and G...Enterprise Information Management: In Support of Operational, Analytic, and G...
Enterprise Information Management: In Support of Operational, Analytic, and G...FindWhitePapers
 
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...FindWhitePapers
 
Data Quality Strategy: A Step-by-Step Approach
Data Quality Strategy: A Step-by-Step ApproachData Quality Strategy: A Step-by-Step Approach
Data Quality Strategy: A Step-by-Step ApproachFindWhitePapers
 
Data Migration: A White Paper by Bloor Research
Data Migration: A White Paper by Bloor ResearchData Migration: A White Paper by Bloor Research
Data Migration: A White Paper by Bloor ResearchFindWhitePapers
 

Plus de FindWhitePapers (20)

The state of privacy and data security compliance
The state of privacy and data security complianceThe state of privacy and data security compliance
The state of privacy and data security compliance
 
Is your data at risk? Why physical security is insufficient for laptop computers
Is your data at risk? Why physical security is insufficient for laptop computersIs your data at risk? Why physical security is insufficient for laptop computers
Is your data at risk? Why physical security is insufficient for laptop computers
 
Closing the gaps in enterprise data security: A model for 360 degrees protection
Closing the gaps in enterprise data security: A model for 360 degrees protectionClosing the gaps in enterprise data security: A model for 360 degrees protection
Closing the gaps in enterprise data security: A model for 360 degrees protection
 
Buyers Guide to Endpoint Protection Platforms
Buyers Guide to Endpoint Protection PlatformsBuyers Guide to Endpoint Protection Platforms
Buyers Guide to Endpoint Protection Platforms
 
VMware DRS: Why You Still Need Assured Application Delivery and Application D...
VMware DRS: Why You Still Need Assured Application Delivery and Application D...VMware DRS: Why You Still Need Assured Application Delivery and Application D...
VMware DRS: Why You Still Need Assured Application Delivery and Application D...
 
The ROI of Application Delivery Controllers in Traditional and Virtualized En...
The ROI of Application Delivery Controllers in Traditional and Virtualized En...The ROI of Application Delivery Controllers in Traditional and Virtualized En...
The ROI of Application Delivery Controllers in Traditional and Virtualized En...
 
The Economic Impact of File Virtualization
The Economic Impact of File VirtualizationThe Economic Impact of File Virtualization
The Economic Impact of File Virtualization
 
Geolocation and Application Delivery
Geolocation and Application DeliveryGeolocation and Application Delivery
Geolocation and Application Delivery
 
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS Attacks
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS AttacksDNSSEC: The Antidote to DNS Cache Poisoning and Other DNS Attacks
DNSSEC: The Antidote to DNS Cache Poisoning and Other DNS Attacks
 
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...
Lean Business Intelligence - How and Why Organizations Are Moving to Self-Ser...
 
Inventory Optimization: A Technique for Improving Operational-Inventory Targets
Inventory Optimization: A Technique for Improving Operational-Inventory TargetsInventory Optimization: A Technique for Improving Operational-Inventory Targets
Inventory Optimization: A Technique for Improving Operational-Inventory Targets
 
Improving Organizational Performance Through Pervasive Business Intelligence
Improving Organizational Performance Through Pervasive Business IntelligenceImproving Organizational Performance Through Pervasive Business Intelligence
Improving Organizational Performance Through Pervasive Business Intelligence
 
IDC Energy Insights - Enterprise Risk Management
IDC Energy Insights - Enterprise Risk ManagementIDC Energy Insights - Enterprise Risk Management
IDC Energy Insights - Enterprise Risk Management
 
How to Use Technology to Support the Lean Enterprise
How to Use Technology to Support the Lean EnterpriseHow to Use Technology to Support the Lean Enterprise
How to Use Technology to Support the Lean Enterprise
 
High Efficiency in Manufacturing Operations
High Efficiency in Manufacturing OperationsHigh Efficiency in Manufacturing Operations
High Efficiency in Manufacturing Operations
 
Enterprise Knowledge Workers: Understanding Risks and Opportunities
Enterprise Knowledge Workers: Understanding Risks and OpportunitiesEnterprise Knowledge Workers: Understanding Risks and Opportunities
Enterprise Knowledge Workers: Understanding Risks and Opportunities
 
Enterprise Information Management: In Support of Operational, Analytic, and G...
Enterprise Information Management: In Support of Operational, Analytic, and G...Enterprise Information Management: In Support of Operational, Analytic, and G...
Enterprise Information Management: In Support of Operational, Analytic, and G...
 
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...
Enabling Strategy and Innovation: Achieving Optimized Outcomes from Planning ...
 
Data Quality Strategy: A Step-by-Step Approach
Data Quality Strategy: A Step-by-Step ApproachData Quality Strategy: A Step-by-Step Approach
Data Quality Strategy: A Step-by-Step Approach
 
Data Migration: A White Paper by Bloor Research
Data Migration: A White Paper by Bloor ResearchData Migration: A White Paper by Bloor Research
Data Migration: A White Paper by Bloor Research
 

Dernier

Mihir Menda - Member of Supervisory Board at RMZ
Mihir Menda - Member of Supervisory Board at RMZMihir Menda - Member of Supervisory Board at RMZ
Mihir Menda - Member of Supervisory Board at RMZKanakChauhan5
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessAPCO
 
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfTalent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfCharles Cotter, PhD
 
Live-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarLive-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarNathanielSchmuck
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxJemalSeid25
 
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGlokeshwarmaha
 
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...AustraliaChapterIIBA
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023Steve Rader
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsIntellect Design Arena Ltd
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...Brian Solis
 
Project Brief & Information Architecture Report
Project Brief & Information Architecture ReportProject Brief & Information Architecture Report
Project Brief & Information Architecture Reportamberjiles31
 
Data skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsData skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsyasinnathani
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfAnhNguyen97152
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBBPMedia1
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentationbaron83
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examplesamberjiles31
 
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003believeminhh
 

Dernier (20)

Mihir Menda - Member of Supervisory Board at RMZ
Mihir Menda - Member of Supervisory Board at RMZMihir Menda - Member of Supervisory Board at RMZ
Mihir Menda - Member of Supervisory Board at RMZ
 
WAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdfWAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdf
 
Investment Opportunity for Thailand's Automotive & EV Industries
Investment Opportunity for Thailand's Automotive & EV IndustriesInvestment Opportunity for Thailand's Automotive & EV Industries
Investment Opportunity for Thailand's Automotive & EV Industries
 
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for BusinessQ2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
Q2 2024 APCO Geopolitical Radar - The Global Operating Environment for Business
 
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdfTalent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
Talent Management research intelligence_13 paradigm shifts_20 March 2024.pdf
 
Live-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry WebinarLive-Streaming in the Music Industry Webinar
Live-Streaming in the Music Industry Webinar
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptx
 
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
 
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
IIBA® Melbourne - Navigating Business Analysis - Excellence for Career Growth...
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023
 
Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking Applications
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
 
Project Brief & Information Architecture Report
Project Brief & Information Architecture ReportProject Brief & Information Architecture Report
Project Brief & Information Architecture Report
 
Data skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsData skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story points
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
 
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John MeulemansBCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
BCE24 | Virtual Brand Ambassadors: Making Brands Personal - John Meulemans
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentation
 
Intellectual Property Licensing Examples
Intellectual Property Licensing ExamplesIntellectual Property Licensing Examples
Intellectual Property Licensing Examples
 
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
The Vietnam Believer Newsletter_MARCH 25, 2024_EN_Vol. 003
 

A Roadmap to Data Migration Success

  • 1. WHITE PAPER A ROADMAP TO DATA MIGRATION SUCCESS CONTENTS Data migration is the one-time movement of data from a legacy source, or multiple 2 Planning and Scoping the Migration sources, to a new target database. This simple concept and requirement can 2 Allowing for Dependency Management 3 Establishing Data Governance and drive a scope that is much larger than expected. A data migration requirement Stewardship can be driven by a range of initiatives, such as an application replacement or 5 Building the Migration Team upgrade, the need to consolidate data within a data warehouse, or a requirement to 6 Project Planning Considerations create a single view of product within an organization. In cases where migrated 6 Mapping Business Needs data is transformed for new uses, your project team encounters some very 6 Assessing Infrastructure 9 Understanding Data specific management and technical challenges. For example, a team that has been writing extraction, transformation, and load (ETL) code for a data warehouse faces a new set of challenges when migrating data to a live, operational system. Although a 2% error rate may be acceptable for aggregate reporting, it is not acceptable for customer contact data—in this example, we would fail to recognize one out of 50 customers when they call! Many significant business initiatives and large IT projects depend upon a successful data migration. Your goal is to minimize as much of your risk as possible through effective planning and scoping. The objective of this paper from Business Objects, an SAP company, is to provide insight into what issues are unique to data migration projects and to offer advice on how to best approach them.
  • 2. PLANNING AND SCOPING THE MIGRATION As with any project, success depends upon a good plan. Data migration projects have a distinct methodology and project approach. Too often, project managers make the costly mistake of thinking that migrating data is a simple task within a larger project and hand it off to a development team. Table 1 lists the migration tasks that must be considered. Table 1. Migration Tasks Migration Element Activities Business needs mapping Ensure that your migration plan is driven by the expectations and needs of the users Infrastructure There are unique infrastructure requirements that should be sorted as early as possible in the project timeline Identifying data sources and targets Validate the system of record for your source data and identify the data needs of your target database Evaluating the data quality Assess the quality of your data to meet the target application requirements and business needs Gap analysis between source and target Identification, and plan for mitigation, of any gaps between available source data and target application data requirements Impact of multiple sources of data Estimate the challenges of consolidating similar data from several sources, or integrating dissimilar data Mapping assessment Understand the effort required to accurately identify source data at column level detail, including transformation specifications Migration assessment Understand the effort required to design, code, test and implement the data migration In addition to these tasks, the following sections detail two additional project management considerations that are essential to any data migration – dependency management and data governance planning. ALLOWING FOR DEPENDENCY MANAGEMENT Don’t forget that most risk is outside of your control or scope. Numerous other projects, initiatives, and issues may influence a data migration project. Your project manager or team leader must be a good communicator and be aware of the decisions and changes happening in the larger environment. The following scenarios can change your plan and affect the scope of your data migration. Scenario: Larger Projects Data migration plans and schedules typically are driven by larger projects—for example a master data management (MDM) initiative, a new business process, or a data consolidation effort that supports a new reporting environment. Business Objects. A Roadmap To Data Migration Success
  • 3. Any changes to the parent project schedule affect the data migration schedule, including changes to production deployment, and the changes to test and beta deployment schedules. The parent project may change its approach to implementation, such as moving from a regional phased approach to a product phased approach. For example, in a cell phone company, rather than convert customers by region, the team converts all customers internationally that have the newest service plan, followed by a family service plan, and so on. What may be a straightforward change for the larger project could exponentially increase the scope of the data migration effort. It is not uncommon for resources to shift roles during a project. For example, the priorities of the larger project could drive key players away from the migration tasks, which are deemed less important than other development. Thus, your schedule is delayed and your risks increase. Scenario: Know Your Data The success of your migration project largely depends upon how well you know the data content and business rules. Your data experts help with identifying business needs, analyzing source systems, and testing, covering the full life cycle of your project. Experts likely will be from the business and tasked to the migration project on an as-needed or part-time basis. Scenario: Changing Source Systems and Business Rules The source data and the business rules that govern it may change during the course of your project. There could be other initiatives in the organization, new product offerings, or business process changes that impact the data. The source system itself could change, due to application or server upgrades. Application upgrades are particularly risky, since the application features and data model of the source may be altered. ESTABLISHING DATA GOVERNANCE AND STEWARDSHIP Involving business users and data experts in the migration project is the single biggest factor driving your success. Have you ever seen a data intensive project implemented successfully without organized, committed business user involvement? In order to fulfill your data migration needs, you must include a data governance strategy that provides leadership and direction for user input. In your organization, you may not refer to the strategy as data governance or stewardship, but regardless of what you call it, the key is that the data experts—often business users—are involved in all data migration phases and tasks. Business Objects. A Roadmap To Data Migration Success
  • 4. Your data migration project team must plan for the participation of data experts, set expectations for resource commitments, and initiate the identification of measurable success factors. Project planning must consider the two key roles for this team: 1) ensuring that the project team is aware of the business needs being met by the target application, and 2) providing detailed data expertise and ongoing monitoring (ownership) demanded by the new application. Team responsibilities include: • Participating in the identification of “best” sources or “system of record” sources • Actively reviewing, creating, and correcting data to support cleansing and reference (master) data requirements • Monitoring data quality and data auditing status • Sharing and capturing content detail about source system data and business processes • Testing and confirming application functionality that is supported by migrated data The approach to establishing a data governance team is specific to your organization. If possible, leverage the talents of an existing team or committee. Often a data warehouse user’s group is an excellent source of data knowledge, or you may be able to leverage a team that supported a previous project. Consider a metadata tool for the project team. The data migration project is a unique opportunity to capture metadata about your source systems and to initiate a business process for managing data in the target application. What can you leverage that you have in place now? What are the requirements for managing the metadata and the business rules for the data migration project? What are the requirements going forward? Consider: • The need to share content knowledge and business rules for multiple team members • Testing against identified business rules and mapping/transformation specifications • Historical documentation (audit trails) of the decisions made, and logic behind them, for the data migration • Historical documentation about the source of data, especially useful when merging data from multiple sources • Identifying data owners and experts for the migration and future data management requirements Business Objects. A Roadmap To Data Migration Success
  • 5. BUILDING THE MIGRATION TEAM Table 2 lists roles and responsibilities for the planning and scope phase of the data migration project. Table 2: Roles and Responsibilities for Planning and Scoping Team Member Responsibilities Project manager or team leader - Coordinating and communicating task status - Managing issues - Creating and maintaining project plan - Managing dependencies to larger project, target, and source application schedules - Coordinating the creation of a data governance strategy and team Business users (data experts) - Identifying and validating business needs - Analyzing target and source systems - Assessing data quality - Testing - Assessing data mapping and business rules for data transformation Business or data analysts - Infrastructure support, including database sizing, optimization, and life cycle management. Works closely with the database analyst. - Data support, including data analysis, data quality assessment, mapping, and migration assessment. Database administrator - Initial sizing for staging database(s) - Planning for life cycle management of database objects - Management planning for schema changes and user security Data architect - Architecture and methodology - Source and target application data analysis - Data quality assessment, mapping assessment - Data governance planning - Configuration management - Configuration standards and quality assurance Migration lead (for planning and scoping) - Architecture and programming standards - Data quality, data mapping, and data migration assessment - Source to target gap analysis and multiple systems impact assessment Consider using consultants for key roles on a migration project, since it is a one-time, custom effort and your team may not have the specific planning, strategy, or development experience. Focus your project plan on growing the skills needed in your organization, which may include future data migration, data integration, or data quality projects. Business Objects. A Roadmap To Data Migration Success
  • 6. PROJECT PLANNING CONSIDERATIONS The following sections address planning considerations for your data migration project. MAPPING BUSINESS NEEDS Business needs represent why you are migrating data. How many new applications go live but do not deliver the business value expected? You must articulate the business needs that are driving your project. In addition to ensuring that you are meeting the needs of the consumer, you are also controlling the scope of a data migration project, especially source systems analysis and mapping. At first glance, this may seem obvious, but consider as an example the full functionality of a customer relationship management (CRM) application. Organizations pick and choose from the menu of functions and features, and the implementation may be phased over time. Rather than migrating all of the data subject areas of a full customer database, the team can focus on only those tables and columns that support the business needs of the CRM application. A clear understanding of business needs also provides focus for the sizing estimates, data quality assessment, and test planning. Additionally, as unexpected data quality and gap analysis issues arise (and they will), knowing the business needs aids in identifying priorities for the project team. For example, if the customer age is not fully populated, or has inaccurate data (230 years old), the team can reference the known uses of that data to determine the extent of data cleansing needed. In another example, the medical history for an automated patient chart must be of the highest qual- ity when migrated. The business need, which is driven by legal requirements, must be clearly understood by the entire migration team. Ideally, the project team will include data experts and business users as part of a data governance strategy to ensure that this process is as straightforward as possible. ASSESSING INFRASTRUCTURE A data migration project demands a dedicated infrastructure. Don’t make the mistake of using your existing development and test environments. For example, do not underestimate the planning needed for a dedicated staging area or for load testing (yes, load testing!) You might be converting many years worth of data in a single weekend. Far too many organizations wait until the beta or production conversion to discover performance issues that can set the schedule back by weeks or months. Business Objects. A Roadmap To Data Migration Success
  • 7. Current Architecture What is your current architecture? Do you have data quality and data integration tools needed to accomplish a data conversion? Is there a database or server available to host your data and code? When and how will you test the full conversion to ensure that your performance will meet the schedule needs? The team should evaluate what servers, databases, and software tools are available to host a migration project. Leverage what you can and do not reinvent the wheel. Specifically, review: • Requirements documentation and capture • Metadata and data integration mapping repositories • Data integration tools • Data quality tools • Code management tools and processes • Test tools and processes Given that the data migration is a one-time project, you should evaluate what code management processes can be bypassed or customized for this effort. This may sound like a surprising recommendation, but consider the environment surrounding the data migration project. Your schedule is not your own; rather, it is dictated by the larger project. Also the priorities and phased approach can change at any time. The data migration team needs to be very agile. The approach to analysis, coding, and testing should be lightweight and mutable. Target Architecture The dedicated infrastructure for the project should be planned as part of your scope phase. Do not make the error of waiting until the development phase of the project to discover that you need additional resources for integration testing, such as servers or new database instances. The planning for this infrastructure should include: • A staging database for analysis and development. Impact on production services is minimized by the use of a separate migration server and by the use of configured staging and process areas within the service to manage multiple data versions, sources, and status values. Your team can also use the staging area to created needed reference data for the target system. • A sizing estimate for the staging database, which includes any needed replication required to support both migration code development and application Business Objects. A Roadmap To Data Migration Success
  • 8. • A code life-cycle management plan • Performance test requirements • Backup and recovery planning • Scalable connectivity requirements and approaches (for example, how will you extract and load data to the source and target systems?) Planning for Testing The test requirements for a data migration project are unique, as you may have multiple conversions during parallel and beta testing of an application. Adding to the complexity is the fact that the migration team is supporting the application test requirements of the larger project and also testing the conversion code itself. Your migration team needs to first consider the basics of audit and reconciliation. This is missed more often than it should be. Are you looking at the right set of data? Can you trace back to your source system at a record level? Include basic record counts and validation of ETL processes. Many of these processes will be accomplished multiple times, as testing and re-loads occur. The audit statistics need to be reported and evaluated each time. Also, if the project is a phased implementation, you may be running similar data across parallel databases. Audits and reconciliation are needed to ensure that the data is consistent across these applications. You should also plan for performance testing. Many projects are delayed because the development team has successfully converted a test set of data but waited until the last minute to run the conversion of the full set. A customer consolidation project for one of the largest global computer manufacturer’s was delayed by months when the development team attempted to convert (at go-live!) a year’s worth of customer data – using code that had been developed using a day’s worth of test data. For your conversion, there may be millions of records to process, with complex referential integrity. Performance issues for data processing at this level of magnitude can require significant code changes. What to do? Estimate your load and include this consideration when unit and integration testing. Participate in peer reviews, or contract with experienced consultants, to address code optimization, data query tuning, and database tuning challenges. And include these options in your earliest planning. The primary reason for capturing the metadata around your business rules is to test the mapping and transformations. All phases of testing, unit, integration, and target application, should include a validation of the business rules. Business Objects. A Roadmap To Data Migration Success
  • 9. As early as possible in your migration schedule, allow the business users to test the data in the target application. Does it support the features and functionality as expected? Data migration mapping is very abstract, especially if the users have never been part of a similar project. They may not “get it” until the converted data is actually supporting the application features and processes that they are implementing. Allow time in your planning for changes to the mapping, business rules, and re-loads. It is not uncommon for there to be confusion during beta testing and implementa- tion as to where errors originate. It’s all too easy for “bad data” to be blamed for every test error. What can you do as part of your planning to avoid this? Implement the migration with as much metadata and reconciliation as possible. Not only are you ensuring the integrity of your data, you are creating the lineage necessary to support the integration test process. UNDERSTANDING DATA Data migration is, in fact, all about the data. Data related tasks, especially those around quality, will be the bulk of your work. Plan enough time to evaluate the source data, to compare the available data to the needed data, and to drill down to the detail needed for source to target mapping. Your goal is to ensure that the data migration development is the shortest task on your project plan. You must understand the source data and target needs before you start coding. You don’t want to experience the very costly surprise of learning that the source data is not “fit to use” at integration testing or implementation. Evaluate Data Sources and Targets As part of the planning process, your team will take an initial look at the source and target systems. The objective is to know enough about each system to make an estimate of the effort to complete mapping and migration. At a high level, the team will identify subject areas, data entities, and supported functionality. For example, customer address data will come from the billing system, customer phone contact data will come from the data warehouse, and so on. The planning team needs to look for the following risks and complexities: • The commonly agreed to, or popular, system of record may not be the best source of data. Consult as many data experts as possible and look for reconciliation statistics, compliance numbers, and other forms of audit to identify a trusted source. Business Objects. A Roadmap To Data Migration Success
  • 10. • Multiple sources increase your schedule exponentially. Never assume that a second source of data will make your project twice the effort. If you have one source of data you may have to look for duplicate data, but if you have two sources you have to look for duplicate data, harmonize reference data, and run record-matching algorithms to determine correct mapping. (A requirement for sharing master data across multiple applications is a project in itself. The need to harmonize reference data is complex enough to warrant its own white paper.) • Certain types of projects introduce their own challenges. For example, the approach to managing a data migration that is part of an MDM project will be distinct from other kinds of migrations, with a focus on dependency management, data quality, and data governance. An MDM implementation requires initial data migrations, but also ongoing integration and reconciliation. The business rules, matching criteria, and governance processes that are established for migration will be propagated to support the live application. Also, mergers and acquisitions require data consolidation, with a focus on reference data and business processes (that are represented in the data structures and content). Even if the two organizations have the same software the mapping may not necessarily be straightforward. Another challenge is that a data warehouse migration can seem less complicated because the data is relatively static in the target database; however, the expectations of the business users can be extensive. In a well- designed warehouse, one data mart will support multiple business units. Also, the data migration process is likely to be the initial load of an ongoing ETL process and there should be reuse of many components. • Conduct a gap analysis. The features that the business expects from the target system may not have a data source at all, and the migration team may be the designated messenger to carry this bad news. For example, in an MDM customer data project, one region out of five has never captured customer contact history and, when consolidating the customer data, a subset will not have a full history. The business requirement to “establish customer segmentation based on contact history” is seriously impacted. Your planning should include the time needed and the approach for resolving gaps in the source to target data mapping (and a cost benefit of the effort). There are several options, and each will increase the scope of your project: Business Objects. A Roadmap To Data Migration Success 10
  • 11. • The business users and data experts could manually create data. This is not uncommon when creating reference data, such as product types or customer status. The project plan will include the time needed to work with the governance team and facilitate the data entry, updates, and testing. • You may have the option to source data externally—for example, using Dun Bradstreet data to create more consistent customer data. • You could default the data to an unknown value and plan to populate it over time, as part of the target application. Using the customer segmentation example above, if it requires a year’s worth of customer contact history, the business will accept the 12-month delay before having a 100% correct segmentation calculation. Figure 1: A Sample Data Profiling Report that Shows the Frequency Distribution of Values Perform a Data Quality Assessment The biggest factor in your success—and your biggest risk—is the quality of the source data. Many projects fail, or are significantly delayed, upon discovering that the data is not “fit to use.” The most important phase in a data migration project involves the tasks needed to understand the data content. You would not put dirty petrol in your new car—why would you map bad data into your new software application? Business Objects. A Roadmap To Data Migration Success 11
  • 12. It’s unlikely that anyone on your team really understands the current state of the data at the level of detail needed for your project. And even if the source data is pristine, that doesn’t mean that it is fit for the requirements of the new application. Experience shows that unexpected data conditions encountered in load programs can significantly impact the timelines of data migration projects. Early assessment of data quality reduces project risk. Your team will do an initial data quality assessment as part of the planning process. The sooner that you look at the data, the more likely your success. For this initial pass, you are looking for the big-ticket items – missing data, data that is not fully populated, columns names that are inconsistent with content, and so on. Your goal is to know enough to estimate the level of effort to get the migration job done. You have done a review of business needs and this will drive the type of data you review. For example, if the objective is to create a consolidated view of product data across six regional SAP® applications, you may pull a subset of product data from each and attempt to match them. How good is the data? What are the challenges? This should give you a pretty clear idea of the project requirements for mapping and migration. Another example is pulling customer data from two systems, such as billing and CRM. You want to run some high-level reconciliations and audits between the two systems, looking for deltas. What if you have customers who have products but they’re not getting billed? It has happened! What if you have historical data for billing that isn’t represented in your CRM application? When capturing metadata, a tool is a good option – but when assessing data quality, a profiling tool is a must. Use data profiling technology to systematically scan the tables of interest to quantify any data quality and data integrity issues, such as: • Missing values • Pattern analysis • Frequency analysis • Ranges and outliers • Redundant data analysis • Foreign key integrity analysis • Statistics relevant to numeric fields • Parsing requirements to enable mapping of source columns to the target columns Business Objects. A Roadmap To Data Migration Success 1
  • 13. Perform a Data Mapping Assessment Hand-in-hand with data quality is data mapping. What are we moving and what are the rules? How much time will it take the team to complete data mapping? Given your planning, you should have a general idea of what it will take to create the final design and implementation. The captured business needs, the source and target system metadata, and the data profiling results from the data quality assessment all create the information need to understand the mapping effort. Much of the mapping will be data analysis work, if done right. It is not uncommon for teams to accomplish this mapping based on guessing which column names match in a spreadsheet, or worse, jotting a few notes down on a hardcopy of the DDL listing before coding. Be aware that there are development teams capable of this behavior. Where possible, identify the data challenges. One example illustrates how complex your mapping can become. An insurance premium discount for a new membership system is stored in a single column on a single record. However, in the custom source systems, it is stored in multiple locations and averaged according to complex business rules. These rules are captured in the source system code. There are no data experts who understand fully how the logic works. The data migration team must recreate this logic in the data integration code in order to populate the target system value. To make it even more complicated, the data conversion is an ongoing process that happens at policy renewal, and getting the calculation wrong will cost the company money. If the discount is too large, the organization loses money; if it’s too small, the organization may lose customers. Finally, there is the importance of capturing this logic in the metadata or design of the data migration for auditors, actuaries, and the membership support teams. Some of the things to consider when estimating the time it will take to accomplish data mapping are: • Know that some of your target requirements will require significant transformations, or the creation of new data content. For example, the mapping scope should include the effort to create reference data for the target application, such as product types, customer status, and vendor segmentation types. It’s not uncommon for the team, working with the data experts, to create these and set up some cross-reference tables to support the conversion of the data. This should be planned early in the project, and data quality validation should be included, such that all records are converted. Business Objects. A Roadmap To Data Migration Success 1
  • 14. Also, matching and consolidating is required when sourcing from multiple systems. It could be that you need to make a match decision on the best record from multiple source systems, rather than having a single system of record. Or that your reference data is so in your target system that you need to combine multiple records to make a “best of” instance. Data quality matching could also be used to eliminate duplicate data within a single source. This logic could be implemented to select a best of record that will later be evaluated by a data expert. The logic and the process to identify the ‘good record’ should be part of your mapping specification. Figure 2: Requirements Gathering and Mapping Tool Within a Multi-Developer Environment • Allow time to look at the data. Don’t allow your team to fall into the trap of mapping based on column names, copybooks (remember those?), or DDL. For each subject area, you should allow at least a week of time for the analyst to gather information about the meaning of the data, how it is populated (data profiling), and how it supports the target application. Business Objects. A Roadmap To Data Migration Success 1
  • 15. • For high priority data content, the team should learn as much as possible about the business processes that create the data and the target applications business processes. Data experts and business users are an invaluable source for this information. For example, a patient record in a hospital radiology application has some very basic requirements for radiologist review. There may be scheduling and workflow included in the application. If that data is mapped to an “automated patient chart,” it will be used by physicians other than radiologists to make broader diagnoses and evaluations. What are the differences in these processes—and does the data support the new needs represented by the automated patient chart? • The planning should include the evaluation of a mapping tool, or mapping functionality within a data integration tool. Keys things to look for are the ability to easily connect to and analyze the source data as part of the mapping process, the ability to share analysis results across a larger team, and how well the mapping drives the development of code. • Include validation and reconciliation steps in your mapping. • For any matching that you do, or any consolidation, verify that each source record is present in the source systems as designed. Each subject area mapped must have a reconciliation base to the source, validating that all records extracted were processed. For example, if I have 2,000 customers on my billing system, then I should have 2,000 customers on my new CRM system. External to the migration process, you should also have a process that is designed for a checks-and-balance approach. Another approach is to create a high-level mapping to a database that is wholly external to the migration process, thus creating a further audit validation. For example, you may map your in-patient data in an automated chart application to hospital admissions data that is captured by a financial system. Is this overkill? That entirely depends on your business user community and the extent to which you want to guarantee trusted data. Business Objects. A Roadmap To Data Migration Success 1
  • 16. Figure 3: Business Objects Provides the Ability to Standardize, Cleanse, and Match Data Regardless of Type, Source, or Target Perform a Data Migration Assessment Effective planning will allow you to mitigate the risks associated with your project. If you have done the work of scoping the infrastructure, the mapping, and the data quality assessment, then the migration itself will be your shortest task. When it comes time to migrate your data, the people and the processes will make it work. It’s more a team effort than another other type of IT project. Throughout the migration process, you will have checks and balances in place, including a team of data experts available to address the expected data content decisions and to help resolve the unexpected. This can include team members who are clearly responsible for tracking set of audit and reconciliation metrics to evaluate: • Do the record counts match from source to target through each phase of your processing? • Can you reconcile counts and reports to other systems in the organization that host the same data, for example numbers of active customers, or product sales. This is especially important if you are running systems in parallel. Can you track the deltas between the parallel systems and can you understand them? Business Objects. A Roadmap To Data Migration Success 1
  • 17. • Do the data experts have access to review the data for quality and consistency? • Can you create an audit back to your source system that would satisfy financial compliance requirements? • An approach to a phased project and how to keep parallel systems in synch as you migrate forward. This can include ongoing integration or replication between applications for the duration of the project. Agility is key. This is a custom (bespoke) development project and you should apply the rigor that you would to any other development project. However, keep the implementation approach as lightweight as possible. There are many dependencies to other projects and applications within the organization. Plan for the inevitable need to change scope, schedules, and roadmaps as quickly as possible. It‘s not unusual for a new application to source from multiple databases, flat files, or external feeds. Additional sources will increase your complexity exponentially. Consider your approach to rationalizing the data across these sources. Also what is your cutover strategy? And how will these sources impact your schedule? If you are de-duping or consolidating data across sources, consider what your record- matching requirements are—and be certain to evaluate your data integration soft- ware with this in mind. Finally, map all of your data integration requirements and data quality needs against an off-the-shelf toolset. Simple things, such as populating a gender column based on customer name, come out of the package, and save significant amounts of time. The more complicated needs, such as matching and consolidation, are infinitely easer using dedicated tools. Don’t make the mistake of thinking you are saving money by hand coding—you are risking your schedule and the quality of your product. Business Objects. A Roadmap To Data Migration Success 1
  • 18. Figure 4: A Data Integration Tool Provides the Ability to Extract, Clean, Transform, and Deliver the Data Between the Source(s) and Target Applications Perform a Post-Migration Needs Assessment Finally, as part of your planning process, do not forget the needs of your team going forward. These include: • The need for archiving data that is not converted, or needs to be reported on in its historical structure. • The ongoing reconciliation and balancing between the new application and parallel applications. For example, if you implement a new customer data mart, it should include a frequent reconciliation back to its source (lineage) and to another unrelated customer application (trust). • It’s likely that the data governance team has determined resolution of a set of data quality issues can be deferred post-production. Plan to capture as much detail as possible over the course of your project. Business Objects. A Roadmap To Data Migration Success 1
  • 19. DATA MIGRATION SUCCESS If you have successfully implemented your data migration, you should have data governance, standards, integration, and quality processes in place. Leverage these on your next project. Use the migration effort as a route to growing your overall information management skills, which will make subsequent data projects easier, faster, and cheaper. Business Objects. A Roadmap To Data Migration Success 1
  • 20. businessobjects.com © 2008 Business Objects. All rights reserved. Business Objects owns the following U.S. patents, which may cover products that are offered and licensed by Business Objects: 5,555,403; 5,857,205; 6,289,352; 6,247,008; 6,490,593; 6,578,027; 6,831,668; 6,768,986; 6,772,409; 6,882,998; 7,139,766; 7,299,419; 7,194,465; 7,222,130; 7,181,440 and 7,181,435. Business Objects and the Business Objects logo, BusinessObjects, Business Objects Crystal Vision, Business Process On Demand, BusinessQuery, Crystal Analysis, Crystal Applications, Crystal Decisions, Crystal Enterprise, Crystal Insider, Crystal Reports, Desktop Intelligence, Inxight, the Inxight Logo, LinguistX, Star Tree, Table Lens, ThingFinder, Timewall, Let there be light, Metify, NSite, Rapid Marts, RapidMarts, the Spectrum Design, Web Intelligence, Workmail and Xcelsius are trademarks or registered trademarks in the United States and/or other countries of Business Objects and/or affiliated companies. All other names mentioned herein may be trademarks of their respective owners. February 2008 WP3127-A