Data migration: ‘ETL’ reasons for work interruption

1. Incorrect data sequence: Attempt to load a dependent dataset before loading the base dataset. (Master data is loaded before loading configuration data)
2. Incorrectly formatted data: all data that does not conform to the structures of the target system.
3. Invalid data: correctly formatted information, but with incorrect content; usually identified during reconciliation.
4. Data volume: The data load exceeds the expected capacity of the method used to transfer the data.
5. Hardware or network failure: could only load a partial dataset into the new system due to connection delays for some reason.
6. Loading data into incorrect environment: a connection pointed to incorrect information to load.

Data migration based on complexity rating

Type of complexity and their definition:

• Number of input files / tables or types of records / rows
  • The number of data files and / or different tables used as input to the conversion program. The number of snapshots of files and / or records required to process the input files / arrays will also be taken into account.
• Number of output files / tables or types of records / rows
  • The number of different data files and / or tables supplied as output from the conversion program. The number of snapshots of files and / or records required to process the output files / tables will also be taken into account.
• Percentage of data translations
  • The percentage of data translations required in the converter program to translate a field value from the source system file / table to the target system file / table. The percentage is expressed relative to all fields in the data feed.
• Selection criteria
  • The data parameters used to restrict the data records or rows processed by the conversion program. While the selection criteria are often related to the number of records / rows of entry, the extent and type of selection criteria needed to produce the complexity of the conversion program.
• Complexity of data manipulation
  • Data manipulation required as part of a migration program. Data manipulation processing, including calculations and logic flows, will be considered. Calculations will be evaluated based on the number of mathematical functions required per calculation and the origin of the input digits; logical paths will be evaluated based on the anticipated difficulty of the required programming logical structures.
• Propensity of system customization changes
  • The possibility that one or more conversion systems undergo changes which could impact the development of the conversion program. If one or more of the systems involved in a conversion process have been recently implemented or are currently under development, the potential for changes to the new system is greater than for a system with a higher degree of maturity. Changes to a conversion system often initiate changes to data conversion.