Legacy Source Data Import¶
Data from external sources, generated as CSV flies, are loaded to the legacy data container. User can select the mode (load from server or load from client) and it is recommended to use server loading, for large data bulk. For each data load a load sequence will be generated. In order to be used in the Mapping, it must be locked. User will be able to define date format, thousand separator format and decimal format and these will override the formats define in the source.
The first step is to Load the Legacy Data from outside Data Migration Manager. This is done by first define the Legacy Data Header, which is based on the Migration Project ID, the Source from where the data originates from and finally the Legacy Data Table Name.
When the Header is defined, the next step is to define how the Data File is structured, that is, what Character set, Field Delimiter, String Delimiter, Date and Number format is used in the Data File that will be imported. Last step is to define where the data file is located and how to import if, from Server or Client.
When the file is imported, it creates a Load Sequence number value, so it is possible to keep track on different imports of the same Table over time.
Read more on Legacy Table Definition
Loading from Client¶
- Loading from client will read the whole file to the buffer using the default encoding in the user’s machine and based on the delimiter provided it will break it into lines.
- Later it will read line by line and break the line into columns using the string delimiter provided.
Loading from Server¶
For better performance data is loaded using the Oracle’s external tables concept
Database directories must be set at the time of DMM initial installation
In case of loading failure, error message logged in bad file created is read and added to the legacy load message column.
When the same file is loaded next time all the temporary files will be deleted and regenerated.
Note: this option will not be allowed in public Cloud, only Private Cloud
Lock Legacy Load¶
When the File is imported the data Load needs to be Locked before it can be used. Only one Data Load can be locked at the same time for a Legacy Table. This means that, if the Legacy Table has the same structure for each Load, Data Migration Manager can use the same Mapping to handle different loads from the Legacy Table. To make sure that the Data is structured the same way a Legacy Table Definition is created when the Load is locked. Then Data Migration Manager, can compare the different Loads Table Structure and create a warning if they are not the same.