Data Loading from Flat Files

It’s an accepted best practice to always load a flat file into a staging table before any transformations are done on the data in the flat file.

Always use LTRIM, RTRIM functions on string columns before loading data into a stage table.

You can also use UPPER function on string columns but before using it you need to ensure that the data is not case sensitive (e.g. ABC is different from Abc)

If you are loading data from a delimited file then make sure the delimiter is not a character which could appear in the data itself. Avoid using comma-separated files. Tilde (~) is a good delimiter to use.