Data normalisation is the conversion of values into a uniform form so that they can be compared and combined. It is the most tedious yet also the most important step before any analysis, matching, or model deployment. Typically normalised are phone numbers into international format, addresses and municipality names, legal forms in company names, dates, currencies and units, diacritics and letter case, empty values, and the way missing data is recorded. Without this, the classic problem arises where the same value appears in statistics as five different ones. Write down the normalisation rules and use the same ones for every import, otherwise the database will drift apart again. Keep the original value alongside the normalised one, so that what was in the source can be verified retrospectively.
See also: Record deduplication, Data dictionary, Data quality in analytics.
