Validation of collected data is a set of checks carried out before the data reaches reports or further processing. It verifies whether the required fields are filled in, whether the values correspond to the expected format and range, whether there are no duplicates, and whether the total number of records matches the usual state. A useful practice is to manually check a small sample against the original source, because an automatic check reveals missing data but not values systematically assigned to the wrong columns. Records that fail the check should not be silently discarded or forcibly corrected, but set aside in a separate record for review – analysing them is often the best source of information about what changed in the source.
See also: Data quality in analytics, Data normalisation, Missing values.
