Record deduplication is the removal of multiple records about the same entity. In databases of customers and companies it arises naturally – the same company gets entered with a different form of its name, with a different legal form in the name, with a typo, or via a different contact. The consequences are distorted counts, the same person being contacted multiple times, and an incorrect assessment of customer value. The procedure has three steps. Normalising values into a uniform form so that they are comparable. Finding matching candidates by a unique identifier where one exists, otherwise by a combination of characteristics with tolerance for minor differences. And finally merging, with a clear rule for which record is the primary one and which data is carried over. Do the merging reversibly and with a record kept, so that it can be undone.
See also: Record matching, Data normalisation, Master data management.
