API as a data source

An API is an interface through which a provider makes data available in a structured form intended for machine processing. Where it exists, it is always a better choice than scraping content from pages: the data is accurate, the format is stable, the terms of use are clearly defined, and the solution does not break when the website's appearance changes. Access is usually tied to a key and to a limit on the number of calls per time unit, which must be respected when designing the solution. Before deployment it is worth verifying three things: exactly what the interface returns and in what structure, how pagination of larger volumes is handled, and whether the provider guarantees versioning, that is, that an older version of the interface will not disappear from one day to the next. The terms of data use are usually set out in the licence and should be read.

See also: Legal aspects of scraping, Rate limiting and ethical scraping, Data formats.