Legitimizing Scraping As A Data Source For APIs

Harvesting or scraping content and data from other public web sources has been something many do, but few will talk publicly about. While scraping does infringe on copyright in some situations, in many others situations, it is quickly becoming a legitimate way to acquire content or data, for use behind an API.

There is a lot of content available online, where the current stewards do not have the control, resources, or interest in making content available in a machine readable format. In these scenarios, for many developers, if you want the content, you just write a scrape script, and liberate it from the site, to be used as you wish.

In the last year, we are seeing a new breed of API service providers emerge, who assist users in deploying APis from data and content that is liberated though harvesting or scraping. For the first time, I'm seeing enough of these new tools and services, that I'm going to break out into its own area, and make part of this API deployment white paper.