Fork me on GitHub


The harvest or scraping of content from web pages is something many do, few discuss that they do, and some feel it is wrong to scrape content witihout permission..

No matter how you feel, there are numerous approaches to scraping data and then deploy an API. It is something many developers find themselves doing to get the data and content they need.

A new breed of scraping tools are emerging that don't just extract datq and content from websites, but also normalizes and deploys API interfaces for integrating with content--proving to be one possible option for deploying APIs at web scale.

This project is meant to understand the various tools and approaches to deploying APIs from scraped data.


Introducing API search and sharing features

(Posted on )

Were excited to announce new features to help you discover and customize kimono APIs extracting data from across the web: API search: Search our fast growing database of over 50,000 public kimono APIs to find the data you want.Search for real estate, jobs or bitcoin APIs to get started.Existing API panel: Load extraction rules from other APIs built from the webpage you are kimonifying.When you launch kimono, you will see a panel showing existing API templates.Click one to load it in your browser. more.

GeoJSON into ScraperWiki will go!

(Posted on )

Surely everyone likes things on maps?Driven by this thought we’re produced a new tool for the ScraperWiki Platform: an importer for GeoJSON.GeoJSON is a file format for encoding geographic information.It is based on JSON which is popular for web based APIs because it is light weight, flexible and easy to parse by JavaScript – the language that powers the interactive web.We envisage the GeoJSON importer being handy to visualise geographic data, and export that data to software like Tableau using our OData connector. more.

The story of getting Twitter data and its “missing middle”

(Posted on )

We’ve tried hard, but sadly we are not able to bring back our Twitter data tools.Simply put, this is because Twitter have no route to market to sell low volume data for spreadsheet-style individual use.It’s happened to similar services in the past, and even to blog post instructions.There’s lots of confusion in the market about the exact rules, and why they happen.This blog post tries to explain them clearly! more.