ScraperWiki
ScraperWiki is an effort to make web scraping collaborative by collecting and organizing scripts and keeping them online.
From the about page:
There's lots of useful data on the internet - crime statistics, government spending, missing kittens...
But getting at it isn't always easy. There's a table here, a report there, web pages, PDFs, spreadsheets... And it can be scattered over thousands of different places on the web, making it hard to see the whole picture and the story behind it.
It's like trying to build something from Lego when someone has hidden the bricks all over town and you have to find them before you can start building!
To get at data, programmers write bits of code called 'screen scrapers', which extract the useful bits so they can be reused in other apps, or rummaged through by journalists and researchers.
But these bits of code tend to break, get thrown away or forgotten once they have been used, and so the data is lost again. Which is bad.
ScraperWiki is an online tool to make that process simpler and more collaborative. Anyone can write a screen scraper using the online editor, and the code and data are shared with the world.
Because it's a wiki, other programmers can contribute to and improve the code. And, if you're not a programmer yourself, you can request a scraper or ask the ScraperWiki team to write one for you.
collaboration, python
And it's free! Free as in beer!
Added by chrisamico
How do you use ScraperWiki?
Please login to comment