For dirt-cheap web scraping, search Elance for “web scrapping” (!) Good luck!
For reliable and accurate web scraping, go to a professional web scraping company like ScraperWiki or an independent software engineer like me.
I offer pay-monthly managed web scraping starting at $50 per month, depending on size and complexity.
My Web Scraping Experience
I’m a Software Engineer with a background in Electronic Engineering. I write clean and supportable code - see some of my work on GitHub.
I’ve written high-reliability web scrapers for many commercial clients at ScraperWiki as well as for my own projects.
Examples of my web scraping work:
- web scraping company financial reports
- pulling property price data from an API
- web scraping director and shareholder information from eg. Companies House
- web scraping news bulletins from stock exchanges
- processing procurement data to power parts of UK Government
- generating an events calendar from bookings on JustPark
- reliable commodity price web scraping
- accurate utility bill web scraping
- scraping contact details from thousands of websites
Web Scraping as a Service
First and foremost contact ScraperWiki. They have the capacity to take on large, complex projects and their Data Scientists can do additional data visualisation and analysis.
Disclaimer: I worked for ScraperWiki in 2013⁄2014. They’re great and we’re still friends!
If they’re not suitable for you, email me with your specific requirements and we can arrange a call to discuss further.
I particularly enjoy working with charities, NGOs and Governments and I only do “ethical scraping” so no spammers please!
Data Output
After web scraping I can provide a data feed in many different forms, for example:
- Daily / weekly / monthly email attachment.
- JSON API endpoint.
- Regular SQL database dump.
- SMS / Fax / Letter (!)
- Most other ways you can think of!
Ethical Web Scraping
I believe in “ethical scraping”, operating within a set of principles:
- A website that welcomes human visitors should welcome robots too.
- A website should not discriminate against well-behaved robots; one that welcomes Google should welcome others too.
- A scraper bot should try to identify itself as such, preferably with a contact URL and/or email address.
- A scraper bot should not disrupt the service being scraped (it should use rate limiting and backoff).
- A scraper should not be built to gain free access to an otherwise paid-for data product.
- A scraper should not attempt to circumvent the security of a website.
- A scraper should not facilitate plagiarism or copyright infringement.
- A scraper should not help spammers or bad guys.
- A scraper should not use anonymising proxies, Tor or dirty tricks to hide its origin.
If that’s all cool with you, then get in touch.
People found this page while searching for:
- accurate web scraping
- reliable web scraping
- web scraping government
- uk based web scraping
- web scrapping