Introduction
I knew there was a cloud service called ScrapyCloud for the Python scraping library scrapy, but I’d never used it. When I tried it, it was amazing, so let me introduce it.
What’s amazing about it is that with AI-powered scraping, you can scrape without writing any code at all.
Getting started
You need to register a credit card when you create an account, but there’s a free tier, so as long as you stay within it there’s apparently no charge.
How to use it
After finishing account registration, you create a project and create a new Spider. There, you select Zyte’s AI-Powered Spiders.
After that, you just enter the URL of the site you want to scrape, and scraping starts automatically.
You can download the data in formats like csv, json, and xml.
Here is the data that was actually scraped.
Once it’s turned into data, you can do whatever you like with it in Python.
About pricing
It seems the Zyte API is used to make requests. I made 100 requests and $0.72 worth was used. It feels like about 1 yen per request.
Summary
I thought the amazing thing about ScrapyCloud is that with AI-powered scraping, you can scrape without writing any code at all. If you can scrape without writing a single line of code, it might be worth paying for.
