- cross-posted to:
- technology@lemmy.zip
- technology
- cross-posted to:
- technology@lemmy.zip
- technology
You must log in or register to comment.
They’re already ignoring robots.txt, so I’m not sure why anyone would think they won’t just ignore this too. All they have to do is get a new IP and change their useragent.
Cloudflare is protecting a lot of sites from scraping with their POW captchas. They could allow people who pay
How can I do this without Cloudflare?