It’s not just search engines. Lots of people on Mastodon were using robots.txt to block ChatGPT (and any other LLM company they knew of) from scraping their sites/blogs.

I disagree, to a point. I want to be able to control my services to the greatest extent possible, including picking who scrapes me.

On the other hand, orgs as large as Google doing this poses a real threat to how the internet works right now which I hate.