Please stop externalizing your costs directly into my face
120
votes
Showing only topics with the tag "crawlers.web".
Back to normal view
-
-
Trapping misbehaving bots in an AI Labyrinth
40 votes -
FOSS infrastructure is under attack by AI companies
39 votes -
LLM crawlers continue to DDoS SourceHut
11 votes -
Nepenthes: a tarpit intended to catch AI web crawlers
33 votes -
Websites are blocking the wrong AI scrapers (because AI companies keep making new ones)
18 votes -
Robots.txt governed the behavior of web crawlers for over thirty years; AI vendors are ignoring it or proliferating too fast to block
41 votes -
The creators of TikTok caused my website to shut down
12 votes