Maybe I’m way too cynical but I wonder if disrupting these kinds of networks will have any impact on LLM scrapers and crawlers? My memory is imperfect but I feel like I remember conversations on...
Maybe I’m way too cynical but I wonder if disrupting these kinds of networks will have any impact on LLM scrapers and crawlers? My memory is imperfect but I feel like I remember conversations on Tildes in the past year talking about how there was little to no pattern in IP addresses that were used to crawl/scrape websites, which is why some of the “tarpit” traps were developed?
It sounds technically kind of similar to what Tor does? But black hat instead of grey hat.
Maybe I’m way too cynical but I wonder if disrupting these kinds of networks will have any impact on LLM scrapers and crawlers? My memory is imperfect but I feel like I remember conversations on Tildes in the past year talking about how there was little to no pattern in IP addresses that were used to crawl/scrape websites, which is why some of the “tarpit” traps were developed?