8
votes
Bot web traffic has overtaken human web traffic
I've been seeing this claim repeated across social media, blogs, and various online communities these days. However, I haven't yet found a discussion that digs into the evidence behind it or provides reliable sources.
Where can I learn more about this topic?
I'm increasingly skeptical of mainstream media coverage and a lot of what I encounter online, so I'm looking for sources that are as rigorous and unbiased as possible. I'd especially appreciate:
- Academic papers and research studies
- Industry reports with transparent methodologies
- Independent analyses that critically examine the claim
- Any insights from people who work in web infrastructure, cybersecurity, search, analytics, or related fields
If you know of high-quality resources, I'd love to read about them.
I don't have any links for you, but a clarification: Bot traffic very likely first started outpacing human traffic at least a decade ago. Scrapers, search engine spiders, bots sniffying for common vulnerabilities, social media link info grabbers, even SMS and messaging apps send bot hits. A single bot instance can hit multiple pages a second for tiny fractions of a penny in electricity costs, humans never stood a chance.
So bot traffic volumes versus human aren't really that interesting, bots were always going to have higher volume. The interesting question is about content that people are actually consuming/interacting with. How much of that is automated?
Almost definitely bots are already ahead of humans on content creation volume, but traditionally the majority of that is SEO spam that people only interact with accidentally and rarely stay long. Humans are still way ahead when it comes to content that people actually engage with.
Now with LLMs there's a real possibility that could change, but it hasn't happened yet.
Just as anecdata, my little blog running on a raspberry pi on my desk gets between 8000 and 20,000 hits a day from bots.
I get maybe half a dozen hits from actual people.
Depending how you want to count "bot" traffic, I would be alarmed if the scales didn't tip over a decade ago.
To add to this, if you put a page/site online today that isn't linked anywhere on the internet (thus effectively invisible) and you just put a security certificate on it, within days you'll be getting 100s of bot hits/day. Once it's indexed, multiply that by 10, double it if there's a contact form or comment boxes, triple it if it's running software that's a popular target (like wordpress), multiply by 10 again if it becomes a reasonably popular social site with user generated content.1
1 napkin math based on real world sites
It absolutely does not have to be running WordPress. My site isn't running WordPress and bots sure do try hitting all of the standard WordPress endpoints.