12 votes

What just happened to TheNumbers.com should worry us all

Posted by Kremor
Tags: internet, crawlers.web, artificial intelligence, markets.prediction, data, search engines, the numbers, box office, websites, substack.stephen follows, author.stephen follows
https://stephenfollows.com/p/what-just-happened-to-thenumberscom-should-worry-us-all

Link information

This data is scraped automatically and may be incorrect.

Authors
Stephen Follows
Published
Jul 22 2026
Word count
2404 words

1 comment

  1. Kremor
    Link

    The Numbers felt this shift in two distinct waves. The first started around 2024:

    We saw a big increase in crawls as AI training joined the search engine crawlers. The AI crawlers are generally less well-behaved than the search engines, which increased the management tasks for us to keep the site running smoothly.

    And the second wave was stronger and more damaging:

    Around December 2025, we saw another big spike in traffic which I attribute to agentic AI: a combination of AI agents that scrape sites in response to prompts, and people being able to write agents that scrape sites.

    Bruce says that only 10% of their traffic is from humans browsing the site, with the rest coming from AI bots and automated traffic.

    Around 90% of our time was spent keeping the existing site running while we spent our spare moments working on a new and improved system.

    3 votes