Web scraping for me, but not for thee

There are few, if any, legal domains where hypocrisy is as baked into the ecosystem as it is with web scraping.

Some of the biggest companies on earth—including Meta and Microsoft—take aggressive, litigious approaches to prohibiting web scraping on their own properties, while taking liberal approaches to scraping data on other companies’ properties.

When we talk about web scraping, what we’re really talking about is data access. All the world’s knowledge is available for the taking on the Internet, and web scraping is how companies acquire it at scale. But the question of who can access and use that data, and for what purposes, is a tricky legal question, which gets trickier the deeper you dig.