It’s amazing how a theory can take over a brain. Consider the idea that people believe what serves their interests. As heuristics go, it’s a good one. I use it all the time. Yet when Dario Amodei says AI is dangerous, perhaps even an extinction risk, some people conclude he must be running a marketing campaign. That is stupid. Which is more likely, that a useful heuristic sometimes misfires or that “our product might kill you” is a clever way to sell it? Death threats are a poor marketing strategy.
We also have plenty of evidence that the fears of AI experts are sincere. Amodei, Altman and Musk were all publicly warning about AI risk long before they had AI companies to promote. The worry runs well beyond the executive suite; rank and file researchers share it. And it extends outside the industry altogether, to computer scientists with no product to sell, among them Nobel laureate Geoffrey Hinton. Hinton left a high-paying job at Google precisely so he could speak out and Hinton is not in a Berkeley polycule with Eliezer Yudkowsky, at least as far as I know. Whatever else you may say about the belief that AI presents a serious risk, plenty of AI researchers believe it sincerely.
[...]
Notice that classic regulatory capture takes time, it’s a process of erosion rather than a battle, it happens in the shadows, in the backrooms, away from the public’s eye. As Culpepper argues in Quiet Politics and Business Power, business power goes down as political salience goes up. Regulatory capture and lobbying does a good job explaining why roasting coffee beans was defined as “domestic manufacturing”, thereby lowering Starbuck’s tax rate by 2%. It does less well at explaining big cross-industry issues the public cares about such as environmental regulation or race and gender discrimination regulation. Finally, don’t confuse capture with firms making the best of a bad situation. Philip Morris supported the 2009 Tobacco Control Act not because FDA regulation was Philip Morris’s unconstrained ideal but because it knew regulation was coming and it wanted a seat at the table to nudge the rules in its favor. That’s ordinary political bargaining—or rent-seeking—not evidence that the regulator has been captured.
Now let’s evaluate Amodei’s call for regulation in light of regulatory capture theory. AI regulation is in gestation. Public attention is fixed on the industry, and much of that attention is hostile. The big profits in AI lie in automating work, and job loss is a much more salient fear than extinction. AI politics is now loud–precisely the environment in which Culpepper predicts business power will be weakest. A mature industry can bend regulation to its purposes through revolving doors, longstanding relationships and obscure rulemaking. An industry under Sauron’s eye has much less power and faces much greater risk that politics will bend regulation to its purposes. Political actors are eager for an excuse to redistribute AI rents away from capitalists and toward favored groups (ala Peltzman).
[...]
Go ahead: argue that Amodei and other AI experts are wrong about AI risk. Ask whether his proposals favor Anthropic. But calling “our product might kill you” a clever marketing and regulatory-capture strategy isn’t sophisticated analysis. The facts don’t fit regulatory capture theory and trying to make them fit requires epistemically painful Ptolemaic epicycles. Even a dull Ockham’s razor cuts through that story to the obvious alternative: Amodei actually believes what he’s saying.
From the article:
[...]
[...]
I diagree with Tabarrok on a lot of things, but thought I'd post this one since he's a voice of sanity here.