Goodhart wasn’t talking in general about it being a bad idea to have targets. He really couldn’t have been, given the job he was in. In my view, it’s really not possible or serious to be completely opposed to the concept of setting targets in general; it’s too close to the idea that there should be no feedback from output to input.
…
But targets are always misleading, that’s what Goodhart told us! As pointed out above, no he didn’t. Even allowing for the very interesting slip from “statistical regularity” to “measure”, he was talking in the context of monetary base targeting in the UK. The specific problem that Goodhart’s Law was meant to dramatize was that before the policy regime changed, the M0 money supply seemed to bear a reasonably stable relationship to the actual quantities of interest – the level of activity and prices in the economy. When it was used as a target with the hope of manipulating those quantities, it broke down.
…
After all, a thermostat both measures and targets the ambient temperature, but that doesn’t mean that the temperature ceases to be a good measure of what the thermostat is trying to control. If you want to eliminate smallpox, then however complicated the overall ecological and social context, the number of smallpox infections is a very good measure of whether you’re winning or not. Central banks these days have more or less given up on intermediate targets and simply target the actual inflation rate – this has a number of its own problems, but they’re not problems of the sort that Goodhart experienced.
So the message of Goodhart’s Law is that if you’re setting targets, they ought to target the thing that you care about, not something which you believe to be related to it, no matter how much easier that intermediate thing is to measure. That doesn’t guarantee success; the phenomenon of “gaming the system” or the tendency of control systems to be undermined by adversarial activity is much more general and complicated than this single problem.
…
On the other hand, I think there is one major and empirically important case which is pure Goodhart’s Law and where people really could help themselves out a bit by respecting it. As far as I can see, “teaching to the test” is a one hundred and eighty degrees inverted description of a phenomenon that ought to be called “not testing for the outcomes you want”.
From the blog post:
…
…
…