Matt blogs every now and then defending economic forecasters and always seems to draw out at least a couple of comments slagging off forecasters for being inaccurate. His reponse is that it’s not the numbers that matter but the qualitative analysis of risks and market direction. His defence came to mind when I was reading a recent AEJ article about testing the performance of expert advisers.
The authors make the point — which isn’t central to their hypothesis — that there’s a difference between forecasters who get the numbers right and forecasters who are useful to their clients.
…a decision maker must take some default action even in the absence of any expert, and may not appreciate forecasts that suggest the same actions, even if they are provided by an informed expert. On the other hand, if forecasts lead to better decisions, the decision maker may appreciate them, even if they are provided by an uninformed expert.
All of which suggests that trying to judge economic forecasts by the accuracy of their predictions alone may miss much of the value that they provide. Of course, there is probably a correlation between the accuracy of the forecasts and the usefulness of the accompanying advice but the two need to be judged together by their usefulness to the forecasters’ clients. Given that people continue to purchase forecasts it appears that their value in improving decisions is far from negligible.