Question: Have economists been over-confident regarding their ability to predict things?
My unequivocal answer here would be yes – but I’m not asking me, I’m asking you.
Do you guys think that economists have been over-confident about their ability to predict things?
We have repeatedly said on this blog that economic science “frames issues” – but predictions only stem from virtually untestable value judgments (although we can inform these, and narrow the band of judgments, by using data).
However, it appears that many economists have tried to sell their ability to predict – something that has caused issues.
This blog post really sums up how I think people think about economists right now (ht Market Movers). I think the issue is that economists have sold this story to the public about their ability to predict – an ability that doesn’t exist. The risk from this is that the value that economists can add (framing issues, even describing what has happened) may get ignored as a result of this perceived failure.
Interesting take. It’s sad how a few loud pundits, usually with financial stakes, can sully the image of an entire profession. I actually blogged on this very sentiment:
http://akingsmind.wordpress.com/2009/01/23/have-economists-failed/
and the few posts thereafter.
Do you think there was intent on the part of economists to set themselves up as oracles? Or is it that they are veiwed as the most qualified so their opinion gets more credence than it ought to be worth.
I don’t find the question very meaningful. No doubt the man on the street treats economist predictions as an exact estimate, not a median subject to large error bands. But are you saying that economists have underestimated or mislead the public about the size of the error bands? Groups like the Treasury and the RBNZ regularly assess their forecast accuracy and make the results public. And such studies confirm some degree of predictive ability for economists, contrary to your claim.
“It’s sad how a few loud pundits, usually with financial stakes, can sully the image of an entire profession”
Very true
“Do you think there was intent on the part of economists to set themselves up as oracles? Or is it that they are veiwed as the most qualified so their opinion gets more credence than it ought to be worth.”
A very important question. Personally I see it more as the second way – which is unsurprising given that I’m an economist 🙂
Ultimately, at most I think economists just haven’t had the opportunity to discuss the limits of their predictive accuracy – rather than them genuinely misleading people
“I don’t find the question very meaningful.”
Have economists been over-confident with regards to prediction – it is definitely meaningful in some sense. Economists that say we should base the worth of the discipline on predictive accuracy (eg Friedman) definitely thought it was an important question.
“No doubt the man on the street treats economist predictions as an exact estimate, not a median subject to large error bands.”
Agreed – and I don’t think economists have pushed this error, but have they maybe not corrected for it? Personally I don’t think we could have explained the limitations any further – so I’m not blaming economists. However, I wanted to leave this post open to people who wanted to suggest that – which is why I didn’t phrase it either way in the post.
However, I can’t get past the feeling that sometimes economists themselves overestimate the predictive accuracy of their models.
“Groups like the Treasury and the RBNZ regularly assess their forecast accuracy and make the results public. And such studies confirm some degree of predictive ability for economists, contrary to your claim.”
Yes, and it is very good. I really don’t think that the RBNZ and Treasury have been over-confident with there predictive accuracy.
However, lets be honest here – a lot of economists had said things about the “great moderation” and monetary policy “dulling the businesses cycle”. Now to some degree these things are true – but there appears to be a section of economists who seem unwilling to debate things outside of this type of world view.
Furthermore, it is definitely the case that some economists “oversell” their predictions – as that is where their revenue comes from. Surely this is an incentive problem. In the case of such an incentive problem it is conceivable that some economists could get “over-confident” regarding the strength of their models – could it not.
Personally, I think our ability to strictly predicit is over-rated. Although our ability to paint risks and describe the situation is under-rated.
Matt, my problem with economists and prediction is that 99% of economists don’t make them. People seems to think that what economists do, and the only thing they do, is make predictions. Most of us wouldn’t go near a prediction if our lives depended on it! The small minority of economists who do make predictions give the rest of us a bad name! 🙂
While I agree that some do over sell their predictive abilities, I would also say that many buyers of these prediction don’t seem to read the small print. They don’t read the if and but and maybes that go into the prediction. This is especially true of the news media who just want a good headline and don’t care about the details. This distorted view is what gets passed on to the general public.
The other point I would make, however, is that there are buyers for these predictions and these people seem to keep on coming back. So I can only assume that however bad the predictions are, they are better than no prediction at all. Otherwise why do people keep on buying?
“Matt, my problem with economists and prediction is that 99% of economists don’t make them. People seems to think that what economists do, and the only thing they do, is make predictions. Most of us wouldn’t go near a prediction if our lives depended on it! The small minority of economists who do make predictions give the rest of us a bad name!”
I would be apt to agree with that – it is the personalities of some economists that may have been too strong, not an intrinsic issue with the discipline.
“While I agree that some do over sell their predictive abilities, I would also say that many buyers of these prediction don’t seem to read the small print”
“The other point I would make, however, is that there are buyers for these predictions and these people seem to keep on coming back”
I would note that people who actually do consume forecast know how to get value out of them – economists are good at painting risks, and giving favour to what is going on.
This is a little different to the idea that some members of the economics discipline because “over-confident” about their ability to solve a crisis.
My primary concern lies with economists who didn’t believe we could experience a sharp fall in economic activity unless there was a substantial (and easily observable) policy error. There is nothing in our theories that states that output volatility will be low – as even if we were all RBC theorists we assume that technology drives a good proportion of shocks and that technology can move around randomly and violently.
From the article I linked to (and others like it) I get the impression that people believe we are supposed to be able to prevent output volatility. With this post I am really asking the question: Is this random conjecture on societies part, or did some economists sort of push this inflated view of themselves?
Rethinking security has an interesting parallel to this: the yanks keep going on about how accurate their bombs and intelligence are, so when they hit civilians people assume that that’s what they intended to do.
So yes, all the economists who wank on about how wonderful their analysis is and how accurately they understand what is happening deserve to cop it big time when it turns out that actually they can’t tell their arse from their elbow. All the effort put into telling me that I’m deservedly poor because I refuse to follow their instructions (just like Russia and Argentina are poor for the same reason)… then they get it badly wrong and go running to mummy for a handout? Kick them in the teeth, I say, and repeat until they recant.
One obvious solution to the GFC is performance based pay for the whole sector. Get it wrong, don’t get paid (well, get the minimum wage unless you’re a contractor). Bankrupt? The taxpayer will happily become the owner of last resort… a dollar sounds fair. Bankruptcy protection? Sure, but if you broke the law you’re not eligible, you know that. Assets held by related parties? Throw them in too.
http://rethinkingsecurity.typepad.com/rethinkingsecurity/2009/02/a-victim-of-its-own-success.html
I’m reading “debunking economics” at the moment and it’s starting to explain why I struggled so much to understand the subject despite being ok at the maths (studying electrical engineering and started to do economics paper but gave up because it was nonsense)
“Rethinking security has an interesting parallel to this: the yanks keep going on about how accurate their bombs and intelligence are, so when
they hit civilians people assume that that’s what they intended to do.”
Indeed.
“One obvious solution to the GFC is performance based pay for the whole sector”
One problem is that the poor advice I am talking about here is “unpaid”. Economists that are paid to make predictions have the incentive to illustrate risks and discuss the process in more detail – as otherwise they do lose out.
However, the attitude of (some) economists surrounding their knowledge of the economy may have given the general public the wrong idea surrounding the certainty of some assertions (such as the effectiveness of monetary policy in the case of a sharp fall in world demand).
When the authour in the post I linked to said that they took on actions as a result of what they had heard from economists, actions that even at the time would have seemed dubious, it implies to me that there is some communication problem between economists and non-economists.
Now, is this a result of economists “overselling” one section of their discipline, is it the result of economists “value judgments” not being transparent, or is it a result of other people not listening to the disclaimers that economists DO put on their forecasts.
“I’m reading “debunking economics” at the moment and it’s starting to explain why I struggled so much to understand the subject despite being ok at the maths”
I am not a fan of the debunking economics stuff – from what I’ve read it stems more from a misunderstanding of “marginal theory” (which is the basis of microeconomics) then a fair critique of the discipline.
Furthermore, when he offers an “alternative” it tends to be an appeal to arbitrary cycles – rather than the sort of methodological individualistic explanation that economics demands. Sure I enjoy Schumpeter myself, but I don’t think holistic explanations should be the ultimate goal of a progressive research program.
Economics has the same problem as all other science (social and physical) – when a hypothesis is rejected it can be because the core of the theory is wrong, or because auxiliary assumptions are wrong.
In economics, we haven’t been able to test the core (namely the behavioural motives of individuals) which is why we have stuck so strongly to certain methods of analysis – as a result I think behavioural economics will be VERY useful in helping economics analysis evolve.
I think if you didn’t like the economics course it may have been the fact that economists sometimes don’t start at the beginning when teaching – the underlying method of economics is sometimes not explicitly taught (as compared to maths).
Matt, it was more that I was spoiled by engineering lecturers who knew their stuff in depth, so second year economics was a bit of a shock. It just wasn’t possible to keep saying “but why” until you get back to the fundamental, experimentally verified underpinnings the same way it is with engineering (albeit with statistics in some places, thanks to the quantum mechanical underpinnings of semiconductors). I did quite a lot of reading and seemed to be up with the rest of the tutorial on the course material, but I could never make it hang together in my head the way (for example) I eventually understood the differential equations behind antenna design (and if ever there was a bit of EE that involves a lot of handwaving, antenna design is it).
Yeah, behavioural economics has always sounded sensible… it’s not very hard to find cases where homo economicus behaves differently from homo sapiens. The debunking book I’m reading skeptically and thinking about, and I suspect it will lead me into more economics reading. He has a fairly liberal bibliography too, which could be interesting.
FWIW, I read “counting for nothing” as part of feminist studies at uni, and it’s amusing how much of that critique has come to be accepted (at least outside bought media economics commentators).
I guess the ultimate question is: given the obvious flaws in what’s being suggested, why do economists find such a ready audience?
If a bunch of engineers stood up and said “building more roads will reduce traffic congestion” they’d be laughed out of the building. No, wait, um, another example… “reducing spending on the electricity transmission system will increase reliability” (ok, I lie, economists say both those things and most engineers cringe).
Have engineers been over-confident regarding their ability to predict things outside their sphere of expertise?
“given the obvious flaws in what’s being suggested, why do economists find such a ready audience?”
Because talking to engineers about the economy wouldn’t be very enlightening.
Dont you think that every economist wishes that their field of study was as easy as engineering? Imagine it; measureable inputs and verfiable outcomes! Answers to 4 decimal places? Luxury.
I wonder how accurate those engineers would be if at any moment a god could appear and disregard the laws of physics. I doubt those predictions would be all that reliable then. Economists face such a god in government, which is not really subject to any budget constraints as an economic actor. Trying to make predictions in a setting where such an entity exists is impossible. At least the laws of gravity are constant. The laws of the nation aren’t.
I have to agree with this 100%:
“Matt, my problem with economists and prediction is that 99% of economists don’t make them. People seems to think that what economists do, and the only thing they do, is make predictions. Most of us wouldn’t go near a prediction if our lives depended on it! The small minority of economists who do make predictions give the rest of us a bad name!”
Most economists understand the limits of their predictive abilities. We can make general predictions [“easy credit will lead to a bubble”] but we can’t say “the bubble will pop March 17th 2008” because it’s unrealistic. We’re further confounded by government, as mentioned above. By the same token, biologists can’t say “the polar bear will go extinct in the fall of 2020” but they can predict that, if things continue, it’s doomed. That’s accepted as a valid and specific prediction in that field: why is more expected of economists?
My guess is it’s because economic issues are a little closer to home than others are, since they affect us directly. Add the fact that a specific prediction makes better TV/paper/blog than general ones, and the media’s general fascination with showing contrasting viewpoints, and you get a public that seems to think that all economists do is fight with one another because each one is sure about everything. Fortunately for the profession, the perception does not match the reality. Unfortunately, public opinion matters even to economists, and we’re losing that battle.
“I was spoiled by engineering lecturers who knew their stuff in depth, so second year economics was a bit of a shock”
Economics lecturers definitely know their stuff. The problem with teaching a second year course is that you have to make things sound “definitive” – you can’t teach the nuances which a lot of the economics stuff requires. This is what sometimes gives it the feeling of being taught “the wrong way round”. It is an issue – but I don’t blame the people teaching it, or doubt their extremely large knowledge.
“t just wasn’t possible to keep saying “but why” until you get back to the fundamental, experimentally verified underpinnings the same way it is with engineering”
It is because these priors are currently unobservable and untestable – its just something that can’t be done at present. When the opportunity arises hopefully economics will be amiable to it – but at present economists are just making use of their limited information.
“it’s not very hard to find cases where homo economicus behaves differently from homo sapiens”
Actually it depends how you define “homo economicus” – it is easy to define him in such a way that he could capture any form of behaviour – the problem is that doing this would make the description empirically empty.
“I guess the ultimate question is: given the obvious flaws in what’s being suggested, why do economists find such a ready audience?”
Even if the flaws are obvious – the solutions obviously aren’t. Economists find an audience because they are able to frame some things in a better way than non-economists – it is as simple as that. If other people could predict things more accurately they could use that knowledge to drive “economists” out of business.
In the above post I have asked if “economists are over-confident” – in other words are they “predicting too much”. I have also heard another criticism of economists – that we say too little, or that we are two handed. It appears to be a case of “damned if you do, damned if you don’t” to me.
Economics provides a great method here – I think some economists have predicted too much as they became overconfident of their methods, but I don’t take that as a fundamental issue with the discipline. I fear that the general public will though …
“Dont you think that every economist wishes that their field of study was as easy as engineering? Imagine it; measureable inputs and verfiable outcomes! Answers to 4 decimal places? Luxury.”
It would be very very different wouldn’t it Kimble. Imagine actually being able to recommend policy without have to add 101 provisions for potential outcomes 🙂
“Trying to make predictions in a setting where such an entity exists is impossible. At least the laws of gravity are constant. The laws of the nation aren’t.”
The Lucas critique – a true bane on our empirical models 🙂
“I have to agree with this 100%:”
I very much agree with the statement as well. However, there are economists that are very willing to predict things without specifying risks – I don’t know how they can manage to do it, but it happens.
In fact, many of the economists arguing about the stimulus on US blogs appear to be doing just that 🙁
“Most economists understand the limits of their predictive abilities. We can make general predictions [“easy credit will lead to a bubble”] but we can’t say “the bubble will pop March 17th 2008″ because it’s unrealistic. We’re further confounded by government, as mentioned above. By the same token, biologists can’t say “the polar bear will go extinct in the fall of 2020″ but they can predict that, if things continue, it’s doomed. That’s accepted as a valid and specific prediction in that field: why is more expected of economists?”
Completely agree – I find it annoying that the natural science are given a relative “free-ride” in terms of their predictive accuracy. I guess it is because economics is looking at human nature – and everyone feels that they know a bit about human nature 😉 (I just saw you made this point – I write comments as a read down paragraph by paragraph 🙂 )
“Fortunately for the profession, the perception does not match the reality. Unfortunately, public opinion matters even to economists, and we’re losing that battle.”
I agree, but there is one thing I would add. The current behaviour of a large group of macroeconomists is doing nothing to help the public perception of us – in fact it is making it worse. Does Krugman care that he is spending economists reputational capital when he calls Barro an idiot – no. It makes me sad 🙁
The economic method is brilliant – the level of economic discourse during a recession is poor – the public ranting of some economists (under the guise of “simplifying things for the public) is a dis-service to society and to our discipline.
“The current behaviour of a large group of macroeconomists is doing nothing to help the public perception of us – in fact it is making it worse. Does Krugman care that he is spending economists reputational capital when he calls Barro an idiot – no. It makes me sad 🙁 ”
Now that you say that, I have images of the tragedy of the commons flashing before me. Krugman [and Roubini, and Feldstein, though PK is probably the worst of them all] is simply adding his additional sheep to the commons, which helps him even though it devalues it for everyone else. Of course, at this pace, we’ll get to the point where he destroys his own audience because they decide they can’t trust anyone, including him.
I think it all comes down to the consumer of the prediction. Do people question what they read or take it at face value? I would argue that many people don’t question what they read enough and tend to fall victim to argument by authority.
I jest here but the bloke on tv with a caption under his name of BNZ Chief economist, winner of the Nobel prize in economics will acquire automatic connotations of accuracy and the public are more likely to take on board what is said.
Given the power of the argument by authority, economists when they go on tv in particular need to be more careful of the language that they use. I would much prefer to see sentence structure like, ¨It is our prediction that interest rates will….blah blah..¨ Too often certainty is used where perhaps caution is better advised. Unfortunately certainty is exactly what people crave and those that project it are more likely to get themselves in the public eye. Economists like Krugman in particular who seem to have such a high belief in their own predictive ability seem particularly guilty of this. The Nobel prize hasn’t seemed to added any humility to his prognostications either!
So in essence I think most economists are well aware of their profession’s short comings, unfortunately a some seem to create an artificial air of gravitas where none should exist. Perhaps Nicholas Taleb’s black swan should be compulsory reading in all secondary schools then perhaps the public wouldn’t be so easily fooled.
“Of course, at this pace, we’ll get to the point where he destroys his own audience because they decide they can’t trust anyone, including him.”
Bittersweet I guess 🙁
“unfortunately a some seem to create an artificial air of gravitas where none should exist”
Indeed – although I guess it is a fine line between actually being brave and being cocky.
“Perhaps Nicholas Taleb’s black swan should be compulsory reading in all secondary schools then perhaps the public wouldn’t be so easily fooled”
Maybe just teach economics in 3rd form …