Why Economists (On Average) are Terrible Forecasters
by Mike Kimel
Why Economists (On Average) are Terrible Forecasters
My colleague, Rebecca Wilder, had a post at her site entitled Economists are terrible forecasters – why trust them anyway?. The reason why economists as a general rule are lousy forecasters is obvious: there are no penalties to being wildly wrong.
Prominent examples abound. Dow 36,000 anyone? No housing bubble in 2005. I can go on forever, but these aren’t even as as it gets. At least these are bad forecasts of the future. There are plenty of bad forecasts of the past, or even the hypothetical past, too. My favorite example, in fact, of a bad forecast came in 2002, when a group of prominent policy economists, advisors to the then President, told the world that barring the 2001 recession, the US would have enjoyed double digit growth in fiscal 2002. And nobody said peep. It wasn’t front page in the newspapers. It wasn’t in the newspapers at all! Nobody involved paid any price for it, except the public who had to endure the policies “supported” by such an incredibly inane analysis. In fact, just about everyone involved went on to bigger and better things – Governor of Indiana, Dean of the Business School at Columbia, etc.
If there are no penalties to being wrong, there also usually aren’t any benefits to being right. Consider, well, me for example. Regular readers know I don’t make predictions often, but I like to be right when I do make ’em. I can’t think of anyone else who called both the start and end of the Great Recession, in both cases running against the grain, but you aren’t likely to see me on TV any time soon. (I will admit my forecasts weren’t perfect: I misunderestimated the stupidity of the policy responses of both Bush and Obama and thus didn’t expect it to be quite as bad as it turned out.) I can even think of two forecasts made for a then employer that I suspect together cost me a job, despite the fact that the forecasts turned out to be spectacularly right.
Its been said its better to be wrong in the same way as everyone else than to be right alone. That’s certainly true for economists. Unfortunately, that is a bad thing for anyone who listens to economists.
Hear hear!
That’s why it’s just best to do your own research and think for yourself. Oh, and follow the good ones, like Dean Baker, Randy Wray, Mark Thoma, Spencer, Mike Kimel, Edward Hugh, and others….
Rebecca
The threat of malpractice suits helps to keep professional honest.
No such penalty for economists.
Mike,
What profession would you say is good at predicting the future? It’s a pretty hard thing to do, so you are being unfair to economists.
With uncertainty, the “safest” position to hold is to extrapolate trends and adhere to the consensus. This way you can correctly say “hey, everybody else was wrong too.”
Its been said its better to be wrong in the same way as everyone else than to be right alone. That’s certainly true for economists.
This is where you are dead, flat out wrong. This is where the BIG money is. In retrospect the dot-com bubble and the housing bubble were obvious to any economist, so if you correctly went against the grain you could have made huge money for yourself or your employer, and it would be very likely to “see you on TV.”
Correctly guessing which quarter the recession ends will not win you any acclaim. Going out on a limb and correctly calling major shifts in trends is what will you acclaim.
Sammy,
“Correctly guessing which quarter the recession ends will not win you any acclaim. Going out on a limb and correctly calling major shifts in trends is what will you acclaim.”
Go back and read the comments to the post where I said the end of the recession would come in the first half of 2009. Since that was just about when most of the world was waking up to the fact we were in a recession, and given the #$%@storm it kicked up in the comments, I’d say that was going out on a limb and calling a major shift.
But let’s forget about me. I ask you – if a group of people can backcast (not even forecast, backcast) a double digit growth in the economy in FY 2002, and the folks in that group continue to be taken seriously (they kept making policy, and then each went on to plum positions, like the current deanship of the business school at Columbia and the current governorship of Indiana), then what is the incentive to getting things right? Seriously?
A former teacher with a history of foreclosure and liens for failing to pay income taxes, Walsh ran against runaway spending in Washington and the health-care overhaul law and won his swing district in 2010 by just 291 votes. He defeated Democratic Representative Melissa Bean, who had the support of the U.S. Chamber of Commerce and other business organizations.
Bloomberg News featured this story which should scare the hell out of anyone. This clown ran as a Tea Bagger and fiscal conservative. The morons who elected him should never be allowed to vote again but since this country is on the road to ruin, they no doubt will be hailed as patriots.
“What profession would you say is good at predicting the future?”
Physics. It put a man on the moon.
Chemistry. It predicts that if you mix hydrochloric acid and sodium hydroxide, two highly toxic substances, in equal proportions, you will get salt water that you can drink safely.
Biology. It predicted that life on earth descended from a common ancestor. Molecular biology has confirmed this.
I could go on and on . . .
mike,
I said the end of the recession would come in the first half of 2009.
“More than 90pc of economists predict the US recession will end this year (2009)
http://www.telegraph.co.uk/finance/recession/5391505/Leading-forecasters-predict-US-recession-will-end-this-year-NABE-survey-claims.html
the folks in that group continue to be taken seriously (they kept making policy, and then each went on to plum positions,
This proves my point: “With uncertainty, the “safest” position to hold is to …. adhere to the consensus. This way you can correctly say “hey, everybody else was wrong too.”
BTW, what happened to the economists that predicted <8% unemployment with the Stimulus? Why don't you use that, more recent, example?
then what is the incentive to getting things right? Seriously?
Seriously? I can think of 15 billion incentives.
“How hedge fund manager John Paulson bet against the real estate bubble and made $15 billion in a single year.” http://www.newsweek.com/2009/11/09/the-greatest-trade-ever.html
“I could go on and on . . .”
And so I will:
Engineering. It builds bridges and skyscrapers with only the knowledge of physics and material science, and people risk their lives daily on these creations.
Meteorology. It predicts the weather. Say what you will, you pay attention to it, at least for the next 2-3 days!
Well, sammy, doctors are pretty good at predicting the future if it comes to,”Take your pills or you’ll die.” Refuse to take your nitroglycerine when you have chest pain? Say bye bye! Don’t take your metformin? Ever heard of diabetic ketoacidosis? Insulin shock if you try to lose weight the wrong way? All predictable by competent professionals. Economics is different because it’s not a science and they don’t do commonsense. NancyO
Do you know who have to make predictions as a function of their jobs and who do pay a penalty if they’re right or wrong? Product marketing managers, of any stripe. Even if they’re off because they based their own forecast on the economists numbers, it makes not a whit of difference. Insofar as the business general managers they usually report to, if they’re wrong, they’re gone. They must take responsibility for their forecasts, unlike the academics and bureaucrats and think tank pundits who inhabit most of the economists’ slots in this world, If we saw more economists being forced to live on the street, we just might see better forecasts.
Sammy,
“More than 90pc of economists predict the US recession will end this year (2009)”
Yes. In a survey done in April and May 2009. My call was first half of 2009, made in 2008. Perhaps you can spot the difference.
Second, getting it right doesn’t mean you’ll get $15 billion. Paulson started with enough assets that he could bet quite a bit. I’ve never in my life had what he started with, and I’ve done pretty well with what I started, I think.
Mike, Do economist have confidence based on the test of time or are they confident hoping they are right when subjected to the test of time?
What I think I see is that “vanity publishing” is an avenue to respect. So economists can become influential through the media even while their work is debated at the professional level. How many times do you hear that 100 economists agree that (insert a political outcome) or see that a single opinion on the economy is emphasized by the selection of an economist that is the guest for the night? (To see this from an unbiased corner, substitute global warming for economy.)
Models are built either from the physical theory or from real data. Sometimes both. Projections attempt to look at variables, perhaps under assumptions, and hope that even if all the variables are not fully understood the relationships are embedded and fully described in the existing data. This has been my impression of the confidence of economists. There seems to be too much “faith based” in their arrogance.
Can they afford to be independent scholars when there is so much tied up in being in the right club?
Nancy O,
I’m not sure doctors are that great at predictions either. There are plenty of studies that show that subject to the right incentives, doctors prescribe suboptimal treatments and/or unnecesary surgeries.
PrahaPartizan,
I’m not sure I agree. I’ve seen businesses that are well run and businesses that are poorly run. (And I note, the profitability or success of the business isn’t always correlated with how well its run.) I’m becoming very fond of the Reagan aphorism: you don’t have to outrun the bear, you just have to outrun the other hikers.
Anna Lee,
I can’t speak for anyone else. I am paranoid, and I keep assuming I’m wrong, so I keep coming back and checking my results a bunch of different ways.
But I think, realistically, there are too many incentives to get it wrong and, pace Sammy, few incentives to get it right unless. I’ll give you an example… throughout my career, I’ve often been asked to talk about economic conditions facing a given company. I often like to broaden the presentation, and I like to present variations of that bar chart I put up at AB so many times that shows the historical relationship between marginal tax rates and subsequent economic growth rates. I can’t tell you how many people fairly high up in corporations respond with something like: “how can an economist believe such a thing?” (Actually, that exact quote has come up at least once.)
Now, that’s what you get if you just show the data in a bar chart, unadulterated by commentary or some fancy statistics. A sizable percentage of economists get the message quickly and learn what they should believe if they want to eat well. And the well fed economists have more intellectual offspring.
I gave a presentation once in which I indicated that conditions weren’t right for rapid growth. I showed a variation of that simple bar chart I often put up
I’ve been operating a soft dollar portfolio strategy service for almost 25 years. My clients are portfolio managers.
Over that period I’ve lost more clients for being right than for being wrong.
For example, I forecast the 1987 bear market and told my clients to go back into a fully invested position in December 1987. Yet, in early 1988 I lost almost every client at renewal time.
Admittedly, at that time the consensus was that the market crash was forecasting a recession and everyone was pulling in their horns and battening down the hatches in preparation for hard times.
I feel you are implying/assuming that economists get things wrong on purposes — the price is very high — their personal credibility. Since it is all they have as the basis for their credibility, I think it’s significantly more of a price than say the price of say a blog posting…
Mike, I agree that doctors’ predictions and recommendations can be influenced by economic or other considerations. No question about it. However, that doesn’t change the way insulin works or doesn’t work under some conditions. Clinical practice is ad hoc and largely empirical.
But, there is a good deal of science underlying many common treatments for hypertension, diabetes, common digestive disorders and the like. There you have the bulk of the general practice population.
As for surgeries, try to convince me I don’t need my stents and my angioplasties were unnecessary. 20 years ago I wouldn’t have received any treatment for coronary heart disease and I’d be dead now. It’s my experience that even frequently questioned procedures can be life-saving. But, studies show……;-) NancyO
SHB,
1. Consider Glenn Hubbard. From what I can tell, he’s just shy of the “tax cuts cure three kinds of cancer” school of thought. He isn’t espousing tax cuts to be wrong on purpose. Maybe he believes it, maybe he doesn’t, but either way, he knows its a damn sight more lucrative to espouse that line than to espouse the other.
2. Why do you think nobody, but most especially Robert Barro, has run this:
growth (t to t+1) = f(marginal tax rate (t), marginal tax rate (t) squared)
using BEA data for GDP which goes back to 1929, and marginal tax rates from the IRS?
I mention Barro since its a pretty straightforward extension of most of his work. Laffer is another obvious choice. I’ve posted that on this blog… its pretty easy to do.
How robust are your prognosticative skills?
At the end of September, what will be the difference between the 30 year Treasury bond yield and the 90 day Treasury Bill? How about the 5 year and the 10 year?
For 2011 Q3, what will be:
-the average rate of inflation according to the CPI?
-the annual GDP growth rate?
-the net number of jobs created (or lost)?
What will be the final statistics for 2011 in:
-the % change in CPI for household energy?
-the value added to the nation’s GDP from utilities?
If you’re accurate with these predictions, you’ll get a book deal and a couple of TV appearances.
Kevin,
“If you’re accurate with these predictions, you’ll get a book deal and a couple of TV appearances.”
Based on what happened following past forecasts, I predict you are incorrect. Which means I also predict there’s no gain to performing on demand unless someone is paying me to do it. Wait ’til I feel inspired to make predictions on those topics and I’ll write the post. But if you want to hurry up the predictions they will cost you. (I gotta earn a living too.)