Show Me the Empirical Success of the Empirical Method in Economics
In the comments of my latest post, as well as the second front of this war going on here, we’ve been raising the usual issues that come up in this debate on the empirical method in economic science. I have to be brief, but let me summarize my perspective:
(1) Many people simply assume that to be scientific, a discipline must use what we learned as “the scientific method” in fifth grade. Namely, the idea that economists must use their models or theories to generate falsifiable predictions, and then look at the actual observations to see which theories perform the best. We can never know an economic law for sure–just like we can’t “prove” a law of physics–but that’s what it means to “do science.” Well, if that’s your view, then I am saying economics is not a science. However, I would dispute your view; I claim that there can be a science–economics for example–where what we learned as “the scientific method” is not a fruitful approach.
(2) “What’s the alternative?!” you ask in shock. My answer: Everything you get in a good introduction to “thinking like an economist.” Consider the classic essays from the great French liberals. Consider Henry Hazlitt. Consider Steve Landsburg’s Armchair Economist–it’s in the very title, for crying out loud. In fact, go look at the textbook and syllabus that a free-market economist who champions the empirical method uses for his/her first-year students. I will bet you that the vast majority of what they teach involves getting the students to think a new way, rather than saying, “We’ve gone out and tested these propositions numerous ways; trust me, this is how the economy works.” The way you get someone to “believe in” free trade isn’t to show numerous empirical studies. No, you give him something by Bastiat (or Thomas Sowell) and he’ll say, “Whoa, I never thought about it like that before.”
(3) What is the empirical success of the empirical method in economics? Eighty years since central banks caused the last worldwide depression–according to various schools of thought, though the details differ widely–we now have central banks, under the expert leadership of Ivory Tower-trained economists, causing the second worst worldwide economic calamity in modern times, and again according to economists from across the spectrum (with details varying). Is there anything remotely analogous in physics or chemistry?
The supreme claim of the engineers, chemists, and physicists is, “We put a man on the moon.”
In contrast, Ben Bernanke, Larry Summers, and Christina Romer can say with pride, “We put a man in an empty region of space where we erroneously thought the moon would be. Of course that man is dead now, but what do you want from us? This is macroeconomics, not rocket science.”
(4) Don’t misunderstand: I’m not knocking counterfactual analysis. What I’m saying is that if you can’t predict very well, then using the empirical method in constructing your counterfactual is very dubious. It is precisely the logical, deductive approach that makes you very confident in your counterfactuals, in a world of highly unpredictable absolute movements.
Let me give an example: Suppose a guy is on trial for embezzling $1 million from his firm. He doesn’t deny that he took the money, but he claims the company benefited. His evidence is that the shareholder equity and stock price of the firm went up, all during the period of his embezzlement. The prosecutor is at first rattled by this, but then says, “Well of course, the company would be even richer had you not taken that $1 million!” But the defendant says, “How do you know that? Have you conducted empirical investigations of this assertion? I want scientific evidence for your counterfactual claims.”
Now sure maybe the prosecutor could try to come up with something, but can we all agree that the best course of action here–the truly “scientific” thing–would be an appeal to the pure logic of mathematics and accounting, rather than digging up studies of past embezzlement cases, where issues of correlation/causation have been satisfactorily handled?
Thoughts on Job
In my nightly Bible reading, I have hit the Book of Job. At first glance, this is a very troubling story for a Jew / Christian, because it apparently casts God in an unflattering light. Here’s the setup (Job 1:6-22):
6 Now there was a day when the sons of God came to present themselves before the Lord, and Satan[a] also came among them. 7 And the Lord said to Satan, “From where do you come?”
So Satan answered the Lord and said, “From going to and fro on the earth, and from walking back and forth on it.”
8 Then the Lord said to Satan, “Have you considered My servant Job, that there is none like him on the earth, a blameless and upright man, one who fears God and shuns evil?”
9 So Satan answered the Lord and said, “Does Job fear God for nothing? 10 Have You not made a hedge around him, around his household, and around all that he has on every side? You have blessed the work of his hands, and his possessions have increased in the land. 11 But now, stretch out Your hand and touch all that he has, and he will surely curse You to Your face!”
12 And the Lord said to Satan, “Behold, all that he has is in your power; only do not lay a hand on his person.” So Satan went out from the presence of the Lord.
13 Now there was a day when his sons and daughters were eating and drinking wine in their oldest brother’s house; 14 and a messenger came to Job and said, “The oxen were plowing and the donkeys feeding beside them, 15 when the Sabeans[b] raided them and took them away—indeed they have killed the servants with the edge of the sword; and I alone have escaped to tell you!”
16 While he was still speaking, another also came and said, “The fire of God fell from heaven and burned up the sheep and the servants, and consumed them; and I alone have escaped to tell you!”
17 While he was still speaking, another also came and said, “The Chaldeans formed three bands, raided the camels and took them away, yes, and killed the servants with the edge of the sword; and I alone have escaped to tell you!”
18 While he was still speaking, another also came and said, “Your sons and daughters were eating and drinking wine in their oldest brother’s house, 19 and suddenly a great wind came from across[c] the wilderness and struck the four corners of the house, and it fell on the young people, and they are dead; and I alone have escaped to tell you!”
20 Then Job arose, tore his robe, and shaved his head; and he fell to the ground and worshiped. 21 And he said:
“Naked I came from my mother’s womb,
And naked shall I return there.
The Lord gave, and the Lord has taken away;
Blessed be the name of the Lord.”22 In all this Job did not sin nor charge God with wrong.
But upon this reading, these events don’t seem so odd to me, because–as I often say on these Sunday posts–if you’re going to evaluate whether God is good or bad, you have to take the depiction of Him in its entirety. Obviously, if some prince did the above, he would be evil. But if God does it, things are different, and not because I’m saying a mere definition–“the acts of God are good because He’s God”–but because the context is different. Just as, if I say it’s OK for Jim to smash a car because he’s the owner of it, but it’s not OK for Mike to smash the same car, I haven’t just admitted to moral relativism. So, please at least recognize that I’m explicitly denying that there is one set of morality for God and another for humans; if you’re going to accuse me of that in the comments, spell out why my defense doesn’t work, please.
Anyway, back to my main point. Imagine Job dies and is united with the Lord in paradise for eternity. His kids (the ones who died) are all there too. God says:
Well done Job! I know I asked you a lot during your time on earth, but now from this perspective you see that your kids got to be with me sooner. I could have killed them with a heart attack, or a lion, or kidney failure at age 68, but instead I allowed them to be killed by a windstorm that knocked a building on them. No matter what, I ultimately was going to kill them and end their mortal existence, and I decided that it would best educate the future of humanity to do it this way. I knew you would rise to the challenge, and give a model of how people should respond to such calamities. You are now one of the most famous human beings in all of recorded history; My people have held you up as a role model for countless generations. As much as I asked of you, it was less than I asked of My own Son, who was tortured to death. It’s unfortunate that your and His suffering were necessary, but you know how people are: they need human role models to follow.
I submit that seeing the kind of person Job was on earth, if God said something like the above to him–and the kids who had been killed in the windstorm are there, beaming at him–he would respond, “Thank you my God, I am incredibly honored that you chose me for this purpose.”
So suppose for the sake of argument I’m right, and everybody who was “wronged” by God in His challenge to Satan thinks it was wonderful, after the fact: they think the lesson it taught the world totally justified the nanosecond of misery it caused them in their mortal existence. If we stipulate that for the sake of argument, then is it still so “obvious” that the God of the Christian Bible is fickle and immature? I don’t think so.
The more I read the Bible, the more I understand why Jesus said–referring to the “God of the Old Testament”–that no one is good but Him.
“No War With Syria” Rally in Nashville
I was out of town when this occurred, but anyway…
Money for Nothing Trailer
Does anybody know about this? I am surprised by the strength of some of the statements from Fed officials in this trailer.
Here’s the website for the documentary.
The Intellectual Bankruptcy of (Mainstream) Macroeconomics
I borrowed that title from Russ Roberts (but inserted the qualifier “mainstream” since, of coure, I love how the Austrians do “macro” topics like business cycle theory). You may recall that Russ was the inspiration for my recent post where I chortled over (what Russ and I thought was) a clearly botched prediction from Krugman that the “fiscal doomsday machine” (Krugman’s term) of the sequester would probably cost 700,000 jobs.
Now in the comments of my post, a few people pointed out that Russ and I (and Scott Sumner, since he has been having fun with the post-sequester economic statistics too) are too quick to claim victory. In particular, surely Krugman didn’t mean that the absolute level of employment would suddenly fall by 700,000 in one month. Rather, Krugman was saying that over the length of the sequester, total employment would eventually be 700,000 lower than it otherwise would have been. Thus there were issues of a counterfactual, and timing, that we would need to consider, before evaluating Krugman’s prediction. As “Joe” pointed out, if we use the Total Nonfarm Employment seasonally adjusted figures from the BLS, and if we choose our baseline of job growth from November 2012 through February 2013 (just before the sequester kicked in), then the average is 247,000 jobs added per month. In contrast, if we look at March 2013 through July 2013 (the latest point available), then the economy under sequestration added only 173,000 jobs per month. So if that discrepancy holds up, it would only take 10 months for Krugman’s prediction to be vindicated.
I absolutely love this sort of thing, where presumably smart guys (Russ, Scott Sumner, and me) can look at the data and walk away with one interpretation, where other presumably smart guys (“Joe” didn’t make any calculation mistakes) can look at the same data and reach a diametrically opposite conclusion. In fact, this is an overriding theme on both my blog and Russ’ posts at Cafe Hayek: There are so many moving parts in macroeconomics that you can ex post fit the observations to just about any theory you want, and you won’t be “lying.” As I like to put it, believing is seeing. That’s why we’re still arguing about the Obama stimulus package, and indeed about the Great Depression. We can’t turn back the clock and re-run the same experiment, this time (say) staying on the gold standard in the 1930s.
My Point Is to Show the Bankruptcy of the Empirical Approach
I know that some Austrians are going to jump on me in the comments, so let me say upfront:
MY POINT IN DWELLING ON THIS STUFF IS TO SHOW THE BANKRUPTCY OF THE EMPIRICAL APPROACH TO ECONOMICS. So for people who say, “Bob, what are you doing?! You’re conceding everything to Krugman when you play the game on his terms!!” I would just say, we have a difference of opinion on the best way to get outsiders to change their minds. Superficially, it sounds entirely reasonable and scientific to take David Friedman’s approach as he laid it out in our debate last summer. It does indeed sound like the Misesians are being weird and medieval if they insist that we can deduce economic laws in an a priori fashion, and no empirical evidence would shake our convictions. Isn’t it the mark of scientific objectivity and humility, to be willing to make falsifiable predictions based on your “model,” and then change your views accordingly when the facts come in?
So, to try to get someone who thinks this way, to understand why it doesn’t work in economics the way it does in physics or chemistry, I think it is entirely worthwhile to show that Krugman et al. aren’t being nearly as “objective” as they think. The Obama stimulus is the best example, of course, where the Romer/Bernstein predictions were hilariously wrong, but no Keynesian batted on eye: “Huh, I guess the economy was worse than we realized, now it’s really good that we ran up the deficit, timid though the stimulus package was.”
Different Baselines Cast “Doomsday Machine” In Different Light
I’ve already relayed “Joe”‘s analysis above, which showed one way that you could vindicate Krugman’s prediction. OK, but Krugman never specified the relevant baseline. As I pointed out to Joe, he conveniently stopped going back in time at the best possible month, because job growth in October 2012 was only 160,000 (below the average under sequester). Is that why Joe started his baseline in November 2012? Only Joe can know for sure.
Let’s pick a less arbitrary baseline. Suppose we look at the 12 months prior to sequester. In that case, using the same government data set that Joe and I were citing in the previous post (and again, I’m not endorsing government stats, I’m just showing how the Keynesians can be wrong even on their own terms), average monthly job growth is 188,000 (compared to the average post-sequester of 173,000). So, if the prior year is the relevant baseline, then Krugman will eventually be right (assuming current trends continued) in 47 months, i.e. in almost four years. Is that what he was trying to say? We’ll come back to that question shortly.
What if we go back two years prior to sequester? In that case, the relevant baseline is average job growth of 163,000/month. Oops, that’s lower than average growth under the sequester, so clearly we can’t go back that far in constructing our baseline.
Finally, my personal favorite, what if we look at average job growth from September 2009–you know, the month after Krugman said that Big Government had saved the economy–until February 2013, when Krugman told us to brace for the fiscal doomsday machine? In that baseline, average job growth during the entire recovery (up to the sequester) was 123,000 jobs per month, a full 50,000 jobs per month less than the growth we’ve seen since the sequester.
Here’s an even more fun fact: In the five months after Krugman declared that Big Government had spared the economy from another Great Depression, the economy lost an average of 131,000 jobs per month. This is in contrast to the average monthly gain of 173,000 jobs, in the five months after Krugman declared that “one of the worst policy ideas in our nation’s history” kicked in.
To be clear, there is no outright contradiction in any of the above. But notice how much Krugman is relying on his Keynesian models, with their counterfactual projections of an alternate universe that we never get to observe, when he so confidently tells us that IS/LM has soared through this crisis with flying colors. A naive examination of the raw data would show that the economy was awful when the stimulus kicked in, and isn’t so bad after the sequester kicked in. The only reason it’s “obvious” to Krugman that the stimulus helped, and the sequester hurt, is that he is using his own Keynesian model to tell him what the world would have been like had there been no stimulus, and no sequester. Fine, but you can’t then use these observations to vindicate the Keynesian model! The model looks vindicated, only if you already thought it was true at the outset. This is why Austrians in the Misesian tradition think economists are fooling themselves when they try to copy the methods of the physicists.
And the Grand Finale: The Macroeconomic Advisers Analysis That Krugman Endorsed Was Totally Wrong
I’ve saved the best for last. All of the above could understandably be construed as quibbling by a Krugman fan. Ah, but what if we click Krugman’s link and actually look at the analysis that generated the “700,000 jobs” figure? Then we’ll see the exact context of that prediction, and realize that it has been repudiated by observations–at least in the same way that the Romer/Bernstein report was totally repudiated.
The Macroeconomic Advisers report first constructs a baseline projection of GDP growth without the sequester, than overlays it with their projections of how GDP growth will be reduced if the sequester happens. This forms the basis of their projections of slower job growth: lower government spending ==> lower output growth ==> fewer workers needed to create that output.
So, if you flip to the tables at the end of the report, you’ll see that its baseline forecast of GDP growth in the second quarter of 2013 was 2.4 percent. But, if the sequester kicks in, in this “Alternative Scenario” they were projecting GDP growth in 2q2013 of only 1.1 percent. Okay, you can see that the sequester was modeled to have a humongous impact on the economy then, in the second quarter of 2013–a reduction of growth of 1.3 percentage points, almost cutting the baseline growth forecast inby more than half.
Now, leaving the world of the Macroeconomic Advisers Keynesian model, what actually happened in the real world? Well, according to the BEA, the best estimate right now of 2q2013 GDP growth is 2.5 percent.
Does everyone see the absolute deliciousness of this? The actual GDP growth in 2q2013 was higher with the sequester than the Macroeconomic Adviser report said it would be without sequester. It is thus the mirror image of the Romer/Bernstein projection of the stimulus package’s impact on unemployment.
Conclusion
Of course, we can adopt the same trick that the Keynesians used back in 2009. We can now say, “Phew, the economy was better than we realized back in February 2013, and so the awful sequester is not causing as much misery as we had feared. Still, we have slower GDP growth, and less job creation, because of the sequester. It’s still a dumb policy, we just got lucky.”
But at some point, maybe the Keynesians should consider that having government officials spend money, isn’t the path to recovery? Maybe that’s why these “baseline forecasts” have to keep shifting around, in such a way as to justify the Keynesians predictions ex post?
NOTE: If you look at the projections of first quarter GDP growth, then things are better for the Macroeconomic Advisers analysis. But surely a better test of their model is to look at 2q2013, which is all sequester.
“Paul Krugman’s Problem”
The creator of this video, Skyler Lehto, sent it to me when he first posted it, but I lost track of it on my browser and didn’t actually watch it until Danny Sanchez popped into my office (I’m at the Mises Institute for a few days) and mentioned it.
This video certainly won’t convince Krugman fans, but I’m blown away by how work Lehto put into this. He has a bigger Paul Krugman Problem than even I do!
Upcoming Speaking Engagements
You’ve been reading Free Advice for years, but have never had the pleasure of seeing me in person? Well now’s your chance!
==> I will be speaking at the Libertarian Party of Illinois’ convention. On Friday September 20 I’ll be hosting “Freedom Karaoke” (really), and on Saturday September 21 I’m giving a talk on Keynesian versus Austrian economics. Tom Woods is the keynote. Details here.
==> The next week I’m headed to San Jose, where I will bask in the beauty of the Salinas valley and wish I could write like John Steinbeck. Then, I will give the keynote talk in celebration of Mises birthday on Saturday September 28. Details here.
The Really Important Thing About Syria…
…is that it rights the Bush Record, at least according to some people on Facebook who just posted this:
I’m posting this simply because I am in awe of it. Just off the top of my head:
==> I don’t remember President Bush, Don Rumsfeld, or Colin Powell saying, “There are WMDs in the Middle East.” I think they were a lot more specific about them being in Iraq, since that was the reason given for invading that particular country.
==> Is the only explanation for Syria having chemical weapons in 2013, that Saddam had them back in 2003 and moved them in advance of the US invasion?
==> Let’s assume that the people on Facebook who liked the above poster are right; they were saying things like, “and at the time bush said saddam moved them to syria. hmmm” Fine. Suppose Bush had said the following back in early 2003:
“My fellow Americans, we and the rest of the free world face a grave threat from the regime of Saddam Hussein. That’s why I propose sending in American forces, thousands of whom will die over the next ten years, so that the chemical weapons in Saddam’s possession will end up in the hands of Assad, who will use them in horrifying attacks on his people, so that (a decade from now) my successor will come back to you, explaining the urgency of using American forces to take over another country. God bless America.”
Are you telling me he would’ve gotten the green light? Yet the above is actually the defense of Bush that these people are giving, and giving it with smugness to boot. Like they can’t believe they’re surrounded by such idiots, that they have to point this stuff out about what a great decision it was to invade Iraq a decade ago because of those chemical weapons.
Recent Comments