No One Takes Caplan Seriously on Bayes Law, Not Even Bryan
Back when TARP was first passed, Bryan Caplan said something that I considered absurd at the time:
Ex ante, though, basic Bayesian reasoning doesn’t allow us to claim that whatever happens confirms our position. If the occurrence of X raises the probability of A, then the occurrence of not-X must reduce the probability of A. See Eliezer’s classic essay.
…
I’ll bite first. If the bail-out happens, and unemployment stays below 8% for the next two years, I’m going to become less confident that the bail-out prevented disaster. After all, even a near-miss with disaster should look pretty ugly. Alternately, if the bail-out happens, and unemployment hits 8% or higher during the next two years, I’m going to become more confident that the bail-out prevented disaster. I still won’t be convinced, but I’ll be less skeptical than I am now.
Because had thus boxed himself in, he was forced to say in March 2009:
Now that unemployment has passed 8%, I now admit to having underestimated the severity of the threat, and marginally reduce my skepticism about the effectiveness of the bail-out. I still think it’s a bad idea, but at least it’s no longer much ado about nothing.
I’m not going to walk you through the specific flaws in Bayes’ Law that he (must) have made. Just try a different example: A guy says, “You have a fever? Take my special blend of rat poison as medicine!” Then the guy takes it and his fever goes up. Bryan says, “Hmm, my a priori hunch was that rat poison wouldn’t help, but man, it turns out this guy was on his deathbed after all! I’ll need to reevaluate my initial skepticism.”
OK but it’s good to see that Bryan actually now (apparently) thinks that when the economy kept getting worse, even after the TARP and stimulus, that that was prima facie evidence that free-market economic theory is right. Today he writes:
I’ve been against bail-outs from the beginning. So should have all economists. It’s reasonable to debate the merits of contracyclical monetary policy. It’s not reasonable to debate the merits of rewarding failure on a grand scale.
Alas, in “practical politics” almost no one’s interested in figuring out whether we took the wrong course two years ago. Instead, it’s all about the latest crisis – and the next crisis on the horizon. It really does seem like the crises just keep getting bigger: Wall St., Greece, then what? Italy?
I am not sure how you can be an economist, who believes There Is No Such Thing As A Free Lunch, and give any credence to “the merits of contracyclical monetary policy.”
Guys, this stuff is really simple– there is no monetary fairy! That’s it. No debate necessary.
Bob, we’ve been over this before, and despite your sincere claims of “getting” Bayesianism you really, really don’t understand this one.
All Bryan was saying back then is that you can’t *both* claim that A should increase the probability of your view, and that not-A should increase it as well. For the hundredth time, it is perfectly acceptable to say that both A and not-A have no impact on the probability you assign to a belief. And, yep, that’s the belief you hold: no matter what the unemployment rate in X months, you would not change your confidence in the belief that “TARP was necessary to avert disaster”.
Bryan’s post was an attempt to pre-empt those who are planning to BOTH say, “Hah, 4% unemployment after 18 months — obviously TARP wasn’t necessary to avert any disaster!” and also planning, should the opposite happen, to say “Hah, 15% unemployment after 18 months — obviously TARP had no ability to avert disaster!” His post was targeted at those who were planning to user future stats as evidence. You weren’t planning to do that? Then his post wasn’t demanding that you say how you would update your beliefs on what stats.
Also, note his change the in *confidence* with which he disagrees with TARP is not the same thing as him saying he thinks TARP was a good idea; it’s just that the degree of belief changed a bit.
What specifically do you disagree with, or find flawed about Bayesianism, in all of that? You have plenty of time to answer this simple question, but you’re probably going to ignore it because you have no answer.
How about a mea culpa?
Silas, I don’t know why you put that in bold, since I agree with that.
I totally agree with you that that’s what Bryan thought he was doing. But I thought he was being ridiculous for then taking the stand that if unemployment went up, he’d modify his views and think maybe TARP was a good idea with slightly more confidence (though not above 50%). So already I think that was a mistake, though it was not evidence that he didn’t get Bayes Law.
OK, and now to the point of my present post, which I do believe you completely ignored here. Bryan is writing as if the evidence shows TARP didn’t work. Yet by Bryan’s own ex ante test, TARP did everything it could to show that it worked!
Do you at least concede that minor point, that Bryan’s post today is completely inconsistent with his earlier thoughts on how to judge the effectiveness of TARP?
OK, and now to the point of my present post, which I do believe you completely ignored here. Bryan is writing as if the evidence shows TARP didn’t work. Yet by Bryan’s own ex ante test, TARP did everything it could to show that it worked! … Do you at least concede that minor point …?
No, I don’t. Where on earth do you get that TARP did “everything” it could to show it worked? Bryan listed *one* datapoint, and the (sign of the log) likelihood ratio he gave to it. His post was addressing, in advance, how to use that datapoint, because it would be commonly cited. It does not mean he believed this is the *only* datapoint relevant to the whole matter; it just means he is adhering to good “epistemic hygience” in stating his likelihood ratio on that datapoint *in advance*.
The fact that he didn’t go over his likelihood ratios for other informative evidence doesn’t mean he thinks there are none. And the fact that he still puts the probability of TARP being helpful below 50% does not mean he is being insensitive to the evidence — he updated in the direction he promised.
So no, I don’t see where his subtle rejection of Bayesianism is.
Remember what we talked about: explain your assumptions *before* crucially relying on them for arguments, rather than assuming people read your mind.
OK Silas I don’t hope to get you to say, “Yikes, sorry Bob!” but I am surprised you don’t at least see where I am coming from on this one. So here goes:
Suppose Paul Krugman read EconLog as much as I do. He would be perfectly justified in saying, “Caplan are you nuts? You are chastising everyone for not looking at history, when you yourself offer the *a priori* argument that it’s dumb to reward failure! So what about my a priori argument, that if the banking system collapses then the economy is going to be a basket case? Not only that, but if we comb your writings to see what an empirical test of TARP would look like, the one criterion you set up, shows that TARP gave evidence of being effective!!”
Now Silas, you’re right that you can come up with an explanation such that Bryan didn’t literally contradict himself, a la “black is white.” But you yourself acknowledge that his whole mission with the Bayesian posts, is to get people to state *beforehand* what types of evidence will count in favor of their prediction. So just because Bryan probably could come up with other empirical results to justify his implicit statement that TARP has “demonstrated” its failure, doesn’t get him off the hook. He hasn’t blogged about those other tests, and so he is committing the very same bad habit that he lectures everyone else on periodically.
This is my main beef, and why I wrote the present post the way I did. I think mainstream economists (including Caplan) periodically stop to tut-tut everyone else on how they are so much more rational thinkers, when in practice they do the same “mistakes” that everybody else does.
It’s like the (perhaps apocryphal) story of the Ivy League economist who got a job offer from a rival school. He was agonizing over the decision, and asked his colleague for advice. “Well, why don’t you set up a constrained optimization problem, and figure out which decision maximizes the PDV of your lifetime utility function?”
“C’mon, this is serious,” replied the first economist.
Okay, there’s where you go wrong: Caplan has Bayesian *priors*; that’s not the same thing as having *a priori* positions in the sense Austrians mean it (though of course this is further complicated by the fact that Caplan suggests Austrians treat their a priori views *as* priors and updated them on evidence).
So he didn’t offer an a priori position. Bayesians accept that others can have priors wildly different from them. Bayesians only dispute your *likelihood ratios*, i.e. how you move your beliefs based on the evidence, not where they might have been to begin with. And so, in good Bayesian practice, he’s saying, “*whatever* your prior, what would you do with *this* piece of evidence? Tell me now, before you get it so I know you’re being consistent.”
So Krugman has a different prior (NOT a priori view). So what? How would that justify why someone *else* should move their beliefs *toward* Krugman’s? Did Krugman say what likelihood ratio his hypothesis puts on certain scenarios? Did he do it in advance? Did it happen in a way that favors him? Then why would the Krugman argument you list be convincing? Why would it be parallel to what Bryan did?
Moreover, if you want to go further back in people’s belief structures, you could look at the evidence for (likelihood ratios on) “rewarding failure is destructive” and see how that reference class compares to the strength of the evidence in this case. This would show why he needs stronger evidence to reverse his position on TARP. (And I’m *sure* you dug up all his other posts on the metrics he proposed, right?)
Caplan actually provided a great example that others should follow: he said what he would do to his beliefs based on a datapoint, and did it. So it doesn’t reverse his position? At least he’s admitting his beliefs move, which is much more than we can expect from the commentariat.
At first I was going to condemn Silas… Then, I realized that he’s right here. Now, I think that Bryan’s updating scheme is a bit wacky – which is the point that your rat poison example makes – but, like Silas says, Bryan does leave room for you to pick whatever updating scheme you like – as long as you don’t say “No matter what happens, after the data comes in, I’m going to have a STRONGER belief that the bailouts didn’t help.”
Now, he’s just saying that, even after revising the odds of the bailouts helping upward, it’s still low enough for him to oppose them.
Where he seems to be making a mistake in Bayes’ Law is when he says “It’s not reasonable to debate the merits of rewarding failure on a grand scale.”
If that’s true, it seems that the ex ante probability of bailouts helping is zero. If that’s true, then no data can change that belief. But, I might just be reading too much into Bryan’s use of “reasonable”…
Yes Lucas I think you and I are close to agreement here. What I’m saying is that Bryan wanted an investigation into the past success or failure of the bailouts two years ago. He wasn’t saying, “Let’s think through the a priori logic of bailouts.” He wanted policymakers to be empirical, look out the window and see if TARP worked or not.
Since he thinks they’re idiots for doing more bailouts, then presumably Bryan thinks the evidence points to bailout FAIL.
And yet, that violates Bryan’s original (silly in my opinion) stance.
Just to clarify for people who can’t go to Silas and my black belt level in internet debate: Whether unemployment went up or down after TARP, I would have thought it was bad. This is because economic laws are counterfactual for one thing (e.g. even if it went down, it might have gone down faster without TARP), and also because unemployment is not the sole indicator of the success of an economic policy.
But I am not violating Bayes Law by saying that. My confidence in my belief that TARP was bad, would not have been affected by the new data on unemployment rolling in.
Silas is right, and so is Bryan, that morons on both sides would have cited any stat to justify their original views. Just like pro-Bush people said, “See? No terrorist attacks since 9/11, Cheney is awesome,” and anti-war people said, “See? No terrorist attacks, I guess waterboarding was unnecessary.”
Even so, today Bryan seems to have reverted to the commonsense view that if anything, the continued sluggishness in the economy is evidence that TARP didn’t work. (And here is where I would get empirical: I know that left to its own devices, market economies tend to recover after a year or two in recession. So if a given recession lasts a lot longer than that, and it goes hand in hand with unprecedented gov’t intervention, then that makes me more confident in my theoretical views.)
Took me a few reads to get what you were saying here – mostly because my interpretation of what Bryan was saying is a bit different from yours, but on further thought, I think you’re right. So, let me make sure I’ve got it.
(1) Bryan’s “Alas” is basically a call for a study of whether or not the bailout worked. In context, he’s suggesting that an empirical study would show that it didn’t.
(2) Bryan’s Bayesian discussion suggests that the empirical evidence would point to a “Yes the bailout helped” verdict, if the priors put an equal (or nearly equal) probability on “Yes” v. “No”. Here, Silas’s point – lots of data other than unemployment – is good.
But, I agree that it does strike me as a bit strange – except that it’s as easily explained by carelessness as by some deep internal contradiction in Caplan’s mind.