I finally met Scott Sumner at the APEE conference this week. We shook hands, exchanged pleasantries, and basically were civil human beings. But that will not stop me from making this post. The fate of the US economy hangs in the balance. We must not let personal associations cloud our duty.
In a recent EconLog post Scott wrote:
This post is about the way I think about extreme outliers. It’s very unscientific, but I hope the comment section will help me to better understand this issue.
Suppose you have two variables, X and Y, which are (supposedly) positively correlated. But the very highest value of X is associated with very the lowest value of Y. Or assume the two variables are supposed to be negatively correlated, but the highest value of X corresponds to the highest value of Y. Should that make us suspicious of the alleged relationship?
Suppose you believed that direct democracy led to bad political outcomes, because philosopher kings were much better than mob rule. In that case, how likely it is that the one country with by far the most direct democracy in the entire world, would also be arguably the best governed in Europe, and perhaps the world? Possible, but how likely?
Suppose you noticed that America scores higher on happiness rankings than does Europe, on average. So you developed a hypothesis that social welfare states are less happy. How likely would be that a country which by some measures has the world’s most generous social insurance system, is also the world’s happiest country?
And while we are on the subject, suppose you thought deregulation and privatization made people unhappy. How likely would it be for the world’s most free market economy (excluding level of taxation and government spending) to also be the happiest?
Suppose you thought that the East Asia tiger economies were successful because they rejected the neoliberal agenda coming out of Washington, and instead had state directed development strategies. If that were true, how likely is it that the two very richest East Asian economies would also be number one and two in the world in the Heritage Ranking of Economic Freedom?
Suppose you believed that monetary policy was ineffective at boosting NGDP at the zero bound. In that case, how likely is it that the fastest 4 month stretch of NGDP growth in American history would occur during a period of near-zero interest rates, right after a easily identifiable monetary shock (March-July, 1933).
And while we are at it, suppose you believed that the credit channel explains why growth is slow during and after a banking crisis. How likely is it that the fastest stretch of NGDP growth would occur during a period right after America’s worst banking crisis, and during a period when 1000s of banks were still closed down? Again, not just growth during the financial crisis, but perhaps the fastest NGDP growth ever, during arguably the worst banking crisis ever.
Suppose you thought that inflation was caused by bottlenecks in the economy, and deflation was caused by slack. How likely is it that the price level (WPI) would rise by 20% during a period of 25% unemployment (1933-34)?
And speaking of the zero bound, just how likely is it that the biggest two day stock rally in US history would (just randomly) occur immediately after Hoover announced a proposal to allow the Fed to print more money, for each ounce of gold backing.
Suppose you thought that Mexican-Americans had a propensity to rape and murder. (Hmm, where have we heard that theory?) How likely is it that America’s most Mexican major city (of the top fifty) would also have the lowest murder rate, and perhaps the lowest violent crime rate?
Here’s how I look at it, and I want you to tell me why I’m wrong. If you have only a few observations, then extreme outliers are no big deal—but your study is also not very reliable. If your study includes a large number of observations, the odds of the most extreme value of X and Y being correlated in the opposite direction from the actual relationship seems very low. Am I too suspicious of extreme outliers? What do you think?
You get Scott’s point, right? He’s (of course) being a bit tongue-in-cheek, and saying that these theories are all goofy. He thinks it’s incredibly awkward for the people who hold these theories.
I have one more to add to the list:
Suppose you thought that the Fed’s tight money policy caused the Great Recession. How likely is it that the monetary base grew far, far more from mid-2008 to mid-2010 than in any other two-year period in US history?
P.S. Of course Scott will say, “In this particular case, the growth in the monetary base doesn’t correlate with loose money, even though it normally does. It’s a poor indicator.” Right, but that misses the whole point of Scott’s post. Any defender of any of the theories Scott is mocking can explain those “outliers” on other circumstances too. They sleep soundly at night, thinking themselves good scientists, just like Scott sleeps soundly at night, thinking “tight money” describes 2008-2010.