Disappointment about the war on worms in the American South 100 years ago

On GiveWell.org, I just blogged a new study revisiting the evidence on whether the campaign in the 1910s to rid the South of hookworm brought major benefits. A great 2007 paper by Hoyt Bleakley suggests that it did: after eradication school attendance rose disproportionately in historically hookworm-heavy areas; and adult earnings of babies born in affected areas also later rose. The new study revisits Bleakley’s original by reconstructing its database from primary sources, and replicating and revising the analysis. I ended up strongly questioning the original study’s conclusion. These two pairs of graphs show why. The first graph in each pair is from the  original study, the second from the new version. The original graphs seem to show jumps in outcomes of interest—school attendance, earnings—but the new ones do not.   I have also reanalyzed Bleakley’s study of the impacts of malaria eradication in the U.S., Brazil, Colombia, and Mexico. In that case my results match the original much more. Write-ups coming...

Who has the power to count chickens?

In March I recounted how former colleagues Michael Clemens, Steven Radelet, and Rikhil Bhavnani wrote an excellent paper in 2004 on the impact of foreign aid on economic growth, “Counting Chickens When They Hatch.” The idea captured in that title is that it is important to think about the likely timing of the impacts of aid. Don’t design your analysis as if you expect that funding for teaching 6-year-olds will raise economic growth in four years. Match the follow-up period to the type of aid. Count your chickens only when they hatch. Some years later, and joined by another CGD recruit, Samuel Bazzi, those authors overhauled their paper and published it in the top-flight Economic Journal (ungated version). The final version is quite different but also excellent (it won the journal’s best-article prize). Instead of doing its own econometrics afresh, it modifies the three most-cited studies in the aid-growth literature in light of the “counting chickens” insight. Although those studies disagree on whether and when aid “works,” in the sense of boosting growth, Clemens, Radelet, Bhavnani, and Bazzi (CRBB) conclude that revising the studies to take timing into account causes all results to converge, to a ginger but positive appraisal. (Listen to Michael speak cautionarily about “Chickens” in this Library of Economics and Liberty podcast.) I say “excellent” and I mean it. But, true to type, I actually doubt the econometric reasoning. I am not persuaded by these results that “aid inflows are systematically associated with modest, positive subsequent growth.” In 2010, the journal Public Finance Review inaugurated a section for replication studies, in order to increase incentives to...

Rejoinder to Pitt

The April issue of the Journal of Development Studies includes the final version of my article with Jonathan Morduch replicating the study of the impact of microcredit in Bangladesh by Mark Pitt and Shahidur Khandker. Properly, the journal also carries a reply from Mark Pitt. (Ungated versions of the dueling documents are here and here.) To my surprise, JDS did not solicit a rejoinder from us the way they did in a nearly identical situation involving a JDS editor as replicating author. Perhaps this is a sign of the strength the editors see in our paper…which is to say, maybe I should have just chilled. But as usual, Pitt’s arguments are strongly worded even as their subject remains technical. So the average reader will absorb the style more than the substance, and wonder, I fear, who are these fools Roodman and Morduch? So for the public record, here is a rejoinder from yours truly. It’s a quote-and-response. “RM [Roodman & Morduch] have backed off many of their prior claims and methods.” No. The first version of our paper questions the exogeneity of the core intent-to-treat variables; highlights that an asserted discontinuity in treatment, central to Pitt & Khandker’s (PK’s) claim to quasi-experimental status, is absent from the data; observes that the magnitude of the impact estimates depends on an arbitrary censoring choice for the “log of 0”; and demonstrates that a more-robust linear estimator produces no evidence of impact. Those arguments stand. Of course, I have also learned from the debate with PK. RM (2009) failed to replicate PK’s original impact estimates, getting opposite signs. Pitt (2011) showed us how to match, by swapping in a missing control and adopting a different censoring value. As it...

Comment on “Counting Chickens”

In 2012, the top-flight Economic Journal published an article by my former CGD colleagues about the impact of foreign aid on economic growth in receiving countries. It is called Counting Chickens When They Hatch: Timing and the Effects of Aid on Growth (ungated version on cgdev.org). The journal’s publisher, the Royal Economic Society, awarded the authors its annual prize for best article. I think you’ll see why, if you read the paper. It’s sharply written and firmly argued, yet judicious in its conclusions. Those conclusions are that “aid inflows are systematically associated with modest, positive subsequent growth”; and that “the most plausible explanation is that aid causes some degree of growth in recipient countries, although the magnitude of this relationship is modest, varies greatly across recipients and diminishes at high levels of aid.” One of my first projects at CGD was to assist Bill Easterly and Ross Levine in replicating and questioning (ungated) an older paper favoring the theory that aid causes growth. Perhaps this baptism into econometrics permanently converted me to skepticism. Perhaps it merely reinforced my distrust as a mathematician of a structural tendency in the social sciences to use equations full of Greek letters to claim objectivity and superior insight. At any rate, I’ve doubted such aid-growth studies ever since. Here is a comment that explains why I’m not persuaded by “Counting Chickens.” A version of this comment has been accepted for the new replication section of the Public Finance Review. Here is the code needed to reproduce the results. It runs off the CRBB data sets. Perhaps the authors of “Counting Chickens” will respond. And perhaps they will change my mind. For now, this...