We use cookies to improve our service and to tailor our content and advertising to you. More infoClose You can manage your cookie settings via your browser at any time. To learn more about how we use cookies, please see our cookies policyClose

I have read the responses of Hall et al. and the replies of Ludwig et al. Admittedly I have no special expertise in these subjects, but I believe I have identified a serious flaw in Dr. Hall’s arguments.

He says that his analysis is correct, while Dr. Ludwig’s is incorrect. But Ludwig’s analysis gives a highly significant result. Dr. Hall, if he is right, should be able to explain why Ludwig’s analysis gives a highly significant result. The only explanation implied by Hall’s arguments is that it is random chance. We know “p-hacking” can occur if there are many possible analyses. Then one can choose the analysis that gives a significant result by random chance. This is clearly not the situation here, since this is the primary endpoint, and there only a small number of possible analyses.

Hence, since random chance is very unlikely, there should be a reason that Dr. Ludwig’s analysis gives a significant result. Dr. Ludwig has a compelling explanation for this. Dr. Hall, if he is to be convincing, should give an explanation for the significant result. The non-significant result of Hall is explained plausibly by both Hall and Ludwig, so that result does not distinguish who is correct. The significant result is explained much more plausibly by Dr. Ludwig, so it is compelling evidence that Ludwig is correct and Hall is wrong.