Tag Archives: unreplicated studies

The Guardian’s head­line was “US doc­tors cure child born with HIV.” It con­tin­ued, “Doc­tors in the US have made med­ical his­tory by effec­tively cur­ing a child born with HIV … the child has a nor­mal life expectancy and is highly unlikely to be infec­tious to oth­ers, doc­tors believe.” The researcher was described as “stunned” at this “extra­or­di­nary” out­come. The story ended with a note that patients should not stop tak­ing anti­retro­vi­rals and that pre­ven­tive treat­ments for preg­nant, HIV pos­i­tive women were of proved effec­tive­ness. There was no reminder of the unpub­lished nature of the report and its lack of peer review or the lack of replication.

Sarah Bose­ley in the same news­pa­per the next day explained the lim­i­ta­tions of the research: “Is this the big one? Have doc­tors stum­bled across the cure for HIV? Unfor­tu­nately not. This is progress … but the impli­ca­tions for those already infected or even the still sig­nif­i­cant num­bers of babies born with the virus in the devel­op­ing world are sadly prob­a­bly slight.”

Psy­chol­ogy is his­tor­i­cally a field in which repli­ca­tions are rare. Sought after jour­nals with a high impact fac­tor typ­i­cally refuse to pub­lish repli­ca­tions and until recently, there has been lit­tle to no incen­tive for researchers to con­duct research which will not fur­ther their careers and likely would not even ever get read. Thus neg­a­tive find­ings have van­ished from exis­tence while pos­i­tive find­ings — which may be the excep­tion rather than the rule, receive all the atten­tion. This is a prob­lem that has become known as the File Drawer Effect in the world of psychology…

…Accord­ing to Ioan­ni­dis, we are cur­rently pass­ing through an extra­or­di­nary age of per­verse incen­tives in sci­ence. The prime motive of researchers is placed firmly on new dis­cov­er­ies and chas­ing sta­tis­ti­cal sig­nif­i­cance at all cost. Ioan­ni­dis describes a night­mare sce­nario, “Planet F345, Androm­eda Galaxy, Year 3045268″ in which the entire process of sci­ence is dis­torted by numer­ous per­verse incen­tives placed on researchers by dic­ta­to­r­ial jour­nal pub­lish­ers and finan­cial offi­cers “recruited after suc­cess­ful careers as real estate agents, man­agers in super­mar­ket chains, or employ­ees in other cor­po­rate struc­tures where they have proven that they can cut cost and make more money”.

Schiz­o­phre­nia researcher Tim­o­thy Crow wrote in 2008 that mol­e­c­u­lar genetic researchers inves­ti­gat­ing psy­chotic dis­or­ders such as schiz­o­phre­nia had pre­vi­ously thought that “suc­cess was inevitable-​one would ‘drain the pond dry’ and there would be the genes!” But as Crow con­cluded, “The pond is empty.” Four years later the psy­chi­atric dis­or­der and psy­cho­log­i­cal trait “gene ponds” appear to have been com­pletely drained, and there are few if any genes to be found. Twenty years ago, how­ever, lead­ing behav­ioral geneti­cists had high expec­ta­tions that mol­e­c­u­lar genetic research would soon “rev­o­lu­tion­ize” the behav­ioral sciences.