Comments on: Replicate vs. reproduce (or vice versa?)http://languagelog.ldc.upenn.edu/nll/?p=36717
Tue, 14 Aug 2018 21:18:46 +0000hourly1By: DCAhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546541
Fri, 16 Feb 2018 22:42:31 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546541Perhaps (off the cuff without yet reading the paper), the distinction can be clarified by the noun used? I'd say "reproducible research" as opposed to "replicable results" or "replicable findings". So far as I know "replicate" as used (for a very long time) by philosophers of science means "do the whole thing over from scratch and get the same answer", where "the thing" they have in mind is a lab experiment: so, different data because you collected the results anew.

I'm certainly biased by having read Claerbout's paper soon after it came out, and thinking, I should aim to make this possible for myself: his focus was on, can you reproduce what you yourself did a few years back?

]]>By: Nedhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546494
Thu, 15 Feb 2018 20:42:32 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546494For biomedical scientists there is another definition of replicates, as opposed to repeats. Suppose you extract something (RNA, protein, blood sample, whatever) from a mouse, and then run some analysis on this material in triplicate. The assay in triplicate can tell you something about the reliability of the assay, for example if you made a pipetting error with one of the three. However, as far as *biological* variability is concerned you still have n = 1 mouse, and you need to analyze more mice to provide evidence for the reproducibility of the results. For statistical analysis, 5 mice assayed in triplicate is n = 5, not n = 15.

As someone who deals with a lot of manuscripts I still find authors who are not clear on this. I generally suggest that they read this paper from David Vaux: Vaux DL, Fidler F, Cumming G. Replicates and repeats–what is the difference and is it significant? A brief discussion of statistics and experimental design. EMBO Rep. 2012 Apr 2;13(4):291-6. doi: 10.1038/embor.2012.36. PubMed PMID: 22421999; PubMed Central PMCID: PMC3321166.

Vaux would call the results of the triplicate assays replicates and those with different mice repeats.

]]>By: KevinMhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546493
Thu, 15 Feb 2018 20:22:52 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546493And don't forget the Journal of Irreproducible Results.http://www.jir.com/
For what it's worth in the usage survey above, this scientific humor magazine was founded in 1955 by a virologist and a physicist.
]]>By: Ethanhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546487
Thu, 15 Feb 2018 18:18:55 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546487As a biomedical scientist, I agree with the comment of mgh. However I think there are actually two distinct uses of "reproduce" that are relevant in this field and are generally clear in context. One refers to recalculation or reevaluation of conclusions from the same original data, for example when a referee notes "I cannot reproduce the claimed level of significance from the values reported in table 3". This is a case of "same data, same method, different scientist" (B1). The second use, as noted by mgh, is an attempt by different scientists to re-perform the original experiment and analyze the new data using the original method (B2). I would not expect the word "replicate" to be used for either of these cases.
]]>By: mghhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546484
Thu, 15 Feb 2018 17:26:00 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546484This passage from the linked arxiv article is not consistent with placing biomedical research in B1 (rather than A):

"The National Institutes of Health (NIH) maintains a website titled “Rigor and Reproducibility,” but does not provide terminology.4 It set up requirements for strengthening reproducibility of biomedical research in January 2016, providing instructions for grant proposals. NIH focuses on rigor (concerning methods and materials) and transparency (concerning publication). Their defini- tion of reproducibility is the straight combination of those two notions, while making no particular distinction with the word ‘replication’"

It also does not seem factually correct, as the NIH does seem to be clear that it is using reproducibility to mean different scientists following the same experimental design:
"Two of the cornerstones of science advancement are rigor in designing and performing scientific research and the ability to reproduce biomedical research findings. The application of rigor ensures robust and unbiased experimental design, methodology, analysis, interpretation, and reporting of results. When a result can be reproduced by multiple scientists, it validates the original results and readiness to progress to the next phase of research."

"Replicates" are usually used (at least in biomedical research) to mean the same scientists performing the same experiments with the same equipment (ie, technical replicates). As reflected in the NIH standards, "reproducibility" usually refers to different scientists performing the same experiments with different equipment.

]]>By: John Spevacekhttp://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546477
Thu, 15 Feb 2018 11:55:44 +0000http://languagelog.ldc.upenn.edu/nll/?p=36717#comment-1546477Interesting work. I hope that a third party will be able to re____________ the results.
(Sorry, couldn't help myself. )
]]>