Tag Archives: Peer Review

Reproducing the results from a scientific paper can be challenging due to the absence of data and the computational tools required for their analysis. In addition, details relating to the procedures used to obtain the published results can be difficult to discern due to the use of natural language when reporting how experiments have been performed. The Investigation/Study/Assay (ISA), Nanopublications (NP), and Research Objects (RO) models are conceptual data modelling frameworks that can structure such information from scientific papers. Computational workflow platforms can also be used to reproduce analyses of data in a principled manner. We assessed the extent by which ISA, NP, and RO models, together with the Galaxy workflow system, can capture the experimental processes and reproduce the findings of a previously published paper reporting on the development of SOAPdenovo2, a de novo genome assembler.

Results

Executable workflows were developed using Galaxy, which reproduced results that were consistent with the published findings. A structured representation of the information in the SOAPdenovo2 paper was produced by combining the use of ISA, NP, and RO models. By structuring the information in the published paper using these data and scientific workflow modelling frameworks, it was possible to explicitly declare elements of experimental design, variables, and findings. The models served as guides in the curation of scientific information and this led to the identification of inconsistencies in the original published paper, thereby allowing its authors to publish corrections in the form of an errata. »

« Developments in emergent technology offer innovative solutions for facilitating a hybrid review process; we examine a unique combination of private–peer and open–public review uniquely relevant for disseminating the scholarship of teaching and learning (SoTL) through development of the Journal of Instructional Research (JIR). An analysis of the hybrid review process (combining the strengths of traditional peer review with an integrative public review process) revealed substantial reviewer participation that contributed to a well–rounded, engaged review process. Public review feedback constructively addressed the value and relevance of the implications, methodology, content and written quality of the manuscripts; an additional layer of private, peer review further refined the manuscripts to determine suitability for publication. This, in turn, created a space where refinement of content, structure, and design of SoTL research was achieved through an interactive process of scholarly inquiry and dialogue. »

« The article presents one of the main findings of an international study of 4,000 academic researchers that examined how trustworthiness is determined in the digital environment when it comes to scholarly reading, citing, and publishing. The study shows that peer review is still the most trustworthy characteristic of all. There is, though, a common perception that open access journals are not peer reviewed or do not have proper peer-review systems. Researchers appear to have moved inexorably from a print-based system to a digital system, but it has not significantly changed the way they decide what to trust. They do not trust social media. Only a minority – although significantly mostly young and early career researchers – thought that social media are anything other than more appropriate to personal interactions and peripheral to their professional/academic lives. There are other significant differences, according to the age of the researcher. Thus, in regard to choosing an outlet for publication of their work, young researchers are much less concerned with the fact that it is peer reviewed. »

« Peer review of data is an important process if data is to take its place as a first class research output. Much has been written about the theoretical aspects of peer review, but not as much about the actual process of doing it. This paper takes an experimental view, and selects seven datasets, all from the Earth Sciences and with DOIs from DataCite, and attempts to review them, with varying levels of success. Key issues identified from these case studies include the necessity of human readable metadata, accessibility of datasets, and permanence of links to and accessibility of metadata stored in other locations. »

« The purpose of this paper is to consider alternatives to the traditional system of peer review. I will argue that new methods of review should be more in accordance with the principles of Open Science. Current modes of carrying out peer review are functioning as barriers against more transparent ways of doing research. I will focus on peer reviewing as it is done in the humanities. These sciences seem to be clinging particularly tight to traditional ways of publishing and doing peer review. After looking at traditional peer review and the troubles related to it, I will discuss alternative ways of reviewing scholarly material. The anonymity of reviewers and authors, the appropriate time to make papers public, and how to reward reviewers are topics that are of importance in this context. »