This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.

Abstract

The internet era, large-scale computing and storage resources, mobile devices, social media, and their high uptake among different groups of people, have all deeply changed the way knowledge is created, communicated, and further deployed. These advances have enabled a radical transformation of the practice of science, which is now more open, more global and collaborative, and closer to society than ever. Open science has therefore become an increasingly important topic. Moreover, as open science is actively pursued by several high-profile funders and institutions, it has fast become a crucial matter to all researchers. However, because this widespread interest in open science has emerged relatively recently, its definition and implementation are constantly shifting and evolving, sometimes leaving researchers in doubt about how to adopt open science, and which are the best practices to follow.

This article therefore aims to be a field guide for scientists who want to perform science in the open, offering resources and tips to make open science happen in the four key areas of data, code, publications and peer-review.

Author Comment

This is a preprint submission to PeerJ Preprints.

Additional Information

Competing Interests

Paola Masuzzo is a Research Data Alliance early career fellow, an OpenCon alumna, and is funded through an EC H2020 project that aims to create an open data exchange ecosystem. Lennart Martens is an Academic Editor for PeerJ.

Author Contributions

Data Deposition

The following information was supplied regarding data availability:

The research in this article did not generate, collect or analyse any raw data or code.

Funding

The authors acknowledge funding from the European Union's Horizon 2020 Programme under Grant Agreement 634107 (PHC32-2014). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

1

So, I learn a couple of new things, and I liked a lot the preprint. As somebody that came from a wet lab, I can not stress enough the importance of the "experimental setup" that was already mentioned in a previous comment. This part is also relevant, as there are a lot of tricks that nobody writes (I think is called "Dark data") and equivalent to make the code open. After all, this experimental setup is necessary as for peer review pre- and post- publication.

To another point, when you talk about the preprints itself in the open access section there a contrast that maybe you can consider. I would like to read something as the opposition to them, but in the perspective that it can become a source of noise due to a large number of preprints published making harder to find the important data/code/tool. This point was raised by one of my colleagues when I make a case to preprint our work; he spent too munch time looking to preprints in arXiv to find the one that was relevant to the code. But that's up to you.

So, during the reading you mention these badges, in table three you mention Publons, as these guys give the award of "sentinel of science" just last year maybe it be worth mention. At least it was a splendid idea to recognize the peer review effort (even my advisor liked, and he is ancient school and he's not that old). Also, you could expand a couple of lines to the bad side, I don't remember which journal was ready to put like ranking to the reviewers for how fast they review and how many.

The point you raise about the 'experimental' side of the research cycle is crucial, and we will certainly make sure to stress this in a future version of the manuscript.

As for the preprints point, I think it is true that it might become difficult to track back exactly what you are looking for, especially if the number of preprints published online keeps increasing. However, I suppose this is the same for peer-reviewed articles. What I believe is really important is to have adequate systems for proper tracking and citation (e.g. robust DOIs), and aggregation services based on keywords, disciplines etc. (on the model of what the OSF has been busy with: https://osf.io/preprints/, and that we will incorporate in the next version of the manuscript).

As for Publons, I was not aware of the award, certainly a nice recognition! Perhaps, however, not too appropriate to report it in the preprint, as it could be interpreted as an 'endorsement' sign or something similar.

Thank you for taking the time to read the preprint and leave your feedback!

2 - Open Science isn't just four pillars; there are a lot of other things that should be identified and made open/reproducible/credited, such as protocols, workflows, reagents, cell lines, etc.

3 - Open science isn't just science; it also applies to engineering, humanities, etc. I understand that open science is the buzzword you have to use, but the intro should at least say that this is really open research, not just open science.

2 - You are right that there are many more other things that could be identified and made open, but of course we could not include 'everything' in this preprint. We did decide, however, to focus on the four pillars described and give somehow a deep level of detail, rather than mentioning more areas with less details.

3 - Absolutely! Open research is a much better expression, we will definitely use it in the next round of editing.

0

I think it is a very timely article that should be distributed widely and discussed to a great extend! I understand it is impossible to cover the whole Open science controversy and reach in the introduction, but I would modestly suggest (on top of what Daniel has said before) that you comment on a few things:

- You mostly consider the part of Open science that is by and for researchers only. I agree Open research is a more appropriate term in this context, but still, Open science is useful as it covers more than Open research. The other aspects of Open science should be discussed very quickly in the introduction to give the reader an overview of the potential. You cite the book chapter by Benedikt Fecher & Sascha Friesike on the five schools, but bury this important aspect in the "four pillars" chapter. Maybe adding a section "Open science: more than Open research" or something similar would allow to explain this quickly.

- The paragraph on the "history of Open science" is a bit naive (no offense). Some historians of science are uneasy with this simplistic view (Science was closed, then open, then closed again). Maybe it is not so important to discuss this, as it is to point to the reasons for the current crisis (paywalls, pre-digital formats still in use, slow process of knowledge dissemination).

- Having 4 pillars is convenient, but I would no underestimate the importance of tools, both software (not to confuse with code) and hardware. Could these fall under a fifth pillar ("experimental setup") with software, reagent, workflows, material, methods, etc.

- "Papers" is a popular term that refers to the actual 20th century paper support. Wouldn't it be interesting to look for an alternative? I do not have a suggestion here, but is something published on a platform such as PeerJ still a "paper"?

- Maybe you should reuse the terms you used to describe the 4 pillars in the titles of the next chapters, not to confuse the reader (when you go from "paper" -> "open access", etc.)

- When you write "One of the basic premises of science is that it should be based on a global, collaborative effort, building on open communication of published methods, data, and results" you only account for an idealistic view of science. In reality, Open science has an enormous opportunity cost for 1. researchers themselves (hence the importance of credit and citation) 2. institutions 3. countries (somehow secrecy is believed to be a competitive advantage). In the past (and still today to a large extend), science was done for the benefit (prestige, economic or power advantage) of researchers, but also benefactors, universities, nations, etc. not the whole community. I love the idea that we need to insist on the "communism" dimension of research, but we should not ignore the obstacles to Open science and the fact that funders are mostly national agencies supporting national interests. One paragraph about the opportunity costs should not justify inaction, but show the complexity of the topic.

Box 1 is great! And could be expended a very little, with a minimal addition of details.

Also a lexicon would be very useful.

And of course, feel free to ignore my comments if not relevant.

Looking forward to see how this manuscript evolve! I'll certainly use it in my own work.

Thank you very much for taking the time to leave your feedback on the article!

I comment on your points hereby, and I will definitely take them into account in the next evolution of the article.

- I totally agree with this point. As Daniel already suggested, Open Science is definitely more than what the article describes. Please keep in mind that when we conceived this manuscript, we thought somehow of a practical guide for scientists that felt overwhelmed with the amount of information on Open Science and wanted to have some guidance on the matter. This explains the focus of the article. However, adding a section on 'Open Science: more than Open Research' can certainly add value to the overall message. I will attempt at writing it in the next version of the article.

- Offense not taken :) I need to admit my knowledge on the history of science is not too deep, and I did not have time nor resources to acquire it before writing this article. I agree with you that this is not really relevant in the context of this manuscript, and I hope readers will not mind too much.

- A fifth pillar with software, research materials, methods and so on... This is a really nice suggestion, thanks!

- Yes, this question keeps arising at conferences, meetings and discussions all the time. We should not call these 'papers' anymore, you are right. I guess it is a matter of habit. But the habit needs to change, so I might replace 'papers' with something else. 'Articles', perhaps?

- Good point. Could definitely improve clarity!

- Yes, this is also a good point. I must admit I wanted to elaborate on this aspect (opportunity costs), but it felt a bit tricky, and I was afraid it would send a wrong message to the community (i.e. justify inaction, as you say). I will think about this more, and try to write a section that would highlight the complexity of the topic without discouraging action and engagement.

0

This is a nice overview of resources for open science, and I learned a few things as well. Here are my comments/suggestions:

1- I agree with the previous comment that "open research" or "open scholarship" might work better, although "open science" has more demonstrated usage.

2- I also agree that "paper" is a bit outdated. Other possibilities might be "report" or "peer-reviewed article."

3- The usage of the "goo.gl" shortlinks reminded me of research showing that many of these links used in legal decisions no longer work. There are so many links here that it is perhaps inevitable that some will not work in the future. However, journals might consider a best practice of using more persistent links in papers that refer to web pages. This preprint could serve as an exemplar by utilizing perma.cc or one of the other persistent link services (at least for the shortlinks).

4- In the open access section, you state that preprints have DOIs assigned, but I think this is not always true. You could say "some preprints." In any case, I think there would be a timestamp in the repository regardless of whether there was a DOI.

5- It is probably impossible to have a comprehensive list of preprint servers, but you could include OSF and the services built on it like SocArXiv, as well as preprints.org from MDPI. Also in this table, you use the Australian mirror URL for arXiv, when it should just be arxiv.org.

6- The list of resources for open access seems very wide-ranging, and I am not sure if the listing is in any order. To place more emphasis on useful tools, you could move DOAJ and Sherpa/Romeo to the beginning.

7- The section on open peer review is far more susceptible to misunderstandings than the other sections, since it is defined so many ways. Your table helps clarify this somewhat, but you could also refer to/borrow from the schema at the end of this OpenAIRE blog post: https://blogs.openaire.eu/?p=1371.

Thanks for the interest in our preprint and for taking the time to leave your feedback. You raise very good points, I try to reply here and we will definitely keep in mind your suggestions for the next version of the article.

1- Yes, open research might work better indeed. This seems to be the common 'concern' on the preprint so far, and we agree that the scope of open practices is broader than science.

2- Why not just saying 'publication'?

3- Thanks for this point, perma.cc is certainly a valuable resource, and I was not aware of it! I will look into using it for more robust and persistent links.

4- I did assume, maybe wrongly, that every preprint service would assign a DOI to an item. I will do some research and amend the text if necessary.

5- Yes, the OSF preprint service should definitely be in! Same goes for the MDPI resource.

6- No, the resources for open access are not really listed in any specific order. Maybe we could indeed try to organize this list with a certain criterion, to make the catalog of resources easier to digest/use.

7- This is a very good point. The debate on open peer-review is very alive and definitions keep evolving and extending. The schema you suggested is clear and to the point. We can definitely refer to it or adapt it to make the section much clearer.

0

As studies into researchers’ attitudes regarding open science/research data management often point to a need for practical tips, this is a welcome guide containing a wealth of useful resources and advice from a researcher’s perspective, especially for those working in the life sciences.

Below are some comments/suggestions:

- Open Research could indeed be a more inclusive term than Open Science (although in some cultural contexts “science” is actually understood in a much broader sense than in the Anglosphere – see for example ‘Literaturwissenschaft’ in German: literally ‘literarature science’).

- While the authors rightly focus on the important pillars of open data, open code, open papers and open peer review, I was wondering what their thoughts are on the perhaps more recent calls for open lab notebooks (https://en.wikipedia.org/wiki/Opennotebookscience)? I guess this also touches on previous comments regarding openness of the experimental setup as an important dimension of open science.

- It is suggested that “privacy sensitive data” do not belong in the category of public assets available to the public. Although personal data can indeed not be made openly available for legal and ethical reasons, sometimes it can nevertheless be possible to legally and ethically share research data containing personally identifiable information, albeit usually under more restricted conditions (e.g. some trustworthy data repositories are equipped and have the right procedures in place to offer researchers restricted access to personal or otherwise sensitive data). While such data would of course not constitute fully open data in the sense of the Open Definition, maybe the data sharing story should be presented as somewhat more nuanced than a binary choice between either fully open or fully closed data?

- Technically speaking, there is a third type of data repository, namely the institutional data repository (although this kind is usually less relevant for research domains characterized by greater data volumes and larger degrees of standardization and international collaboration – as these domains often build their own international infrastructures).

- Some (well-known) data repositories focus more on publicly disseminating data than on their preservation, so when selecting a repository it’s usually a good idea to also check whether it has an explicit commitment to/policy regarding long-term preservation (of course, certification will provide a strong indication of this, but not all repositories in re3data.org are certified). Other sensible criteria to take into account can be found here: https://www.openaire.eu/opendatapilot-repository. One that is worth emphasizing is whether the repository assigns persistent and unique identifiers, because this is vital to enabling a culture of data citation (which in turn gives researchers credit for making data available).

- For publicly shared data, standard licenses are in principle more interesting than bespoke licenses, because they allow for legal interoperability. An interesting tool to help you select an appropriate standard license for data (or software) is the EUDAT License Selector (http://ufal.github.io/public-license-selector/), although it also includes licenses that are not conformant with the Open Definition.

- The FAIR data concept is indeed gaining prominence among data sharing advocates, and it may be useful to point out one of its distinctive features, namely its emphasis on making data findable, accessible, interoperable and reusable to humans as well as machines. Although discussions about the FAIR data concept’s implementation and operationalization are still very much ongoing and although appropriate metadata are definitely a crucial element, it also involves other things such as persistent identifiers, user licenses, non-proprietary formats and standard vocabularies. So maybe the FAIR data concept shouldn’t just be mentioned as part of the section on metadata?

- As regards open access to publications: besides posting preprints, another option for authors to make their work open while still publishing in subscription-based journals is to deposit post-prints in their institutional repositories, although some publishers require an embargo period before the post-print can be made open access. As enablers of the self-archiving, “green” route to open access, institutional repositories are nevertheless a vital part of the open access ecosystem.

- The Open Access Directory (http://oad.simmons.edu/oadwiki/Main_Page) and OpenAIRE (https://www.openaire.eu/) might be other useful resources.

- In the context of the general lack of credit for non-traditional research outputs (such as peer reviews), it might also be worthwile pointing to new initiatives attempting to address this issue such as the RIO Open Science Journal (http://riojournal.com/), which publishes a wide variety of research outputs (including e.g. grant proposals and data management plans).

Overall, a great guide for those who want to start practicing open science but don't know where to start!

I draft here some responses, and we will make sure to address your points in a later version of the article.

- Yes, Open Research is more than Open Science, other comments have pointed to the same 'semantic' issue. We will perhaps still talk about Open Science, but explicitly mention that this is put in a much bigger context.

- Yes, we are aware of the Open Notebook Science practice (https://en.wikipedia.org/wiki/Opennotebookscience), and indeed this falls into the broader context of experimental setups and tools, which will certainly discuss in the revised version of the article.

- This is a very good point, yes. We could indeed present the data sharing story as a more nuanced one, than as a binary choice: open or close. If you have any idea about any nuanced situation, I'd love to hear more about it.

- Yes, understood. Just out of curiosity, do institutional data repositories allow access/visibility of data to third parties? Namely people who are not affiliated with the institution? I believe the answer is no, but I would like to be sure.

- Thank you for this great resource! We will definitely stress more the importance of long-term preservation, and the assignment of unique and persistent IDs.

- The EUDAT License Selector is definitely another great resource, and we will incorporate it in the next round of edits!

- I see why talking about the FAIR concept in the metadata section could arise confusion. Perhaps it is indeed better to elaborate a bit more on it, and shift its discussion in a more generic section on open data.

- Yes, we should indicate indeed more explicitly the possibility to deposit postprints in institutional repositories. Thanks for pointing this out.

- As regards the more nuanced story of data sharing: basically, apart from fully open access, various other levels of access exist. An example are the three access levels provided by the UK Data Service (open, safeguarded or controlled access), depending on what is suitable given the nature of the data and the permissions that are in place. See: https://www.ukdataservice.ac.uk/deposit-data/how-to/regular-depositors/negotiate

- regarding institutional data repositories: usually these are set up to disseminate the digital data assets of an institution, so they will focus on publicly sharing research data under open access conditions. Some institutional repositories will also offer more restricted levels of access to data that cannot be made fully open for legal, contractual or ethical reasons. It is possible that some of those restricted data are only accessible to people within the institution (but even then, the metadata record for the research data may still be publicly visible).

Add your feedback

Before adding feedback, consider if it can be asked as a question instead, and if so then use the Question tab. Pointing out typos is fine, but authors are encouraged to accept only substantially helpful feedback.

Follow this preprint for updates

"Following" is like subscribing to any updates related to a preprint.
These updates will appear in your home dashboard each time you visit PeerJ.

You can also choose to receive updates via daily or weekly email digests.
If you are following multiple preprints then we will send you
no more than one email per day or week based on your preferences.

Note: You are now also subscribed to the subject areas of this preprint
and will receive updates in the daily or weekly email digests if turned on.
You can add specific subject areas through your profile settings.