May 6, 2018

Five years of blog silence! Good news: I’m not dead! :) Working hard on Unpaywall these days. Making no promises about any additional blogging, but what the heck, feeling it today, so here goes :)

Was digging into some publisher pages today, and I noticed a trend. Links to Public Access author manuscripts that CHORUS says are publicly available thanks to funder mandates are often very difficult to actually find on publisher pages. Want to see what I mean?

It’s in the margin, beside “SUBSCRIPTION REQUIRED”, under “Buy Article”, and under “Log in to Institution”. Nope, not “Available via CHORUS”, that’s takes you to the CHORUS website. Under that. Yup! Pretty clear once you see it, it’s true, but is the Public really going to go looking for that when they don’t know to look?

On this one though? https://aip.scitation.org/doi/10.1063/1.4962501
On this one the publishers are *counting* on you to know that if you click on “CHORUS” you will get to a free version of the paper, because that’s the only label the free version is given. If you naively think clicking “PDF” or “Full Text” would be the best way to get you to the full text, you just get these options… with no indication you can also read the author manuscript for free.

Not very happy-making.

And the thing is the Author Manuscripts are peer-reviewed versions! They have all the same info as the PDFs these journals want to selling us, they are just missing the prettying up etc. So Publishers aren’t doing an important quality gate-keeping role here — they are just making it harder than it needs to be for people to find free versions of articles, the free versions that funders have mandated be made available. I have no problem then if they want to sell the Public a pretty PDF version. Heck I might even buy it sometimes. But let it be an informed decision. The publishers are telling funders “oh yes of COURSE we link to the free articles on our page” but then doing it a way that makes it really unlikely it will actually improve Public Access. #notcool

It’s pretty disappointing, but not very surprising. Counting on toll-access publishers to implement our #openaccess infrastructure is kinda likely to end up this way, yeah?

Anyway, I don’t know if anyone is looking at these issues systematically, but I think doing so would be a great idea so we can start making noise to our funders and our publishers to do better.

Value all research products

A new funding policy by the US National Science Foundation represents a sea-change in how researchers are evaluated, says Heather Piwowar

What a difference a word makes. For all new grant applications from 14 January, the US National Science Foundation (NSF) asks a principal investigator to list his or her research “products” rather than “publications” in the biographical sketch section. This means that, according to the NSF, a scientist’s worth is not dependent solely on publications. Data sets, software and other non-traditional research products will count too.

There are more diverse research products now than ever before. Scientists are developing and releasing better tools to document their workflow, check each other’s work and share information, from data repositories to post-publication discussion systems. As it gets easier to publish a wide variety of material online, it should also become easy to recognize the breadth of a scientist’s intellectual contributions.

But one must evaluate whether each product has made an impact on its field — from a data set on beetle growth, for instance, to the solution to a colleague’s research problem posted on a question-and-answer website. So scientists are developing and assessing alternative metrics, or ‘altmetrics’ — new ways to measure engagement with research output.

The NSF policy change comes at a time when around 1 in 40 scholars is active on Twitter1, more than 2 million researchers use the online reference-sharing tool Mendeley (see go.nature.com/x63cwe), and more than 25,000 blog entries have been written about peer-reviewed research papers and indexed on the Research Blogging platform2.

In the next five years, I believe that it will become routine to track — and to value — citations to an online lab notebook, contributions to a software library, bookmarks to data sets from content-sharing sites such as Pinterest and Delicious. In other words, to value a wider range of metrics that suggest a research product has made a difference. For example, my colleagues and I have estimated that the data sets added to the US National Center for Biotechnology Information’s Gene Expression Omnibus in 2007 have contributed to more than 1,000 papers3, 4. Such attributions continue to accumulate for several years after data sets are first made publicly available.

In the long run, the NSF policy change will do much more than just reward an investigator who has authored a popular statistics package, for instance. It will change the game, because it will alter how scientists assess research impact.

The new NSF policy states: “Acceptable products must be citable and accessible including but not limited to publications, data sets, software, patents, and copyrights.” By contrast, previous policies allowed only “patents, copyrights and software systems” in addition to research publications in the biography section of a proposal, and considered their inclusion to be a substitute for the main task of listing research papers.

Still, the status quo is largely unchanged. Some types of NSF grant-renewal applications continue to request papers alone. Indeed, several funders — including the US National Institutes of Health, the Howard Hughes Medical Institute and the UK Medical Research Council — still explicitly ask for a list of research papers rather than products.

Even when applicants are allowed to include alternative products in grant applications, how will reviewers know if they should be impressed? They might have a little bit of time to watch a short video on YouTube demonstrating a wet-lab technique, or to read a Google Plus post describing a computational algorithm. But what if the technique takes more time to review, or is in an area that is outside the reviewer’s expertise? Existing evaluation mechanisms often fail for alternative products — a YouTube video, for example, has no journal title to use as a proxy for anticipated impact. But it will definitely receive a number of downloads, some ‘likes’ on Facebook, a few Pinterest bookmarks and discussion in blogs.

Tracking trends

Many altmetrics have already been gathered for a range of research products. For example, the data repositories Dryad and figshare track download statistics (figshare is supported by Digital Science, which is owned by the same parent company as Nature). Some repositories, such as the Inter-university Consortium for Political and Social Research, provide anonymous demographic breakdowns of usage.

Specific tools have been built to aggregate altmetrics across a wide variety of content. Altmetric.com (also supported by Digital Science) reveals the impact of anything with a digital object identifier (DOI) or other standard identifier. It can find mentions of a data set in blog posts, tweets and mainstream media (see go.nature.com/yche8g). The non-profit organization ImpactStory (http://impactstory.org), of which I am a co-founder, tracks the impact of articles, data sets, software, blog posts, posters and lab websites by monitoring citations, blogs, tweets, download statistics and attributions in research articles, such as mentions within methods and acknowledgements5. For example, a data set on an outbreak of Escherichia coli has received 43 ‘stars’ in the GitHub software repository, 18 tweets and two mentions in peer-reviewed articles (see go.nature.com/dnhdgh).

Such altmetrics give a fuller picture of how research products have influenced conversation, thought and behaviour. Tracking them is likely to motivate more people to release alternative products — scientists say that the most important condition for sharing their data is ensuring that they receive proper credit for it6.

The shift to valuing broad research impact will be more rapid and smooth if more funders and institutions explicitly welcome evidence of impact. Scientists can speed the shift by publishing diverse research products in their natural form, rather than shoehorning everything into an article format, and by tracking and reporting their products’ impact. When we, as scientists, build and use tools and infrastructure that support open dissemination of actionable, accessible and auditable metrics, we will be on our way to a more useful and nimble scholarly communication system.

Great question. The first two are actually also available OA, under a CC-BY license (though I was disappointed to see the publishers didn’t make the CC-BY license clear, and I didn’t know that they would also appear in a format that makes it look like they are subscription only… I’m unhappy about that but oh well).

The final one was upon invitation from Nature. I hemmed and hawwed about whether to do it, since it wasn’t OA, but decided ultimately to go for it because (a) it was an opinion piece not primary research, and (b) if opinion pieces about open things are only published in open places, they miss many audiences that aren’t yet convinced about the value of open. I tried to mitigate lack of open by publishing as much preprint about it as I could, negotiating hard for no paywall for its first week, and I’ll post the preprint as soon as I can (I think July 10th? will double check)

The purist-loving part of me does wish I only published in OA places and is unsettled when I don’t. More than that, though, I want to change the world to be more open, and I think I can do that best by sometimes publishing in places that aren’t OA. (yet. ;) )

Edited: moved the “sometimes” in the last sentence to better capture my meaning.

More and more of us are being choosy about where to place our publishing-related efforts: we say no to reviewing requests for non-OA journals, we preferentially choose to publish in OA journals, we refuse to publish in Elsevier journals… we figure out where our principles meet our pragmatic needs and we make decisions accordingly.

I’ve heard people wonder whether they should describe *why* they are declining a review or an article invitation, instead of just saying no. A few months ago I was invited to contribute a paper to an Elsevier journal as part of a special collection. I responded as such:

Thanks for the invitation. It sounds like a great special issue!

That said, I’ve signed the Elsevier boycott. In the spirit of encouraging you to understand how seriously some scholars dislike Elsevier’s current policies and wish you would move your journal to a truly Open publisher I’m not willing to write anything for your publication.

Hopefully we’ll have a chance to collaborate another way some day.

Sincerely,

Heather

And here, in part, was the response from the guest editor (posted with permission):

Thanks! I think this is great. We will make a special point about how two have refused to publish with Elsevier and how the quality of Elsevier journals suffers because of its policies and lobbying. That way your message will reach the audience (and editors).

If we want people to hear that we — as scholars, librarians, students, the public — want change, then we need to speak up. We’ll be surprised how often our voices reinforce those of others and are forwarded on.

May 13, 2013

We have a huge and valuable opportunity to honor hard work and dedication in our community: the White House is calling for nominations for “Open Science” Champions of Change.

Awards matter. They feel good, they help people get taken seriously, and they make it easier to get funding. Let’s run with this opportunity!

Nominations must be in by May 14 2013 (tomorrow!) and you can nominate as many people and organizations as you like. It isn’t clear, but it seems like multiple people will be honored. The nominator and nominated must be both be affiliated with a US address. Under “Theme of Service,” choose “Open Science”.

Here are my personal open science champions of change. I hesitated to post this, because of course the list isn’t complete: I’ve got a bad memory so I’ve no doubt forgotten many, it is likely skewed to people whose tweets and blog posts I’ve recently read, and it emphasizes my biomed-centric, research data, coding, textmining view of the world. That said, just because I can’t do it all is no reason not to make a start, right? (what is the real quote for that? open science words to live by if there ever were any), so here’s a go:

May 11, 2013

Does your society want to embrace Open Access but not know where to start? Maybe this will help.

I was invited to be part of an OA Exploration Taskforce last fall: AMIA is/was considering OA for its society journal JAMIA and deciding what it wanted to do, if anything. As the person on the taskforce most familiar with current OA trends I was in charge of reporting back on possible OA options. Others on the taskforce surveyed current membership, did financial modelling, and so on.

I was surprised how confusing and time-consuming it was to gather and sift information on options, so hopefully blogging the report here (belatedly) will save others time and spur conversation. Thanks very much to all the publishers I contacted for their information, and for permission to post this information, especially the specifics provided by BMJ and Ubiquity Press.

Here is my initial post with notes, thoughts, and contact details, and the organized list of options is below. Much has changed in the OA landscape since last fall when this was written, so some of this might already be out of date…. please check with the publishers for current information. Sorry it has taken me months to post this; it has been busy.

I’m not actually sure of the current status of JAMIA and OA, since my participation on the task force has ended. I know they took a subset of these options to the Board and there was active consideration of options. Nancy Roderer summarized the situation to me this way, a few months ago:

“I believe they are committed now to OA, but worry about the finances, and what you have given helps in convincing them that if they say what the want and put out a request, publishers will make some good offers and the best ones will come from the newer publishers.”

I’m very glad to have contributed to the task force. I felt honoured to be invited, and it was empowering to have an opportunity to help make AMIA the open society I think it can and should be.

Overview of options (details follow)

Continue with current OA offering

Increase openness within current framework

change Unlocked license to CC-BY

make all papers CC-BY when they become free in PMC

shorten time until papers are free in PMC

lower Unlocked fee

offer discount Unlocked fee to AMIA members

offer “coupons” for Unlock fees to organizations that subscribe to JAMIA

Introduce new OA journal possibly to complement JAMIA

through BMJ

through F1000Research and F1000Posters

others

Transition JAMIA to 100% OA with a publisher

through BMJ

through BioMed Central

through Ubiquity Press

others

Publish JAMIA within AMIA as 100% OA

hosted internally

hosted by a journal-management system hosting service

others

Other options

probably out of scope: publishing cooperative, overlay journal, etc.

related revenue and cost-saving possibilities

OA Options for a society journal: details

Edited to add: a number of people have said my original post didn’t highlight the financial positives from openness. I’ll try to add this. In the mean time, I’ve added a “positives from openness” placeholder to possible financial implications below. The potential financial positives include increased membership numbers due to more exposure and future-embracing leadership, more interest in publishing in the journal due to wider readership and openness, and so on.

JAMIA authors may self-archive after 6 months on their personal website or through their institution’s repository, but may not post the publisher’s version or PDF

2. Increase openness within current framework

Some small changes could increase the Openness of JAMIA content.

– Change Unlocked license to CC-BY

“Unlocked” articles could be made Open under a CC-BY license rather than a CC-BY-NC license. CC-BY is generally considered the best OA practice because it facilitates most reuse. NC means “non-commercial”: it prohibits reuse by companies without further negotiation. Startup and established commercial companies often want to reuse article text for indexing, building research tools, text-mining for research, and disseminating reprints. Restrictions can be surprising: images and text can not be used in Wikipedia when licensed CC-BY-NC, for example. For more discussion on NC clause, see:

– Make all articles CC-BY when they are available for free in PMC

JAMIA could choose to make articles available under a CC-BY license once they are available for free to PMC. “Free in PMC” is not the same as Open Access because it does not allow Open reuse and redistribution; moving to a delayed OA license would facilitate this reuse. RUP uses this model: subscriptions are needed to view the content for the first N months, then the articles are OA.

– offer discount Unlocked fee to AMIA members

– offer “coupons” for Unlock fees to organizations that subscribe to JAMIA

JAMIA could offer coupons to subscribers to offset Unlock fees. This would make it easier for authors of subscribing institutions to publish their work as Unlocked articles. The Royal Society of Chemistry has recently proposed a similar idea to facilitate community transition to OA.

3. Introduce new OA journal, possibly to complement JAMIA

– through BMJ

AMIA could launch a companion journal through BMJ to complement the existing journal. It could take the same form as BMJ Open: 100% open access, peer-reviewed on scientific rigor but not anticipated impact or novelty, online-only, and agile with respect to emerging publishing practices. Authors could submit directly to “JAMIA Open”, and also editors of JAMIA could suggest JAMIA Open to authors of papers submitted to JAMIA that are sound but not appropriate for JAMIA. JAMIA Open would be a standalone journal which would be indexed.

– through F1000Research and F1000Posters

JAMIA could launch a novel publishing collection through F1000Research and F1000Posters. Authors could submit papers, data papers, posters, and presentations to F1000Research and F1000Posters: these could then be consolidated into a “JAMIA page”, similar to the emerging institution pages. This would be a diverse collection. Many of the research papers would be indexed in traditional indexes (based on post-publication peer review status), but the collection itself would not be a standalone journal.

Financial implication: positives from openness; more discussion would be necessary with F1000. Submission to F1000Research will have a fee.

– others

There may be other possibilities to partner with PLOS Currents, PeerJ, etc. More research would be needed.

4. Transition JAMIA to 100% OA with a publisher

– through BMJ

AMIA could transition JAMIA to a 100% OA journal with BMJ. This would retain the existing infrastructure with BMJ, but eliminate the subscription base and require that all articles (except those with waivers) pay an Unlock fee. BMJ does not do this with any of its other society journals at the moment — its only 100% OA journals are BMJ and BMJ Open — but could be open to this. The Unlock fee could change, depending on how much revenue JAMIA wanted.

– through BioMed Central

AMIA could transition JAMIA to a 100% OA journal with a publisher who publishes many 100% OA society journals. BioMed Central (BMC) is such a publisher (OA society journals it publishes). See feature list BMC has provided to the JAMIA OA explorations taskforce.

Detailed financial and operational terms would require a conversation about AMIA’s goals and requirements, but BMC has offered example terms based on its experiences with other societies.

Financial implications (rough estimates: bespoke agreement would be based on detailed discussion):

‘standard’ article processing charge (APC) is currently £1075.

This increases by £175 if editorial office support is required and by £195 if copy editing, provided by the publisher, is required. If both services are required the there is a discount, and the APC increases by £360

The Standard APC level provides a small honorarium to the society. The APC can be increased, by a surcharge, to raise the potential revenue available to the society

Typically the society receives a 50% share of advertising revenue where they have brought the advertiser/contact to BMC.

No reprint revenue because OA

AMIA could offer discounts to members, or prepay APCs for other reason. Discounts are available when APCs are prepaid. For example, a society depositing $100,000USD would automatically qualify for at least a 15% discount on the APCs it covers for its members

Print can be a major cost and revenue stream so knowing about JAMIA’s plans for print will also help us tailor our proposal more appropriately. BMC does not print any journals on a monthly or issue basis but we often prepare special and anniversary printed collections for a number of our titles, such as Arthritis Research & Therapy. If regular printing was an absolute requirement, we could discuss providing it but have found that the limitations of print – in terms of article length, publication delays etc – are challenging for all concerned (authors, the society as well as the publisher).

– through Ubiquity Press

AMIA could transition JAMIA to a 100% OA journal with an emerging publisher of OA society journals. Ubiquity Press is such a publisher. See feature list Ubiquity Press has provided to the JAMIA OA exploration taskforce.

Detailed financial and operational terms would require a conversation about AMIA’s goals and requirements, but Ubiquity Press has offered example terms.

Financial implications (rough estimates: bespoke agreement would be based on detailed discussion):

A suggested article-processing charge (APC) of $200

At an APC of $200, UP production and hosting would cost between $1,000 per year (production and hosting only) to $15,000 per year (production and hosting plus peer review management, copyediting and marketing)

print-on-demand is available and could be a source of revenue for the journal

back issue conversion is available, with cost TBD

– others

Wiley Open, Elsevier, Copernicus, and MedKnow would also be interested publishing JAMIA as an OA journal. They require a detailed conversation on JAMIA’s needs before suggesting financial terms. Wiley Open and Elsevier would both be willing to publish with a CC-BY license (Copernicus and MedKnow: unknown).

Wiley details: fully gold OA journals pricing ranges between $1650 and $3500 at the moment. APC’s are set on a range of factors including field of research and level of funding, IF, brand and positioning of the journal, pricing of other titles in the discipline, etc. Could be flexible in terms of model based on discussion of needs and goals.

Copernicus details: We finance our Open Access publication services by charging Article Processing Charges (APCs) on page basis from the authors. Therefore, these APCs depend on the length of the article as well as on the journal format (traditional vs. interactive). In most of the cases this results in the APCs range between €500 and €1000 per article. If a journal is already established, and has an Impact Factor, as JAIMIA has, we suggest to start with reduced APCs and then by stages raise them to the full level. However, the journal owners have to decide upon the range of the APCs because they would need to subsidise any reduction.

5. Publish JAMIA within AMIA

– hosted internally

AMIA could run an online-only JAMIA journal itself, using self-hosted open source journal management software.

Financial implications can be assessed based on expenses and time estimates in

March 13, 2013

I recently posted about why Google is not a good enough solution for searching the academic literature (because can’t build on the results! and read the comments on that post for more).

It is sad indeed, then, that PMC and Publishers forbid scientists and others from spidering/indexing/mining their content…. while giving Google privilege to do exactly this.

Check out the robot.txt files for PMC for /pmc/articles/ and notice that GoogleBot is allowed, Bing and a few others are allowed, but User-Agent:* (the rest of us) are not. The same is true for ScienceDirect robots.txt: Google may textmine everything, experimenting scientists, nothing. (hat tip to Alf Eaton on twitter)

Is this defensible on the grounds that Google knows what it is doing but The Rest Of Us Can Not Be Trusted? I sure hope not. Scientists are routinely trusted with a lot more than writing a script that won’t bring down a server. There are other ways to ensure someone won’t bring down a server than a global robots.txt ban.

Perhaps a ban is the only way to prevent unauthorized redistribution of large numbers of papers gathered via spidering? Nope. Require people to register. Monitor use. Clearly state what may be redistributed, what may not, and what actions will be taken if people behave badly.

Maybe they are just waiting till Scientist-initiated indexing projects gets Big and Important and Ask Nicely and then they will write them in as an allowed user. Maybe. But restricting play and experimentation is a pretty poor way to bring about that future and we should not accept this as the default behaviour of the keepers of our scientific literature.

PMC calls its prohibition against bulk downloading a “copyright” issue. That doesn’t make any sense to me. Sounds much more like a Terms of Use issue than a copyright issue. Am I wrong? If so, educate me in the comments. If I’m right, then I think we should ask PMC to change its wording because calling this a copyright issue just muddies already muddy waters.

It does appear to be, at least in part, a contract issue. In the contract between publishers and PMC (http://t.co/EhZP5SrS1i point 16, ht again to Alf Eaton), PMC volunteers in its terms that PMC will prohibit bulk downloading. Why does PMC include this sentence? Is it part of the NIH Public Access law that PMC has to include this sentence? If not, isn’t it capitulating an awful lot to publishers… basically undermining the ability for scientists to build enhanced searching tools, etc?

(and, how, given this, does Google get access? Don’t get me wrong. I think Google is fantastic! I want Google to keep having access! I just want all responsible systems to have the possibility of the same access to our publicly funded and hosted research, so that someone will build infrastructure that properly supports research and research tools.)

Anyway, these spidering policies strike me as unfair, and something that people should be talking about and complaining about and doing something about, especially as we start to craft new policies for how people and computers can access our Public Access research output under the new OSTP policy.

March 5, 2013

People often ask: why all the fuss about Search for academic papers? Google does a fine job, we can find everything we need, what’s the problem?

I gave an answer to this in a comment on Mike Taylor’s blog and it got a bit of twitter pickup, so reposting my comment here for this audience. Summary: no one can build on the results!

Google isn’t an acceptable answer to Searching across academic papers (toll access, green OA, gold OA, whatever) because it doesn’t support a way for people to digest the search results, add value, and apply the results in new and innovative ways. Google search results can only be used on Google’s website manually, or embedded as-is in other websites.

Neither Google nor Google Scholar offer an API — for love nor money, as far as I can tell, point me to it if I am wrong — that would let us do a Google Search and then sort/filter/enhance the results to add value and use in research and in scholarly tools.

Totally unacceptable as a search solution for the scholarly literature. Think of the opportunity cost to research and research tools, and all the things that better research tools facilitate.

It doesn’t have to be this way. Search results can be openly available for reuse (see the search APIs and API terms of use for PLOS, PMC, etc).

February 28, 2013

It looks like PLOS Biology doesn’t ask reviewers to help uphold their data availability policies…. and I’m sure they aren’t the only journal missing this step.

I just send this email to PLOS Biology. When you review a paper, check the material you are sent to see if you are asked to assess appropriate availability of materials, and if not (or not with sufficient emphasis) please make your voice heard. You are welcome to use my dashed-off email as a template if it helps, needless to say.

Hi PLOS Biology,

I’m reviewing a paper for you now. I’ve just realized that your email to reviewers contains several important prompt questions, but no prompts asking us whether data+software have been made appropriate publicly available, as per PLOS guidelines or community norms, whichever are stricter.

Your author instructions say: “All appropriate datasets, images, and information should be deposited in public resources”… but there does not appear to be any reviewer check?

Seems a pretty big lost opportunity: reviewers are very well placed to make recommendations about what data should be made available. The “detailed protocols” mentioned in your reviewer guidelines are unlikely to suggest datasets or software to most people.

Sincerely,

Heather

a big fan of data

Update: a few days later, PLOS Biology responded as such (and gave me permission to post their response), with a CC to four internal employees:

Dear Heather

Many thanks for raising this issue with us. We are actually working on two fronts that will, we hope address your concerns in the near future (although not absolutely immediately). One is a general review of our policies, instructions and guidelines for the PLOS journals around data issues, and the other is improvements to the instructions and forms we use with reviewers. Both of these give us good opportunities to improve what we ask for and stipulate around data, which as you suggest is not yet optimal. It would be great if you let us know of other opportunities you think we’re missing, or any other suggestions you have in this area.

[..]

Theo

and I replied:

Thanks for the response, Theo!

This all sounds good, though I hope you don’t hold off on easy small improvements (adding a sentence or two to reviewer instructions to ask whether existing author instructions on data have been followed) until large changes are thoroughly designed and implemented. [..]

Sincerely,

Heather

Summary: journals want to hear from us. It is definitely worth the time to raise these issues. Please write to your journals too!

Comments Off on Do your review instructions ask if data+software are available?