Archive for the 'Exceptions' Category

Our workshop on the 25th proved a practical and thought-provoking day, generating some excellent feedback. Of particular interest was HEFCE’s stance on exceptions, the opportunity to share current approaches to recording exceptions, and an exploration of reporting REF compliance with Europe PMC and arXiv. A special thanks to all our speakers. A summary of all sessions and links to slides follows.

JISC UPDATE

Hannah DeGroff provided an overview and update on Jisc’s OA Good Practice Programme, touching on the recently released REF Compliance Checker plugin for EPrints, and calling for institutions to join the Publications Router service (email PubRouter@jisc.ac.uk for info). The Router will populate institutions’ repositories and CRISs with their researchers’ papers. Ben Johnson from HEFCE later emphasised that publishers are unlikely to sign up to the service if institutions haven’t already. Hannah’s slides are available here.

REF EXCEPTIONS: KEY ISSUES

Catherine Sharp from UCL reminded us of the exceptions, and summarised the key issues with each, including: difficulties in securing author manuscripts of multi-author papers, problems institutions have in tracking Gold OA papers paid directly by authors or other institutions, how to demonstrate that authors have considered “appropriateness” when publishing in a non-compliant title, who should record exceptions (authors or OA Teams) – and how, e.g. using various publications management systems. Alan Bracey outlined an analysis of UCL’s 2014 REF submission with regard to embargo and deposit exceptions, and presented a community list of non-compliant publishers and journals (see this blog post for more). Slides are available here.

HEFCE

Ben Johnson outlined HEFCE’s position on exceptions. He reiterated that the scholarly publications system is broken, relies on out-of-date technology, and needs to be changed. He clarified some key points:

REF2021 has been committed to by the government in the Green Paper. The OA policy needs to be simple, flexible, but firm. The goal in the first instance is to increase open access.

HEFCE wishes to empower institutions to make judgements with regard to policy interpretation and implementation. “Best endeavours” in point 44 of the policy should be interpreted as best efforts by institutions to establish a framework to maximise compliance, rather than best efforts by academics themselves. Claiming exceptions will not be made difficult.

HEFCE understands that researchers’ deposit rates are unlikely to be sufficient without intervention by institutions, who in turn need reliable publication alerts to prompt action.

The requirement to deposit manuscripts within three months of publication (begins April 2016) could be interpreted as applying to final publication in an issue. (However, HEFCE wants to avoid introducing delays into the deposit process.) This, and the requirement to deposit within 3 months of acceptance (due to start April 2017), will be reviewed in Autumn 2016.

If academics aren’t aware of journal policies when publishing, they should be. Work is needed to make some publishers’ OA policies clear – these publishers are not supporting open access on the ground (despite claims otherwise). Unclear or missing publisher policies is unacceptable, and passive non-compliance by publishers should be challenged. This will take time.

The “appropriateness” of a journal is for the academic to decide, following discussion with their institution. HEFCE will want to see evidence of a system in place to enable researchers to understand this requirement.

SHERPA REF

Tom Parsons updated delegates on HEFCE’s new Sherpa REF service. Sherpa REF will provide researchers and institutions with a quick yes/no answer to the question of a journal’s compliance. Designed with a simple interface, a stable beta release is due in February. Tom explored some of the complications involved in boiling 4000 words of policy to a checklist for assessing journal policies. Sherpa REF entries have been entered manually, since publishers do not make machine-readable policies available.

EXCEPTIONS BREAKOUT SESSION – SUMMARY

Although the potentially small number of exceptions makes them manageable, institutions still need a process in place, rather than dealing with each exception as it comes up. Most institutions are not keeping formal evidence of exceptions, although one is keeping copies of acceptance emails.

In general, institutions are not communicating exceptions to researchers (it’s already hard enough communicating the bare bones of the policy). Institutions are not recording exceptions widely yet – workflows aren’t ready. A few are actively recording mainly deposit/embargo exceptions, with Excel, Symplectic’s OA Monitor, or Pure. Recording exceptions is problematic in publications management systems/CRISs. An added complication with the embargo exception: when does it start? Some publishers specify date of acceptance (e.g. RSC), some publication.

Agreement that exceptions should be managed by library staff rather than authors or departments – don’t want to offer a ‘get-out’. General preference for dealing with exceptions once manuscripts are deposited. Contact researchers if needed, but keep the message as simple as possible. However, there are certain exceptions that only researchers will know about in relation to a paper (e.g. unlawful/security risk). Also preference for central upload of manuscripts in a department by one person e.g. library staff or research administrators.

No consensus on how/where Sherpa REF fits in yet. It might include too much information to be helpful to researchers; the focus should be on deposit.

Some departments keep a list of recommended publications for researchers: it’s unlikely that a library’s REF blacklist/whitelist would influence this. There’s concern over librarians telling researchers where to publish. There’s a risk of directing researchers to/away from journals who subsequently change their OA policy. However, a REF blacklist may be useful for pressuring publishers to change policies.

There’s a general preference for reporting to HEFCE via upload from Pure/Symplectic/equivalent system, in RIOXX-compliant form.

Compliance of journals (embargoes) depends on the panel to which the paper is submitted. Institutions can use their 2014 submissions as a guide to which journals are submitted to which panels, however an article may subsequently be entered to panel A or B, instead of C or D.

There’s uncertainty on how to handle the Gold OA exception. HEFCE seem not to require evidence. There’s general preference for tracking Gold so institutions know which papers won’t need to comply. It’s difficult to track papers paid for by researchers themselves/other institutions. There’s also potential for mixed messages with the Gold exception, since deposit/action isn’t required from the researcher. Is the REF policy still a Green OA policy? Some institutions with their own OA funds are still encouraging Green deposit when paying Gold fees.

EUROPE PMC REPORTING AT THE WELLCOME TRUST

Cecy Marden gave an overview of how the Trust uses Europe PMC to monitor compliance of funded journal papers with its open access policy, provided some tips and tricks for reporting, and discussed compliance rates. EPMC struggles to check OA licences accurately, because of errors in publishers’ metadata. The current solution is using the Cottage Labs tool to determine the type of licence used. Cecy’s slides (including example searches) are available here.

EUROPE PMC & PMC FOR REF COMPLIANCE

Catherine Sharp presented results of testing Europe PMC and PMC to identify author manuscripts. Europe PMC allows searching for author manuscripts, but currently provides unreliable results. A fix for this is forthcoming, and other developments to aid REF reporting are being discussed. PMC successfully finds manuscripts, although numbers are low. Catherine’s slides (including example searches) are available here.

ARXIV FOR REF COMPLIANCE

Danny Kingsley led a discussion of the limitations of and possibilities for using arXiv to record REF compliance. ArXiv is an international subject repository hosting nearly 10,000 new open access papers a month. It generated over 14 million downloads in Nov 2015. But it would need changes to enable reporting for REF. Cambridge and other Russell Group institutions have discussed the potential cost and timeframe: about $75,000 to instate acceptance date and version, with 6 months’ development time. The cost could be shared by Russell Group universities and/or Jisc. However some problems have been identified: the risk of setting a precedent for funding changes to other subject repositories, that a significant proportion of deposits may turn out to be pre-prints/submitted versions instead of accepted manuscripts, and concerns over interfering with a successful community tool for the sake of reporting funder compliance. It was generally agreed that more needs to be known about how researchers use arXiv before committing to funding changes. Discussions are ongoing, and participation is invited from institutions who are not already involved. Slides are available here.

As part of our work on exceptions to the REF policy, we’ve done an analysis of UCL’s 2014 REF submission in relation to the following exceptions:

39 (b): embargo periods that are longer than the permitted REF maximums

39 (c): journals that do not allow deposit

We were keen to get an idea of how many of these exceptions we would have had to report to HEFCE had the policy applied in 2014. We also wanted to see how far our findings agreed with HEFCE’s estimate that 96% of 2014 REF submissions covered by the policy had embargo periods that could have complied (see para 54 of the consultation outcomes document here). As well as answering these questions, we’ve produced some statistics on non-compliant journals: https://goo.gl/JwH7IV. We’ve also put together an editable list of all the non-compliant journals we found, by panel, which we hope will be a useful community resource: https://goo.gl/YJ9H3V.

What we did

UCL submitted a total of 9,413 publications to REF 2014. Of these, 8,048 (85%) would have needed to comply with the new OA policy, i.e. they were papers published in journals or proceedings with an ISSN.

We boiled the 8,048 eligible publications down to a list of 2,510 ISSNs, and checked each title’s Green OA policy. The work consisted of manual checks of publisher webpages, Sherpa Romeo record checks, and using some embargo period data supplied by the Sherpa REF project team.

We recorded journal embargo lengths, where available, and if deposit wasn’t allowed. We also recorded where a Green OA policy was unclear, or missing. Then we checked our findings against the maximum embargo lengths permitted by HEFCE, and created a list of non-compliant publications. From this, we extracted lists of non-compliant journal/proceedings titles, one for each panel. We’re making the lists available here: https://goo.gl/YJ9H3V. They’re open to anyone to edit, correct and supplement.

A selection of our findings

531 of the 8,048 publications would have fallen under either the ’embargo length’ (39b) or ‘deposit not allowed’ (39c) exception. This includes publications in titles without a clear policy, which constituted the majority of the exceptions (281).

93.4% of publications submitted to REF2014 that would have fallen under the policy would have complied (leaving aside other exceptions) if they’d been deposited in time. This is close to HEFCE’s estimate of 96%.

The majority of our exceptions (54%) were in our panel A submission.

Elsevier was UCL’s top non-compliant publisher. 15% of the exceptions were in their journals. They lead on this score by a wide margin, with Wiley, the 2nd-highest publisher of non-compliant papers, publishing 9% of the exceptions. We suspect that society journals will feature in the lists of these publishers’ journals – but we haven’t checked, yet.

Of all UCL’s non-compliant papers, more were published in The Journal of Immunology than any other title. Its publisher, The American Association of Immunologists (AAI), was the 4th-highest non-compliant publisher in our list.

What next?

We’re running a workshop to look at all the REF exceptions in January. We’re asking delegates to contribute their experiences of working with the exceptions so far. However, we’d welcome feedback from everyone, and will include comments from the whole community in the exceptions toolkit that we’ll be producing. Please add your comments to this document: https://goo.gl/QxjMcd.