Impact for REF2021

Impact for REF2021

In terms of defining impact, RCUK specify that:

"Economic and societal impacts embrace all the extremely diverse ways in which research-related knowledge and skills benefit individuals, organisations and nations by:

  • Fostering global economic performance, and specifically the economic competitiveness of the United Kingdom
  • Increasing the effectiveness of public services and policy
  • Enhancing quality of life, health and creative output."

Information about Research impact can be found on the Pathways to Impact web pages

Impact Policy

    REF Guidance Issued November 2017 "REF2021 Decisions on Staff and Outputs ", Impact

 

    "REF 2021 submissions will include a total of one case study, plus one further case study per up to 15 FTE staff returned to the REF, for the first 105 FTE returned. The requirement will decrease after 105 FTE, to one further case study per up to 50 FTE returned. Submissions will therefore include a minimum of two case studies."

Guidance

Guidance and additional policy information will be provided here once it has been published by REF2021

For information about Impact for REF2021 please contact

Marlis Barraclough or Dawn Foster

Resources

Open Access for REF

Open Access for REF

Updated Guidance from the Funding councils

The funding councils recently published initial guidance for the next Research Excellence Framework, REF2021.  The guidance has confirmed the current requirements for open access for the REF which are:

  • All journal articles, and conference proceedings with an ISSN number, that have been accepted for publication since 1 April 2016 need to made publicly available within 3 months of acceptance or, in cases where that is not possible, within 3 months of the publication date.  Authors are required to place the final accepted manuscript on the institutional repository. 

  • All journal articles, and conference proceedings with an ISSN number, that have been accepted for publication since 1 April 2018 need to made publicly available within 3 months of acceptance or, in cases where that is not possible, a valid execption must be applied.  Authors are required to place the final accepted manuscript on the institutional repository.

  • Publications under embargo are acceptable to REF, as long as the embargo period does not exceed 12 months for STEM subjects and 24 months for HASS subjects, and the output has been placed into the repository within the REF timeframes.

A number to exceptions to this rule apply – in the main, these are:

  • Any output made publicly accessible through the Gold open access rule is compliant, as long as its details are entered into the institutional repository within three months of publication.

  • Any output published in a journal whose embargo periods exceed 12 months for STEM and 24 months for HASS subjects, or journals that do not allow any open access publishing, are compliant if the journal represents the most appropriate forum for publication of the specific article.  The bibliographic details of the article must normally be made publicly available on the institutional repository within 3 months of acceptance, and where that is not possible, within 3 months of publication.

  • Outputs by authors who were not employed by a UK institution at the time of acceptance are not within the scope of this open access policy. 

Researchers at the University of Aberdeen are able to comply with these policy requirements by forwarding their acceptance e-mail and final accepted manuscript to paperaccepted@abdn.ac.uk .  Library colleagues will check your journal’s open access permissions, create the Pure entry and upload the manuscript for you.

 

More information on open access and the support available through the library is available on https://www.abdn.ac.uk/library/support/open-access-212.php Or directly from the REF website :  www.ref.ac.uk/publications 

 

We have also updated a list of  frequently asked questions around open access for your information.  https://www.abdn.ac.uk/library/documents/REF%20Open%20Access%20FAQ.pdf

If you have any further questions, please contact Nykohla Strong (n.strong@abdn.ac.uk) or Joanna Adams  (j.adams@abdn.ac.uk).

HEFCE Open Access Policy

OPEN ACCESS MANDATE FOR REF 

The HEFCE REF open access mandate for journal articles came into force on 1 April 2016.  This means that all journal articles and some conference proceedings have to be publicly accessible within three months of acceptance for publication in order to be eligible for submission for the next REF.  Failure to meet this requirement renders the output ineligible for submission to a future REF Assessment.

How do I comply with the mandate?

You may be aware that we have been working on institutional processes to comply with the REF open access mandate for some time.  It is a three stage process:

Before submission, check the access permissions of the journal in which you are planning to publish on http://www.sherpa.ac.uk/romeo/index.php.  If it allows the deposit of the final accepted manuscript in an institutional repository, then the journal is REF compliant.  If it is not (and we estimate that around 80% of journals are compliant), please speak to your College Director of Research in the first instance.  If you are unsure about the access permissions, colleagues in the library will be able to advise: openaccess@abdn.ac.uk

On acceptance, please forward your final accepted manuscript and acceptance e-mail promptly to paperaccepted@abdn.ac.uk.  Colleagues in the Library will enter the detail into Pure and enable open access where possible.  Where the publisher requires an embargo, colleagues in the Library will set the embargo for you.  Please let us have details of all papers, even those that cannot be made publicly available so we can ascertain whether an exception applies.

On publication, please let the Library know on paperaccepted@abdn.ac.uk, so that the publication record in Pure can be completed and embargoes, where required, can be activated.

The rules allow for some exceptions which are detailed in the policy: http://www.hefce.ac.uk/pubs/year/2014/201407/

Recording and monitoring compliance with the REF open access mandate

We will only be able to select journal articles for submission for the next REF from among the articles that meet the REF open access requirements.  For REF 2014, 80% of the outputs submitted were journal articles (around 1,800).  In order to select the best we will have to make sure that as many of our journal articles comply with the REF open access mandate as possible.  The mandate cannot be applied retrospectively – if we fail to make an output accessible within the period specified by HEFCE, or to provide a valid justification within the same period, then the output is not eligible for submission.

We recently upgraded to a new version of Pure which includes a module that will assist us in the preparations for REF 2021.  It allow us to record all eligible outputs, and enable researchers to propose outputs for submission.  You will find a blue button inviting you to propose outputs on your personal pages.  The open access status of each output will be clearly marked by the open access symbol.

If your article is also subject to the Wellcome/RCUK/medical charities open access mandate

If you have paid an article processing charge to make your article publicly accessible (Gold Open Access), then it is REF compliant.  However, we will still need information on when it was accepted by the publisher, and when it first became publicly available. The easiest way to comply is to forward your final manuscript with the acceptance e-mail to paperaccepted@abdn.ac.uk and indicate that an APC will be payable from the RCUK or Charities Open Access Fund.  You can find out more about funders’ open access mandates and how to comply with them on https://www.abdn.ac.uk/library/support/open-access-212.php

If you have any questions about the REF and open access please contact Marlis Barraclough, m.barraclough@abdn.ac.uk, or, for open access questions openaccess@abdn.ac.uk

Enhance your visibility

Enhance your visibility

There are many things you can do to enhance the visibility of your research:

  • Deposit your publications in the university repository (PURE)
  • Create an ORCID and link it to PURE
  • Publish Open Access
  • Produce a short video pitch on your main research topic
  • Blog and tweet selectively on your research topics
  • Share an early version of your paper as pre-print (ArXiv, bioRxiv, preprints.org, Cognet, RepEc, SSRN, PeerJ Preprints etc.)
  • Share your data (FigShare, Dryad, UK Data Archive, Institutional Repository (PURE), etc.)
  • Use a stable and full author name and affiliation
  • Use your ORCID when publishing and applying for funding
  • Use research profiles to unambiguously link publications to you
  • Analyse who is using your research and through which channels
  • Avoid journals that are not well-indexed
Guidance

The Post 2014 REF Open Access policy requires that, in order to be eligible for submission in the next REF, all journal articles and conference proceedings with an ISSN number that have been accepted for publication on or after 1 April 2016 must be publicly accessible through an institutional or subject repository within 3 months of acceptance for publication.

The policy does not apply to monographs or other long form publications, to non-textual outputs, research data, conference proceedings published with an International Standard Book Number (ISBN), or publications that must remain confidential for security or commercial reasons.

There are a limited number of situations where an output may be exempt from the REF open access policy, which fall under the headings deposit, access and technical exceptions. HEFCE expect that such cases will be extremely rare.  More information on exceptions to the policy is in the detailed summary here.

Any output that falls within the scope of this policy and is submitted to REF2021 but does not meet the requirements without a valid exemption will be given an unclassified score and will not be assessed.

To help you prepare for the Open Access Policy we have prepared some resources

  • A downloadable guide, accessible here 
  • An FAQ document, accessible here
  • An open access workflow chart, accessible here,
  • you can also visit the Library pages for more information

A summary of the REF Open Access Policy is below:

Outputs

Repository requirements

Time lines

Version(s)

Journal articles

Institutional repository (AURA; deposit through Pure)

Within 3 months of acceptance for publication  or

Where embargo applies, deposit must be made and metadata must be discoverable within 3 months of acceptance for publication

Acceptable embargo periods:
12 months Panels A, B (STEM)
24 months Panels C,D (HASS)

'final author version' or 'post-print' which can be replaced with the final published PDF version at a later date.

Conference proceedings with ISSN number

Subject repositories – arXiv; Pubmed Central

 

What should researchers do?

Any questions?

E-mail openaccess@abdn.ac.uk or find out more on the Library pages

Visit the library

 

Resources

Policy and FAQ links

Helpful Tools and Compliance Guides

Information Links

For further information, please visit the Library pages or contact Marlis Barraclough.

All you need to know about Open Access Publishing, including the University's Open Access Publishing Policy, and forms to use to apply for Article Processing Charges can be found on our Library Pages
Key Dates for REF 2021

Key Dates for REF2021

Date

Action

1 August 2013 Start of period for income and impacts
1 January 2014 Start of period for outputs

Autumn 2017

Appointment of sub-panel chairs

Nominations invited for panel members

Recruitment of panel secretariat

Further decisions on the arrangements for submitting staff and outputs

Winter 2017-18

Appointment of panels

Consultation workshops on impact and public engagement

Spring 2018

Panels meet to develop criteria

Summer to Autumn 2018

Publish draft guidance, and consultation on panel criteria

Winter 2018-19

Publish final guidance and criteria

2019

Complete preparation of submission systems

2020

Submission phase

31 July 2020 End of assessment period (for research impacts, the research environment and related data)
November 2020 Closing date for submissions
31 December 2020 End of publication period for publication of research outputs and outputs underpinning impact case studies

2021

Assessment phase

December 2021 Publication of outcomes
Spring 2022 Publications of submissions and reports
Guidance for REF 2021

REF 2021: Background

In December 2015, Lord Stern was requested by the Government to complete an independent review of the REF2014. As part of the review, institutions were invited to comment on the final report of that review and recommendations for future exercises was published in July 2016.

  • Stern Consultation report
  • University response to Stern consultation
  • Stern report on REF review: Building on Success and Learning from Experience

 Following on from the Stern, HEFCE published a consultation on the proposed changes to REF

  • HEFCE Consultation document

Initial guidance 

The initial guidance on REF2021, published by HEFCE 1 September 2017, can be found here. This includes a letter summarising the decisions.

In November 2017, HEFCE published key decisions on staff and outputs for REF2021; these include the number of outputs required per member of staff, the model for portability of outputs and the rules around the volume of impact case studies required.

REF 2021

The Research Excellence Framework (REF) is a sector wide assessment exercise of publicly funded research in the UK. It was last held in 2014, when UK institutions were invited to submit outputs, impact case studies and evidence about the quality of their research environment for assessment. Panels of peer reviewers assessed the institutional submissions in research areas, or units of assessment.

The outcome was a quality profile for each unit of assessment which rated the quality of outputs, impacts and the research environment as world leading (4*), internationally excellent (3*), recognised internationally (2*) or recognised nationally (1*). For the general institutional quality profile, the elements of assessment were weighted as follows: publications 65%, impact 20% and research environment 15%.

Read more about the Research Excellence Framework below -

REF 2021

In the Green Paper Higher Education:  teaching excellence, social mobility and student choice the Department for Business, Innovation & Skills confirmed their intention to hold the next Research Excellence Framework (REF) assessment exercise before 2021.

What is the REF?

The REF is a sector wide assessment exercise of publicly funded research in the UK. It was last held in 2014, when UK institutions were invited to submit outputs, impact case studies and evidence about the quality of their research environment for assessment. Panels of peer reviewers assessed the institutional submissions in research areas, or units of assessment. The outcome was a quality profile for each unit of assessment which rated the quality of outputs, impacts and the research environment as world leading (4*), internationally excellent (3*), recognised internationally (2*) or recognised nationally (1*). For the general institutional quality profile, the elements of assessment were weighted as follows: publications 65%, impact 20% and research environment 15%.

The REF is the latest iteration of research assessment exercises held by the UK government since the 1980s, each 6 years or so apart from the next. It has always been a selective exercise, in which institutions were invited to submit their best (not all) of their research. It is a measure of research quality as well as research intensity, with research intensive institutions submitting the majority of their eligible staff and others submitting smaller groups of researchers. In the 2014 exercise the largest submission came from University College London which submitted 2,565.6 FTE of their staff (more than 90%), and the smallest came from St Mary’s University College, a teacher training and liberal arts institution which submitted 3 FTE (around 6%). Our own submission rate in 2014 was just over 71%; we submitted 597.2 FTE.

Why does the REF matter?

The REF outcome has become an internationally recognised standard of research quality for UK institutions, and feeds into many institutional ranking and benchmarking exercises. It therefore has great reputational value to UK institutions.

The REF results also feed into the formula used by the funding councils to calculate the annual  research block grant or Research Excellence Grant (REG). 

About two thirds of the REG is calculated through a formula which multiplies the weighted percentage of 4* research and the percentage of 3* research achieved in the REF by each unit of assessment with the number of staff submitted to work out our institutional share of the resource the Scottish Funding Council identifies for distribution each year. The remaining third of the REG calculated on the basis of our share of the sectoral total research income in Scotland over the last three years.

Our REG is currently around £20m a year and is distributed through the school budgets in accordance with numbers of staff submitted and REF quality profiles achieved. 

What do we know about REF 2021

For the next REF, which is likely to take place in 2020 or 2021, the funding councils have not yet announced any detailed guidance. In December 2015, the Government set up a review of the Research Excellence Framework, chaired by Lord Nicholas Stern.  The aim of the review is to ensure that future research funding is allocated more efficiently, offers greater rewards for excellent research and reduces the administrative burden on institutions. The review will draw on views articulated by stakeholders while responding to the recent Green Paper on Higher Education but also sought comments on specific questions posed by the Review Steering Group. The sole Scottish member of the Review Steering Group is Professor Anton Muscatelli, Vice-Chancellor of the University of Glasgow.  It will report in summer 2016.

More information on the Stern Review is available at https://www.gov.uk/government/consultations/research-excellence-framework-review-call-for-evidence

However, the funding councils have indicated that the open access requirements for outputs are likely to remain in place.  Impact will remain an element of assessment, the weighting of which in the overall quality profile may well increase. We may be asked to submit more impact case studies than in 2014, and some of the rules around underpinning research and evidence may change. Some other rules, including those on eligibility and portability of outputs, may change. Citations may play a more prominent part in the assessment of research outputs, at least for some panels. Compliance with voluntary instruments is likely to be an important indicator of the quality of the research environment, including Athena Swan compliance, compliance with the UK Research Integrity Office Code of Practice for Research, RCUK Concordat to Support the Career Development of Researchers, RCUK Concordat for Engaging the Public with Research and the RCUK Concordat on Open Research Data.

Whatever the changes that will be introduced we will need to be able to submit high quality outputs and high quality impact case studies, so the focus of activity within schools and units of assessment is on high quality and impactful research.

What are the REF open access requirements and how do I comply?

The REF open access policy requires that, in order to be eligible for submission in the next REF, all journal articles and conference proceedings with an ISSN number that have been accepted for publication on or after 1 April 2016 must be publicly accessible through an institutional repository within 3 months of acceptance for publication. Where publishers do not allow open access through a repository within the REF acceptable timelines, the output still needs to be discoverable and a valid reason for closed deposit will need to be recorded.

We are therefore asking that researchers should check the REF compliance of the journals to which they are planning to submit an article for publication – this can be done at  https://ref.sherpa.ac.uk/. Where possible, compliant journals should be chosen.  

On acceptance, or as soon as possible, please send your acceptance e-mail and final accepted manuscript to paperaccepted@abdn.ac.uk or upload both documents into Pure along with the bibliographic data available at that point. 

This applies to all articles, regardless of whether they can be enabled for open access or not.  Colleagues in the library will check the data, enable open access, set embargoes or record an exception and will make sure that the item is discoverable and accessible in accordance with the REF rules.

Find out more about the policy, how to comply and acceptable exception on /staffnet/research/impact-ref-and-open-access.php 

How will the University put together the REF 2021 submission?

The Vice Principal for Research, Professor Marion Campbell, is responsible for the institutional REF submission. She works with a small team within Research & Innovation who keep up to date on policy developments and support the systems that record and manage the data required for the submission. She is supported by the three Directors of Research who oversee the preparations for submissions to the main panels relevant to their own research areas: Professor Gary Macfarlane for Main Panel A (Life Sciences and Medicine), Dr Richard Neilson for Main Panel B (Physical Sciences and Engineering) and Professor Michael Brown for Main Panels C and D (Social Sciences , Arts and Humanities). 

Heads of School and School Directors of Research have identified a lead researcher for each unit of assessment, and a researcher responsible for co-ordinating the impact element of the submission. Some larger units of assessment have a small team of researchers working together on REF preparations. If you want to know who the relevant leads of your unit of assessment are, please contact either your School Director of Research or m.barraclough@abdn.ac.uk

What do I have to do

We are asking all REF eligible researchers (under the 2014 rules until the 2021 guidance has been published) to propose outputs for submission in Pure.  A short guide on how to do this is available here.

If your research has had, or has the potential to have, impact beyond academia, please discuss with your unit of assessment lead or impact lead in the first instance. The University supports the creation of impact of various kinds through

  • Research & Innovation – commercialisation, licensing, patenting , enterprise, knowledge exchange, contact Dr Ann Lewendon (a.lewendon@abdn.ac.uk; ext. 2477);
  • Public Engagement with Research Unit, (peru@abdn.ac.uk; ext. 3739); or
  • Public Affairs Team - engagement with policy makers/influence on public policy contact Sue Bird (sue.bird@abdn.ac.uk, ext. 2088) or Godfrey Brown (a.g.brown@abdn.ac.uk, ext. 3231)

Any further questions

Please contact Marlis Barraclough, Senior Policy Advisor 

Contacts

Marlis Barraclough 
Senior Policy Advisor for Research 
Tel: +44(0)1224 272038 
Email: m.barraclough@abdn.ac.uk

April 2016

 

REF 2014

The page contains key information relating to the REF2014.

What is the REF2014?
The REF2014 is a sector-wide assessment exercise of publicly funded research in the UK. The REF2014 replaces the Research Assessment Exercise, which last took place in 2008 (RAE2008). All Universities that receive funding from any of the Higher Education Funding Councils are invited to present selected research for assessment in to the REF2014 exercise. The assessment is carried out by panels of peers and research users in 36 subject areas, referred to in the REF2014 as Units of Assessment. In total there are four Main Panels and 36 sub-Panels. The exercise is selective, with the expectation that institutions will select only excellent work for submission, and not all research. At the end of the assessment, each institution will receive a quality profile for each Unit of Assessment to which it submitted.

The University REF2014 Steering Group

The University’s REF2014 submission will be coordinated at both College level and centrally, and will be overseen by the institutional REF2014 Steering Group.

Download details of the institutional REF2014 Steering Group

Defining Research in a REF2014 Context

The definition of research within a REF2014 context, as given by HEFCE in the Assessment Framework and Guidance on Submissions document, is as follows:

For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.

It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.

It includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports

Why is the REF2014 Important?

The REF2014 is hugely important for a number of reasons. First, it allows for benchmarking standards of research against other institutions across the sector within the UK. This brings with it a reputational issue, and should be seen as an opportunity for research Units within the University to demonstrate their research excellence. The quality profile achieved by the University in the REF2014 will also inform the block research grant allocation it receives from the Scottish Funding Council (SFC) from 2015 onwards.

For any additional information on the REF2014, please visit the HEFCE REF2014 website.

REF 2014 Results

The results of the Research Excellence Framework will be published by the REF team on 18 December 2014. More information about the publication arrangements.

What will be Assessed?

The REF2014 will assess three key research areas, as follows:

  1. Research Outputs (65%)
  2. Research Impact (20%)
  3. Research Environment (15%)

The most important of these is the assessment of research outputs. Each academic member of staff who is selected for submission will normally have to provide four research outputs for inclusion. These will be assessed by the sub-Panels. Approximately half of the sub-Panels (mainly in the STEM subjects) will take into account the citation counts of the submitted research outputs to inform their decisions. Assessment of research outputs will account for 65% of each submission’s overall quality rating. The citation counts will be provided to the REF panels via Scopus, and will refer to the research outputs submitted.

After the research output element of the assessment exercise, the non-academic impact of research is the next most significant area to be assessed. This will also be assessed by Unit of Assessment, via submission of a pre-determined number of “impact case studies” (number of case studies varying dependant on FTE per Unit) and by assessing how the research environment and research strategy within each Unit supports and facilitates non-academic research impact. The assessment of research impact is new to the REF2014 and will account for 20% of the overall quality profile within each Unit.

The research output and impact elements of each submission will be accompanied by a narrative which describes the research environment of each Unit of Assessment. The “research environment” element of the assessment accounts for 15% of each submission. The research environment element assesses research facilities, research strategy, research metrics, career development opportunities, with particular emphasis on early career researchers and postgraduate students, and on equality and diversity issues.

For further information on how submissions will be assessed, please access;

Alternatively, these documents are available on the HEFCE website.

What do the Quality Ratings Mean?
For the overall quality profile for each Unit of Assessment the quality ratings are as follows:

Rating

Definition

4*

Quality that is world leading in terms of originality, significance and rigour

3*

Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence

2*

Quality that is recognised internationally in terms of originality, significance and rigour

1*

Quality that is recognised nationally in terms of originality, significance and rigour

Unclassified

Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment

For each of the three elements of the assessment – outputs, impact and environment – sub-panels will develop a sub-profile, showing the proportion of the submission that meets each of four starred quality levels. 

  • View the assessment criteria and the definitions of the starred levels for the sub-profiles
How will the University Select Staff for Submission?

Final decisions on the selection of staff for submission to the exercise will be made by the institutional REF2014 Steering Group, in consultation with the Colleges. The REF2014 Steering Group has overarching responsibility for managing and coordinating the University’s submission to the REF2014. Decisions on inclusion for submission will be based entirely on research quality.

In making decisions on inclusion, the University, via the REF2014 Steering Group, will adhere to its internal REF2014 Equality and Diversity Code of Practice, as required by the Funding Councils. The University's REF2014 Equality and Diversity Code of Practice was approved via the University Court at its meeting on 6 December 2011 and has now been submitted to the national REF2014 Team (HEFCE) for formal approval.

Key Documentation

Please find a number of documents key REF2014 below.

Impact Related Links:

In order to access the additional documentation on Impact listed below, as released by HEFCE following the Impact Pilot Exercise (2010);

  • HEFCE Impact Pilot Exercise Expert Panel Findings.
  • Feedback from the Higher Education Institutions Involved in the Pilot.
  • Examples of impact case studies best practice, as submitted to the pilot exercise

Please visit the following link: http://www.hefce.ac.uk/rsrch/REFimpact/

Contacts
RAE 2008

The University's research excellence is reflected in the results of the Research Assessment Exercise (RAE) 2008.

World Leading

  • 89% of Aberdeen’s research activity is of international quality and 55% is world leading or internationally excellent. (4* and 3*)
  • All of our 33 submissions have activity rated as world leading or internationally excellent

1st in the UK

  • 1st for Theology, Divinity and Religious Studies
  • 1st equal for Health Services Research

Outstanding Performance

Almost three quarters of our staff were submitted to units where at least half of the research activity was rated as world leading or internationally excellent (70%+)

  • Computer Science and Informatics (70%)
  • Health Services Research (80%)
  • Other Hospital Based Clinical Subjects (75%)
  • Theology, Divinity and Religious Studies (80%)