In July 2018, the REF team published the following guidance for REF 2021:
- Draft guidance on submissions
- Consultation on the draft panel criteria and working methods
- Draft guidance on codes of practice
In August 2018, the University organised information sessions for staff which summarised the main points arising from the guidance and provided an opportunity for staff to discuss any queries and raise any issues that may help to inform our institutional response to the consultation. Staff were also informed of the next steps in our institutional preparations for REF 2021.
Please do not hesitate to contact Marlis Barraclough, Senior Policy Adviser (firstname.lastname@example.org) if you have any queries regarding REF 2021 or wish to contribute to the institutional response.
- REF 2021
What is the REF?
The REF is a sector wide assessment exercise of publicly funded research in the UK. It was last held in 2014, when UK institutions were invited to submit outputs, impact case studies and evidence about the quality of their research environment for assessment. Panels of peer reviewers assessed the institutional submissions in research areas, or units of assessment. The outcome was a quality profile for each unit of assessment which rated the quality of outputs, impacts and the research environment as world leading (4*), internationally excellent (3*), recognised internationally (2*) or recognised nationally (1*). For the general institutional quality profile, the elements of assessment were weighted as follows: publications 65%, impact 20% and research environment 15%.
The REF is the latest iteration of research assessment exercises held by the UK government since the 1980s, each 6 years or so apart from the next. It has always been a selective exercise, in which institutions were invited to submit their best (not all) of their research. It is a measure of research quality as well as research intensity, with research intensive institutions submitting the majority of their eligible staff and others submitting smaller groups of researchers. In the 2014 exercise the largest submission came from University College London which submitted 2,565.6 FTE of their staff (more than 90%), and the smallest came from St Mary’s University College, a teacher training and liberal arts institution which submitted 3 FTE (around 6%). Our own submission rate in 2014 was just over 71%; we submitted 597.2 FTE.
Why does the REF matter?
The REF outcome has become an internationally recognised standard of research quality for UK institutions, and feeds into many institutional ranking and benchmarking exercises. It therefore has great reputational value to UK institutions.
The REF results also feed into the formula used by the funding councils to calculate the annual research block grant or Research Excellence Grant (REG).
About two thirds of the REG is calculated through a formula which multiplies the weighted percentage of 4* research and the percentage of 3* research achieved in the REF by each unit of assessment with the number of staff submitted to work out our institutional share of the resource the Scottish Funding Council identifies for distribution each year. The remaining third of the REG calculated on the basis of our share of the sectoral total research income in Scotland over the last three years.
Our REG is currently around £20m a year and is distributed through the school budgets in accordance with numbers of staff submitted and REF quality profiles achieved.
What do we know about REF 2021
For the next REF, which is likely to take place in 2020 or 2021, the funding councils have not yet announced any detailed guidance. In December 2015, the Government set up a review of the Research Excellence Framework, chaired by Lord Nicholas Stern. The aim of the review is to ensure that future research funding is allocated more efficiently, offers greater rewards for excellent research and reduces the administrative burden on institutions. The review will draw on views articulated by stakeholders while responding to the recent Green Paper on Higher Education but also sought comments on specific questions posed by the Review Steering Group. The sole Scottish member of the Review Steering Group is Professor Anton Muscatelli, Vice-Chancellor of the University of Glasgow. It will report in summer 2016.
More information on the Stern Review is available at https://www.gov.uk/government/consultations/research-excellence-framework-review-call-for-evidence
However, the funding councils have indicated that the open access requirements for outputs are likely to remain in place. Impact will remain an element of assessment, the weighting of which in the overall quality profile may well increase. We may be asked to submit more impact case studies than in 2014, and some of the rules around underpinning research and evidence may change. Some other rules, including those on eligibility and portability of outputs, may change. Citations may play a more prominent part in the assessment of research outputs, at least for some panels. Compliance with voluntary instruments is likely to be an important indicator of the quality of the research environment, including Athena Swan compliance, compliance with the UK Research Integrity Office Code of Practice for Research, RCUK Concordat to Support the Career Development of Researchers, RCUK Concordat for Engaging the Public with Research and the RCUK Concordat on Open Research Data.
Whatever the changes that will be introduced we will need to be able to submit high quality outputs and high quality impact case studies, so the focus of activity within schools and units of assessment is on high quality and impactful research.
What are the REF open access requirements and how do I comply?
The REF open access policy requires that, in order to be eligible for submission in the next REF, all journal articles and conference proceedings with an ISSN number that have been accepted for publication on or after 1 April 2016 must be publicly accessible through an institutional repository within 3 months of acceptance for publication. Where publishers do not allow open access through a repository within the REF acceptable timelines, the output still needs to be discoverable and a valid reason for closed deposit will need to be recorded.
We are therefore asking that researchers should check the REF compliance of the journals to which they are planning to submit an article for publication – this can be done at https://ref.sherpa.ac.uk/. Where possible, compliant journals should be chosen.
On acceptance, or as soon as possible, please send your acceptance e-mail and final accepted manuscript to email@example.com or upload both documents into Pure along with the bibliographic data available at that point.
This applies to all articles, regardless of whether they can be enabled for open access or not. Colleagues in the library will check the data, enable open access, set embargoes or record an exception and will make sure that the item is discoverable and accessible in accordance with the REF rules.
Find out more about the policy, how to comply and acceptable exception on /staffnet/research/impact-ref-and-open-access.php
How will the University put together the REF 2021 submission?
The Vice Principal for Research, Professor Marion Campbell, is responsible for the institutional REF submission. She works with a small team within Research & Innovation who keep up to date on policy developments and support the systems that record and manage the data required for the submission. She is supported by the three Directors of Research who oversee the preparations for submissions to the main panels relevant to their own research areas: Professor Gary Macfarlane for Main Panel A (Life Sciences and Medicine), Professor Richard Neilson for Main Panel B (Physical Sciences and Engineering) and Professor Michael Brown for Main Panels C and D (Social Sciences , Arts and Humanities).
Heads of School and School Directors of Research have identified a lead researcher for each unit of assessment, and a researcher responsible for co-ordinating the impact element of the submission. Some larger units of assessment have a small team of researchers working together on REF preparations. If you want to know who the relevant leads of your unit of assessment are, please contact either your School Director of Research or firstname.lastname@example.org .
What do I have to do
We are asking all REF eligible researchers (under the 2014 rules until the 2021 guidance has been published) to propose outputs for submission in Pure. A short guide on how to do this is available here.
If your research has had, or has the potential to have, impact beyond academia, please discuss with your unit of assessment lead or impact lead in the first instance. The University supports the creation of impact of various kinds through
- Research & Innovation – commercialisation, licensing, patenting , enterprise, knowledge exchange, contact Dr Ann Lewendon (email@example.com; ext. 2477);
- Public Engagement with Research Unit, (firstname.lastname@example.org; ext. 3739); or
- Public Affairs Team - engagement with policy makers/influence on public policy contact Sue Bird (email@example.com, ext. 2088) or Godfrey Brown (firstname.lastname@example.org, ext. 3231)
Any further questions
Please contact Marlis Barraclough, Senior Policy Advisor
- REF 2014
What is the REF2014? The REF2014 is a sector-wide assessment exercise of publicly funded research in the UK. The REF2014 replaces the Research Assessment Exercise, which last took place in 2008 (RAE2008). All Universities that receive funding from any of the Higher Education Funding Councils are invited to present selected research for assessment in to the REF2014 exercise. The assessment is carried out by panels of peers and research users in 36 subject areas, referred to in the REF2014 as Units of Assessment. In total there are four Main Panels and 36 sub-Panels. The exercise is selective, with the expectation that institutions will select only excellent work for submission, and not all research. At the end of the assessment, each institution will receive a quality profile for each Unit of Assessment to which it submitted.
The University REF2014 Steering Group
The University’s REF2014 submission will be coordinated at both College level and centrally, and will be overseen by the institutional REF2014 Steering Group.
Download details of the institutional REF2014 Steering Group
Defining Research in a REF2014 Context
The definition of research within a REF2014 context, as given by HEFCE in the Assessment Framework and Guidance on Submissions document, is as follows:
For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.
It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.
It includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports
Why is the REF2014 Important?
The REF2014 is hugely important for a number of reasons. First, it allows for benchmarking standards of research against other institutions across the sector within the UK. This brings with it a reputational issue, and should be seen as an opportunity for research Units within the University to demonstrate their research excellence. The quality profile achieved by the University in the REF2014 will also inform the block research grant allocation it receives from the Scottish Funding Council (SFC) from 2015 onwards.
For any additional information on the REF2014, please visit the HEFCE REF2014 website.
REF 2014 Results
The results of the Research Excellence Framework will be published by the REF team on 18 December 2014. More information about the publication arrangements.
What will be Assessed?
The REF2014 will assess three key research areas, as follows:
- Research Outputs (65%)
- Research Impact (20%)
- Research Environment (15%)
The most important of these is the assessment of research outputs. Each academic member of staff who is selected for submission will normally have to provide four research outputs for inclusion. These will be assessed by the sub-Panels. Approximately half of the sub-Panels (mainly in the STEM subjects) will take into account the citation counts of the submitted research outputs to inform their decisions. Assessment of research outputs will account for 65% of each submission’s overall quality rating. The citation counts will be provided to the REF panels via Scopus, and will refer to the research outputs submitted.
After the research output element of the assessment exercise, the non-academic impact of research is the next most significant area to be assessed. This will also be assessed by Unit of Assessment, via submission of a pre-determined number of “impact case studies” (number of case studies varying dependant on FTE per Unit) and by assessing how the research environment and research strategy within each Unit supports and facilitates non-academic research impact. The assessment of research impact is new to the REF2014 and will account for 20% of the overall quality profile within each Unit.
The research output and impact elements of each submission will be accompanied by a narrative which describes the research environment of each Unit of Assessment. The “research environment” element of the assessment accounts for 15% of each submission. The research environment element assesses research facilities, research strategy, research metrics, career development opportunities, with particular emphasis on early career researchers and postgraduate students, and on equality and diversity issues.
For further information on how submissions will be assessed, please access;
- The REF2014 Assessment Framework and Guidance on Submissions Document
- The REF2014: Amendments to the Assessment Framework and Guidance on Submissions Document (January 2012)
- The Panel Criteria and Working Methods documentation (which also details variations between assessment criteria by Main Panel)
Alternatively, these documents are available on the HEFCE website.
What do the Quality Ratings Mean? For the overall quality profile for each Unit of Assessment the quality ratings are as follows:
Quality that is world leading in terms of originality, significance and rigour
Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence
Quality that is recognised internationally in terms of originality, significance and rigour
Quality that is recognised nationally in terms of originality, significance and rigour
Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment
For each of the three elements of the assessment – outputs, impact and environment – sub-panels will develop a sub-profile, showing the proportion of the submission that meets each of four starred quality levels.
- View the assessment criteria and the definitions of the starred levels for the sub-profiles
How will the University Select Staff for Submission?
Final decisions on the selection of staff for submission to the exercise will be made by the institutional REF2014 Steering Group, in consultation with the Colleges. The REF2014 Steering Group has overarching responsibility for managing and coordinating the University’s submission to the REF2014. Decisions on inclusion for submission will be based entirely on research quality.
In making decisions on inclusion, the University, via the REF2014 Steering Group, will adhere to its internal REF2014 Equality and Diversity Code of Practice, as required by the Funding Councils. The University's REF2014 Equality and Diversity Code of Practice was approved via the University Court at its meeting on 6 December 2011 and has now been submitted to the national REF2014 Team (HEFCE) for formal approval.
- Download a copy of the University's REF2014 Equality and Diversity Code of Practice, as submitted to the REF2014 Team.
Please find a number of documents key REF2014 below.
- Assessment Framework and Guidance on Submission
- Full Panel Criteria and Working Methods
- HEFCE Decisions on the Assessment of Impact
Impact Related Links:
In order to access the additional documentation on Impact listed below, as released by HEFCE following the Impact Pilot Exercise (2010);
- HEFCE Impact Pilot Exercise Expert Panel Findings.
- Feedback from the Higher Education Institutions Involved in the Pilot.
- Examples of impact case studies best practice, as submitted to the pilot exercise
Please visit the following link: http://www.hefce.ac.uk/rsrch/REFimpact/
- RAE 2008
The University's research excellence is reflected in the results of the Research Assessment Exercise (RAE) 2008.
- 89% of Aberdeen’s research activity is of international quality and 55% is world leading or internationally excellent. (4* and 3*)
- All of our 33 submissions have activity rated as world leading or internationally excellent
1st in the UK
- 1st for Theology, Divinity and Religious Studies
- 1st equal for Health Services Research
Almost three quarters of our staff were submitted to units where at least half of the research activity was rated as world leading or internationally excellent (70%+)
- Computer Science and Informatics (70%)
- Health Services Research (80%)
- Other Hospital Based Clinical Subjects (75%)
- Theology, Divinity and Religious Studies (80%)