Measuring impact – what are your thoughts?
Recently we wrote about Altmetric, an organisation that looks beyond citations as a measure of academic research impact. Instead, it measures the shares, downloads and links that a particular paper receives online.
This week we’re going to look at proving research impact in a bit more depth. Here in the UK we’re all well aware of the Research Excellence Framework (REF) and the fast-approaching November submission deadline, but will the ‘Impact Statements’ required by the REF look the same next time around, in 2020?
REF came about because the British research councils want to encourage researchers get more involved in thinking about how their work will have real world impact, how they can explore pathways for realising that impact, and therefore achieve research excellence.
However, this initial round of REF assessments has been criticised on two accounts. The first is that ‘impact’ has been deemed to mean impact outside the institution in which the research is undertaken, with some arguing that this undermines academic freedom. Secondly is the debate around what metrics can be used to measure impact that would be regarded as fair and impartial.
We have been looking into the different ways of measuring impact, and we’re grateful to Dr John Lamp, a senior lecturer at Deakin University, for drawing our attention to this paper from the Australian government (PDF). The paper uses a method which splits impact into ‘internal’ and ‘external’, somewhat addressing some of the concerns with the REF in Britain.
The paper lists citations as an internal measure of impact, and as Professor Lamp highlighted to us, it also proposes three external impact measures:
scientific impact – effect on other professions;
professional impact – effect on non-academic professionals;
social impact – broader economic, social and environmental benefits
Clearly measuring these external impacts is never going to be straightforward or easy. The paper states that the assessment of research benefits must ‘verify claims that have been made for how a given research activity has contributed to national well-being, productivity growth and/or the solution to national and global challenges’.
The Australian government is still going through consultations around this workstream, and is looking at the metrics used to measure research engagement and benefits. Here in the UK we’re about to go through our first REF assessment.
Countries all around the world are all going through this process. They’re consulting with academics, reaching their own conclusions and introducing new systems.
We’d love to know some of the ideas you have or systems for measuring impact that you have experienced. What has worked, what hasn’t, and has anything surprised you? As always – do comment below and share your ideas or experiences, or tweet us at @citeab.
– The CiteAb Team
