Introduction

The evaluation of research quality is a feature of most advanced higher education systems and is valued as a driver of international comparison and league tables. Research is central to notions of prestige in academic life and in the competition for students, income and reputation (Blakemore 2016).

But how do we judge research’s quality? Evaluation methods vary internationally and there is an ongoing debate in various countries about how to improve the current systems to make them work for everyone. For example, Australia is reviewing whether to design a new comprehensive system from scratch (Australian Research Council (ARC) 2022); meanwhile, the Nordic countries have a more selective approach, with the Swedish Research Council assessing disciplines as required rather than as part of an all-encompassing exercise (Nature 2023).

The San Francisco Declaration of Research Assessment (DORA 2012) is an influential global initiative to which more than 2,800 institutions in 160 countries are signatories. Its aim is to advance practical and robust approaches for assessing research worldwide and across all scholarly disciplines. Its underlying principle is that evaluation should not rely on journal impact factors as a measure of research quality but should call for broader representation of researchers that can address structural inequalities in higher education.

In the UK, there is also increasing recognition that successful research requires a culture that provides equality of opportunity, thereby enabling researchers to thrive and do their best work. Research England has incorporated this principle into the outline design for the next Research Excellence Framework (REF) (REF 2023). The REF applies to research in all four nations and the next version will supersede both the original and updated REFs (REF 2014, 2021). Please note that all these documents discuss ‘REF 2028’, but the next REF has now been deferred to 2029.

REF 2014 and REF 2021 had three main elements: ‘environment’, ‘outputs’ and ‘impact’. These have been renamed in REF 2029, with their content adjusted and their weightings rebalanced (Box 1).

Research England is currently responding to concerns raised during the sector-wide consultation on the first element. It has begun work to coproduce indicators through extensive engagement with the research community. Given the decision to extend the time frame of the REF to 2029, there is an opportunity for more in-depth feedback and pilot testing of indicators. This work is ongoing so what works for different disciplines is currently unknown.

But even though it has not decided the precise indicators yet, it has set out the direction of travel. It is therefore possible to plan now for the cultural and organisational changes REF 2029 will require.

This article focuses on how universities, particularly those offering nursing programmes, can prepare for the new framework. It does not discuss the assessment of research outputs or impact, but focuses on equality, diversity and inclusion (EDI) as the bedrock of an optimum research culture.

Background

I write this opinion piece having spent a long career in higher education starting off in a joint appointment doing my PhD while practising half-time as a district nurse. I then built my career, alongside having a family, through postdoctoral work in a medical school, as a research leader at King’s College London and finally as dean of a large multidisciplinary healthcare faculty at Kingston University and St George’s, University of London, where I was able to influence career opportunities for others.

When I started in the 1970s and in the early days of building nursing as an academic discipline, there was little in the way of structural support, systems of development or funding schemes available for nurses; even finding mentors was a matter of luck. Despite that, I was fortunate to be encouraged by extraordinarily generous leaders and role models, many of whom were doctors. I have been able to draw on this experience to contribute to institutional change. More recently, I was a member of the REF Equality and Diversity Advisory Panel (EDAP), and this article reflects on some of that learning.

I acknowledge the difficulties around the language of race, which is contested and evolving. Many documents in UK higher education policy use abbreviations such as black and minority ethnic (BME) or black, Asian and minority ethnic (BAME).

In this article I will use the language in the policy documents or research I cite; where I make general points, I will use the term ‘race’ or ‘global majority’. The term ‘global majority’ is used to refer to all ethnic groups except white British and other white groups, including white minorities. This includes people from black, Asian, mixed, and other ethnic groups who are often racialised as ‘ethnic minorities’.

Equalities in context

The UK Research Councils are unequivocal about how important EDI is in fostering excellence in research and innovation (UKRI EDI Strategy 2023a, 2023b). They have set out four principles for growth and change that are fundamental for a high-quality research system ‘by everyone for everyone’: diversity, connectivity, resilience and engagement. However, while there has been change in some cases, the reality often falls short of this ambition. If we take funding for fellowships as an example, the gender gap has narrowed in recent years, with half of fellowships now awarded to women; but there remain stark and stubborn disparities for people from BME backgrounds and people with a disability (National Institute for Health and Care Research 2022). Researchers from global majorities in particular face specific barriers in their career journeys in doctoral training, postdoctoral research, leadership development and other areas.

Implicit in REF 2029 is the idea that a strong, credible, publication record showing promise of international reach is essential to successful promotion in research careers. This is best nurtured in an inclusive culture and job security, where you can bring your whole self to work. More enlightened nursing research environments also recognise professional leadership and roles that influence national and international change for patients’ benefit and greater well-being.

However, there are serious questions if we are to crack the concrete ceilings for all – specifically, how can we help researchers with ‘protected characteristics’ (Equality Act 2010) to be their most productive? Do culture and environment open or close barriers to them? What can we do about those barriers?

REF 2021 found there had been improvement across the board in the equality of research environments and cultures. This was especially true for women, probably because of the wide implementation of the Athena SWAN charter (Xiao et al 2020). However, there was also evidence the system perpetuated disadvantage, specifically in relation to race (Khan et al 2022).

Universities are legally accountable for widening access for and supporting the success of under-represented groups. Higher education’s efforts to improve access and widen participation have successfully focused on diversifying the undergraduate student population in areas such as healthcare (Health Education England 2022). However, this has not translated uniformly into corresponding improvements in outcomes.

This is exemplified by the persistent, sector-wide degree awards gap, whereby BAME students are less successful than their white counterparts at attaining first class or upper second-class degrees; this is true even when entry qualifications, field of study and type of institution have been factored in (Ross et al 2018, Office for Students 2021). This affects entry into postgraduate, doctoral and postdoctoral training, which depends on the student obtaining a good first degree. Other social, economic and systemic factors affect BAME students’ choices as well, such as the financial pressures associated with the costly and lengthy training required by postgraduate and doctoral work, the competitive environment for funded fellowships, and the precarity of postdoctoral work (London Higher 2022).

With race such a barrier to starting a career in research, it is unsurprising there are so few people of black heritage currently working in the higher education sector who are research leaders: out of the 22,855 professors working in higher education in 2022, 28% were women and 7% were of Asian heritage while only 1% were black (Higher Education Statistics Agency (HESA) 2022).

Who judges the quality of research in the REF?

The REF requires quality of research to be assessed by peer review, with assessors nominated by external professional or academic bodies. ‘Unit of Assessment 3’ assesses all aspects of the allied health professions, dentistry, nursing, midwifery and pharmacy. The relevant bodies in nursing may include the RCN, the Royal College of Midwives, the Institute of Health Visiting and the Council of Deans of Health, as well as stakeholder groups and public representatives. Bodies and organisations nominating individuals to panels are asked to take equality and diversity into account.

For REF 2021, EDAP reviewed the characteristics and representativeness of panels at two time points (REF EDAP Final Report 2022). The panel concluded that while nominating bodies took sexual equality seriously, they often overlooked disability, sexual orientation, race and other protected characteristics. A slightly greater proportion of panel members were from BAME backgrounds compared to REF 2014, but this was still lower than the proportion in the permanent academic population at the time.

This analysis matters because REF depends on the principle of peer review. The individuals appointed to panels should therefore be expected to represent multiple perspectives, views, life experiences and research methods. But if panels disproportionately represent one segment of the academic population or one research paradigm, bias may creep into decision-making despite efforts to mitigate it.

REF (2022) recognised the importance of expanding opportunities for under-represented groups who face barriers to progression and have not been in the right place at the right time to develop the ‘usual markers of seniority’ or esteem to be nominated. It suggested that funding bodies consider nominating additional observers to sit on panels to gain insight and experience into research assessment (REF 2022). However, REF (2022) only identified broad themes across all the panels. Therefore, extrapolation to nursing should be circumspect.

Uncoupling individuals from the team

UK Research and Innovation (2023a, 2023b) recognised that high-quality researchers are supported by sustainable teams, groups working together and nurturing workplace cultures. If their proposals are implemented undiluted, staff will be submitted if their employment contracts say they have ‘a significant responsibility for research’. These will be drawn directly from the HESA staff records, which will reduce universities’ administrative burdens and the potential for gaming. Gaming is the possible bias that creeps into the process of selectivity of staff deemed ‘REF submissible’. In other words, this process can lack transparency in the internal assessment of high-quality outputs.

These proposals are radical in that they shift from the old mindset of assessing individual outputs to focus on the wider discipline. If implemented, they will result in the complete decoupling of researchers from research outputs. The assessment of quality will then migrate in focus from the individual and the connotations of assessing performance to the impact of the discipline, group and the unit on advancing knowledge.

Some disciplines may challenge these changes. Philosophy and other traditions have an established and respected tradition of individuals doing great work alone to produce research books and papers. However, it is still likely that these lone researchers will have honed, expanded and developed their ideas through conversations and engagement with colleagues and students, as well as doctoral students in tutorials, lectures and seminars.

Most knowledge in nursing and professions allied to medicine is produced through collaboration in teams and through research programmes. Indeed, research activity is often organised into themes and programmes that may have cross-cutting relationships with sociology, policy, medicine and other disciplines.

In my own research, the questions I asked were mostly multifaceted and required interdisciplinary and collaborative groups to answer. Therefore, it was central and non-negotiable to build collaborations across academic disciplines, alongside service users and the wider public.

Bringing the outside inside

Collegiality, collaboration and inclusion can strengthen creativity and positive outcomes: Farrell (2001) studied group dynamics in six collaborative circles ranging from the French impressionists to Sigmund Freud’s psychoanalysts. The author’s findings led him to argue that ideas and creativity in contested fields come through friendship and colleagueship, which sustain motivation and excellence.

Collaboration is often a tool for success in healthcare. For example, members of a faculty trained in different fields may practise clinically outside their university then work together to integrate knowledge, skills and perspectives into collaborative research (Hager et al 2016).

My experience of participating in developing and implementing university research strategy is that it is hard for university submissions to recognise and include clinical experts or public representatives, even if they had been active partners involved in winning grants and writing for publications as co-authors. Indeed, there have been disincentives to recognise such research partners, as it came with few rewards and required organising part-time employment contracts for them, which may compromise service users’ independence as ‘critical friends’ in the research process.

Partnerships with the NHS, charities, social care and other organisations are central to nursing research, and the changes proposed for REF 2029 will make it easier to embrace this ‘outside’ in the REF submission – institutions will be permitted to submit outputs when there is a demonstrable and substantive link to them within the REF assessment period.

To prepare for this, it will be useful for you to audit current partnerships to determine those where clinical experts or public representatives are part of the research activity. You can then offer them employment contracts lasting at least six months and working two days a week. This will facilitate their entry in your submission and thereby strengthen it as well as promote voice, belonging and inclusion in your research endeavours.

Using data for improvement

Accurate, reliable data is a powerful way of driving improvement. The new focus on research culture in REF 2029 means it is important to identify appropriate benchmarks and baseline measures so we can shine a spotlight on groups of people who may face barriers to progression. Suitable data may include pay, progression, retention, funding awards and workload distribution in research groups.

Equality impact analyses are a useful way to document progress from baseline measures. For example, we can measure the time it has taken people to be promoted to chair and record the stages in their journey, to establish whether there are discrepancies for gender, sexual identities, race and people with disabilities. We can also record the interruption to research productivity caused by parental or carer leave, or absence due to long-term illness and the impact of this on research outputs.

Here I will give a personal example to illustrate my point. When I came back to work after my third baby and joined a research meeting with two senior (male) professors from the medical school to discuss the progress of a grant on which I was a co-applicant, I was told: ‘You have had a baby; we haven’t got time to fill you in.’

This was a long time ago and I was ill-prepared and unsupported to deal with this micro-aggression, which was reinforced by inequalities of gender, seniority and positional power. I believe we are doing better now – but that we can do more.

For example, we could evidence inequalities and require institutions to acknowledge and record the impact of parental leave or extended sick leave on outputs. We could then expect them to act by staging returns, acknowledging possible modifications to workload allocation and offering catch-up writing time, for example.

I want to see a future that facilitates reasonableness and fairness, where researchers do not have to ‘steal’ writing time from their families and significant others by working evenings, weekends and even holidays.

Finally, and most importantly, institutions need to review and expand capacity-building and accelerate leadership development for research staff at all levels. This should not be targeted just at staff on research contracts but should be available to everyone who expresses through their performance development reviews (PDRs) an ambition to conduct research to inform their teaching and practice.

We should not forget the line managers who are crucial to making this work. Many may not be research-active themselves but should be motivated to support their institutions’ research strategies to enable research cultures and first-class outputs that will benefit the whole.

For example, managers could argue for resources to support:

Personal reflections from REF 2021 to inform the future

The REF system is an enormously powerful lever for change: universities will comply and dance to its tune because success under REF will boost their league table positions and help them in their mutual competition for students, funding, reputation and prestige. Therefore, the higher education sector will have a real incentive under REF 2029 to create more equal environments and dent disadvantage.

Looking back, nursing as a discipline should be proud of how much it has reduced disadvantage in research careers. This has not been easy, because nursing is a practice-based profession and so has had to fight its corner and garner respect from established academic elites.

I remember a vice chancellor once asked me if nurses did research and assumed I must be a medical doctor as I had a doctorate. I hope we have moved a long way from that kind of view.

However, we know there is a lot of work to do, especially around the iceberg of disadvantage for people from global majorities. This is apparent in not just research careers but in the wider healthcare system (Kline and Warmington 2024) and discrimination in appointments to first jobs (Harris et al 2013, Hammond et al 2017, 2022) as well as in higher education and the disadvantage in awards gaps.

Conclusion

The reasons for the inequalities discussed in this article are complex, multifaceted and long-standing. They are influenced by an interplay between societal, systemic, institutional, relational and personal factors. Much research tends to problematise inequalities at the level of the individual rather than external conditions. It is therefore perhaps unsurprising that improvement strategies tend to target individuals’ deficits rather than institutions.

This article has tried to suggest wider and institutional change. However, many will say that changing cultures to create more inclusive research environments is slow and needs seismic change; others will argue that efforts to promote inclusivity are misguided, the icing on the cake or even ‘woke’.

Rhetoric about diversity is not enough. We must take positive action (Box 2), increase awareness of the scale of the problem in nursing and healthcare research, assess the starting point, and get support and funding to reach out to the talent already in organisations.