The Joint Research Centre is engaged in a long-term project focused on how to support evidence use in policy making across the EU. They invited me to spend some time thinking about the role of the science system, and how we might know what a ‘good’ science system looks like. This is partly about mapping science systems and trying to understand what makes them work; and also knowing how we can tell if they are functioning, by using indicators.
There are two main problems with setting up an indicator dashboard to assess science systems. First some relatively straightforward technical issues: we often measure what is measurable, which isn’t necessarily the same as what is useful to know; and the potential for perverse incentives, or even just over-attention to what is measurable, is always a risk.
The second challenge is more existential. Does the science system have a goal? Visualising and mapping science systems perhaps inevitably gives the impression that the system was designed, can be controlled and adjusted, and that it operates towards a shared objective. None of this is true of evidence/policy systems, which tend to have evolved organically. And members of this system have their own agendas and interests - not all researchers produce policy-relevant research, for instance. So, no – in itself, the system does not have a clear goal.
However, the JRC project is about supporting evidence use in policy. And the literature on evidence-informed policymaking does have a relatively shared understanding of what this means: the delivery of relevant, robust evidence to people with the capacity to absorb and fully understand it in order that it might inform their decision-making. This is not a simple goal - big questions about what evidence is produced, what evidence counts for policy, how evidence and expertise reaches decision-makers, and how they can and do respond to this evidence all need to be considered; and all these questions have political, socio-technical and philosophical aspects to them.
It is possible, though, to think about which types of organisations and individuals are involved in evidence production, mobilisation and use in decision-making; and to ask about their connections and activities. In fact, we can start to think about the total set of actors and connections within the entire system through which scientific knowledge is acquired, synthesised, translated, presented for use, and applied in the policymaking process. This is my working definition of the science-for-policy system. In the table below, I set out my preliminary thoughts about what these different groups might do in service of evidence-informed policymaking, and how we might know they’re doing it (well). How these signs of activity might translate into measurable indicators, I am not yet sure.
Type of actor | Activities – what might they need to do? | Possible indicators |
Funders | - To foster a culture of professionalisation in evidence production and mobilisation, supporting and promoting skills and careers to engage effectively and ethically
- To engage with, respond to, and shape governmental knowledge needs and researcher-led priorities
- To support and use research into effective knowledge systems
| - Engage regularly and effectively with the research communities and with relevant policy teams to support policy-relevant research through shared problem-framing
- Offer appropriate training and capacity building to research organisations and teams, who include support staff
- Dedicated research and knowledge exchange funding for research into evidence production and use
- Balance between responsive and policy challenge-led research funding
|
Research organisations | - To support a diverse workforce equipped with skills and expertise relevant to policy challenges through relevant training, and capacity building opportunities
- To reward and enable research engagement with policy and practice organisations
| - Support diverse and skilled research workforce by offering appropriate degree and other training and capacity-building support
- Reward and recognise policy-relevant research and engagement activities to policy
- Supporting a diverse workforce with capabilities in knowledge exchange with policy
|
Researchers | - Produce interesting and novel research evidence relevant to policy challenges
- To seek to support effective knowledge systems
| - Appropriately skilled and diverse workforce
- Produce interesting and novel research, or relevant syntheses
- Routinely engage with evidence users as a legitimate part of their work
|
Intermediaries | - To support effective functioning of the system through providing convening and network opportunities
- Synthesise and mobilise evidence in range of ways
| - Have skilled workforce able to synthesise and mobilise evidence in range of ways, network, and support and advocate for evidence use
- Recognised and resourced specialist units or organisations dedicated to knowledge exchange
- Support shared problem-framing and deliberation between policymakers and stakeholders
|
Policymakers | - Internal capacity and capability to engage with shared problem framing, identify and seek out a range of diverse evidence types, and able to assess and comprehend evidence findings and implications
- To state clearly their knowledge needs
- Have clear and transparent evidence-informed mechanisms to seek diverse and appropriate evidence
| - Clearly stated knowledge needs
- Capabilities within government to assess and absorb evidence of different kinds
- Transparent mechanisms to solicit and engage with evidence and experts
- Internal reflection and scrutiny of advisory systems
|
Parliament, media and other scrutiny bodies | - To provide effective scrutiny of effective systems functioning, include gaps between knowledge needs and produced research evidence, workforce diversity, and assessment systems
- To enable government to identify what it does not yet know it needs to know
| - Existence of, and mandate for independent, evidence-capable body(ies) to scrutinise science-for-policy system and its elements, and hold to account
- Science and evidence capabilities for general scrutiny of evidence use in policy-making at national, regional and local levels
|
Although there are probably missing groups, activities, and indicators, mapping these out shows at least 3 interesting things:
- Whether the groups and organisations involved agree on the goal of evidence-informed policymaking, or want to act in its service, all groups currently contribute towards this goal
- Mapping out the connections in this way offers potential points of intervention which could help improve systems functioning
- But interventions which seek to improve evidence use usually target one element or relationship, rather than at a systems level (which tends to add to a chaotic mass of activity rather than improving systems functioning)
More thinking on these activities, goals, and indicators is needed, particularly around what we want our science-for-policy systems to do, for whom, and how.
If you see gaps in the groups, activities, and indicators, or have thoughts on what we want our science-for-policy systems to do, please get in touch.
