Written by:
Jenny Bird
Have you ever said “let’s write a policy brief” without first stopping to consider what you hope it will achieve? I am loathed to admit it, but I know deep down that I have. The good news for me is that it seems I am not alone. The Transforming Evidence team recently studied 428 organisations active in the area of promoting research-policy engagement activities and concluded that ‘the picture is of a vast and increasing mass of rudderless activity, which is busy rather than effective’ in which ‘many providers may have an inappropriate or poorly-articulated goal’.
Goal-focused engagement
Since our inception three years ago, UCL Engineering’s Policy Impact Unit has tried to adopt a goal-focused approach to policy engagement. After all, it seems plausible that setting a goal before embarking on any policy engagement activity will help to sharpen its design and therefore increase the chances of achieving impact. We quickly discovered, however, that unlike advice on ‘how to write a policy briefing’ (of which there is a huge amount), guidance on ‘how to set an engagement goal’ is very thin on the ground.
REF
We started by looking at the examples of ‘impacts on public policy’ that are provided in the REF 2021 Panel criteria and working methods to see if we could adopt these as goals. The list contains quite a number of useful illustrative examples, but they are not provided in a systematic way, which makes it harder to use as a tool for identifying suitable goals. In addition, it does not make any distinction between impacts that might be reasonably achievable with fewer resources or limited time such as ‘research helps parliamentarians and staff to identify inquiry topics, shape the focus on inquiries, inform questioning of witnesses, and underpin recommendations’ and those that would require significant work over a long timescale: ‘risks to the security of nation states have been reduced’.
Third-sector thinking
We also looked at resources available in the third sector. We found a helpful framework in some guidance on monitoring and evaluating advocacy produced by Save the Children and the Open University. The framework usefully distinguishes between intermediate outcomes (such as coalition building and shaping the policy agenda) and long-term outcomes (such as changes to policy and legislation). In our view, this is a helpful perspective as firstly, it recognizes that policy change often takes a long time and that the road to impact can be long. Second, it makes it easier to identify what the individual milestones along the way might be. This in turn can be helpful in putting together a strategy for delivering long-term goals by breaking it down in to a series of smaller, intermediate goals.
We also found the policy funnel (described by the think tank E3G) and analysis by the Institute for Government to be helpful in pinpointing the different types of impact goals that might be adopted at different stages of the policy process. (I have written more about this in a blog for UPEN.)
Building a framework
By combining these approaches and incorporating some ideas based on our own experience, we have produced our own framework. We are sharing it here in the hope that we might start a conversation between research-policy engagement practitioners on how to approach goal setting.
Using the framework
Our framework is intended to provide a prompt to stimulate ideas for possible engagement goals. It is not intended to be prescriptive, nor is it comprehensive; we treat this as a living document that we can add to over time based on our experience or input from others with expertise in this area.
The framework provides examples of goals at different levels: immediate outcomes, intermediate outcomes and long-term impacts. Ideally, an engagement strategy should plot a pathway to show how desired outcomes at each level can build towards the longer-term impact goals. However, there might be times when it is not reasonable to aim for this level of impact (for example because of resource constraint or because the policy context is changing very quickly). In such cases, goals can be selected from the immediate outcome or intermediate outcome sections.
We have also put together a ‘toolkit’ of the types of activities (and associated outputs) that we might use to achieve these goals. It is important not to confuse these with impacts and so we have kept the two lists separate.
Our framework is very much a work in progress and we would welcome any feedback on it. We are also keen to hear how other practitioners are approaching the question of setting goals for research-policy engagement work. We hope that this blog can be the beginning of a conversation and invite you to share your thoughts in the comments below.