It is often thought that evaluation comes at the end, to see what has been achieved. Evaluation at the end will be essential so that the complete process can be assessed. However, it is too late to start thinking about evaluation at the end; the evaluation process needs to start much earlier.
It is essential to evaluate early. Many of the important outcomes sought by participation are intangible (e.g. improved relationships, changed perspectives) and the evidence on which one can measure success is often highly contextual and subjective. Data therefore needs to be collected at the time of the engagement, as well as after the engagement has concluded and the impacts are known, to achieve sufficient richness to be meaningful.
A simple plan for designing a public engagement process might involve the following steps:
Step 1 – Set up a planning / design group to scope the engagement. It helps to have a small group that is concentrating on the engagement exercise specifically, although it will need to link in closely with the overall policy development planning groups. This group will agree the objectives of the exercise, the methods, the scale and scope (what can and what cannot be considered as part of the exercise) etc.
Step 2 – Agree detailed project plan. This will need to cover the scale, timescales, key dates and actions, resources available and needed, location(s), communications (see Step 4 below).
Step 3 – Implementation. From booking venues and preparing briefing materials, if appropriate, to inviting participants, planning catering, recording discussions and decisions, reporting back to the participants what they agreed / said and what was done with those agreements / comments.
Step 4 – Communications. Essential for making participants feel they are taking part in something important (where media coverage can help), to ensuring that the wider community (interest group or geographical or the whole country) knows how they can get involved if they want to (again, if appropriate).
Step 5 – Using the results of the engagement exercise. Feeding the outputs of the engagement exercise into the rest of the policy-making process. How this is done will depend on the particular characteristics of the policy and process in question.
Step 6 – Feedback. If the participants were not directly involved in the decision-making, they should be told what has happened to their input, what difference it has made and, if specific points made in the engagement are not being taken forward, why not. Wider communications may also be appropriate at this stage.
The plan for the evaluation should come in at the very earliest project planning stage – Step 1.
Evaluation can help set objectives that can be measured – so that those running the exercise can see whether the whole process has been a success and in what ways. The evaluation then runs throughout the exercise, collecting data and reviewing progress as the project continues. The relationship between the evaluation process and the overall engagement planning process is shown in the following diagram.
