The content of each evaluation will be different but the basic checklist below describes what needs to be covered in most cases. The evaluation report needs to provide a detailed picture of the whole engagement process and the policy process within which it fits (if it is related to a specific policy process), as well as to assess its success.
Objectives of the engagement process, e.g.
- what were the original stated objectives
- are there any ‘implicit’ objectives that have not been fully articulated
- how were the objectives set, and by whom
- did they change; if so, why and how
- have the objectives been met.
Context, e.g.
- is the process stand-alone or part of a wider programme
- what else relevant was going on at the same time; how did the process under evaluation relate to other relevant initiatives
- what historical, geographical, political, economic and social factors have affected the process.
Levels of involvement, e.g.
- type of involvement sought (e.g. from inform to empower); why and how that level was chosen
- assessment of whether that level of involvement was achieved
- assessment of whether that level of involvement was appropriate in the circumstances.
Methods and techniques used, e.g.
- what methods and techniques were used
- who decided on these
- were they appropriate to the objectives
- what worked well, and what worked less well.
Who was involved, e.g.
- review of any stakeholder analysis done (whose involvement sought; assessment of whether achieved; and whether appropriate)
- numbers of people involved
- analysis of type of people involved (e.g. by socio-economic group, educational qualifications, age).
Inputs (costs), e.g.
- monetary costs (e.g. staff time, expenses, event costs, publicity)
- non-monetary costs (e.g. time contributed by participants, unpaid staff time, training time)
- risks (e.g. to reputation, uncertainty, stress, conflict, loss of control)
Outputs (products and activities), e.g.
- participatory events such as workshops (e.g. numbers attending, feedback)
- information events such as exhibitions (e.g. numbers attending, feedback)
- questionnaires (e.g. numbers and results)
- newsletters and other printed materials (e.g. numbers circulated, feedback)
- interviews undertaken (and results).
Outcomes (benefits / impacts), e.g.
- changes in policy (e.g. different ideas incorporated)
- changes in people (e.g. new skills, greater confidence, increased networks, greater willingness to participate in future)
- changes to organisations (e.g. changed structures, different priorities)
- wider social changes, such as
- new groups or organisations set up
- greater public support for programme
- better public services (e.g. because needs met more effectively)
- greater social cohesion (e.g. because people get to know and trust each other)
- better governance (e.g. greater accountability of government, better information flow, more engagement)
- continued learning (e.g. learning from the process, people go on to do other qualifications)
As well as going through the full checklist of potential issues to cover, you may want to identify some more general points, such as:
- what are the main lessons learnt from the whole thing, and why?
- what should you never do again, and why?
- what was the best / most successful aspect of the whole thing, and why?
- what is the most significant change / biggest impact that the process has had, and why?
When planning the data collection, it works best to explicitly ask these types of general questions rather than infer findings on these issues from more specific data.