intelligent measurement

Impact – how feasible for evaluation?

October 9, 2007 at 9:28 am 1 comment

As I mentioned in an earlier post, people often confuse “impact” with “results”. Is it possible to measure “long term impact” of projects? It is, however for most projects it is unrealistic to do so for two reasons: time and cost.

To evaluate impact, you would usually need to wait some 12 months after the major elements of a project have been implemented. Many organisations cannot simply wait that long. In term of costs, an impact study requires a triangulation methodology that uses various quantitative and qualitative research methods which could be costly. However, if time and cost are not issues, an impact evaluation is possible, keeping in mind the following points:

Was the impact desired defined at the beginning of the project?

For example, greater organisation efficiency; change in the way a target audience and/or an organisation behaves; or improvements in how services for a given audience are managed?

What have been the other elements influencing the impact you want to measure?

Your project cannot be viewed in isolation; there must have been other factors influencing the changes being observed. Identifying these factors will help you to assess the level of influence of your project compared to other factors.

Do you have a mandate to measure impact?

When assessing impact, you will be looking at long term effects that probably go outside of your own responsibilities and into the realms of other projects and units – you are looking at an area of the wider effects of your organisation’s activities and this needs to be taken into consideration. For example, if you are looking at the longer term effects of a training program, you would want to look at how individuals and the organisation as a whole are more efficent as a result of the training. Do you have the political mandate to do so? – As you may discover effects that go way beyond your own responsibilities.

Evaluating impact is a daunting but not impossible task. For most projects, it would be more realistic to focus on measuring outputs and preferably outcomes – and think of short term outcomes as I have written about previously.

Glenn

Entry filed under: Communication evaluation, Evaluation methodology, PR evaluation, Training evaluation.