Previously, we discussed why evaluation is important for your nonprofit. In this article, we talk about the need for both process and outcome evaluation and the pitfalls of over-emphasizing outcomes over process.
There are many types and ways of evaluation: needs assessments, monitoring activities, formative, participatory and summative evaluations – all of which are explained in TSNE’s Executive Directors Guide.
The type of evaluation most commonly requested by foundations is called outcome evaluation. Outcome evaluations assess the effectiveness of a program in producing change. Outcome evaluations focus on difficult questions that ask what happened to program participants and how much of a difference the program made for them.
- Outcome evaluations assess the effectiveness of a program in producing change.
- Process evaluations help stakeholders see how a program outcome or impact was achieved.
Impact or outcome evaluations are undertaken when it is important to know whether and how well the objectives of a project or program were met.
For example, outcome questions for a smoking cessation program might include:
- Did the program succeed in helping people to stop smoking?
- Was the program more successful with certain groups of people than with others?
- What aspects of the program did participants find gave the greatest benefit?
I have always felt that over-emphasizing outcome evaluation at the cost of other types, especially process evaluation, is a disservice to nonprofits and the sector. Why? Because process evaluation allows a nonprofit to look at how it develops itself, its structures, its supporting programs like communications and marketing, even fund development in order to get to the outcomes everyone wants it to achieve.
In other words, process evaluations document the process of a program's implementation. Process evaluations help stakeholders see how a program outcome or impact was achieved. The focus of a process evaluation is on the types and quantities of services delivered, the beneficiaries of those services, the resources used to deliver the services, the practical problems encountered, and the ways such problems were resolved.
Taking process evaluation a step further, it can also look at the the processes of program, management and infrastructure together that is the capacity of an organization to deliver on its promised outcomes.
For example, process evaluation questions might include:
- What specific interventions were put into place by the program in order to fight the problem being tackled? Did the interventions work or not – and how and why?
- What were the kinds of problems encountered in delivering the program – were there enough resources from the beginning to do it well? Was it well managed? Were staff trained or educated to the right level of the program design? Is there skill at facilitating the program processes from beginning to end? Was there adequate support to the program?
Why Process Evaluation Is Important
Information from process evaluations is useful for understanding how program impact and outcome were achieved and for program replication. Looking at outcomes – without analyzing how they were achieved – fails to account for the human capital (over-worked staff) involved in getting to good outcomes and the true costs of the program.
Evaluating the “input” (the very first column in a logic model) is just as valid as evaluating the last columns about outcomes. It is called a “logic” model after all – and logically there is a chain of cause and effect which means, if we have the right resources at the very beginning of the chain (inputs) than we assume we will be able to get to the outcomes to which we aspire.
Get the Whole Picture
If process evaluation was used in combination with financial analysis much clearer pictures of the cost of doing business would emerge – as would the case for more general operating and infrastructure support.
Interested in how to integrate evaluation into your organization? Download Evaluation for Executive Directors.