The importance of articulating assumptions

Evaluation PracticeI do some evaluationEvaluation is my main job
Written By Stephanie Jones

This article is rated as:

 

If you’ve been delving into the world of evaluation and Theory of Change (ToC), you are likely to have come across the concept of “assumptions.” Assumptions have a reputation for causing confusion in evaluation. Articulating assumptions can be a tricky task. 

Documenting assumptions about why a program will contribute to change is very important in order to evaluate the success of a program and its outcomes. In this article, we: 

  • Describe what assumptions are in evaluation; 

  • Explain why you should document assumptions; and, 

  • Briefly explain how to reflect on your assumptions when collecting and analyzing evaluation results. 

We also provide some practical examples of how to include assumptions in your own evaluations. 

What are assumptions?

Assumptions are ideas about how a program is expected to influence or contribute to change. These beliefs focus on how change is expected to happen in the short, medium, or long term of a program and its context (i.e., the conditions of the system in which the program is operating). Assumptions are often informed by worldviews and a stakeholders’ prior experience. They can also sometimes be situated in scientific evidence.  

An example of an assumption for a Health Promotion Program is: 

  • Outcome: Health Promotion Program participants lead healthier lives.

  • Assumption: Once people have the information, they will make better choices.

This assumes that participants currently do not have the information needed to make better choices about their health. By creating and sharing this information with participants, the Health Promotion Program is expected to contribute to change. 

For a deeper dive into the many different types of assumptions (e.g., contextual, normative, diagnostic, prescriptive, etc.), check out Apollo M. Nkwake’s book which includes a detailed discussion on working with assumptions in international development program evaluation. 

Assumptions lie outside the direct control of the program. Assumptions are also outside the program’s interventions and activities, but they are critical to the program’s success. 

Why should I document assumptions when evaluating or discussing a program’s logic?

There are a few reasons why an evaluation should clearly document assumptions: 

  1. Documenting assumptions will pinpoint external factors that can affect how a program contributes to change. 

  2. By making assumptions clear from the start of an evaluation they become hypotheses which can be tested, challenged, and refined by different stakeholder perspectives. Doing so can support learning on how change is expected to happen. 

  3. Documenting assumptions can also inform future activities, collaborations, and a program’s strategic decisions through the increased awareness of how change is expected to happen. 

  4. Documenting assumptions provides clarity on whether change will be realized in the future, therefore helping to focus activities and identify opportunities. 

  5. Documenting assumptions are therefore critical in helping to articulate, validate, and assess the strength of a program’s theory. 

  6. Documenting assumptions can support explanations if things do not go as planned, for better or worse.  

How and when should I document assumptions?

Documenting and testing assumptions is central to developing a ToC or a Log Frame. See our previous ToC tip sheet and ToC template. When creating an evaluation plan and formulating evaluation questions, assumptions can be identified about the change the program is expected to contribute to. The perfect time to start documenting assumptions is at the program planning stages (e.g., a ToC development workshop). If the program is already underway and you’re starting evaluation partway through, make sure that your evaluation kick-off meeting includes the identification of program assumptions.

Here is our 5-step process for identifying assumptions in your next meeting: 

  1. Have each stakeholder/member write and share their assumptions about the expected change. 

  2. You can use facilitating questions such as “Why is this change expected to happen?” and “What needs to happen for this change to be realized?” 

  3. Discuss each stakeholder’s assumptions as a group; refine and aggregate each of the assumptions as needed. This participatory process of documenting assumptions supports clarity among stakeholders about how a program is expected to contribute to change. It also allows stakeholders to identify factors that are critical to achieving desired results. 

  4. Identifying assumptions in this participatory way can be challenging. Thinking about change in relation to underlying assumptions may be a new perspective for stakeholders. Therefore, evaluators should recognize these difficulties and be prepared to ask the right questions to support stakeholders in articulating their assumptions. 

  5. Assumptions can be actor-specific to ensure they are grounded. For instance, in the example above, we focus on Health Promotion Program participants as the actor group. This helps us when thinking about who is expected to be doing something differently because of the intervention. 

Documenting assumptions and justifying change is a continuous process. It should not be forgotten about following the initial kick-off meetings! Assumptions should be re-visited and re-questioned throughout the evaluation. 

Considering assumptions when analyzing results

It is important that your assumptions are also considered in the data collection processes and when analyzing your results. Collecting evidence on your assumptions can highlight common pitfalls about change underlying a program in contrast to the realities of how change happened. This provides the opportunity for a great learning process! 

Evidence for assumptions can be collected through methods such as interviews, focus groups, and/or surveys. Targeted questions for each of the assumptions can be added to any of these methods that are already planned for your evaluation. 

For the example above, a targeted interview question to gather data on the assumption could be: “Why have you started to make better choices about your health?” 

This will uncover some insight into the causal processes that contributed to participants making better choices about their health. It will also help you to understand whether the information provided by the Health Program played a role in this. 

For those wishing to dive a little deeper into the world of assumptions, check out the book Working with assumptions in International Development Program Evaluation by Apollo M. Nkwake.

Have you worked with assumptions before, or have questions? Comment on this article or connect with us on LinkedIn or Twitter! 

Sources: 

 
Stephanie Jones
Previous

How to use Calendly to schedule interviews like a pro

Next

Interpreting themes from qualitative data: thematic analysis