Chapter 4 The why question as a method

A method: formulating a single question about what influences what, and posing it to many sources or respondents to elicit many Theories of Change, or fragments of theories; ideally the data can then be analysed in Theorymaker, a tool for comparing and combining those fragments.

4.1 Steps to asking “the why question”

4.1.1 Formulate …

… a single question about what causally influences what. There are various different scenarios.

  • QuIP-like: “What factors influence this important thing, and what influences those factors?”
  • Appreciative: “Tell us a story about the most important positive effect the project had (what were the enabling factors, what were the consequences)”
  • Impact: “Tell us a story about most important positive or negative effects the project had (what were the enabling factors, what were the consequences)”
  • Research: “What do you think are the most important causes and consequences e.g. of this thing [e.g. climate change]”
  • Goalfree (Sensemaker-like) “What is the most important issue right now, and what are its causes and consequences”

In each case, you can tweak in various ways

  • emphasise causes and/or consequences,
  • specifically ask about causes of causes and/or consequences of consequences (i.e. more than one link away from the main focus – though people often mention these things anyway).
  • specifically ask people for how beneficial (or detrimental) the variables are. For example asking at the end, “… and which of these things are most important to you?”
  • ask people for another example when they have finished the first (and even then ask for a third or fourth example)
  • code only for structure (“X influences Y”) and parameters (“X is a strong but negative influence on Y”) of the causal network, or you can also code information about actual levels of variables (“right now, X is very high”) and even contrasting or counterfactual levels of variables (“X would be low if Z was happening”).

4.1.2 Pose the question

… to several sources or respondents. You can always add, for example, official documents or even your own considered opinion to the list of evidence and perhaps assign them a high level of trust. You’ll usually ask a couple of additional questions about each source like gender, age, status.

4.1.3 Get the answers into a spreadsheet

… with a column with the replies and a columns for the additional questions. (If you have allowed people to go back and provide another example, it would be good to have an additional column to record the ID of the source)

4.1.4 Start analysing

e.g. copy and paste these answers into the first tab of the app. Theorymaker imports them and breaks each answer into individual sentences.

4.2 When would you not want to use this approach?

  • If you can guess the main answers - which / how many sources mention which things, you might be better off just formulating those issues as closed questions
  • If this is a theory or theory of change which has already been discussed a lot, so there is already pretty much a consensus

So this works best with causal structures which we don’t yet know too much about. This probably comes under the buzzword “complexity-aware”.

4.3 “Changes”

I don’t think the right way to elicit information about someone’s causal map of a particular domain (e.g. in order to find out how a project influenced that domain) is to ask about changes since a certain time point. One obvious counterexample:

the government stopped fixing the riverbank, but our NGO intervened to keep fixing it regularly. If someone doesn’t fix the riverbank, there are more floods.

If you ask about changes, in the literal sense, there won’t be any, because the intervention served to maintain the status quo. It might be there are contrast situations like other regions where the government withdrew and no-one stepped in, or a contrast with some previous situation, but there might not be. Either way, the question about changes will only work, if it works at all, because people understand that we really want to know about the whole causal map, not just a change over time. So why not ask that directly? Why not ask, “how do things work around here vis-a-vis floods? Why are there sometimes more or fewer floods?” This is what I like to call “the why question”.

BSDR has a whole wealth of experience and caveats re how to pose these questions, what works and what doesn’t as a question. As an evaluator I’ve often asked “how does X work around here?” and found it fine.