Section 26 Variable-based and propositional-based approaches to causal diagrams: chalk and cheese??
Causation: the relation between mosquitoes and mosquito bites (Michael Scriven)
I want to present two different approaches for coding and combining causal links: the “proposition-based” aka “mosquito-bite” approach, versus the “variable-based” approach. The second is more general, more fiddly and less intuitive. The first is simple, plausible, and limited. I think some of the difficulties we’ve had in discussing how to do QuIP coding come back to this distinction.
Classic QuIP coding – as I think it was, say, a year ago (see the Appendix to Copestake, Morsink, and Remnant (2019)) e.g. before any innovations like the coding of multiple influence items or positive versus negative links – is very close to the first, though it does have a couple of elements of the second; and recently we/you have been trying to extend it a bit further and more consistently.
For any given QuIP project, I think you have to choose. Stick to simple and intuitive “proposition-based” coding, which is what you are doing now, and do without the snazzy extras like multiple causes, necessary conditions, adding up influences, etc. Or switch to “variable-based” coding, which is more powerful but perhaps often not what you want. My app can do either, and you can adapt Excel to do either. But I don’t think you can mix the approaches within one project.
It is hard to just add a couple of features to a proposition-based approach. It’s one or the other. It’s difficult because of the logic, not because of the tech.
But James has an interesting suggestion: code individual responses in a “proposition-based” way, but with the option to generate variable-based theories when aggregating these propositions, for example perhaps deducing where there are necessary conditions. But at the bottom level, when coding individual responses, you would still be unable to code multiple causation, necessary conditions, etc.
Logical Frameworks (Logframes) lean heavily on the proposition-based approach too.
In the proposition-based approach the items are propositions like “I lost my job” or “the weather is getting worse”. The propositions can stand alone but most often we want them linked together, like “I lost my job –> I sold my cow”.
One natural way to get statements for this approach is to ask “what’s changed here” and then ask “what caused that?”. The answers are specific statements about what led to what – what actually happened – in a particular context.
The propositional approach has many advantages: above all, it is a part of a very natural way to ask about and record causal influences and which gathers evidence with high plausibility. I also suspect that its simple “causal chain” model is easier to translate for different settings and cultural contexts. However it is nothing like as general as a variable-based approach and has trouble dealing with common cases like multiple influence variables.
Terminology: I will follow the terminology in the Glossary in BSDR (2017), together with our recent additional Glossary (also, I will call the nodes of the causal maps “items”, which is a more neutral and appropriate term than “variables”).
26.1 The variable-based approach
This approach is what I’ve already been suggesting for QuIP coding; I’ve described it in some detail at http://www.pogol.net/_causal_mapping/index.html. It’s just a more qualitative extension of mainstream causal modelling in social science such as we see in structural equation modelling (SEM) or as described by Judea Pearl; and it’s close to the sort of causal networks we see in Fuzzy Cognitive Mapping.
In the variable-based approach, the items can be e.g. level of unemployment, level of retention of cattle. These variables can be binary (yes/no) or continuous but the most important difference is that they are not true or false; they don’t make any claims. Specification of the actual / current / predicted value of the variables is an additional, optional step. The variable-based approach is more general; the proposition-based approach is a special case of it. A proposition like “I sold the cow” can be expressed using the variable-based approach (more long-windedly) as “The variable did I sell the cow, yes/no takes the value yes”; or even as “There is a variable called did I sell the cow, yes/no; and this variable takes the value yes”. But most often, we don’t need to bother with this kind of thing because we are encoding statements like “loss of job –> sale of cow” and we don’t need the information about whether a particular respondent actually did the selling or not.
One natural way to get statements for this approach is to ask “what causes what around here?” or “What has to happen for a farmer like you to get a good harvest?”. The answers are general statements about what leads to what in a particular context. That context can be quite narrow – just these farmers in this valley – but the statements are general: they don’t necessarily make any claims about what actually happened. In this approach, we encourage people to tell us their knowledge about the whole domain regardless of what happened to them.
26.2 More on the proposition-based approach
Below is an imaginary causal chain derived from some corresponding statement(s) from a single source. It is a good example of a “mosquito-bite” or “propositional” chain. Each item can be said to cause the next one, down the chain, like dominoes falling over. This idea of “X causes Y” is strong in this approach, a familiar idea in ordinary language. You can’t say “X causes Y” so easily with variables. It is transitive: causality flows down the chain, and if X –> Y and Y –> Z, then X –> Z too. It captures the idea that we can sometimes even see causation happening – the ball hits the window, which smashes. The mosquito bites the person, who feels the jab. The two parts of the causal link are even hard to separate: it is the ball which “smashes” but the glass which “is smashed”; two sides of the same coin. We feel strongly that we don’t need any more waffle, that this is just causation happening in front of our eyes. It a strong kind of evidence, it is “not circumstantial”.
In the mosquito-bite approach, every item contributes 100% to the item downstream of it (and to those preceding that item), and every item can be attributed 100% to the item upstream of it (and to those preceding that item). Is this a simplification? Of course, but every model/approach to causation is a simplification.
Propositional-based causal thinking is so sweet and neat that we’d all love it to be good enough at least for everyday thinking about how the world works and in particular for looking at the role played by a project.
My main thesis here is that it is difficult to tweak this “mosquito-bite / propositional” approach to allow for desirable features like strength of connection or multiple causation without ending up with a variable-based approach.
26.2.1 The items are propositions, not variables
The items (the boxes or nodes between which arrows are drawn) are propositions and not, for example, variables. The items themselves are specific empirical (but not causal) claims made in and about a specific context (such as “X happened” or “Y has got worse”) and are, broadly speaking, more or less true or false in that context.
The claims have a varying amount of subjectivity, from a fairly subjective “this has got worse for me” to a fairly intersubjective “the rains have got worse”; nevertheless they are grammatically all claims, which can in principle be more or less true or false.
This also means that the items have no gradation. They are always one thing or the other. There are no possibilities in between.
26.2.2 The items have been individually experienced
In general, the propositions are / have been personally true for, or at least personally experienced by the source – BSDR (2017) doesn’t say this explicitly, and focus group data asks for information on “people like me”. So, seeing that the absence of X leads to the absence of Y does not give me the right to claim that X leads to Y (even if I have seen X leading to Y in relevantly similar cases, but not my own case.)
26.2.3 The propositions have contrastive implications
I think that this way of thinking does at least concur with the a mainstream view of causation that a causal claim always implies a “counterfactual” contrast. If someone makes a claim of the form that “X caused Y”, but then says that actually even if X had not happened in that way, this wouldn’t have made any difference to Y, and that Y can happen or not happen irrespective of X", we would conclude that the speaker does not share our understanding of causality, proposition-based or not, and we should not code the initial statement as a causal claim.
For example, if someone says that the reason my crops were good is because it is the Year of the Dolphin, and the reason my neighbour’s crops were poor is also because it is the Year of the Dolphin, if they have not named further explanatory factors we would probably not code this/these as causal claims at all.
This is quite another thing from saying that the people making it have to be in possession of corresponding evidence, and is compatible with a proposition-based as well as a variable-based approach. Mosquito-bite claims are based on a more fundamental experience of causation than comparing a state of affairs with a counterfactual. But they do have counterfactual implications.
26.2.4 Two-sidedness of propositions
With variables, all the values of the variable have to be clearly defined. So if we have a variable like “% unemployment” then we know what 100% means, we know what 20% means, etc. If we have a variable “Lost job, yes/no” then we know what the “yes” situation looks like and we know what the “no” situation looks like.
A strength and weakness of propositions in ordinary language is that we might know what their “yes” side looks like but we don’t necessarily know what the negation looks like. So we don’t always know the negation of “I sold my cow”. Is it “I didn’t sell my cow, I kept it” or “I sold my pig” or “I lent my cow to my neighbour”. This kind of ambiguity makes these one-sided propositions useless for explanation. I think that QuIP coding does understand propositions in a two-sided way. That is what makes them amenable to contrastive explanations, see above.
26.2.5 Aggregation
The “proposition-based approach” has, by design, less taste for aggregation and simplification, preferring to say “dig into the data yourself”. That’s fine. But classic QuIP coding does do one important kind of aggregation out-of-the-box, namely when coding two different items from two different sources as “the same”, using in QuIP-speak “the same tag”. This issue of deciding whether, say
- “I sold my cow”
- “I sold some cows”
- “I sold some livestock”
- “I got rid of a cow”
should be coded under the same “tag” and if so which one, is of course a challenge of all social research. The propositional approach has a somewhat harder time of it because there is no way to code the idea of “same thing, different degree”.
26.2.6 The items can express changes, states or events
A strength of this kind of proposition-based analysis of causal statements is that it quite naturally captures the kinds of causal explanations which people actually give. One could argue that we shouldn’t even be worrying about what format the antecedent causes take (events, states or whatever), because they are just whatever explanations people give. But this would be problematic. The coder needs to understand the items within a claim, rather than just reproducing them verbatim – first in order to be able to judge whether a different claim by a different source involves the same item or not, and second when reflecting on the other statements made by a respondent changes our understanding of the original statement, and third, most importantly, to check that they really mean causation here.
QuIP seems here to be more restrictive than the proposition-based approach, sometimes suggesting that only changes are relevant, though the definitions in the Glossary in BSDR (2017) are not completely consistent. So, a Driver is “An action or state (X or Z) behind outcomes (Y).” and Outcomes are “Changes (positive or negative) reported by respondents”; as “outcomes can also become drivers of change”, this means that the apparent distinction between “actions or state” on the one hand and “changes” on the other hand is blurred (because any one item can be both).
Perhaps classic QuIP coding requires only that final (“no-child”) outcomes which have to be expressed as changes in the reference period, whereas other items can be expressed in a variety of ways (though still as propositions). But see below.
26.2.6.1 Changes
Changes are familiar items in QuIP coding, for example ..…
Each item can be seen as a comparison between states at at the beginning and end of a reference period.
26.2.6.2 Events
Events are perhaps the archetypical kind of thing to appear in a proposition-based causal chain.
Though there is no hard-and-fast distinction, it can make sense to distinguish events from changes. In this case, the event can also be parsed as a change, from lower to higher food taxes. But …
… in this case, the triggering event can hardly be described as a change. It is a clear example of a one-off event.
“Events” pass the contrastive test (see above): if the incursion hadn’t happened, we wouldn’t have lost (so many) cattle, but it did, and we did.
26.2.6.3 States
Putative static causes like “unemployment” are things whose status may not be changing, but still pass the contrastive test. (Of course, causal claims about changes in an influencing factor like “increasing unemployment” are also possible.)
“Unemployment” perhaps remained at a high level throughout the reference period. Perhaps the source even explicitly mentioned “if we’d had jobs, we could just have worked overtime to compensate”, contrasting unemployment only with its absence, namely employment. Here, unemployment is not mentioned as something which changes over time; rather, it is contrasted with another situation which might have existed outside the reference period or simply as a desirable but non-existent situation.
The role of “unemployment” here should be distinguished from its role as an “enabling factor” in this example (which in any case can probably not be coded in the propositional approach):
26.2.6.4 “Differences” as a general way of expressing events / states / changes
The distinctions between events, states and changes are not very clear and maybe not important. A response would be to replace these ideas with the broader idea of “Differences” or “deltas” (https://www.linkedin.com/pulse/dear-world-bank-its-differences-changes-we-look-impact-steve-powell/).
All the examples above can be covered by the idea of “Differences”; things being one way rather than some expected or possible contrast: a state of unemployment rather than a state of employment, a status quo in flood levels rather than the feared contrast of floods, an incursion of terrorists rather than none, and so on.
As we already mentioned, QuIP understands propositions in a two-sided way, i.e. their negations are clearly defined. This makes it possible to express Differences using propositions too.
26.2.7 Golden threads
One big advantage of QuIP-style coding is that it has the concept of an entire causal chain which is not reducible to a bunch of individual links. Indeed, attribution codes are given to whole chains, not to individual links.
It’s of practical benefit that BSDR has now adapted the coding process to be built up from individual atomic maps, rather than chains. But this shouldn’t mean that chains have disappeared from the coding process.
It is essential for QuIP to be able to track from outcomes of interest back to project interventions within individual causal claims from individual respondents. I understand that this is gold for QuIP; we can call them “gold threads”. It is much more valuable than a case in which one respondent mentions a link from the project to some intermediate item X, and another mentions a link from X to an outcome of interest. This cannot be done if we just construct a causal map for all respondents from a pile of atomic links. The information about which chain each link belongs to has to be additionally retained, in order to be able to dig for QuIP gold.
Golden threads are valuable because of their direct plausibility; actual individuals spontaneously mentioning a causal chain from an intervention to an outcome of interest. Donors love this stuff.
On the other hand, it is still possible to use QuIP (and proposition-based approaches) for questions which are not focused on a particular project.
26.3 Statistics?
Isn’t a variable-based approach just SEM? Well the extension of SEM-type thinking I’m talking about here has no parametric assumptions at all, so it is more general, weaker, and potentially more computationally expensive; it has to be able to deal with ideas like “a bit more than”. And most importantly it isn’t statistics, in the sense that it isn’t conceiving of respondents’ causal claims as some sample out of a larger set which approximate the truth. It says, even if we believe these statements are actually the truth – they are already the conclusions which statisticians try to reach from their empirical data – and after all they are based on people’s actual causal knowledge, not just on a few observations – how can we code and combine them? In this sense it is no different from the propositional approach. But yes it is different in the sense that we are trying to encode and aggregate theories, models, with explicit counterfactual implications, rather than direct, one-off reports of “this happened and it made this happen”
26.4 Conclusion
The “proposition-based” aka “mosquito-bite” approach is simple, plausible, and limited. The “variable-based” approach is more general, more fiddly and less intuitive. I think some of the difficulties we’ve had in discussing how to do QuIP coding come back to this distinction. The two are quite different and can’t be mixed in one set of codings. Perhaps, as James suggests, coding can be done with the propositional approach and then a variable-based theory built up from that. But this does not get round some of the important limitations of the propositional approach in the very first step – basic coding. I will look at these at in my next contribution.