Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms

This chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence in the existence of a mechanism is increased or decreased through a process that can be open, transparent, and shared with the public or groups of stakeholders, reducing cognitive biases, and improving internal validity and consensus around the existence of given mechanisms. The approach is showcased in the evaluation of a learning partnership whereby a knowledge product released by a research organization influenced policy at the municipal level.