Assessing the effectivness of policy measures with the help of qualitative modeling

Abstract

The aim of a recent project of the UBA was to get a systemic and hence a better understanding of the effectiveness of measures that are meant to help achieve more sustainability in our country. Questions were raised as to why some measures had little effect and also why many measures known to be effective were not being implemented. It was also important to determine what additional measures could be taken. We began the cause and effect model by taking some predefined factors that described the overall goal of becoming a sustainable country. We then applied the KNOW WHY Method to systematically create a model that would include the crucial factors. We did this by repeatedly asking what would directly lead to more of a given factor, and what would directly hinder it, both today and in future. These are the so-called KNOW WHY questions, and we asked them for each and every factor in our model. This resulted in us being able to determine early cross connections, and through them feedback structures became apparent. The model included policy measures, social and psychological factors, as well as economical and environmental aspects. Qualitative modeling makes visible the connections that exist between so-called factors, which carry information about the direction of impact (positive or negative), the strength (weak, middle or strong) and any possible delays in terms of time (short term, medium term or long term). Taken all together, these connections can then be analyzed in so-called Insight Matrices that make it possible to compare the short, middle and long term impact of factors, and hence to see what factors are involved in creating a greater or a lesser impact – in the case of this project, this meant determining what measures promised to be more or less effective and what might hinder the success of these measures to a greater and lesser degree, both now and in the future. In our approach, the factors and connections are not mere visualizations of predefined knowledge gained by modeling experts, but the result of collaborative modeling done by experts from different fields with the aim of obtaining new insights and a deeper understanding of the complex challenge at hand. Therefore, the approach is comparable to that of grounded theory or qualitative social research where scenarios of possible developments cannot be based on empirical data from the past either. Ultimately, the model consisted of over 100 factors and had more than 1 million feedback loops. The results gained by taking this approach shed some light on why the process of change in our society on its way to becoming more sustainable is so slow. The results also explained how and why policymakers, consumers, companies and the media are dependent on each other, and made clear what obstacles the first movers among them face. The model offered an explanation for a widespread phenomenon: rationally knowing what should been done and yet. being emotionally satisfied by engaging in non-sustainable behavior. And finally, the model offered a lever, an entry into the cycle of passive, interdependent players: we need to make sustainable consumption and hence non-consumption emotionally felt through a system that scores behavior. In this short article we will provide one concrete example of how we reflected on the effectiveness of a common policy measure, i.e. the introduction of a resource tax, and how we then assessed it and determined possible impacts and constraints.