The OECD’s fragility index is surprisingly fragile and difficult to reproduce

In its 2015 States of Fragility report, the Organization for Economic Co-operation and Development (OECD) decided to complicate its usual one-dimensional list of fragile states by assessing five dimensions of fragility: Violence, Justice, Institutions, Economic Foundations and Resilience. This change shows conceptual progress toward an understanding of what exactly fragility means and how the international community should tailor its response to different types of fragility. With the new framework, it is clear that Benin — vulnerable in resilience and economics — has different development needs than Cambodia — vulnerable in justice and institutions.

Unfortunately, something went wrong during the calculations. In my attempts to replicate the assessment, I found that the OECD misclassified a large number of states, a mistake that could have real-world repercussions. Recent research by Judith Kelley and Beth Simmons shows that international indicators are an influential policy tool. Indicators focus international attention on low performers to positive and negative effect. They cause governments in poorly ranked countries to take action to raise their scores when they realize they are being monitored or as domestic actors mobilize and demand change after learning how they rate versus other countries. Given their potential reach, indicators should be handled with care.

Download paper here