Measuring what matters: Alternative measures of progress for Northern Ireland


What is the yardstick by which the progress of countries and societies ought to be measured and judged? This is the question which, for some years now, has acted as the lynchpin behind the many conversations taking place across the globe concerned with whether or not we are measuring what really matters for our people, our economies, our societies or our environment.

While it might come across as a cliché to say that it is what you measure that matters - economic growth, measured using gross domestic product (GDP), has for over 60 years now been relied upon as the key snapshot measure of societal progression. Judgements on the progression of society have hence been based on whether this figure was going upwards or downwards, with the view that a growing economy will result in a better society. In recent decades, however, and in particular in recent years, people have begun to question if economic growth really means progress and have started to identify the limitations of GDP as an indicator of progress.

Picking up on this Professor Diane Coyle of The University of Manchester, one of the leading scholars looking at this issue comments that using GDP to assess economic or societal progress is like ‘driving a car using only the rearview mirror’. Certainly, rather than being a measure which captures progress, GDP is more accurately a measure of marketed economic activity, which broadly speaking captures how ‘busy’ an economy is. It is typically measured, within a particular geographical area, by adding together a nation’s personal consumption expenditures (payments by households for goods and services), government expenditure (public spending on the provision of goods and services, infrastructure, debt payments etc.), net exports (the value of a country’s exports minus the value of imports), and net capital formation (the increase in value of a nation’s total stock of monetised capital goods). All in all, it assesses spending or output, not economic or societal progress.

Highlighting her concern with the over reliance on this figure, Professor Coyle argues that to rely on fluctuations in GDP to assess progress amounts to nothing short of using one of the ‘least useful’ economic indicators. Importantly, Professor Coyle is careful to say that to question the sufficiency of GDP is not to suggest that we entirely abandon it as a measure, but rather argues that its current role poses a number of problems. She is not alone in her critique of this reliance on GDP and joins many other leading academics, statisticians and policy makers to bring to the fore the numerous limitations of GDP as a measure of progress. A major issue is that because it counts every expense as a positive, it does not distinguish between activities which might enhance the progression of a society, from those activities which might reduce it. For example, an oil spill increases GDP because of the associated cost of cleanup etc., but it obviously is a pollutant and damaging to the environment. Other commonly cited limitations include that:

(1) GDP is incapable of taking into account a range of non-market activities that are important, including household activities such as childcare, cooking or cleaning, government-provided services such as health or education, and unpaid activities such as volunteering;
(2) GDP fails to reflect the distribution of the benefits of economic growth. Thus, an increase in national income that goes to a small minority of wealthy families will not increase societal progress or bring benefits to the majority of the population, as if it were distributed more equally;
(3) GDP does not take into account of externalities or changes to natural capital, such as pollution or environmental deterioration and so distorts how much market prices reflect the marginal contribution of certain items to current and future societal progress;
(4) GDP excludes changes in asset values, despite the fact that such changes influence what an individual can consume without becoming worse off.

Beyond these substantive limitations, Coyle also highlights problems and uncertainties with its measurement. She comments that because the GDP figure is the outcome of a very complicated process of collecting data from many different sources, once adjustments are made for seasonal changes, inflation and so forth the margin of error is sometimes bigger than the headline number. Moreover, she points out, revisions to the figure frequently occur – and so suggests that we take with a pinch of salt quarter to quarter changes to this figure.

In taking together these shortcomings, Professor Coyle points out that there is nothing inevitable or natural about using GDP as a measure of economic or societal progress and joins a number of international and national institutions, policy makers, academics etc. in exploring the different ways which we might measure society’s progress. This recent interest in alternative measures of progress is part of a second wave of concern which has been building since around 2000, and boosted in more recent years by a number of authoritative reports and initiatives – most notably the report from the Stiglitz Commission, the OECD’s ‘Better Life’ initiative and the European Commission’s ‘Beyond GDP’ initiative.

Taking together all of this interest the debate about alternative measures of progress has centered around three areas: the first considers how GDP might be extended or modified to correct for some of its limitations. The second is concerned with sustainable growth, with arguments around the unsustainability of the quest for exponential economic growth and calls for zero growth being made here; and the third around a desire to equate economic and societal progress with the (much debated) concept of well-being and measure it as such.

Reflecting this interest numerous initiatives have emerged both at international and national levels in recent years, with the aim of more accurately capturing economic and societal progress, or in the words of the highly influential Stiglitz Commission ‘the full range of factors that make life worth living’.

Indeed, the Northern Ireland Executive is one of a growing number of countries which have recently embarked on a journey to move beyond a sole short-term focus on economic growth by seeking to adopt in its recently published draft Programme for Government (2016-2020) a commitment to equate economic and societal progress with the concept of well-being. Improvements to well-being will be sought via an outcomes-based approach to governance.

Simply described, the new framework proposed by the Executive seeks to offer a ‘whole-of-government’ approach by focusing on delivering improvements using a dashboard of well-being outcomes covering multiple aspects of people’s lives. More specifically, the draft Programme for Government framework includes a statement of purpose to ‘improve well-being for all – by tackling disadvantage, and driving economic growth’, which it will assess across a set of fourteen outcomes. Taken together, the draft framework explains that these outcomes set a clear direction of travel and enable continuous improvement on the essential components of wellbeing. A set of 42 indicators were proposed to assess progress (or regression) towards these outcomes.

Importantly, what is most unique about this draft Programme for Government (2016-2020) is the new format. The framework is not at all specific in terms of the inputs, outputs or targets is seeks to achieve. In doing this the NI Executive seeks to mirror the Scottish model which encourages the Government to pool efforts and function as a single organisation, working towards a single overarching purpose which would be achieved by making progress toward a dashboard of outcomes, over the long term. The argument made is that by focusing on short-term inputs, outputs and targets the conversation is unnecessarily detracted from what really matters - that is, the actual results achieved in terms of improving well-being.

Since its publication however the draft Programme for Government has been subject to much critical skepticism. A lot of the early commentary, whilst not necessarily opposed to an outcomes approach to governance based on a single unified purpose, expressed concern
about the lack of content and detail in this new format Programme for Government. Critics commented that they were unclear about how or why the outcomes, indicators or measures were chosen or how the Executive were going to go about achieving progress across these outcomes. Another key concern was with the lack of clarity over time-frames or actions to be taken in achieving progress towards these outcomes. Others still expressed concern with the choice of particular outcomes, indicators or measures.

On the 28th of October, the NI Executive published the response to the first public consultation on the draft Programme for Government, and launched a consultation on the second stage which will close on the 23rd of December. There is of yet been little scrutiny of this document from media or political commentators. Given how expansive this document and the many linked documents to this it is highly likely that many are still trying to work their way through these before they make public their remarks or observations on the next stage. Nonetheless, the lack of public debate or discussion on this issue is disconcerting.

The upcoming Annual Donál Nevin lecture on the topic of ‘Measuring what matters: Alternative measures of progress’ delivered by Professor Diane Coyle on Monday 12th December at 5pm in The Great Hall, Queen’s University Belfast hopes to provide a forum to inform discussion on this document and encourage a debate on this second stage of consultation on the new draft Programme for Government (2016-2020) towards a new approach to governance in Northern Ireland. In these beginning stages of a move towards alternative measures of progress it is imperative that we engage with this so that we do not make the same mistakes of the past. Most fundamentally, we need to ensure that in this new approach that we measure the right things, in the right way.

Share this blog:

Dr Lisa Wilson

Lisa Wilson is a Senior Economist at the Nevin Economic Research Institute and is based in the Belfast office. Her main research interests lie in the areas of labour markets, income distribution, poverty, public expenditure, living standards and well-being. 

Since taking up her position in the NERI Lisa has been among other things carrying out work concerned with job quality and its economic and social importance, the future of work, the gendered nature of employment, and housing and its impact for living standards in Northern Ireland.

Lisa is a native of Donegal, and a graduate of Ulster University and Queens University, Belfast. Lisa completed her PhD in Queen’s University, Belfast focusing on income inequality and well-being.

Contact: [email protected] or 00 44 28 9024 6214.