Advice, Money, Results – Rethinking International Support for

June 2020

ADVICE, MONEY, RESULTS

Rethinking International Support for

Managing Public Finance

Report by an International Working Group

Advice, Money, Results

Rethinking International Support for

Managing Public Finance

Report by an International Working Group

New York University

Robert F. Wagner Graduate School of Public Service

June 2020

CONTENTS

ACKNOWLEDGMENTS ...........................................................................................................................i

ABBREVIATIONS ................................................................................................................................... iii

ABSTRACT ................................................................................................................................................iv

1. INTRODUCTION: SETTING THE AGENDA ...................................................................................1

Scope of inquiry ............................................................................................................................................1

The state of PFM in 2019 ..............................................................................................................................2

Four problems for the PFM community .......................................................................................................2

Guiding framework: an “open” model of PFM ...........................................................................................6

Developing some responses ..........................................................................................................................7

Organization of the report ............................................................................................................................8

2. MANAGING PUBLIC FINANCE: A REAPPRAISAL ....................................................................10

Managing public nance and PFM: meaning and denitions ....................................................................10

Public nancial management in context ..................................................................................................... 11

PFM as a “professional discipline” and its conventions ...........................................................................12

Guiding objectives of PFM and their evolution ..........................................................................................19

Moving on from here ...................................................................................................................................26

3. BUILDING ON THE STRENGTHS OF THE “CLOSED” PFM MODEL .................................... 27

Faster learning about external support for PFM .......................................................................................27

Contestation and choice in PFM advice .....................................................................................................32

Moving on from here ...................................................................................................................................39

4. RETHINKING MANAGING PUBLIC FINANCE IN AN “OPEN” SYSTEM .............................40

Linking PFM to service delivery and development results .........................................................................40

Harnessing PFM for policy change ............................................................................................................47

Moving on from here ...................................................................................................................................54

5. PUTTING IDEAS INTO PRACTICE ................................................................................................56

An agenda for change: some proposals for the PFM discipline ................................................................. 56

Building a persuasive case for change: some initial ideas .........................................................................57

Moving on from here ...................................................................................................................................59

CHAPTER REFERENCES .....................................................................................................................60

ANNEXES .................................................................................................................................................75

ENDNOTES.............................................................................................................................................118

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

ANNEXES, TABLES, AND FIGURES

Annex A: Key concepts and terms used in the report .................................................................................75

Annex B: A framework for public policy—government policy choices, government actions,

and development results ..............................................................................................................................82

Annex C: Public nancial management reforms in context .......................................................................90

Annex D: PFM focus within World Bank sector activities .........................................................................93

Annex E: The quality of data on development nancing for PFM .............................................................95

Annex F: Mapping of PFM assessment frameworks ..................................................................................97

Annex G: Relating the institutional and economic perspectives on PFM—a thought experiment ..........101

Annex H: A partial summary of PFM research .........................................................................................105

Annex I: The elusive hope of incorporating context.................................................................................109

Annex J: Evolution in theories of externally supported change in the public sector ................................ 112

Annex K: Putting external contestation of policy ideas into practice ....................................................... 115

Table 1: Proposals for a new direction for managing public nance ............................................................ 7

Table 2: Refreshing the PFM objectives – proposals for discussion ..........................................................24

Table 3: The basis for recommending specic PFM conventions ..............................................................45

Table 4: A recap on the problems and proposals ......................................................................................... 57

Table 5: Terms of art used in the report ......................................................................................................76

Table 6: Government actions for public services range from direct provision to regulation......................88

Table 7: Recent public sector reforms.........................................................................................................90

Table 8: Mapping of PFM assessment tools ...............................................................................................97

Table 9: Capabilities of an effective PFM system and their relationship to public nance objectives ....103

Table 10: A partial summary of key research ............................................................................................ 105

Table 11: Adaptive Management, Doing Development Differently, and Thinking and Working

Politically vs. Berg’s ideas from the early 1990s ......................................................................................110

Table 12: Stylized theories of growth and of externally supported change in public management .........112

Figure 1: “Opening up” perspectives on means and ends in managing public nance ................................6

Figure 2: External funding for PFM, comparison of ve-year totals (US$ billions) ..................................14

Figure 3: Internal and external funding by PFM reform area for a basket of countries (US$ billions) .....14

Figure 4: Links and interactions between the dimensions of public policy ................................................83

Figure 5: Ngram plot of references in literature published in English containing the terms “service

delivery” and “development assistance” .....................................................................................................85

Figure 6: Proportion of activities with a signicant PFM component in the Health, Nutrition, and

Population Global Practice .........................................................................................................................93

Figure 7: Proportion of activities with a signicant PFM component in the Education

Global Practice ............................................................................................................................................94

Figure 8: Proportion of activities with a signicant PFM component in the Governance

Global Practice ............................................................................................................................................94

Figure 9: Causal chains and attributions to be considered ........................................................................105

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

An International Working Group on Managing

Public Finance, operating with guidance from

an Advisory Group of eminent persons, was

convened between January 2018 and August 2019

to investigate and debate the effectiveness of the

development assistance architecture in supporting

public nancial management and to consider

future directions for international support. This

report is the result of that work.

The initiative was hosted by the Robert F.

Wagner Graduate School of Public Service at

New York University and was funded through a

research grant from the Bill and Melinda Gates

Foundation. The views expressed in the report do

not necessarily reect those of either organization.

The usual disclaimers all apply.

Members of the Working Group include Edward

Hedger, Roger Koppl, Florence Kuteesa, Nick

Manning, Barbara Nunberg, Jana Orac (Project

Manager), Allen Schick, Paul Smoke (Project

Director), and Duvvuri Subbarao. Contributions

by Marco Cangiano, who served as a member

of the Working Group until 2018, are gratefully

acknowledged.

Members of the Advisory Group include Vicente

Fretes Cibils (Inter-American Development

Bank, or IADB), Neil Cole (Collaborative Africa

Budget Reform Initiative, or CABRI), Salam

Fayyad (Woodrow Wilson School, Princeton

University), Keith Hansen (World Bank), Homi

Kharas (Brookings Institution), Rathin Roy

(National Institute of Public Finance and Policy,

India), Antoinette Sayeh (Center for Global

Development), Seth Terkper (PFM-Tax Africa

Network), and Debbie Wetzel (World Bank).

Particular thanks are due to Thomas

Kalogeropoulos, Suley Ouattara, and Jocelyne

Uwayo, who provided valuable research

assistance, and to Vishal Gujadhur and Chisom

Okechukwu of the Gates Foundation for their

strategic guidance. The idea for the Working

Group originated from Philipp Krause at the

Gates Foundation, whose advice and feedback

throughout the work is gratefully acknowledged.

Amanda Green edited the report with meticulous

care.

The Working Group consulted a large number

of experts in public nance and development

cooperation, who gave generously of their time,

ideas, and advice. Those individuals include Awo

Ablo, Mike Ablowich, Tera Allas, Richard Allen,

Jean-François Almanza, Katherine Baer, Owen

Barder, Erik Berglof, Jon Blondal, Jurgen Blum,

David Bontempo, Martin Bowen, Chiara Bronchi,

Jim Brumby, Kalipso Chalkidou, Chris Chalmers,

Matt Davies, Cem Dener, Paolo de Renzio, Shanta

Devarajan, Bill Dorotinsky, Riccardo Ercoli,

Thomas Feige, Manal Fouad, Adrian Fozzard,

Verena Fritz, Erica Gerretsen, Sanjeev Gupta,

Sierd Hadley, Tobias Haque, Dan Honig, Alan

Hudson, Richard Hughes, Sachiko Imoto, Phil

Keefer, Stuti Khemani, Maia King, Rajesh Kishan,

Shivin Kohli, Jens Kristensen, Vincent Larrouze,

Edwin Lau, Andrew Lawson, Mary McCarthy,

Larry McDonald, Mark Miller, Rob Mills, Ian

Mitchell, Hakon Mundal, Piroska Nagy-Mohacsi,

Peter Nicholas, Aimée Nichols, Jason Orlando,

Steve Pierce, Carlos Pimenta, Mark Plant, Vivek

Ramkumar, Carolina Renteria, Andrew Rogerson,

Jacek Rostowski, Carlos Santiso, Bill Savedoff,

Salvatore Schiavo-Campo, Rosmarie Schlup,

Robert Schoellhammer, Renaud Seligmann, Bing

Xun Seng, Rebecca Simson, Gregory Smith,

Nicola Smithers, Franziska Spoerri, Stephanie

Sweet, Rob Taliercio, Teresa Ter-Minassian,

Chris Tinning, Rachel Turner, Sanjay Vani, Clay

Wescott, Alan Whitworth, Ellen Wichmann, and

Jennifer Widner.

Important insights were provided by nance

ministry ofcials from Ghana, Lesotho, Liberia,

and Nigeria at a CABRI workshop in Pretoria in

May 2018. Valuable feedback on draft versions

was obtained from presentations made to the

Public Expenditure and Financial Accountability

(PEFA) Steering Committee in June and December

ACKNOWLEDGMENTS

2018, and at a symposium organized by New

York University, Princeton University, and the

Overseas Development Institute (ODI) in May

2019 in Washington, D.C.

The Working Group drew from a signicant body

of published research. The bibliographies for each

chapter are not exhaustive, but they do highlight

the most signicant references used as part of this

work.

The Working Group welcomes reactions to this

report. Please direct any questions or comments to

the principal authors:

Nick Manning ([email protected])

Edward Hedger ([email protected])

Allen Schick ([email protected])

Please copy the director of the overall initiative,

Paul Smoke ([email protected]).

Acknowledgments

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

iii

ABBREVIATIONS

ADB Asian Development Bank

BMZ FederalMinistryofEconomicCooperaonandDevelopment(Germany)

CABRI CollaboraveAfricaBudgetReformIniave

CPIA CountryPolicyandInstuonalAssessment

DAC DevelopmentAssistanceCommiee(OECD)

Danida DanishInternaonalDevelopmentAgency

DeMPA DebtManagementPerformanceAssessment

DFAT DepartmentforForeignAairsandTrade(Australia)

DFID DepartmentforInternaonalDevelopment(UnitedKingdom)

EC EuropeanCommission

EU EuropeanUnion

FMIS FinancialManagementInformaonSystem

G20 GroupofTwenty

GDP GrossDomescProduct

GIZ GesellschafürInternaonaleZusammenarbeit(GermanSocietyfor

InternaonalCooperaon)

IADB Inter-American Development Bank

IEG IndependentEvaluaonGroup

IFI InternaonalFinancialInstuon

ILO InternaonalLabourOrganizaon

IMF InternaonalMonetaryFund

IPSAS InternaonalPublicSectorAccounngStandards

IT InformaonTechnology

MDB MullateralDevelopmentBank

MTEF Medium-TermExpenditureFramework

NGO Non-governmentalOrganizaon

NYU NewYorkUniversity

ODA OcialDevelopmentAssistance

ODI OverseasDevelopmentInstute

OECD OrganisaonforEconomicCo-operaonandDevelopment

OOF OtherOcialFlows

PEFA PublicExpenditureandFinancialAccountabilityprogram

PEMNA PublicExpenditureManagementNetworkinAsia

PEMPAL PublicExpenditureManagementPeer-AssistedLearningnetwork

PFM PublicFinancialManagement

PIMA PublicInvestmentManagementAssessment

SARA Semi-AutonomousRevenueAuthority

SDGs SustainableDevelopmentGoals

SNA SystemofNaonalAccounts

TADAT TaxAdministraonDiagnoscAssessmentTool

TOSSD TotalOcialSupportforSustainableDevelopment

UNDESA UnitedNaonsDepartmentofEconomicandSocialAairs

UNDP UnitedNaonsDevelopmentProgramme

UK UnitedKingdom

US UnitedStates

USAID UnitedStatesAgencyforInternaonalDevelopment

The current paradigm of public nancial

management (PFM) in international development

has proved remarkably successful as the basis for a

common approach, supported by a wide consensus

among practitioners, advisers, and researchers.

That approach emphasizes conformity with a set

of processes and conventions aimed at achieving

a standard trinity of PFM objectives: scal

discipline, allocative efciency, and operational

efciency. The approach is powerful and has

provided practical means for those seeking to

strengthen the institutional foundations of PFM.

Improvements are still needed, however, to make

external interventions on PFM more effective and

to connect PFM more strongly to all levels of

public policy.

Further enhancing the effectiveness of external

interventions within the current paradigm will

require better sharing of evidence and knowledge

about intervention design. This could arguably

be facilitated by a stronger role for debate and

government choice among external advice options.

This is a demanding technical agenda.

Strengthening links between institutional

arrangements for public nancial management

and country-specic public policy raises more

challenging conceptual questions, however. From

a policy perspective, the PFM paradigm is at

the limits of relevance and utility in its current,

inwardly focused form. It is constrained in its

capacity to engage with real-world choices and

dilemmas.

The current approach treats PFM as a “closed”

system, in which the PFM processes and

conventions that drive operational performance

are considered to be unchanging ends in and of

themselves. Viewing PFM instead as an “open”

system that interacts more uently with all aspects

of public policy—namely, government policy

choices, government actions (especially service

delivery), and development results—offers the

potential for developing a new generation of

approaches to managing public nance. This

broader perspective would better reect the

complex realities of context and the unavoidable

nature of trade-offs in policy objectives.

This “open” conception of PFM, proposed by

the International Working Group on Managing

Public Finance, is distinguished by a willingness

to revise and expand the guiding objectives for

managing public nance and to reappraise the

conventions that drive the form and content of

external technical advice in this area. It sees

conventions about the operational performance

of PFM processes as useful benchmarks, without

con-sidering them to be set in stone or remaining

blind to their purpose of informing and delivering

wider public policy outcomes.

The report motivates and frames this ambitious

and challenging agenda. It identies problems

with current approaches to external assistance in

PFM and offers specic proposals on how to move

toward a new, more “open” model of managing

public nance that is t for the 2020s and a post-

COVID world grappling with climate change.

ABSTRACT

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Scope of inquiry

Advice, money, and results are central concerns

in international support for managing public

nance. Technical advice is the mainstay of

external assistance for strengthening scal

institutions. Effective use of development nance

and domestic resources is a priority for donors

and governments alike. Development results are

ultimately the raison d’être of managing public

nance. As a raft of recent reviews, evaluations,

and policy initiatives indicates, these matters are

persistent preoccupations for the international

community when considering lower-income

countries, and there is strong demand for ideas,

innovation, and progress (Coady & Murgasova,

2019; IMF, 2018a, 2019; IEG, 2018; Lagarde,

2019; Shanmugaratnam et al., 2018; World Bank,

2019b). This report speaks directly to the question

of how the self-described domain of public

nancial management (PFM) can contribute more,

and more quickly, to development results.

The report argues that now is the right time to

review and re-evaluate the scope, objectives,

and results of the PFM agenda and the

development assistance that supports it. It sets

out some of the frontier challenges confronting

PFM and considers how the PFM community

should respond. In doing so, it poses several

questions. First, long-standing PFM objectives

have shaped the design of external technical

advice based on an increasingly standardized set

of PFM processes and conventions. To the extent

that these conventions reect a widely accepted set

of institutional practices, how can external support

for reforms be made more effective? Second,

greater choice by client governments is often

suggested as a way to drive up effectiveness. How

could greater contestation of ideas and intervention

approaches be deployed to support institutional

reforms? Third, a strong PFM system is assumed

to contribute to service delivery and sector

outcomes. What is actually known about those

connections and the ways in which PFM makes a

difference for development results? Fourth, major

policy concerns such as tackling climate change

and boosting infrastructure development will

require massive reprioritization in budget policies.

Are the standard PFM processes and conventions

up to the task? What changes are needed? What

can PFM do to support better analysis of the trade-

offs in policy objectives posed by socioeconomic

concerns such as inequality, which are regaining

political salience but are crowded out by other

scal objectives?

The analysis and recommendations presented

here draw on 18 months of in-depth and wide-

ranging deliberations by an International

Working Group on Managing Public Finance.

The questions examined by the Working Group are

at the forefront of debates among policy makers,

practitioners, and academics concerned with

international support to PFM. They intentionally

push at the boundaries of what has conventionally

been considered the terrain of PFM and its primary

areas of concern. For the purposes of this report,

PFM is treated in broad terms as a set of processes

that connect public resources with public policy

while serving the wider nancial purposes of

accounting and accountability. Although it is a

highly technical topic, discussion of PFM is often

accompanied by terms that are generalized and

ambiguous rather than specic and consistent. To

avoid confusion, this report employs a standard

lexicon of relevant “terms of art,” which is set out,

with examples, in Annex A.

1. INTRODUCTION: SETTING THE AGENDA

The state of PFM in 2019

The scope and objectives of PFM are not

timeless; they can and should be reviewed,

refreshed, and expanded to reect the full

range of contemporary policy concerns and

goals. Fiscal discipline, allocative efciency,

and operational efciency have been adopted

widely by the international PFM community

as the explicit aims of the PFM system—the

“holy trinity” of standard PFM objectives. These

objectives have had a powerful impact on the

content of technical advice to governments over

the past two decades. They have, however, created

a bias toward short-term measures to ensure scal

discipline. More generally, the broad consensus

around these proximate and technical objectives

has masked an underlying normative and value-

based conict about the aims of government

policy. Obscuring the inherent tensions among

scal objectives and broader socioeconomic

outcomes does not remove those tensions. They

are always present, and it is a fundamental task

of democratic politics to try to mitigate or resolve

them. Refreshing and expanding the objectives

of PFM—as proposed in this report—would play

a part in helping to rebalance policy priorities in

a way that allows these normative conicts to

surface and be resolved through consultative and

political processes.

International support for PFM has assumed a

distinctively strong place in development, with

its own professional discipline and a strong set

of conventions about desirable practices. While

it is challenging to impose a single framework on

the analysis of public nance writ large, PFM has

emerged as a more distinct and tractable domain

within this larger and conceptually more uncertain

context. The term leaned strongly toward the

management of general government expenditures

in the 1990s, but most standard denitions of

PFM have broadened since then to encompass

the revenue side of the budget, the wider public

sector, and the relationship to scal policy.

PFM has risen rapidly in importance within

development assistance overall, and a distinct

PFM “discipline” has emerged. The associated

disciplinary approach encompasses a set of

strong normative conventions about what PFM

institutional arrangements should be, built on the

premise that conformity with these conventions is

necessary, if not sufcient, to achieve the trinity

of objectives. The discipline has more recently

evolved to recognize the signicance of problem-

based and adaptive approaches, which are offering

increasingly nuanced advice about sequencing the

standard PFM objectives and how conformity with

PFM conventions might realistically be achieved.

Four problems for the PFM

community

The central argument of the Working Group

is that the paradigm of PFM in international

development has become too inwardly focused

and needs to reconnect with broader policies

and results. Although the current disciplinary

approach has been of considerable value and still

has much to offer in accelerating improvements

in PFM, further progress could be made through

better sharing of evidence and knowledge about

intervention approaches and by introducing a

stronger role for debate and government choice

over external advice options. This approach would

strengthen PFM within the terms of its current

framing, but is insufcient on its own. The current

overall framework for PFM is limited and self-

referential. It assumes a “closed” system in which

PFM processes and conventions are considered to

be ends in themselves, independent of variations

in policy objectives. While these conventions

may provide useful guideposts, they are a weak

alternative to a more “open” model, in which the

inuences of PFM on government policy choices,

government actions (especially service delivery),

and development results are the central focus

(see Annex A for denition of specic terms,

and Annex B for an elaboration of the integrated

public policy framework used in this report).

Refreshing and expanding the objectives of

PFM to bring in these wider agendas is a starting

Chapter 1. Introduction:

Setting the Agenda

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

point in highlighting the complex trade-offs

involved and thus strengthening the role of PFM

in helping policy makers to address real-world

priorities, choices, and decisions. Treating PFM

as an “open” system that interacts uently with

the policy domain would bring the development

sector into line with how managing public nance

is viewed in emerging and advanced economies

(Schick, 2019).

Improving support for strengthening PFM on

its own terms and restoring the inuence of

PFM on wider development outcomes are the

twin concerns of this report. While the ambition

of the Working Group is to motivate a change

in the PFM paradigm and in the mindset of the

PFM community, it is clear that such an evolution

may take time. In the meantime, there remains an

immediate opportunity to improve the effective-

ness of technical advice on current terms. The

challenges addressed by this report reect this

two-track approach, focusing rst on how to

build on the current “closed” PFM system and

then contemplating the shift to an “open” model

of PFM.

Rethinking international support for managing

public nance embodies a vast agenda, but some

clear and strong contentions have emerged

about a set of top-order issues. Drawing on

extensive consultations with practitioners,

academics, and policy makers engaged in PFM

across a wide spectrum of organizations, countries,

and perspectives, together with an in-depth review

of the literature and available evidence, this

report takes a focused and propositional approach

by selecting and examining four interrelated

challenges or “problems” in PFM that respond to

the key questions explored above.

Problem 1: The pace of learning

about how external assistance can

better support improvements in

PFM is too slow

Countries have made progress in strengthening

their PFM processes, but the contribution of

external support is hard to disentangle. Some

countries have shown improvements in PFM

according to assessment frameworks such as the

Public Expenditure and Financial Accountability

(PEFA) indicators, which reect the current

disciplinary conventions. The evidence is limited,

however, and the picture is mixed. Lower-income

countries have made gains from a low base, and

improvements link closely to country income, in

that countries with higher income per capita show

evidence of a stronger PFM system (de Renzio,

2009; Fritz, Verhoeven & Avenia, 2017). Countries

may be making progress, but it remains unclear

how international assistance providers can best

support those improvements. Assuming there is

no universal and unchanging approach waiting to

be revealed, the focus needs to be on a continuous

process of discovery and learning across different

contexts and requirements.

Technical advice is at the heart of external

support for PFM. It may be provided by

international organizations, donor governments,

consulting rms, non-prot organizations, or

individual advisers. Often it is funded, designed,

delivered, and evaluated by a combination of them.

Technical advice is embedded in many forms of

intervention: technical assistance missions or

projects, investment projects, nancial support

through government systems, pooled funds,

policy or structural reform conditions, and results-

based funding. These interventions are inuenced

increasingly by problem-driven and adaptive

approaches to institutional reform. Surprisingly

little is known, however, about the effectiveness

of any of these instruments and approaches.

Proposals for improving the effectiveness of

external interventions are often based more on

orthodoxy or belief than on evidence. Can the

evidence base be strengthened and mobilized so

that learning and improvement happen faster?

Problem 2: Development agencies’

preference for following a common

approach has reduced opportunities

to test ideas through competition

Debates about donor coordination and

the risks of competition among assistance

providers are long-standing. There is a growing

range of new sources of development nance

and technical assistance to governments (Custer,

DiLorenzo, Masaki, Sethi & Harutyunyan, 2018;

Greenhill, Prizzon & Rogerson, 2013; Hudson

Institute, 2016; Schmaljohann & Prizzon, 2015).

Despite renewed efforts and initiatives to improve

donor coordination and agree on common rules

and standards (Shanmugaratnam et al., 2018),

effective coordination remains hard to achieve

(Annen & Moers, 2012; Bourguignon & Platteau,

2015; Koch, Leiderer, Faust & Molenaers, 2017).

Approaches to external assistance for PFM have

sought to reduce conicting donor requirements

through a “strengthened approach” that emphasizes

government leadership and “coherent, co-ordinated

and programmatic” support by donors (OECD,

2006, p. 76). But there are critiques about the

realism and success of attempts at harmonization

and alignment (Booth, 2011). The debate about

whether and under what conditions some degree

of competition in development assistance might

be preferred to coordinated approaches—and the

desirable balance between these approaches—

remains a live one, without any clear-cut answer.

Rather than chasing a fully coordinated

approach, which seems unattainable, a more

promising avenue may be to promote greater

contestation among alternative intervention

design options and to ensure greater choice

on the part of client governments. While

competition among technical assistance providers

could in theory drive those effects, it would

carry risks and require donors to unbundle their

funding and advice or to submit themselves to

genuine competition (Barder, 2009; Easterly,

2002, 2009; Klein & Harford, 2005). That is

simply not realistic. Instead, a more modest and

possibly more powerful way forward may be to

ensure the development and rigorous assessment

of alternative intervention options proposed by a

particular organization or group of coordinating

development agencies to foster proper debate

and allow for an informed choice among these

options. This might incidentally involve a larger

number of funders or providers, but it need not do

so. It would strengthen the emphasis on ideas and

evidence about why different PFM interventions

and approaches might be expected to work. Would

the introduction of such approaches have positive

results? How could greater contestation and choice

realistically be put into effect and tested?

Problem 3: The ways in which PFM matters

for service delivery and development results

are not well understood

Despite the growing inuence of problem-

based and adaptive approaches to development

assistance, the PFM arena remains dominated

by certainty rather than experimentation.

Standardized PFM assessment frameworks are a

growth industry, and there is a propensity to look

for “scal machinery” that will be applicable across

all countries in supporting government actions and

development results (PEFA Secretariat, 2018).

The obvious risk is that one size is assumed to t

all contexts, with the underpinning assumption

that the desirable “look and feel” of PFM

processes are xed rather than remaining subject

to continuous reappraisal based on new evidence

about what matters for development results. One

way in which the PFM system can be envisaged as

more “open” is by reviewing how processes and

conventions can evolve in relation to evidence

and objectives. Could extrapolating signicantly

beyond emergent approaches in adaptation and

learning help to identify new or alternative PFM

conventions and manage the life cycle of existing

ones to better understand linkages with service

delivery and development results (Andrews, 2013;

Pritchett, Samji & Hammer, 2013)?

Experimentation is needed to learn about the

impact of different PFM conventions on the

Chapter 1. Introduction:

Setting the Agenda

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

efcient and effective provision of collective

goods and services. The case is easily made

that strong PFM is necessary if the public sector

is to be able to ensure effective service delivery,

whether through direct provision, commissioning,

or only government funding (see Table 6 in Annex

B for an elaboration of these alternative modes of

delivering public services). Strong procurement

systems contribute to efciency in service delivery

by ensuring better value for money in government

purchases, while effective public investment

management arrangements underpin successful

infrastructure development. Analysts have found it

distinctly harder to specify exactly which features

of PFM, when combined with other management

capabilities, are sufcient to ensure that these

functions are undertaken well. The link between

PFM and service delivery is typically described

without any specic explanations or evidence, and

it often remains generic across sectors with widely

varying characteristics (Welham, Hart, Mustapha

& Hadley, 2017, p. 18).

How can the ways in

which PFM matters for the provision of collective

goods and services be investigated and identied?

Problem 4: The standard objectives and

conventions of PFM are not equipped to

deal with political trade-offs or major

policy change

The pace and scale of political demands for

policy change to respond to new priorities

will test increasingly the versatility of PFM

systems. Recent announcements or proposals to

address climate change and inequality in countries

of the Organisation for Economic Co-operation

and Development (OECD) are emblematic of

some of the radical new policy directions that are

envisaged. Budgeting is inherently an incremental

process, however (Wildavsky, 1992). Efforts to

promote large-scale policy change—such as the

massive investments implied by climate change

mitigation measures (Gilmore & St. Clair, 2018)

or accelerating progress toward the Sustainable

Development Goals (SDGs)

—raise concerns that

PFM is not well-suited to facilitating signicant

reprioritization and reallocation across sectors—

especially at a time of slower economic growth.

PFM processes may need tailoring, or even more

fundamental adaptation, to facilitate these new

policy directions (Schick, 2013).

How could

PFM processes and conventions be re-envisaged

to better support or enable major policy change?

Mounting pressure on politicians to tackle

economic inequality is exposing the tensions and

trade-offs among policy objectives. The erosion

of real incomes (ILO, 2018; Rios-Avila, 2015)—

particularly in a context of declining trust in

government (Manning & Wetzel, 2010) and rising

intolerance—is making countries more vulnerable

to populism and authoritarianism (Dijkstra,

Poelman & Rodríguez-Pose, 2018; OECD, 2018).

Current policy debates across OECD countries

make an empirical case for more progressive,

comprehensive, and redistributive social agendas

(Milanovic, 2016; van der Weide & Milanovic,

2014). The tensions and trade-offs in lower-income

countries are just as acute—and may be rising

despite progress in overall poverty reduction and

favorable economic growth rates.

These tensions

are most apparent in the debates about how to

secure inclusive growth that addresses inequality

at the same time as overall prosperity (Lagarde,

2019). A similar tension exists in how to balance

scal discipline against the need for massive

scaling up of public investment, especially for

infrastructure. Understanding the impact of PFM

on policy objectives beyond the trinity of standard

PFM objectives exposes unavoidable policy

trade-offs. Can the PFM system be envisaged as

more “open” to debates about trade-offs and the

possibility of major policy change?

Guiding framework: an “open”

model of PFM

A more “open” conception of PFM is proposed

as the guiding framework for rethinking

international support for managing public

nance. It would respond to knowledge and

learning from outside the PFM discipline, be

sensitive to the impact of PFM conventions

on public policy (choices, actions, and results)

(Annex B), and recognize the inevitability of

trade-offs among policy objectives. The current

archetype of PFM has been powerful and

productive, creating a culture of inquiry within the

PFM discipline about means and ends—namely

the institutional arrangements for achieving a

given set of objectives. Yet, as revealed by the four

problems identied by the Working Group, that

inquiry takes place within a constricted space. The

objectives that PFM aims to support are limited to

a narrow and standard set formulated in the mid-

1990s. Moreover, the conventions for operational

performance of PFM processes that are considered

necessary to support these objectives follow the

Figure 1: “Opening up” perspectives on means and ends in managing



Means are informed

byslowlyevolving

consensusamong

PFMpraconers

(codiedinPEFA,

DeMPA,TADAT,and

otherframeworks)

Convenonsarealso

informed by

knowledgeand

learningfromoutside

the PFM discipline

Impact of PFM

convenonsonpublic

services and

socioeconomic

outcomes

is considered

Ends are informed by a

xedsetof“desirable

budgetaryoutcomes”

(aggregatediscipline,

allocaveeciency,

operaonaleciency)

Recognionthatthere

aremulplepossible

objecvesrelangtoa

well-performingPFM

system

Possibilityoftrade-os

amongpolicy

objecvesisregarded

as inevitable and

necessary

–– Current “closed”

concepon of PFM

Addional

consideraons in

a more “open”

concepon

Means: the convenons for

operaonal performance of

PFM processes

Ends: the stated objecves

of the PFM system

Source:Authors’formulaon.

Chapter 1. Introduction:

Setting the Agenda

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

consensus of PFM practitioners. The eld of

PFM in development has become focused on

standardized assessments of these conventions,

with less active consideration of the ways in which

the conventions relate to policies and results.

In sum, while the current conceptual model for

PFM has underpinned considerable progress,

its limitations are now evident. It is detached

from the realities of policy making and closed to

new ideas and evidence from outside the PFM

discipline. In response, this report envisages a new

model of PFM, with a more “open” conception,

as the analytical lens for proposed future inquiry

by the PFM discipline and the wider development

assistance community (Figure 1).

Developing some responses

Based on the Working Group’s deliberations,

and in response to the identied problems,

this report sets out seven proposals for a new

direction of travel for the PFM discipline in the

2020s (Table 1). The rst set of three proposals

builds on the strengths of the existing “closed”

model of PFM. These proposals argue for more

rigorous assessment of technical advice offers,

based on evidence supporting the validity of the

intended approach and plans for the dissemination

of results, as well as for establishing practitioner

groups to share sensitive information condentially

so that they can begin to identify ways to improve

performance. In other domains these groups are

known as “data clubs” (Chapter 3). The proposals

also recommend experimenting with “managed

choice” in the provision of technical advice to

give client governments a greater say in deciding

among intervention options. A second set of four

proposals envisages PFM as an “open” system

that learns and interacts with the broader public

policy domain. These proposals argue for a review

of the current conventions embedded in standard

PFM assessment frameworks to test their validity,

and for the collection of data and evidence “at the

frontier” about which aspects of PFM operational

performance matter for service delivery. They

propose investigating how PFM conventions

can better support major policy change through

strategic reprioritization in budget policies, and

expanding the PFM toolkit to analyze trade-

offs among policy objectives and to assess

socioeconomic impact.



Building on the strengths of the current approach to PFM in development

1. Assessoersoftechnicaladviceagainststandardsfortheprovisionofinformaonandevidencejusfyingthe

approachandfortheplanneddisseminaonofdataaboutimpact

2. EstablishpraconergroupstosharesensiveperformanceinformaononPFMintervenonsonavoluntary

andcondenalbasistoidenfywaysofimprovingperformance

3.Experimentwith“managedchoice”intheprovisionoftechnicaladvice,withacoalionoffundersandclient

governmentsandthecreaonofadedicatedfundingfacility

Moving toward a more “open” system for managing public nance

4.ReviewthecurrentconvenonsembeddedinstandardPFMassessmentframeworkstotestevidenceofresults

andthedegreeofapplicabilityinadvancedeconomies

5.UsePFMassessmentframeworkstocollectdataandevidence“atthefroner”tobuildinsightsaboutwhich

aspectsofPFMoperaonalperformancemaerforservicedelivery

6.InvesgatehowPFMconvenonsandprocessescouldbeersupportnon-incrementalpolicychangethrough

strategicrepriorizaoninbudgetpolicies

7.ExpandthetoolkitofPFMtoanalyzethetrade-osamongpolicyobjecvesandtoconsiderthesocioeconomic

impactofscalandbudgetarypolicychoices

Building a strong case for change will be

critical in navigating adverse organizational

incentives in development agencies and making

progress on this new agenda. Through this

report, the Working Group sets out a view on the

current state of PFM and proposes an agenda for

substantive change. While the Working Group

has the benet of an independent perspective

and has been informed by extensive consultation

and engagement, not all readers will agree with

the perspective or the agenda. Even where the

arguments are convincing, vested interests may

not wish to see some changes occur. It is hoped,

however, that the ideas developed and discussed

in the report will, in and of themselves, provide

sufcient countervailing motivation to move

forward with practical actions and the piloting

of initiatives.

The public management community might

require more than technical arguments if

the agenda is to be taken seriously. There is a

strategic rationale for proactive leadership to

ensure the relevance of PFM and make external

support more effective, as a means to counter the

risk of losing ground on institutional strengthening

as a development priority. The primary risk

foreseen by the Working Group is that institutional

reform may lose its place as a prominent point

of entry for improving development results.

It could come to be seen as an expensive and

optional agenda, rather than as a precondition

for sustainable outcomes. The incentives could

diminish for development agencies to focus on

complex institutional reforms that cannot readily

be shown to have tangible results and cannot be

explained in terms of bilateral donors’ national

interest. In this possible future, the emphasis on

country-level institutional strengthening would

likely be displaced by a redoubled focus on direct

interventions in sectors and increasing attention

to global public goods, begging many questions

about sustainability.

While the Working Group contends that its

proposals will contribute to the future relevance

and effectiveness of the PFM discipline, they

do pose nancial and opportunity costs in the

short term. In looking at these proposals from

the point of view of an international development

agency, there is a question about why one agency

would invest in signicant new research and

learning that will benet the eld as a whole,

allowing free riding. Leadership will be needed at

an institutional level, along with some willingness

to direct nancial resources to these ideas or new

approaches with the wider strategic rationale in

mind. Active experimentation and learning will be

required from the outset. Leaders within the PFM

discipline might take up these ideas, and progress

might be kick-started by funding for one or more

of the proposals under a pilot approach.

Organization of the report

In the following chapter, we dene PFM and

take stock of how it has evolved as an area of

work and a focus of development assistance

over the past two decades. We describe the

emergence, conventions, and approaches of the

“discipline” of PFM within the overall eld of

international development. Finally, we review the

standard objectives of PFM and consider the case

for refreshing and expanding them.

In Chapter 3, we discuss how to build on

and improve the existing “closed” model of

PFM. We explore how to drive improvements in

external support and how to make choices among

technical advice options more meaningful. The

chapter considers whether there may be a need

and opportunity to accelerate knowledge sharing

and learning in technical advice for PFM and then

discusses whether increased contestation among

different intervention options and ideas could

drive greater effectiveness and encourage more

meaningful government choice.

In Chapter 4, we examine the possibilities that

may emerge if we envisage PFM as an “open”

system and consider questions regarding how

PFM inuences public policy. The chapter begins

with an examination of how PFM can be related

Chapter 1. Introduction:

Setting the Agenda

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

more directly to service delivery and development

results, and the need for PFM conventions to

evolve in response to new evidence. It then reviews

how PFM can support major policy changes and

promote the consideration of trade-offs among

policy objectives, including socioeconomic

priorities.

Finally, in Chapter 5, we put forward seven

proposals for how to put this agenda into

practice. The chapter concludes with some

propositions for further debate and by addressing

the case for change and the incentives of

development assistance providers.

This chapter provides a framing of the core

subject matter to be discussed through the

report—managing public nance, broadly

dened, and public nancial management

more specically. It sets out key denitions and

takes stock of how PFM has evolved as an area of

work and a focus of development assistance over

the past two decades. It describes the emergence

of a “discipline” of PFM within the overall eld

of international development—complete with its

own set of approaches and “conventions” for the

desirable operational performance of PFM. The

impact and limitations of contemporary ideas about

applying problem-based and adaptive approaches

to PFM are explored. The chapter then reviews

the standard “trinity” of PFM objectives and

considers the case for refreshing and expanding

them. It proposes broadening the objectives to

address additional considerations such as scal

risks, policy change and reprioritization, modes

and quality of service delivery, macroeconomic

stability, fairness and legitimacy, and democratic

accountability.





The phrase “managing public nance” directs

attention toward a large conceptual territory

with little certainty about its boundaries. It

comprises objectives and policies on the one

hand and institutional arrangements and systems

on the other. It covers a broad swath of state

engagement with the economy and society.

Fiscal policy, budgets, budgetary processes,

expenditure management, nancial management,

accountability, and nance ministries all sit within

the domain of managing public nance. Finding

a single organizing framework for all of these

dimensions and elements has proved difcult.

The narrower concept of “public nancial

management” has emerged as a more distinct

and tractable domain within this larger and

conceptually more uncertain context. It is

delimited by the institutional arrangements that

support public nance objectives. In international

development circles, PFM has become an

increasingly proprietary term that is “owned” by

a distinct group of professionals. While there is no

consensus, this report follows a recent formulation

by the International Monetary Fund (IMF):

PFM in the narrowest, and perhaps most

traditional, sense is concerned with

how governments manage the budget

in its established phases—formulation,

approval, and execution.…[I]ts relevance

in scal policymaking has evolved

over time,…broadened its focus…to all

aspects of managing public resources,

including resource mobilization and debt

management, with a progressive extension

to the medium- to long-term implications

and risks to public nances from today’s

policy decisions. The coverage of PFM has

thus expanded from the narrowly dened

central government budget to all levels of

government and the broader public sector,

including state enterprises and public-

private partnerships. Furthermore, PFM

is now seen as an ‘umbrella’ denition,

covering a set of systems aimed at

producing information, processes, and

rules that can help support scal policy-

making as well as provide instruments for

its implementation. (Cangiano, Curristine

& Lazare, 2013b, pp. 1-2)

Using this broad denition, PFM refers to

the systems that connect public resources

with social and economic policy while serving

the wider nancial purposes of accounting

and accountability. In its economic and policy

functions, PFM supports analysis and decision

making for government allocation, distribution,

and stabilization objectives and choices—

principally through the budget (Musgrave &

Musgrave, 1989). In its nancial functions, PFM

covers the use of money for public purposes. It

2. MANAGING PUBLIC FINANCE: A REAPPRAISAL

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

involves an accounting function for revenue and

expenditure and a system of accountability to

support public scrutiny (Hughes, 2018).



context

PFM is not an independent variable. In seeking

to improve development results, factors such as

the wider public management systems in play,

the social and political environment, and the

capacity of government cannot be disregarded. In

considering the long chain of connections linking

government policy choices to development results

(Annex B), there is no institutional dimension

which is primus inter pares.

Broader developments in public

management

The 1990s were a momentous time for public

management in general. Dominant ideas about

how to improve public sector performance,

however that slippery term may have been dened,

were changing—and not just in Anglophone

advanced economies. Prior to that period, views

on improving public sector performance were

essentially “managerialist,” with the idea that

organizations would perform as well as they were

managed and that similar managerial skills were

needed in the public and private sectors (Drucker,

1974). These ideas had been dominant since the

1970s and had themselves seemed revolutionary,

a promising way to open up the previously “self-

contained world” of public administration and

its arcane traditions (Sayre, 1958). Over time,

however, this approach proved unconvincing in

the face of the evident difculties in reforming

large public bureaucracies.

By the 1990s, new

thinking had emerged that the route to public

management improvements lay in rethinking

the institutional arrangements that underpinned

them—the structures and incentives of the public

sector. Research insights about the nature of

public sector institutions (Bakir & Jarvis, 2018)

contributed to new ideas in the public sectors

of advanced economies (Gore, 1995; OECD,

1995) and in developing countries (World Bank,

1997). How public ofcials approached their task

remained important, but this was a time when the

formal and informal rules that motivated those staff

were now thought to be both more fundamental

and more tractable.

This rewiring of public management was more

evolution than Copernican revolution. While

institutional reform became more central as the

point of entry to public sector improvements, not

everything changed. The shift toward institutional

innovation, and away from the stability and

certainty of merit and hierarchy, left in place

several assumptions that still inuence PFM

thinking today. For example, the general approach

to PFM has continued to assume that improvements

can be achieved by lling institutional gaps; that

the public sector is homogenous enough that

lessons concerning policy implementation are

generalizable across sectors, implying that what

has worked in health could work in education;

and that the public sector runs on somewhat

mechanical principles—get the machine right and

it can execute any feasible and legitimate political

directives.

The implications of country context

Allowing for country context and other

distinctive political economy factors has always

been challenging for reform initiatives. The

public sector reform menu during the last three

decades has been extensive (Annex C). But

critics of this period of busy institutional change

point out that, while much was being learned,

the study of public management had become

unhelpfully “decontextualized” (Pollitt, 2008).

They argued that the eld was prone to offering

ill-tting reforms in the name of modernization.

The obvious challenge for those seeking to

learn lessons has always been that, if context is

dominant, few principles can be applied beyond a

high level of generality.

Political and contextual factors mean that

the same institutional arrangements for PFM

may have different effects in different settings.

Although some observers suggest that recent

progress toward a normative eld of public

management has been limited because of the

problem of understanding and controlling for

context (Roberts, 2018), there has been some

progress within the development community—at

least in scoping the territory. Research shows that

seemingly similar institutions work differently

in different contexts. For example, social

accountability and “demand-side” mechanisms

can channel local expenditure exibility toward

higher-priority results, or they can be blocked

(Ringold, 2012; Sheely, 2014),

or they can

be actively distorted in the presence of non-

programmatic political parties and vote buying

(Keefer, 2013; Khemani, 2013).

In the same

vein, similar budgetary institutions can support

ministerial accountability or, by contrast, allow

overspending (Alesina, Hausmann, Hommes &

Stein, 1999).

Political context also affects PFM reform

possibilities. It can be harder, for example, to

implement “basic” reforms that focus on control

of public expenditures (including commitment

control, prevention of leakages in procurement and

payroll, or reliable accounting) in political contexts

where leakages benet well-connected interest

groups than it is to overcome more advanced

technical challenges such as the introduction of

medium-term expenditure frameworks (MTEFs)

(Fritz et al., 2017). Further down the results

chain, Batley and McLoughlin (2015) show that

the political challenges to service delivery reform

are affected by the characteristics of the particular

sector and by the legal assignment of functions

to different levels of government. All in all, it is

hardly surprising that general principles are hard

to nd.

The role of government capacity and

capability

There is a large and occasionally rather

abstract literature concerning the development

of government “capacity,” with a growing focus

now on building the “capability” of nance

ministries. While there are no stable denitions

of either term (Morgan, 2006), capacity can

be usefully seen as the endowment of human,

nancial, technical, and physical resources

available to an organization or a government, while

capability captures the practical reality of whether

or not that capacity results in the achievement of

specic tasks. Recent research has pushed beyond

the traditional focus on capacity inputs, and

whether that capacity should be built or otherwise

acquired, to emphasize the functional capability

of nance ministries as an essential ingredient

of effective PFM (Allen & Grigoli, 2012; Allen,

Grigoli & Howard, 2011; Allen & Krause, 2013;

Dressel & Brumby, 2009; and Krause, Hadley,

Mustapha & Welham, 2016).

There is a risk that denitional uncertainties

leave capacity and capability as the default

culprits when performance failures cannot

otherwise be explained. Such an approach

cannot identify specic changes that need to be

made (Zafarullah & Huque, 2012). That said,

the growing literature on these issues is usefully

highlighting neglected factors in the “production

function” of a central nance agency, “such as the

culture, morale and incentives of the organization;

the quality and experience of its management

and staff; the organizational structure; the

business processes that support and underpin the

organization; and the supporting information and

human resource management systems” (Allen et

al., 2011, p. 4).

PFM as a “professional discipline”

and its conventions

Public nancial management is its own

distinctive eld in international development.

This section considers the reasons for that

distinctive signicance and what underpins it. It

argues that PFM can be viewed as a professional

“discipline” with its own conventions, which have

a powerful effect on how the technical advice

provided by PFM experts is understood, codied,

and deployed.

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

The importance of PFM in development

assistance

Donor countries champion and support PFM

for the rather obvious reason that how public

nances are managed plays an important part

in the achievement, or not, of development

results (Andrews, 2007; Cangiano et al.,

2013b). There is little dispute about the essential

features of PFM institutional arrangements that,

at a minimum, provide nancing for sectors in

line with policy objectives, ensure timely and

predictable resource ows to agencies and between

levels of government, and assist in the effective

management and use of resources at the facility

level (Andrews, 2007; Cangiano et al., 2013b).

These contributions to service delivery are

reasonably assumed to contribute, by extension,

to the achievement of sector outcomes. There

is a matching consensus among development

agencies that arrangements for ensuring aggregate

scal discipline are the key to the long-term

sustainability of government policy actions to

improve service delivery and development results.

As discussed below, however, there are some signs

of growing uncertainty about the right sequencing

of these two objectives.

Donor countries have placed a greater emphasis

on the importance of PFM in lower-income

countries since the 1990s.

Ofcial nance

providers are prepared to invest considerable

effort in assessing the performance of PFM

arrangements, seeking preemptive assurance that

nances will be used as intended and used well.

Arguably, they sometimes trade off development

impact against duciary risk when sacricing

programs that are effective but not accountable

enough (Bain, Booth & Wild, 2016; Derviş,

Chandy, Kharas, Medler & Unger, 2011).

The

move toward general budget support in the

2000s—and then away from it again in the 2010s—

underscored concerns about managing duciary

risk (de Renzio, 2006). Limited risk tolerance on

the part of donor countries, particularly during the

time of austerity and heightened skepticism about

the utility of foreign aid following the global

nancial crisis of 2007–08, has undoubtedly been

a major impetus for the recent focus on PFM.

The scale and composition of international

support for PFM

Measured in nancial terms, PFM is

a substantial program of development

cooperation activity. Recent research shows the

trends in development nance, including ofcial

development assistance (ODA) and other ofcial

ows (OOF) from governments and multilateral

organizations, directed toward technical assistance

for PFM (AlphaBeta, 2018). Rapid growth in the

rst part of the 2000s remained buoyant into the

2010s, with average annual spending of US$ 1.86

billion globally between 2012 and 2016 (Figure 2).

The picture is uneven across geographical areas,

with absolute spending per year in Latin America

increasing by 217 percent between 2002–06 and

2012–16 and overtaking the amounts directed to

Sub-Saharan Africa, which increased by only 6

percent in absolute terms over the same period.

International support for PFM has grown faster

(86 percent) than overall volumes of development

nance (69 percent) over the same period,

suggesting an uptick in PFM’s relative importance.

If general budget support and other programmatic

nancing linked to strengthening the scal sector

were to be included, the gures for PFM support

would be even higher.



(US$ billions)

Source:AlphaBeta(2018).

Notes on Figures 2 and 3

External sources of nancing for PFM reform: Disbursements from external sources in support of PFM

reformwereobtainedfromasampleof18countries.Forthosecountries,in-depthanalysisandadjustment

ofexternalfundingdataforPFMunderthe15111codeontheDACdatabasefrom2002–16,usingaddional

sourcessuchasdonorprojectdocumentsandalternavedatabases,revealedthat46%ofdonor-reported

projectdisbursementsover2002–16wereinfactPFMreform-relatedspending.Theremainingdisbursements

wereremovedfromthetotals.

Analysis of internal spending in Figure 3: The totalsspentby governmentsfromdomesc sourceswere

derivedfromanalysisof7ofthe18countriesandesmaonsfortheother11.

Denion of reform areas: See endnotes.

Figure 3: Internal and external funding by PFM reform area for

a basket of countries (US$ billions)

Source:AlphaBeta(2018).

Chapter 2: Managing Public Finance

A Reappraisal

Core FMIS

infrastructure

Budget execuon

2002-2006 2007-2011 2012-2016

Budget

formulaon &

authorizaon

Budget evaluaon

Expenditure side

reforms

Revenue-side reforms

Accounng and

reporng

Treasury Single

Accounts

Forecasng and

strategic budgeng

PFM legislaon

External audit and

monitoring

Internal audit and

control

External funding (ODA & OOF) for PFM reform 2002-16 in

nominal terms

US$ billions

$10

$5.0

2002-2006

$8.5

2007-2011

$9.3

2012-2016

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Recent attempts to disaggregate PFM reform

spending indicate priority areas and dominant

funders. The most signicant areas of spending

within PFM—looking at a sample of 18 countries

and including governments’ own allocations

as well as development assistance—have been

core nancial management information system

(FMIS) infrastructure and budget execution

reforms in procurement, service delivery, and

payroll (Figure 3). These funding gures may

understate the volume and relative signicance

of external support for PFM, as they include

the OECD Development Assistance Committee

(DAC) category of “public nance management”

but exclude the separate categories for domestic

revenue mobilization, local government nance,

and public procurement (AlphaBeta, 2018).

Yet

all of these categories are considered as part of this

report’s “umbrella” denition of PFM, following

Cangiano et al. (2013b). The most signicant

providers of external nance for PFM, by some

considerable margin, are the World Bank, Inter-

American Development Bank (IADB), and Asian

Development Bank (ADB). Despite the prominent

role and inuence of IMF advice on PFM, the

nancial scale of its capacity development support

is modest (AlphaBeta, 2018).

There is some evidence that PFM is becoming

a larger component of sector assistance

programs as well. The World Bank, as the largest

PFM assistance provider, is a guide. The growing

relative signicance of PFM as an area of focus

for development assistance is illustrated by its

share in other World Bank portfolios (Annex D).

In health and education programs, PFM has grown

to a share of roughly 9 percent over the past 15

years, and in governance it has held steady at

around 50 percent over the past decade.

Data on the volume and disaggregation of PFM

support are interesting, but also unreliable.

Despite apparent trends, particular caution is

required in using or interpreting the data, as there

are well-known problems of collection, recording,

reporting, and comparison. The overall quality

of available data is poor, and this remains a live

research agenda (Annex E). For full discussion of

these data, see AlphaBeta (2018).

The PFM “discipline” in international

development

The emergence of PFM as a professional

discipline

An academic or subject discipline is a common

language, spoken by a select group.

comprises a focus area for discussion, concepts

and theories for organizing that discussion, and

some institutional arrangements for imposing

discipline by correcting or at least identifying

those who have erred.

A professional discipline goes one step further;

it is an academic discipline plus career paths

with opportunities for advancement in work

and reputations (Nikitina, 2005). As such, it

brings together a “sociology of the profession—

career incentives, norms, socialization patterns”

(Naidu, Rodrik & Zucman, 2019, p. 1).

International development has few professional

disciplines. This is not to say that development

is without expertise or research, but it has few

distinctive bodies of knowledge and even fewer

that combine such knowledge with distinct career

paths. Agronomists and epidemiologists draw on

rich bodies of academic work, but do not form

large cadres within development organizations

with clear lines of career advancement. Economics

is the most evident professional discipline found

in the development eld (Stein, 2008). Although

still some distance behind economics, PFM is

emerging as a distinct professional discipline

in international development, with a common

language and an associated career path.

PFM experts act as the guardians of the

professional discipline

PFM expertise is carried by a distinct body of

staff within the development community. Much

of the expertise in PFM in ofcial development

agencies is concentrated in the IMF, World

Bank, IADB, and other international nancial

institutions (IFIs). The degree to which these staff

adhere to both explicit technical knowledge and

tacit knowledge is noted in the IMF case, with a

recent high-level review group concluding that

“…part of the Fund’s success has been that it is

a strongly normed organization, with many staff

deeply committed to the mission and mandate of

the Fund, to professional and disciplinary norms,

and to the often implicit culture and norms of the

organization” (IMF, 2018a, p. 6). Many staff in the

IMF Fiscal Affairs Department are former nance

ministry ofcials and most are economists by

training, with both factors serving to reinforce the

sense of PFM as a select professional discipline.

The nancial management and procurement

cadres in the World Bank form “chapters” of

the PFM discipline. The interchange of staff

between IFIs and client country nance ministries,

especially from developing countries, strengthens

the disciplinary connections (Fritz, Fialho Lopes,

Hedger, Tavakoli & Krause, 2011).

Once located within international cooperation

agencies, many PFM experts remain there

and envisage completing their careers there.

In an exploratory survey of self-identied PFM

specialists undertaken for this report,

the

majority of respondents identied most closely

with general nancial management, economics,

or governance elds. Their related professional

experience was gained almost equally across

public sector, consulting, international

organization, and academic work. Respondents

were overwhelmingly senior (with 15 or more

years of experience), and most had spent their

careers in the public sector or in an international

development organization. Most of those who

responded to the survey stated that the nature

of their work was as staff of a single, generally

multilateral, organization (as opposed to as an

expert working with multiple organizations). A

majority of respondents envisaged their future

careers continuing in an international development

agency.

Beyond the ofcial development agencies, a

large cadre of international PFM experts are

located in consulting and accounting rms,

universities, research institutes, and think

tanks. Although not revealed to a great extent

by the survey, these experts are prevalent in the

extensive and growing literature on PFM issues.

There is also a large community of independent

PFM experts, or those with multiple afliations,

who work as technical advisers on projects and

other assistance interventions in lower-income

countries.

Experts from all of these different constituencies

within the PFM discipline learn from each

other. Respondents identied the views of

counterparts and other stakeholders, along with

their own professional practitioner experience,

as the most signicant in forming their views.

Almost all survey respondents were familiar with

formal assessment frameworks (Annex F). The

most commonly cited were PEFA, the IMF Fiscal

Transparency Code, and the Tax Administration

Diagnostic Assessment Tool (TADAT).

Strong professional associations offset some

gaps in theory and evidence

PFM experts have a common professional

language and frame of reference, underpinned

by three structural features of PFM. First, despite

many political and constitutional differences,

similar PFM processes have developed in most

countries, represented by an annual cycle of scal

decision making, implementation, and review

centered on the budget process.

While there are

many complicating factors, not least that these

stages overlap and that multiple agencies are

involved in each one (Andrews et al., 2014), the

annual milestones in the budget cycle provide a

way of structuring discussion about PFM. Second,

strong forms of professional association have

emerged for specic subgroups in PFM, such as

auditors and accountants (Squire & Beazley, 2016;

Unegbu, 2014). Professional bodies set widely

accepted standards for individual and organizational

behaviors, ranging from the International Public

Sector Accounting Standards

to the Standards

for Internal Controls.

Disciplinary enforcement

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

is primarily reputational, but some additional

force is given by the membership basis of some

of these organizations—usually at the individual

level and with the possibility, at least in theory, of

cancellation.

Third, the articulation of standard

PFM objectives in the late 1990s (described below)

has had a signicant inuence in establishing and

consolidating a sense of shared purpose among

PFM experts.

The challenge for the PFM discipline is to

provide practical guidance, because the

“institutions matter” mantra alone does not

get us far. The underpinning theory for PFM is

at the same high level of generality as in other

governance domains, all of which have found

it difcult to dene a practical approach to

uniting concerns for due process, efciency and

economic growth, and politics and inclusion

(Andrews, 2008a, 2008b; de Renzio, 2013; Plant,

Stalebrink & Vasavada, 2010; Thomas, 2010).

The evidence base concerning what works where

and under what circumstances remains too patchy

to supplement the theoretical shortcomings,

and

the PFM discipline thus nds itself without a strong

theoretical or empirical anchor for its practical

recommendations (Key, 1940). In response, it has

found two ways of navigating past these gaps to

develop recommendations for action: focusing on

a general set of conventions for the operational

performance of PFM processes and, increasingly,

emphasizing local, context-specic problem

solving as an approach to PFM reform.

Strong conventions as the mainstay

of PFM

The specication of a set of conventions

concerning the desired operational performance

of PFM processes has emerged as the dominant

disciplinary approach. The increasing emphasis

on consensus-based conventions in PFM is

reected in the swelling numbers of assessment

frameworks and tools (Annex F; PEFA Secretariat,

2018b). Their proliferation does not signal an

end to contentious debates such as that on the

degree to which performance incentives should

be emphasized in PFM. Some of the dimensions

covered in these frameworks are based on

internationally agreed standards, but many are not.

While the organizations or groups responsible for

devising and rening the frameworks are clear that

they are intended as a basis for discussion rather

than as conclusions regarding actions that need to

be taken,

they are undoubtedly normative.

These frameworks and the conventions

they reect provide a powerful and succinct

codication of the tacit knowledge of many

generations of practitioners, gained over

many years and in many contexts.

Proposing

conformity with the conventions is tantamount to

learning from others about how to strengthen the

“scal machinery” that is put in place to address

the common pool problem in public expenditure

management (Annett, 2002; Hallerberg &

Yläoutinen, 2010; von Hagen & Harden, 1995;

Wyplosz, 2011).

A similar picture emerged

for tax administration in the 1990s and 2000s,

where a standard set of reforms such as semi-

autonomous revenue authorities developed based

on ideas surrounding new public management and

observed practices in advanced economies (Dom

& Miller, 2018).

While there has long been some

acknowledgment that different PFM objectives

require different emphases in PFM systems,

there has been an equally long-held assumption

among technical assistance providers that, for

all practical purposes, the PFM conventions are

independent of any sector policies and objectives

(Premchand, 1993).

A specic denition and usage of “PFM

conventions” is applied for the purposes of

clarity in this report. We use PFM conventions to

refer to the normative assumptions about desirable

“operational performance of key elements of the

PFM system” based on “international standards

and good practices on crucial aspects of PFM, as

identied by experienced practitioners” and as

expressed in standard PFM assessment frameworks

such as PEFA and TADAT (PEFA Secretariat,

2016, pp. 1–4). Those assessment frameworks

typically include graduated performance scores.

There is a normative assumption that a stronger

and more capable PFM system is one in which

processes and procedures fulll requirements

at the top end of the measured scales. The PFM

conventions might thus be represented by the

highest-scoring attributes captured by the various

assessment frameworks. A distinction is drawn

here between the PFM conventions, the PFM

assessment frameworks that incorporate and

represent them, and the PFM processes that form

the technical building blocks for the overall PFM

system (Annex A).

While the PFM conventions are generally

viewed in terms of their contribution to a

standard set of PFM objectives, evidence to

support that relationship is limited. Various

studies nd no statistically signicant relationship

between a PEFA-based measure of overall PFM

quality and the scal balance—for both non-

fragile and fragile states (Mustapha & Long,

2019; Fritz, Sweet & Verhoeven, 2014; Alesina et

al., 1999; von Hagen & Harden, 1995). Evidence

is more positive on the impact of MTEFs on scal

discipline (World Bank, 2012a). The imposition

of scal rules (numerical limits on budgetary

aggregates), driven by recent scal consolidation

experiences (IMF, 2010; Lassen, 2010; OECD,

2010), has had at best a marginal impact on actual

behaviors. Finally, countries that credibly enforce

penalties for non-compliance do tend to collect

more taxes (Mustapha & Long, 2019).

Problem-based and adaptive approaches

Problem-based and adaptive approaches are

increasingly supplementing the disciplinary

emphasis on PFM conventions. The “new

realist”

approach did not originate in PFM but

has found particular resonance there (CID, 2014;

Andrews, 2013; Andrews, Pritchett & Woolcock,

2012; Blum, Manning & Srivastava, 2012; Booth,

2014; Booth & Unsworth, 2014; World Bank,

2000, 2012b). The approach rests on two insights.

First, it is hard to predict what will change the

behavior of public ofcials in a given context.

Conformity with the PFM conventions might

have a positive impact on behavior or, given

complex local conditions, it might not. Second,

it is expensive and difcult to nd out what is

really happening. We can perhaps see what the

“concentrated agents” at the center are doing, but

it is hard to see what behaviors may be changing

among “distributed agents” such as “budgeters,

accountants, and such in sector ministries,

provinces, and districts” (Andrews, 2014, p. 1).

The information problem undermines the

“conformity as lesson learning” idea at both

ends. On the theory side, the behaviors and

procedures identied as preferred standards

might represent the distilled essence of what

was claimed rather than what was done. On the

practice side, the emulation of those changes might

not be genuine, with changed rules disguising

unchanged behaviors. Proponents of a problem-

based and adaptive approach argue that, while

an understanding of performance failures can be

informed by wider lessons and experiences, the

specic causes and purposes of a given apparent

dysfunction have to be determined case by case

through “guided incremental experimentation”

(Pritchett et al., 2013, p. 48). They assert that

institutional reforms are addressing “adaptive”

rather than “technical” problems.

In its

draft guidance on improving PFM, the PEFA

Secretariat (2019, p. 27) highlights contingency

to country context in deciding whether a formal

comprehensive reform strategy or an informal

iterative approach would be more appropriate.

These approaches have emerged amid an

increasingly nuanced understanding of the

signicance of political context. The insight

that PFM is entangled with politics is not new.

There has long been discussion of “big politics,”

with identiable elites driven by the self-interest

of remaining in power or in ofce and of self-

enrichment. That phenomenon is well-documented

in weak governance environments with “extractive

institutions” (Acemoglu & Robinson, 2012) or in

“limited access orders,” in which the consensus

among elites about rent distribution is unstable

(North, Wallis, Webb & Weingast, 2007). There

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

is also a growing awareness of the signicance of

the “small politics” reected in interministerial

rivalries, union concerns, and cadre and

bureaucratic rivalries.

Recent work in this area

is moving beyond broad analyses, pointing the

way toward more grounded theories about why

institutions are working as they are and about

how change might be feasible (Booth, 2011; Fritz,

Levy & Ort, 2014; Fritz et al., 2017; Yanguas &

Hulme, 2014).

Bridging conventions and adaptation

Using PFM conventions as reference points in

assessing performance and following adaptive

approaches to reform design need not be in

tension. Adaptation and agnosticism are not

necessarily good things in themselves. They

are part of the general tactical case for cautious

incrementalism in reforms, along the lines of

the observations made by Lindblom (1959)

concerning “muddling through” within large

bureaucracies and by Pressman and Wildavsky

(1973) on policy implementation. Those are

rational ways of managing complexity and the

uncertainty inherent in predicting the exact

consequences of reforms at each stage. But

reforms have to begin somewhere. It is necessary

to “develop initial responses which are then

modied over the life of the project” (Brinkerhoff

& Ingle, 1989, p. 490). After all, it is “difcult

to hold the attention of those eager for progress

and clear answers with responses that amount to

‘Well, I’m not sure, but let’s explore this more and

perhaps we can generate some ideas…’” (Grindle,

2013, p. 400). Conformity with conventions can

provide that starting point.

“In reality, reform

strategies need to be both problem-driven and

solution driven” (Allen, 2018, p. 402). This is a

point echoed in a recent study of PFM reforms in

eight East Asian countries, which concludes that

“‘innovative’ means—by denition—not holding

unwaveringly to predetermined plans and not

insistently using standard techniques in standard

ways” (So, Woolcock, April, Hughes & Smithers,

2018, p. 4).

Guiding objectives of PFM and their

evolution

The purpose of PFM must drive the content

and the approach. The PFM discipline offers

its own distinctive set of objectives which the

discipline holds up as the ultimate justication for

the conventions it recommends. These standard

PFM objectives have come to be seen as universal,

transcending the specicities of country and

context. There is a real world case for refreshing

and expanding them. This section proposes a

broadening of the objectives to address additional

considerations such as scal risks, policy change

and reprioritization, modes and quality of service

delivery, macroeconomic stability, fairness and

legitimacy, and democratic accountability.

The rise and dominance of standard

PFM objectives

PFM has achieved a broad normative

consensus that conformity with the established

conventions is necessary, if not sufcient, to

achieve a standard set of objectives. Over the

last 25 years, PFM reforms in lower-income

countries have been widely assessed in terms of

their contribution to what is often referred to in

the literature as “desirable budgetary outcomes”

(Campos & Pradhan, 1996; PEFA Secretariat,

2016; Schick, 1998a; World Bank, 1998).

Although originally described merely as desirable

outcomes, they have come to dene the purpose

of the PFM system and have become the de

facto objectives. They are widely viewed as the

gateway through which PFM contributes to other

government objectives.

In one of their classic formulations, the three

standard objectives are:

• “Aggregate scal discipline – Budget totals

should be the result of explicit, enforced

decisions; they should not merely accommodate

spending demands. These totals should be set

before individual spending decisions are made,

and should be sustainable over the medium term

and beyond.

• Allocative efciency – Expenditures should

be based on government priorities and on

effectiveness of public programs. The budget

system should spur reallocation from lesser to

higher priorities and from less to more effective

programs.

• Operational efciency – Agencies should

produce goods and services at a cost that

achieves ongoing efciency gains and (to the

extent appropriate) is competitive with market

prices” (Schick, 1998a, p. 2, developed from

the original framework by Campos & Pradhan,

1996).

These objectives emerged originally to

cover public expenditure management—

as a narrower domain than contemporary

PFM. It is useful to recall that these objectives

were the product of their time. They reected a

preoccupation with scal stability, top-down

expenditure control, and efforts to bring order

and restraint to bottom-up budgeting approaches.

They emerged as attention was beginning to shift

from government policies toward the institutional

arrangements that enabled or incentivized policy

making and implementation. Most particularly,

they came into being at a time when a normative

version of the “Washington Consensus”

(Williamson, 2004)—arguably differing from

its original intention to observe rather than to

recommend—was ascendant. The objectives

were led by an emphasis on scal discipline and

supported by a view, emerging at the time, that

incrementalism might need to be challenged in

order to achieve this discipline.

This standard trinity of PFM objectives is

strongly value-laden, but was not intended

as hierarchical. The objectives were originally

envisaged as interdependent, without any

hierarchy or proposed sequencing (Campos &

Pradhan, 1996; World Bank, 1998). Contrary

to this vision, however, they are often treated

in practice as three levels of priority, with scal

discipline at the apex. This point is illustrated

by the proliferation of numerical scal rules and

targets over the past 15 years, which “have often

been seen as a means for addressing all scal

problems—chiey rising decits and debt levels

but also accountability and responsibility gaps—

while relegating other vital PFM elements to a

supporting role” (Cangiano et al., 2013b, p. 5).

Some institutional arrangements, including scal

rules, scal councils, multiyear frameworks,

and accrual regimes, actively reinforce scal

discipline as the primary and dominant concern.

This shift arguably represents a misreading of

two formative case studies from the 1980s and

early 1990s—New Zealand and Australia—which

heavily inuenced the objectives, but for which

the relative emphasis among objectives differed

(Campbell, 2001; Keating & Holmes, 1990;

Schick, 1996; Scott, Bushnell & Sallee, 1990).

These standard objectives have a powerful grip

on PFM work, and addressing scal discipline and

related considerations of nancial control is often

the recommended rst step for reforms (Brooke,

2003; Diamond, 2013). The consequence of this

assumed rank ordering of the objectives is that

the focus on scal discipline has the potential to

override values such as improving public services

or moving toward more equal income distribution.

Discussion of these PFM objectives should not

be confused with government policy choices

in the context of their domestic public policy

framework. We refer to PFM “objectives”

throughout this report because that is how they

have been dened and described in the public

nance literature. Annex B sets out a framework

for domestic public policy, and it is clear that these

PFM objectives are at a distinctly higher and more

generic level than the choices, actions, and results

that one particular government might make, or

wish to make. It is more accurate to view the

objectives as an orientation for the PFM discipline.

They are a way of structuring problem analysis

in public nance (Dunn, 1988) and a means to

reduce complexity through a set of concepts that

“simplify and abstract from reality with limiting

assumptions” (Love & Stockdale-Otárola, 2017,

p. 17).

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

The objectives of scal discipline, allocative

efciency, and operational efciency have been

widely adopted and applied by the international

PFM community. The consensus around the

three objectives has been entrenched by their

incorporation into standard PFM assessment tools

such as the PEFA framework (PEFA Secretariat,

2016). They have been highly effective in shifting

attention from theoretical concerns to real-world

recommendations for PFM reforms (Andrews,

2007; PEFA Secretariat, 2016). They have proven

remarkably durable, and their original formulation

remains largely unchanged in substance almost 25

years later. Allocative efciency tends now to be

described as the “strategic allocation of resources”

and operational efciency has become “efcient

service delivery” (PEFA Secretariat, 2016), but

they appear to have changed little in substance

from their formulations two decades ago (Schick,

1998a; World Bank, 1998). Although PEFA now

refers to scal risks in its description of scal

discipline (PEFA Secretariat, 2016), it is much less

apparent that the explicit focus on service delivery

has yet succeeded in addressing the dimensions of

quality and accessibility alongside efciency.

The standard objectives remain valid but



PFM is often presented as a set of technical

processes that are neutral to particular policy

directions. The approach to managing public

nances that has held sway for some three decades

views a relatively standard set of institutional

arrangements as essential in limiting the ability of

politicians to use revenues raised from the many to

satisfy the few and thus deny them corresponding

opportunities to privilege their base and overspend

(von Hagen, 2006). The dominant “institutional”

perspective that emerged from studies of budgeting

in OECD countries in the 1990s sees the budget

primarily as an instrument to control decits

and debt levels through a set of institutional

arrangements to negotiate and adjudicate among

competing interests (von Hagen, 1998; von Hagen

& Harden, 1995). Some consensus emerged that

a common “scal machinery” (Premchand, 2000)

for achieving this could be identied and that this

machinery was policy-neutral: it would be t for

purpose across a range of policy directions and

institutional settings.

In reality, the PFM system cannot easily be

disconnected from policy for two reasons:

the need to be responsive to policy change

and the need to support fairness in outcomes.

The standard PFM objectives are silent on these

challenges. This is not a criticism—the objectives

have performed strongly as a guide to expenditure

management and as a framework for budgeting,

and they have set the scene for practical

considerations about the nature of the institutional

arrangements needed to achieve the objectives.

But it is becoming clear that these challenges

cannot be ignored.

Major global and national policy priorities—

such as climate change, inequality, and

infrastructure development—are demanding

a fundamental reorientation or rebalancing

in public nance and how it is managed. By

contrast, PFM systems are designed to orient

scal outcomes toward caution and inertia. PFM

conveys norms that may predispose policy makers

toward short-term scal restraint or retrenchment

in pursuit of stabilization. At the same time,

PFM has always embodied a tendency toward

“incrementalism” in budget formulation, not least

because of the complexity in costing and nancing

signicant policy reorientation (Wildavsky, 1992).

Incremental changes are a pragmatic response

to complexity and a way to reduce conict over

resource allocation. The advent of multiyear

frameworks and accrual regimes has reinforced

multiyear continuity and stability in PFM.

The power of the standard trinity of objectives

has derived from their seemingly neutral

and universally relevant nature. There are

presumably no circumstances under which scal

indiscipline, allocative inefciency, or operational

inefciency would be desirable. Yet policy agility

in the face of pressing economic, social, and

environmental challenges, and the distribution

of income to address questions of fairness and

legitimacy, are also important. The current PFM

objectives are not technically disputed or wrong.

The problem is that they are incomplete, and the

prominence that their success and inuence have

given them is stiing the attention needed to other

valuable socioeconomic objectives.

Updating and expanding the standard

PFM objectives

Recognizing the reality of multiple

perspectives on PFM

Despite the prominence of the standard PFM

objectives, underpinned by an extensive

literature, there is no consensus about their

theoretical basis. At least three different

perspectives have informed thinking about the

purpose and development of PFM. The dominant

inuence on the standard formulation of objectives

has been the institutional perspective (Campos

& Pradhan, 1996; Schick, 1998a; von Hagen,

1998; von Hagen & Harden, 1995), which draws

on the insights of public choice theory and new

institutional economics. Yet de Renzio (2009,

2013) highlights two alternative ways to interpret

budgeting objectives. An administrative perspec-

tive emphasizes the nancial functions of PFM

through management systems and associated

integrity and compliance mechanisms (Cangiano

et al., 2013b; Diamond, 2013; Hughes, 2018). An

economic perspective draws on public economics

to consider scal and budget policies (Musgrave

& Musgrave, 1989). While there are some areas

of overlap among these three ways of considering

PFM, their differing content provides a prima

facie case for considering whether to expand the

standard objectives.

PFM has always been broader than the

budget process, which is the main focus of

the dominant institutional perspective, and it

has continued to expand in its scope since the

1990s. Beyond budgeting itself, the PFM system

represents the regulations and procedures needed

to ensure compliance with budgetary decisions.

Accordingly, “management systems must also

be in place to execute the budget in line with

resource allocation decisions, (i.e. resource use is

accounted for, monitored, reported and audited);

treasury operations exist so this execution takes

place smoothly in terms of cash availability; and

resource use is nanced efciently through debt

management operations” (Diamond, 2013, p. 12).

The purview of PFM has also grown markedly in

recent years to incorporate additional dimensions

and domains. These include resource mobilization

(in other words, the “revenue side” of the budget),

to a “progressive extension to the medium- to long-

term implications and risks,” and to an expansion

“to all levels of government and the broader public

sector, including state enterprises and public-

private partnerships” (Cangiano et al., 2013b, pp.

1-2). Heightened attention to scal risks, scal

transparency, and whole-of-government scal

activity—especially following the global nancial

crisis—has been a major driver of this expanding

scope (IMF, 2018b).

The economic functions of PFM have been

less prominent over the past two decades,

especially outside of OECD countries. The

classic public nance literature is clear about their

centrality. Public nance policy is about the role

of government in the economy and classically

covers the objectives of allocation, distribution,

and stabilization (Musgrave & Musgrave,

1989). Budget policy in the form of revenue and

expenditure policies, and broader scal policy

including public debt and surplus/decit levels,

both concern the choices and actions taken by

government in pursuit of these objectives.

In considering the expansion and updating of

the standard PFM objectives, it may be useful to

step back and consider the objectives of public

nance more broadly. Public nance objectives

are classically dened in another set of three:

• “The provision for social goods, or the process

by which total resource use is divided between

private and social goods and by which the mix

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

of social goods is chosen. This provision may be

termed the allocation function of budget policy.

• Adjustment of the distribution of income and

wealth to ensure conformance with what society

considers a ‘fair’ or ‘just’ state of distribution,

here referred to as the distribution function.

• The use of budget policy as a means of

maintaining high employment, a reasonable

degree of price level stability, and an appropriate

rate of economic growth, with allowances

for effects on trade and on the balance of

payments. We refer to all these objectives as the

stabilization function” (Musgrave & Musgrave,

1989, p. 6).

The three public nance policy objectives, like

the trinity of objectives in the PFM literature,

reect the times in which they were developed.

The notions of allocation, distribution, and

stabilization come from a time that emphasized

government’s ability and duty to implement

policy in the light of “community values,”

including fairness (Case, 2008). This formulation

is arguably also now incomplete as a set of

objectives. While the institutional perspective

omits consideration of distribution, fairness, and

accountability, the economic perspective on public

nance omits reference to management processes

and how services are delivered (Annex G). It is

important to note, however, that the expansion of

PFM objectives to include service delivery is still

relatively recent, as “budgeting has become more

intimately linked with other processes—planning,

operational management, and performance

management” (Pollitt & Bouckaert, 2018, p. 78).

The contribution of PFM processes to service

delivery is acknowledged to be more modest and

less direct than to scal discipline, for example,

owing to the role of other processes such as human

resources management (World Bank, 1998).

Reassessing the PFM objectives for

the 2020s

The trinity of PFM objectives has stood

the test of time impressively well. Certainly,

neither aggregate scal discipline, nor allocative

efciency, nor operational efciency seems a

candidate for retirement as an objective. The

question is whether some expansion and change in

emphasis would better equip the objectives for the

demands of the coming decade. Although it may

seem a big step to suggest that these time-honored

objectives might be changed, the general principle

that such objectives should be dynamic and able to

respond to contemporary challenges, rather than

static and immutable, seems unobjectionable.

The standard PFM objectives have three

apparent limitations:

• First, they may not adequately address all policy

considerations:

o The standard objectives are usually taken to

suggest by their ordering a hierarchy, with

scal discipline as foremost. That objective

dominates as a priority and drives the

sequencing of measures to strengthen the

PFM system.

o Fiscal discipline has become closely

associated, or even synonymous, with scal

tightening rather than allowing for a broader

interpretation of maintaining a sustainable

scal position to support macroeconomic

stability and growth.

o They exclude one of the three classic public

nance objectives, which is concerned

with the distribution of income and wealth.

While this could, in principle, be treated as

an implicit consideration through allocative

efciency, it typically is not the case.

o There is a fulsome treatment of allocative

efciency, but in practice the emphasis on

reallocation is insufcient and incremental

adjustments are the norm. A focus on

strategic reprioritization is missing.

• Second, they may not cover sufciently all

institutional domains:

o The focus is limited to core expenditure

management. The objectives do not include

wider concerns, such as resource mobilization

and the revenue side of PFM, the wider public

sector beyond general government, and scal

risks over the medium to long term.

o They do not cover the delivery choices facing

government for the provision of collective

goods and services. Strategic allocation of

resources decides the level of budgetary

provision, but not the mode of provision.

o They do not consider the accessibility and

quality of public service delivery. Efciency

and cost effectiveness, as covered by

operational efciency, are necessary but

insufcient dimensions in the provision of

collective goods and services.

• Third, they may not be explicit enough about

democratic accountability:

o They do not address the oversight and

accountability purpose of PFM in ensuring

proper scrutiny of the use of public

sector resources. There is no reference to

democratic accountability through the role

of legislatures or public participation to build

state legitimacy.

This report takes the position that the standard

PFM objectives would benet from some

expansion to address the observed limitations.

The insights of Musgrave and Musgrave (1989)

suggest how the objectives might better reect

scal policy objectives, and Cangiano et al.

(2013b) offer an example of how they might be

extended into other institutional domains. Table

2 sketches out how these perspectives suggest

expanding PFM objectives: broadening the policy

objectives to address risk; responding to the need

for major policy change and reprioritization;

broadening the institutional coverage to include

modes of delivery; and adding policy objectives

concerning stability, fairness, and accountability.

Table 2: Refreshing the PFM objectives – proposals for discussion

Standard PFM objecves or “desirable

budgetary outcomes” (oen treated in

pracce as hierarchical or sequenal)

Nature of proposed

update and

expansion

Updated PFM objecves (no assumed hierarchy

or sequence; focus on the overall balance and

trade-os)

AggregateFiscalDiscipline:Budget

totalsshouldbetheresultofexplicit,

enforceddecisions;theyshould

notmerelyaccommodatespending

demands.Thesetotalsshouldbeset

beforeindividualspendingdecisionsare

made,andshouldbesustainableover

themediumtermandbeyond.

Broaden policy

objecves

concerning risk

Fiscal Discipline and Managed Risk:Budget

totalsshouldbetheresultofexplicit,enforced

decisions;theyshouldnotmerelyaccommodate

spendingdemands.Thesetotalsshouldbeset

beforeindividualspendingdecisionsaremade,

and debt and decit levelsshouldbesustainable

overthemediumtermandbeyond.Medium-

term scal risks across the enre public sector

should be assessed and managed.

AllocaveEciency(orstrategic

allocaonofresources):Expenditures

shouldbebasedongovernment

prioriesandontheeecvenessof

publicprograms.Thebudgetsystem

shouldspurreallocaonfromlesserto

higherprioriesandfromlesstomore

eecveprograms.

Broaden policy

objecves

concerning change

Allocave Eciency and Strategic

Repriorizaon: Revenue and expenditure

policiesshouldbebasedongovernment

prioriesandontheeecvenessofpublic

programs.Thescalsystemshouldspur

reallocaonfromlessertohigherprioriesand

fromlesstomoreeecveprogramsacross

the public sector, with the capability to manage

fundamental policy change.

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Standard PFM objecves or “desirable

budgetary outcomes” (oen treated in

pracce as hierarchical or sequenal)

Nature of proposed

update and

expansion

Updated PFM objecves (no assumed hierarchy

or sequence; focus on the overall balance and

trade-os)

OperaonalEciency(orecient

servicedelivery):Agenciesshould

producegoodsandservicesatacost

thatachievesongoingeciencygains

and(totheextentappropriate)is

compevewithmarketprices.

Broaden

instuonal

coverage to include

modes of delivery

OperaonalEciencyand Service Quality:

Government should providegoodsandservices

atacostthatachievesongoingeciencygains

and(totheextentappropriate)iscompeve

withmarketprices.It should use modes of

delivery that opmize service quality and

ensure accessibility.

Add policy

objecves

concerning stability

Macroeconomic Stability: Fiscal policies should

support high employment and price level

stability, and an appropriate rate of economic

growth, with allowances for eects on trade

and on the balance of payments. Trade-os with

other socioeconomic objecves should explicitly

inform scal decisions.

Add policy

objecves

concerning fairness

Fairness and Legimacy: Budget policy choices

should produce a distribuon of income and

wealth that conforms to what society considers

a “fair” or “just” state of distribuon. There

should be sucient transparency, parcipaon,

and accountability for the public to be sased

of the legimacy of processes and decisions.

Add policy

objecves

concerning

accountability

Democrac Accountability: The exercise

and manner of raising revenues and of

allocang, execung, and accounng for

expenditures should contribute to condence

in the capabilies and discreon of the state.

These processes and rounes, appropriately

conducted, should build and sustain capable and

accountable instuons.

*FromCamposandPradhan(1996);PEFASecretariat(2016);Schick(1998a);andWorldBank(1998).Schickformulaon

usedhere.

*DrawingonWorkingGroupdeliberaonsandconsultaons,andliteratureincludingAllen,Hemming,andPoer(2013);

Andrewsetal.(2014);Cangianoetal.(2013b);Cangiano,Gelb,andGoodwin-Groen(2019);deRenzio(2009);Diamond(2013);

Hughes(2018);andMusgraveandMusgrave(1989).

Note:ItalicizedwordsinthenalcolumnmarkchangesoraddionstothelanguageofthestandardPFMobjecves.

Source:Authors’formulaon,drawingonliteraturecitedabove.

Expanding the roster of PFM objectives as

proposed here will inevitably increase the

probability of conict among them. Normative

tensions emerge when considering emphasizing

one objective over another or when pursuing

several in conjunction. While conicts in policy

design are not in fact new (Musgrave & Musgrave,

1989), they have been somewhat submerged or

obscured by the standard PFM objectives. One

consequence of the expansion proposed here is to

restore a political dimension to the consideration

of objectives. While no position is taken in

this report about their relative merits, Chapter

4 explores some of the tensions and potential

trade-offs—especially between stabilization and

distribution. It is a fundamental task of democratic

politics to resolve these normative conicts over

socioeconomic objectives in ways that advance

the overall goals of government.

Moving on from here

This chapter has reviewed the assumptions

inherent in the current approach to

strengthening PFM in development. Put

broadly, the story comprises two strands. First,

the assumptions draw from a set of conventions

concerning the desired operational performance

of PFM processes. This approach underpins

the growing professional “discipline” of PFM

specialists within development agencies,

consulting rms, universities, and think

tanks, or working as independent consultants.

Second, these conventions are seen as the path

to achieving a standard set of PFM objectives

representing a universal statement of the purposes

of strengthening PFM and implicitly prioritizing

scal discipline.

These assumptions represent a “closed”

model for diagnosing shortcomings in the

operational performance of PFM processes

and for developing reform measures to

address those shortcomings. Within this model,

political and contextual variables are important in

understanding reform tactics and sequencing, but

wider considerations about the impact of PFM

processes on public policy, service delivery, and

development outcomes such as citizen well-being

are not major variables in determining desirable

institutional arrangements. The next chapter

examines the inuence and results of this “closed”

model and proposes some ways to build on its

demonstrated strengths.

Chapter 2: Managing Public Finance

A Reappraisal

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

This chapter builds on the signicant body

of practitioner and academic work that has

constructed and reviewed the dominant

current approach to strengthening PFM in

development. That work has usefully set out

the assumed trinity of standard objectives in

PFM and the set of public sector institutional

arrangements and processes that are thought to

strongly inuence the achievement, or not, of

those objectives. As the previous chapter noted,

these assumptions represent a “closed” model

for diagnosing shortcomings in the operational

performance of PFM processes and for developing

reform measures. This chapter considers the utility

of the conventional model and proposes some

ways to build on its strengths in order to improve

the effectiveness of external technical advice. It

addresses two of the problems that were identied

and discussed in the introductory chapter. First,

the pace of learning about how external assistance

can better support improvements in PFM is too

slow. Second, development agencies’ preference

for following a common approach has reduced

opportunities to test ideas through competition.

Faster learning about external

support for PFM

More and better evidence about PFM support

interventions is needed to accelerate learning

about the effectiveness of external technical

advice. Knowledge and learning about the role

and use of PFM conventions in external support

and the evidence of impact both fall short of what

is required to drive real improvements. Within

the “closed” model, the rst question is what

would enable faster learning about how external

assistance can better support improvements

in PFM.

PFM conventions as a route map for

external support

Assessing progress in strengthening PFM is

tough. The sheer complexity of managing public

nance makes analysis difcult, as it is an area

of signicant “causal density” characterized

by “high uncertainty, which is a function of

the numerous pathways and feedback loops

connecting inputs, actions and outcomes, the

entity’s openness to exogenous inuences, and

the capacity of constituent elements (most notably

people) to exercise discretion” (Woolcock, 2013,

p. 8). Most public sector systems are a bewildering

mix of cause and effect in relation to any results

achieved. Finding a way of breaking this down

into manageable pieces, however imperfect, is in

itself a notable accomplishment (Simson, Sharma

& Aziz, 2011). In this respect, the PFM discipline

has been enormously successful at imposing

some simplied order on the conceptual territory

of interventions, institutional reforms, and

intermediate PFM results. The degree to which

this conceptual simplication has enabled real-

world development results is less clear, however.

Conceptual simplication has permitted a

focused discussion about institutional reforms.

The public sector results chain is long and open to

so many inuences that the relative signicance

of each one cannot be assessed. In consequence,

a model for examining public sector results that

attempts to consider all of the relevant inuences

and variables provides domestic reformers and

providers of external support with little practical

guidance to be applied in context. As discussed

in Chapter 2, the response of the PFM discipline

has been to collate and standardize the observed

features of effective operational performance

in PFM processes into a set of conventions that

describe an “effective” PFM system (Simson et

al., 2011). For example, the convention regarding

performance evaluation for service delivery, as

set out in the PEFA framework as the targeted

performance standard, is that “independent

evaluations of the efciency and effectiveness

of service delivery have been carried out and

published for most ministries at least once within

the last three years” (PEFA Secretariat, 2016, p.

3. BUILDING ON THE STRENGTHS OF THE “CLOSED” PFM MODEL

28). Here the test is not whether evaluations are

used in practice to improve service delivery—

or other outcomes that relate to the standard

PFM objectives of scal discipline, allocative

efciency, and operational efciency. The test

instead focuses on the degree of conformity with a

general standard.

Identifying conformity with conventions as the

stepping stone to results and as the proximate

target for external interventions reduces the

number of confounding factors in play. It

offers a shorter and more tractable model of the

public sector results chain. The fault lines in this

simplication are well-recognized, however.

The conventions are not as well-known, as

xed, or indeed as proven to be central to results

as the model suggests. Arguably, removing

consideration of many external political, social,

cultural, and context-specic inuences on the

achievement of results is to move in exactly the

wrong direction.

There is a risk of rendering

PFM as a technocratic domain and disregarding

crucial non-technical drivers of reform (Fritz et

al., 2017). Having said that, many PFM experts

see value in this simplifying archetype. Although

there are signicant limitations and risks in this

“closed” conceptualization (Schick, 2019), its

ability to make a complex territory legible is

clear. Moreover, despite offering little help in

interpreting the signicance of context, it has led

to a culture of inquiry within the PFM discipline

about the nature of institutional arrangements for

PFM and their place in achieving results.

It is sometimes suggested that the development

of PFM assessment frameworks containing

conventions for operational performance in

PFM may encourage gaming (Hadley & Miller,

2016; Krause et al., 2014). Gaming refers in a

strict sense either to strategic, self-interested

manipulation of the scales or criteria used for

measurement (such as selecting the indicators

that give the preferred results) or to manipulation

of the data resulting from those measurements

(Courty & Marschke, 2004; Hood, 2006; Van

Dooren et al., 2006).

In the case of standard

PFM assessments, incentives for gaming may be

attenuated, as the nancial rewards for conformity

with the conventions are not particularly high

and there is considerable transparency within the

frameworks about how the ratings are calculated.

The risk of gaming cannot be entirely disregarded,

however, especially with respect to the replication

or imitation of formal PFM processes in relation to

disbursement triggers for development assistance.

Limited knowledge and the slow pace

of learning

The record of evidence and results is hard

to decipher

Measuring the impact of technical advice on

strengthening PFM is a difcult task. There is

surprisingly little to be concluded from research

concerning the effectiveness of development

assistance in improving PFM. In principle,

research might look at evidence that external

interventions can have a causal role in improving

conformity with PFM conventions or that

strengthening the conventions can be necessary

for achieving development results. The case may

even be made that PFM interventions can in some

other way have a directly causal role in achieving

development results. Given this high “causal

density,” it is important also to look from the other

end of the public sector results chain, identifying

observed improvements and scanning the horizon

of possibilities in considering what caused them.

Annex H offers a partial summary of the literature

reviewed for this report and provides some

representative research summaries.

Recent analyses of the evaluation literature

note that the evidence base concerning the

impact of PFM support is patchy at best (De

Lay, Mills, Jadeja & Lucas, 2015; de Renzio,

Andrews & Mills, 2010; Mills, 2018; Pretorius &

Pretorius, 2009). A major evaluation of “Public

Finance for Development” being undertaken by

the World Bank’s Independent Evaluation Group

(IEG) includes support for PFM. It will assess

whether the design and implementation of World

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Bank interventions have contributed to better

PFM outcomes—both intermediate outcomes,

such as improved predictability and control in

budget execution, and nal outcomes, such as the

achievement of a strategic allocation of resources

(IEG, 2018).

The current state of knowledge about the impact

of external PFM support can be summarized as

follows. There is some, distinctly limited, evidence

that donor interventions—such as investment

projects or technical assistance missions—can lead

to improved conformity with PFM conventions.

For example, in examining a number of case

studies, Orth, Schmitt, Krisch, and Oltsch (2017)

found that budget support (comprising funding

through government systems, conditionality,

policy dialogue, and technical assistance) was

associated with increased government spending in

social sectors, stronger PFM, and less corruption.

There is perhaps more compelling evidence

that interventions can lead to improved results

by linking resource allocation more directly to

policy intentions and by improving transparency,

but the evidence points toward participation and

consultation as the drivers of these improvements

more than institutional reforms (De Lay et al.,

2015). Finally, there is some limited evidence

that, however it is achieved, conformity with

the conventions can improve scal and service

delivery results. For example, Welham et al. (2017)

nd that better public nancial management,

as measured by PEFA, is associated with lower

variance in overall budget execution rates and

improved health outcomes. But there are many

identied cases in which there is seemingly no

connection.

The same challenges affect assessments that

look back up the results chains to detect

the contribution of external PFM support.

Evaluations that look back from improved

development results at the national or sector

level and ask whether these are attributable

to donor interventions in PFM have largely

drawn a blank—whether these concern general

develop-ment progress, as measured by economic

growth and poverty reduction (Radelet, 2015),

or improvements in the health sector (Morrison,

2013). Similarly, evaluations have not shown that

improved development results can be attributed

to improvements in PEFA ratings, for example.

When there has been an identied improvement

in country PEFA ratings, implying improved

conformity with the PFM conventions, this has

rarely been attributable to donor interventions

(Lawson, 2012).

The pace of learning remains frustratingly

slow

The challenges of evaluation in PFM echo those

faced by social science research more broadly.

Empirical work on public sector management and,

within it, PFM offers just one particular case of

a larger difculty. The denitional challenges in

capturing institutional changes and measuring

results are enormous. The hunt for quasi-

scientic “single treatments” that might lead to

specic results is undoubtedly misplaced. With

so many contextual factors in play, the attribution

issues are confounding. The time lags between

interventions and effects are outside the scope

of most available data. Controlled experiments

are seldom feasible, and natural experiments are

rare.

PFM, at least, is a domain in which at least

some of the measurement challenges have been

addressed, particularly in relation to assessing

conformity with conventions and with scal

results (Schick, 1998a). Nonetheless, there has

been disappointment that empirical ndings have

not emerged more rapidly. PFM interventions may

be good enough, but they could still be better. The

question is how to achieve that.

There have been some promising ideas about

how to accelerate improvements in knowledge

about PFM support, but no breakthroughs. The

question of how to embed political and contextual

considerations in technical analyses has been the

subject of intense debate in recent years (Annex

I). Ideas about “adaptive management,” “doing

development differently,” and “thinking and

working politically” in governance reforms have

become increasingly sophisticated (Andrews,

2012; Green, 2018; Teskey & Tyrrel, 2017) but

show some clear similarity to ideas from the early

1990s (Berg, 1993). Similarly, deliberations about

optimal PFM reform sequencing have found little

to agree on beyond an entrenched assertion that

basic disciplines matter (Diamond, 2012; Fritz,

Hedger & Lopes, 2011; USAID, 2014).

The

practical result has been a series of relatively

distinct ideas about supporting and sequencing

institutional change. Each one has seemingly been

clear and self-standing until the next one arrives

(Pretorius & Pretorius, 2009) (Annex J).

Some ways forward: better data

collection and knowledge sharing

If better and faster learning is the goal, then

action is needed on the part of both individual

agencies and the wider PFM discipline.

Individual agencies can make a difference, such

as when the World Bank acted as a catalyst for

research on economic growth in the 1990s.

That example showed that detailed tracking

of country-level institutional reforms through

high-quality case studies, coupled with greater

experimentation with impact evaluations (in the

occasional cases when “natural experiments” can

be found), can be a useful contribution. But the

biggest gains could come from mobilizing the

PFM discipline as a whole. If assistance providers

and client governments had stronger incentives

to release information about interventions and

results using standardized metrics—moving

away from the current situation, in which most

relevant knowledge remains tacit and unshared—

then richer and more powerful analysis could be

brought to bear.

One key to progress lies in moving beyond the

current fragmentation among agency-specic

research and evaluation initiatives and data

sharing. Open-source data on public sector

institutional change could spur both external

research and internal learning, supporting the

development of a secondary industry of analysis

of project data. Alongside the IMF, multilateral

development banks (MDBs) are the most signicant

providers of technical advice and funding for

PFM (Simson et al., 2011). However, the track

record of deep collaboration among international

organizations is not encouraging, as they each

maintain separate analytical and knowledge

functions. “There are only a few examples

where MDBs have worked together and taken

advantage of the capabilities in sister institutions.

More commonly, they operate independently.

For example, each MDB has its own research

department and independent evaluation ofce.

They each develop individual country strategies.

They have their own assessment frameworks

for development effectiveness” (Bhattacharya,

Kharas, Plant & Prizzon, 2018, pp. 11–12). The

coordination of data collection in other areas

suggests that progress in knowledge sharing across

agencies is likely to take the form of agreement

on principles and classications accompanied by

persistent small steps in the right direction, rather

than a major breakthrough (Inter-Agency Group

on Economic and Financial Statistics, 2019). Two

ideas may hold some promise.

Developing new standards for data collection

and sharing

Proposal 1: Assess offers of technical

advice against standards for the provision

of information and evidence justifying the

approach and for the planned dissemination of

data about impact

One potential way forward is the use of a

standardized framework for data collection

and sharing. Current strategies for learning about

the effectiveness of development interventions are

of limited utility in the face of provider agencies’

incentives to restrict information about their

activities and to make excessive claims about their

impact, and of client governments’ incentives

not to challenge that behavior. Exposing more

clearly the basis on which offers of technical

advice are made and the claims that they entail

could initiate a virtuous circle. Governments

would have a basis for making more informed

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

choices among providers of technical advice and

potential approaches, and providers of technical

advice would have greater incentive to share

data and justify claims. Such a framework could

include standardized data categories to allow for

comparison among design options and among

results achieved.

Assessing how well the operational performance

of PFM processes conforms to PFM conventions

follows a well-established pattern of using

standard frameworks to design external

interventions (World Bank, 2010). The PFM

discipline could develop a tool to assess how well

PFM support interventions conform to open data

standards on program design and results. Such a

tool would ask how clearly the proposals developed

by technical advice providers are elaborated along

two dimensions. First, an efciency justication

dimension would rate proposals according to the

degree to which they show the potential costs

involved, such as nancial and transactional

costs, and the likely returns. Second, an empirical

justication dimension would rate proposals on

the availability of evidence on which they draw

to make their case. Development of this tool could

draw from cross-country and multisector studies

about how this type of reform proposal has worked

out in general, and from impact evaluations or

other research that can test for the salience of

distinctive contextual conditions.

The data assessment tool could be used as

part of “quality at entry” processes within

international agencies. While the tool would

assess the transparency with which project

proposals are justied nancially and empirically,

it would not be used to collect actual project data.

Use of the tool could be combined with some

form of independent, non-binding review or

validation—possibly by an arm’s-length agency

or entity of some form—that the appropriate

procedures for using the assessment tool had been

followed. The “PEFA Check” provided by the

PEFA Secretariat is one such example.

The results measurement component could spur

a much-needed step-change in measuring the

results achieved through complex institutional

reforms, including how to treat different types

of evidence (Bates & Glennerster, 2017; Pritchett

& Sandefur, 2013). The current “publish what you

fund” movement is of value for input coordination.

Here, though, we propose a new “reveal what

you are proposing and what you did” movement,

which would be of greater value in enriching the

PFM discipline with more solid evidence about

successes and failures, perhaps supporting client

governments through access to greater choice

among different technical advice options.

The pressure to publish standard data

might prompt innovations in the means of

data collection and stronger leadership by

international organizations. Developments such

as algorithmic learning and cell phone surveys

could emerge as mechanisms for collating expert

assessments and amassing data on implementation

and results. While the hope may be overblown

that “big data” might rapidly nd meaning within

a mass of metrics about reforms, results, and

contexts (Lazer, Kennedy, King & Vespignani,

2014), some large-scale data collection approaches

could help to indicate which avenues to explore

further (Calvard, 2016). Ofcial fora such as the

OECD DAC and the United Nations Department

of Economic and Social Affairs (UNDESA) might

shift their focus toward assessing whether advice

providers and client governments are willing to

open up their reform approaches to contribute to

the global data pool.

Lowering risk in sharing project shortcomings

Proposal 2: Establish practitioner groups

to share sensitive performance information

on PFM interventions on a voluntary and

condential basis to identify ways of improving

performance

A complementary approach may be to open up

a smaller dataset for more in-depth review and

lesson sharing. Robust and comparable external

measurement of results can seem risky for providers

of technical advice that need to make claims

about the benets of their interventions. Provider

agencies face strong incentives to demonstrate to

their shareholders or funders that they are highly

effective, which means that rigorous, veriable,

and comparable measurement of results by third

parties may not be a high priority (Barder, 2009).

Political pressures on provider agencies instead

emphasize the need to demonstrate visible, short-

term results to shareholders, taxpayers, or other

principals (Gulrajani, 2017; Honig, 2018; Koch

et al., 2017; Power, Scheytt, Soin & Sahlin,

2009; Valters & Whitty, 2017). Risk aversion is

especially high among providers in fragile states

(Dendura & Flynn, 2011; McKechnie & Davies,

2013). Open data, deep knowledge, and sharing of

learning are not likely to be the primary concern.

In some other sectors in OECD countries, these

concerns have been addressed through the

creation of “data clubs,” or private networks

for sharing sensitive data, which operate

along reciprocal lines. This approach could be

considered for the PFM sector as well. Submission

of a base set of data and the commitment by all

participants to treating all data shared in strict

condentiality would be conditions of joining a

PFM data club. Such an approach has been shown

to overcome government concerns and to facilitate

the sharing of sensitive safety data concerning

large city subway systems (Hood, Dixon, &

Wilson, 2009; Wilson, 2010) as well as potentially

damaging information on school performance

(Glaister & Anderson, 2005). Membership would

provide access to an aggregate, anonymized

summary of the data that have been shared by

others, and would allow individual members to

approach each other for more tailored sharing of

information. Club membership could be offered

to entities or individuals, whether from client

governments, international organizations, or

donors and other assistance providers. Opting into

the club would entail careful specication of the

parties or types of parties with whom the data may

be shared as well as whether and when the data

must be anonymized (Glaister & Anderson, 2005).

New advances in blockchain technology could

offer a mechanism for permitting secure sharing

of sensitive information within an approved group

of disparate participants among whom trust needs

to be established (Reinsberg, 2018).

Members, serving as representatives of their

respective agencies, would thus be able to

obtain authoritative technical support and

data and have the credibility to speak critically

about their agency’s approach. They would

decide which data mattered and how to obtain

information for sharing, with an emphasis on

practical and implementable actions and explicit

protocols about how the data could be used in their

respective agencies. Members would establish the

governance arrangements for the club. Agencies

would be able to opt out of the club at any

time, with all references to their data removed

(Anderson, 2005; Glaister & Anderson, 2005;

Hood et al., 2009; Wildy, 2004; Wilson, 2010).

An important distinction is that these

condential data clubs are not peer learning

exercises. Peer learning focuses on the individual

and is intended to contribute to individual-level

gains in knowledge and insights—although with a

view to their making impacts at the organizational,

sector, or national level (Andrews & Manning,

2015). The Collaborative Africa Budget Reform

Initiative (CABRI), the Public Expenditure

Management Network in Asia (PEMNA), and the

Public Expenditure Management Peer Assisted

Learning (PEMPAL) network for the Europe and

Central Asia region are examples of successful

initiatives in the peer group mold (PEMPAL

Steering Committee, 2012; World Bank, 2013).

By contrast, data clubs are intended to provide

insights at the system-wide level.

Contestation and choice in

PFM advice

The growing diversity in providers of

development assistance and the increased

choice experienced by some governments

could strengthen the market for technical

advice. Despite concerted efforts over 15 years

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

to improve donor coordination, success continues

to be limited and sporadic. There may be some

conditions under which greater competition would

be a desirable or even superior approach. This

section explores that possibility with regard to

technical advice on strengthening PFM. It builds

on the consensus of the “closed” PFM model

regarding the operational performance of PFM

processes needed to achieve the trinity of standard

PFM objectives. The core question is whether

greater contestation among intervention ideas and

options, and greater government choice over those

options, could help to improve innovation and

results in external assistance for PFM.

Donor coordination and its

emerging limits

Rationale and future prospects

Effective coordination among donors in the

provision of development assistance has long

been stated as an important principle and

objective. Coordination is pursued primarily to

maximize the impact of development nancing,

either by reducing unintended overlaps in

interventions (Barder, Gavas, Maxwell & Johnson,

2010) or by encouraging compliance with a larger

technical agenda or set of standards determined by

the international community. Harmonization and

alignment of international aid were central features

of the 2005 Paris Declaration and its successor

agreements. The “Strengthened Approach to

Public Financial Management” was put forward

as a common technical agenda and made a strong

case for coordination as part of the PEFA program

(OECD DAC, 2003). Coordination with reference

to a technical agenda was given extra force by the

concern that independent actions by development

agencies “may postpone necessary adjustment

because there is so little conditionality” (Woods,

2008, p. 1207) and so could be wasteful (Klein

& Harford, 2005, p. 3). The arguments for

coordination are bolstered by a transaction

costs argument that fragmented external support

can further burden already overloaded client

governments.

There is a contrary case to be made that

coordination may not always be desirable

or realistic. Coordination, in the sense of

compliance with a common set of assumptions

about technical approaches, seeks by denition to

reduce experimentation and choice. It locks in the

presumption that adherence to current conventions

is the right way to go (Scharfstein & Stein, 1990).

Separately, and regardless of whether coordination

can in principle achieve the aim of more

effective development assistance, there is a more

pragmatic argument that preaching coordination

can be unrealistic, as it is often not delivered in

practice. Despite concerted efforts and a series of

global commitments, the evidence suggests that

coordination efforts have been only moderately

successful. Many attempts at coordination have

failed to impose a rational division of labor among

donors based on comparative advantage (CABRI,

2014; Brookings Institution and CGD, 2011)

because there is no effective “umpire” to impose

discipline (Harris, 2011).

Recent research is pessimistic about the

prospects for improved donor coordination at

a general level in development cooperation.

Incentives to disregard coordination include

the need to pursue the national interests of the

donor country even at the expense of greater

transaction costs, and the need to appease interests

in partner countries in order to maintain a longer-

term relationship (Bourguignon & Platteau,

2015; Olivié & Pérez, 2016). The increase in aid

impact evaluations and the growing prominence

of indices comparing donor performance are

contributing to increased fragmentation, as

these metrics inherently promote competition

among development agencies. Thus, at least

according to some analysts, “efforts to improve

donor coordination are doomed to failure, ‘cheap

talk’, since they do not change the incentives

underlying donors’ strategic interaction” (Annen

& Moers, 2012, p. 30). Although the reality may

be somewhere in between, the track record of

effective coordination without corresponding

change in incentives is not strong.

Structural changes in the supply of

development assistance

Coordination is likely to become more

challenging as new sources of development

nance create greater choice for client

governments. As countries move into middle-

income status and the landscape of development

nance diversies to include new players, the

traditional model of coordination is coming

under strain (Greenhill et al., 2013; Prizzon,

2014; Schmaljohann & Prizzon, 2014). Recent

evidence shows signicant heterogeneity among

a growing group of non-DAC providers of

development cooperation (Lauria & Fumagalli,

2019). This trend suggests that client governments

may gain access to greater and more meaningful

choices among all donors, rather than merely a

binary distinction between DAC and non-DAC

alternatives. Well-established major funders and

advice providers appear to be relinquishing some

of their ownership of and inuence on PFM

advice in the face of alternative development

nance sources—most notably China, but also

Turkey, the Islamic Development Bank, and other

“new donors” (Greenhill et al., 2013; Hanauer

& Morris, 2014).

China, in particular, brings a

corresponding increase in the volume of public

management technical assistance embedded

within projects—especially infrastructure

initiatives (Lum, 2009). The product being offered

may also vary with the type of provider. Such

products could range from the explicit provision

of technical advice by development agency staff

to assist in the implementation of an institutional

reform agenda, to implicit or informal advice

associated with large infrastructure nancing on

near-market terms, to advice provided by a major

consulting rm working under contract on a full

cost-recovery basis.

New sources of technical advice may further

increase the degree of country choice. The

proliferation of small interventions (Fengler &

Kharas, 2010), combined with a growing diversity

of assistance providers, are allowing countries such

as Ethiopia, Rwanda, and Sierra Leone to exercise

greater choice in selecting preferred providers of

technical advice.

There are signs that a growing

number of “challenger agencies” are emerging,

offering advice without funding and espousing

innovative methods or ideas.

Typically these

organizations are small, non-governmental, non-

prot, and independently funded, often connected

to policy think tanks, research institutes, or

universities. These agencies tend to be proponents

of problem-based and adaptive approaches to PFM

support, and they appear less likely to be bound

by the prescriptive inuence of the conventions or

incentives of the PFM discipline. They often work

directly for client governments, rather than merely

with those governments at the behest of other

organizations such as donors and IFIs.

The improbability of achieving a market-

based approach at scale

Donor aversion and demand-side constraints

A more open and competitive market for

technical advice in development is often

invoked and has long been predicted by some

commentators (Barder, 2009; de Renzio &

Rogerson, 2005; O’Keefe, 2007).

A full market

would be characterized by many buyers and

sellers, distinct products, low barriers to supplier

entry and exit, and robust information about prices.

Those features largely do not apply in the case of

technical advice for lower-income countries on

PFM (or indeed on other technical development

concerns). Although there is often, de facto,

considerable competition among development

agencies for projects and inuence at the country

level, the strength of the coordination paradigm

makes explicit consideration of competition in

development assistance an awkward topic and a

contentious proposal.

It is also likely to be too

disruptive and costly to the integrated business

models of the IFIs, which bundle together their

functions of nancing and technical advice—

typically along with broader global public goods.

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Decoupling of funding for technical advice

from the selection of providers would need

to be a central feature of a proper market. A

structural change of that type is highly improbable

at scale in the case of development assistance to

lower-income countries.

Any model in which

funders detach themselves from inuence over

the external providers of advice and assistance to

governments (Barder, 2009; Booth, 2013; Klein &

Harford, 2005) would remove the key device by

which donors provide assurance to their political

principals and their ultimate customers—namely,

domestic taxpayers in donor countries—that

their funds are well used (Booth, 2013; Klein &

Harford, 2005; Martens, 2005; Mosley, 1985).

Donors themselves would resist strongly, since

they “proceduralize” operations, imposing internal

steps and hurdles as a proxy for ensuring quality

in an environment where nal results are hard

to determine (Power et al., 2009). This is only

possible if the funder has some signicant control

over the type of advice that the disbursed funds are

used to purchase. In addition, since a signicant

proportion of donor nancing is ultimately used

to nance the donors’ own professional services

(Browne, 2012), there would be signicant

internal resistance.

Any serious debate about a market for

technical advice must also recognize that there

are signicant constraints on the demand side.

While governments in lower-income countries

may in principle seek to maximize the quality

of technical assistance, they are likely to be

as much or more concerned with maximizing

concessional nancial ows (Greenhill et al.,

2013; Prizzon, 2014; Schmaljohann & Prizzon,

2014).

Governments have mixed and multiple

motives when expressing demand. They may be

reluctant to express clear and reliable demand for

the external support they require when a wider

partnership and nancing agenda are at stake

(Khemani, 2017). Just like provider agencies,

client governments are inuenced by notions

of comparative good practice in other countries

and “cutting-edge” ideas and models, with the

corresponding temptation to choose based on

notions of modernity more than on effectiveness

(Lawson, 2012).

Possible evolution in the longer term

The market for technical advice in lower-

income countries will evolve over the longer

term in response to the inux of new sources

of nance and new providers of advice,

but current donors are likely to resist that

disruption. For the foreseeable future, OECD-

DAC donors and long-standing IFIs will continue

to provide the predominant share of concessional

nancing to lower-income countries, with the

notable exception of China in some cases. These

well-established actors have, in effect, formed a

cartel—both for the external funding of technical

advice, and for determining the providers and

often the content of that technical advice. This

syndrome is observed clearly in the funding and

provision of advice on PFM (AlphaBeta, 2018).

As discussed above, the business model, risk

management, and quality assurance pressures and

requirements faced by donors and IFIs make it

unlikely that they would yield a substantial degree

of choice to client governments about advice,

services, and even providers. The rise of new

sources of nancing and technical advice—for

example, from donors outside the OECD-DAC

or from foundations and non-prot actors—might

eventually start to reduce the dominance of some

long-standing donors and IFIs across all areas of

technical advice on PFM. However, the heads of

those agencies are unlikely to acquiesce to, or

to collaborate readily with, plans that would put

them at odds with the interests of their political

principals and, ultimately, with those of taxpayers

in donor countries (Mosley, 1985; Martens, 2005).

Donors and IFIs are likely also to argue that they

are the guardians of quality assurance and results

attainment when considering requests by client

governments for technical advice on PFM.

Middle-income countries are a different

case. More comprehensive competitive market

approaches are already emerging in middle-income

contexts, with both benecial and challenging

features. Those countries tend to be characterized

by some or all of the following features: greater

scal capacity to self-nance technical advice,

greater access to diversied sources of development

nance (including from international capital

markets), and a higher-capacity bureaucracy

with stronger commissioning acumen. In those

contexts, there is evidence of a wider array of

management consulting rms working directly

for governments. Following this trend, some IFIs

have started to offer technical advisory services

on a directly reimbursable basis, outside of the

main lending relationship (for example, the World

Bank’s Reimbursable Advisory Services), and the

IMF has been scaling up its program of regional

Technical Assistance Centers. The message,

therefore, is that competition is likely to emerge

naturally as countries develop economically—

provided that they are not aficted by conict—

but the transition is necessarily slow.

One way forward: managed options and

greater choice

Contestation of options, not competition

among providers

More active country choice in technical advice

is possible for lower-income countries in the

short and medium term, but the reality is that

funders will continue to manage that choice. The

strongest steps that provider agencies might take

are to develop a comparable range of intervention

options, all of which meet their quality standards,

and to provide a choice to governments among the

set of options that have been assessed and veried.

The essential distinction here is that the number or

diversity of credible providers of technical advice

ceases to be the main consideration, replaced

by the number and heterogeneity of credible

intervention options. This shifts the debate from

donor coordination versus donor competition

toward contestation among ideas. It could, in

principle, apply with equivalent effect to one

provider or many providers.

Choice mechanisms that use reputational

feedback on providers can help to identify

promising or proven types of interventions,

even when a fully competitive market is not

possible. The theoretical and empirical literature

on quasi-markets for public service delivery

in OECD countries shows that greater choice

and competition can lead to improvements in

innovation, diversication, and performance when

users are able to indicate preferences and use the

threat of exit (Le Grand, 2010; Le Grand, 2007;

Lubienski, 2009). Some commentators argue that

the basis for a market in development assistance

is already in place (O’Keefe, 2007),

since client

governments have at their disposal information

about the characteristics and performance of

assistance providers (de Renzio & Rogerson,

2005). It is further argued that existing competitive

procurement methods for selecting technical

assistance providers could be extended radically

through electronic auction formats in tendering

and better information about the reputation and

past performance of providers (Barder, 2009).

What these commentators are really pointing

to is the potential for greater government

choice to stimulate innovation and fresh ideas

and to improve the comparative assessment of

different intervention designs or variations in the

“intervention logic.”

Alternative options proposed by providers of

technical advice could each be analyzed based

on their proposed or implicit intervention

logic. Several overlapping questions are generally

considered explicitly or, more often, implicitly

in the design of external interventions and the

“theories of change” on which they rest.

One

question is the degree to which it is assumed that

the specied input (such as a training program

or new procedures) will lead in a linear fashion

to the intended output and outcome as set out in

a results chain (OECD, 2017), logframe (DFID,

2009; Team Technologies, 2005), or equivalent.

This approach is standard for most technical

assistance interventions. A second, contrasting

question concerns whether the theory of change

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

suggests uncertainty about the consequences

of any intervention and the need for exibility

and participation on the part of local actors

(Valters, 2015), thereby emphasizing the need

for iteration and adaptation. A third question is

whether incentives to achieve a specied outcome

could be combined with exibility concerning

the choice of input (Birdsall & Savedoff, 2010).

Examples of intervention types that follow a pure

outcome-based logic include “cash on delivery”

aid and development impact bonds. Finally, a

fourth question concerns the technical sequencing

of the proposed PFM content (Diamond, 2013),

including whether to strengthen “basic” or

foundational PFM processes rst. These four

questions underpin the variety of intervention

designs that can reasonably be proposed.

In practice, governments are typically offered

few meaningful choices when deciding which

PFM reforms to pursue and the intervention

logic that underpins the reform model. An

evaluation of external PFM interventions in

three African countries found that “government

authorities were presented with a limited range of

options…and were not encouraged either to consult

or to reect more widely on the choice of reform

models….There were apparently neither nancial

constraints nor political constraints determining

the choice of reform model, so we interpret this

as simply as a lack of adequate ideas: a policy

space constraint” (Lawson, 2012, p. 68, 72). The

absence of alternative ideas and propositions leads

to a corresponding lack of debate about different

reform options.

Despite the strengthening of

business case processes, quality-at-entry reviews,

and economic appraisal by many assistance

providers when it comes to project or program

design—including, in some cases, borrowing

from domestic options appraisal techniques

(HM Treasury, 2018)—it is evident from the

examination of development agency project

documentation that the alternatives considered

tend to be treated in a distinctly cursory way.

Projects typically propose one single, rst-best

intervention logic. In light of these challenges, the

Working Group proposes a potential way forward.

Piloting “managed choice” in external

PFM advice

Proposal 3: Experiment with “managed choice”

in the provision of technical advice, with a

coalition of funders and client governments

and the creation of a dedicated funding facility

In a “managed choice” model, providers of

PFM support would prepare more than one

intervention design option as the basis for

comparison, discussion, and—ultimately—

choice by the client government. Each option

would be subjected to a quality assurance process,

for example through robust peer review,

and

would explicitly harness available research

evidence. Offering multiple intervention options

is more expensive than asserting that there is just

one correct or superior way forward. However,

given the lack of certainty about what works and

the need for greater experimentation, it would be

a more honest and open-minded approach. When

offering options to a client government, any

differences in the intervention logic or technical

design would be emphasized and discussed openly.

Accordingly, each option would need to make

explicit the technical content and intervention

logic that underpinned it and the logical and

empirical justication for that approach.

The potential benets of this approach

stretch beyond creating more experiments to

assist in learning about what works in PFM

support interventions. Other benets could

include the incentives that such arrangements

would provide for strengthening relationships

among different parts of government as they

explore the implications of options together,

the increased motivation inherent in giving key

actors the opportunity to express their views and

inuence reform design, and the possibility that

the problems to be addressed would incorporate

locally identied issues more centrally. This

approach also brings risks, however, including

that preparing several options would increase

costs unreasonably and that government might be

unable to achieve a consensus choice—especially

if choices set up conicts along existing fault lines

within government.

While this proposal is still a long way from

creating a competitive market for PFM

support, it would move the PFM discipline

toward greater contestability of ideas. It may

not be known whether, for example, the IMF, the

World Bank, the United Kingdom’s Department

for International Development (DFID), or the

European Commission has the best track record

in supporting a particular dimension of PFM

reform. Equally, it may not be known what degree

of balance between standard technical design on

the one hand and problem-driven and adaptive

assistance on the other hand is likely to be most

effective. As noted in Annex H, the research

evidence is simply not good enough to provide

clear, denitive, or general answers. The benet

of introducing options and choices more strongly

into PFM support is that it would provide more

data to support choices among different types

of interventions and more information on which

approaches are most credible from the perspective

of the client government. It should also strengthen

the quality of the dialogue and relationships

between advice providers and client governments.

More vigorous and explicit contestation of the

ideas put forward by external advisers in their

engagement with government counterparts

would need to cover two facets. First, it must

entail a plurality of diverse viewpoints to

counterbalance the tendency of advisers to argue

for their own agency’s or their own discipline’s

position on PFM reform.

Second, it must shift the

incentives faced by external advisers so that they

can act in a more disinterested fashion. This would

mean that advice would not be unduly inuenced

by other objectives or agendas, such as a parallel

nancial relationship between the agency and the

government or an organizational predisposition

toward a particular approach or reform proposal

(in other words, a “house view”). While truly

disinterested advice may not be a realistic aim,

reducing conicting incentives and introducing

a mechanism for structured contestation of ideas

would be important steps in that direction (Annex

K).

This proposal would ideally be coupled

with improved availability and sharing of

information about the impact of external

interventions and projects (Proposal 1). Even

without much improvement in impact measures,

there is still a prima facie case for harnessing

the data and analysis that already exist. The

increased demand on the part of both technical

advice providers that are developing their options

and client governments that are assessing those

options for evidence of impact could itself drive

the requirement for more open and routine sharing

of assessments. So, while these two proposals

ought to be complementary, introducing choice

among options is still potentially worthwhile even

if better availability of impact measurement has

yet to emerge. In addition, donors that fund the

PFM advice provided by IFIs and consulting rms

could begin stipulating that the funded content

(such as manuals, guidance notes, and forecasting

models) be placed accessibly in the public

domain as a means of making PFM knowledge

more of a public good that is readily available to

governments.

A major consideration about feasibility is how

to create incentives for agencies to develop

alternative intervention designs and offer

genuine choices to governments. This approach

could harness the intellectual energy of those who

are hoping to see a more competitive approach to

technical advice for PFM. At the same time, it is

sufciently modest that it should be able to gain

acceptance among providers that might otherwise

be concerned about the loss of control implied by a

more open market and about the risks of detaching

advice from funding. The starting point is twofold:

appealing to PFM professionals’ desire to improve

their impact and, at the same time, encouraging a

couple of progressive donors to test the proposed

“managed choice” approach.

Chapter 3: Building on the Strengths of the “Closed” PFM Model

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Consultations with bilateral PFM donors

suggest that some choice is implicit in dialogue

with client governments, but typically focuses on

the use of bilateral versus multilateral channels.

Some useful distinctions in the intervention design

and approach are revealed by choices among these

channels—for example, on issues such as the

approach to change management—and the client

government’s preference may be given some

weight in the decision. This is the case especially

when there is already a strong partnership between

the donor agency and the client government.

These are all positive steps in the direction

of “managed choice” and are supportive of a

partnership approach. However, there do not yet

appear to be instances of full implementation

of the proposal, as it is broadly set out here, to

experiment explicitly with “managed choice” in

the provision of technical advice. Further steps to

build on current practices could include making

more explicit the differences in intervention logic

when comparing two delivery channels, and for a

group of bilateral donors to request a multilateral

agency to pilot a “managed choice” approach as

part of granting them trust fund resources.

To bring more powerful incentives into

play, there would need to be a separate and

independent funding mechanism or facility.

With that established, options might include

giving a consortium of donor agencies access

to funds that would enable them to nance the

development and peer review of, for example,

three different approaches to improving PFM.

Alternatively, client governments could be given

a modest nancial incentive if they were able to

show that they had insisted upon the production,

scrutiny, and deliberation by PFM donors of a

credible choice of interventions. Most ambitious-

ly, an entirely separate funding mechanism could

create some degree of split between funding and

advice. A signicant test would be whether ofcial

donors or private foundations would be willing

to give third parties commissioning control over

such a technical advice fund.

Moving on from here

This chapter has proposed some ways to

build on the strengths of the current “closed”

model of strengthening PFM in development.

They respond to two problems identied in the

introductory chapter of this report: rst, that the

pace of learning about how external assistance

can better support improvements in PFM is too

slow, and second, that development agencies’

preference for following a common approach

has reduced opportunities to test ideas through

competition. The rst three International Working

Group proposals are as follows:

(1) Assess offers of technical advice against

standards for the provision of information and

evidence justifying the approach and for the

planned dissemination of data about impact;

(2) Establish practitioner groups to share sensitive

performance information on PFM interventions

on a voluntary and condential basis to identify

ways of improving performance; and

(3) Experiment with “managed choice” in the

provision of technical advice, with a coalition of

funders and client governments and the creation

of a dedicated funding facility.

The next chapter considers how to rethink

managing public nance as an “open” system

in which the inuences of PFM on government

policy choices, government actions (especially

service delivery), and development results are

the central focus. An effort to introduce more

real-world complexity into thinking about PFM

raises many questions. How can the conventions

that provide the benchmark for assessing the

operational performance of PFM processes

be used to investigate the links between PFM,

government actions, and development results?

How can PFM processes better support policy

change and become more responsive to a broader

set of socioeconomic objectives?

This chapter explores ways to rethink the

paradigm of PFM in international development

in order to reconnect it with public policy.

Refreshing and expanding the trinity of standard

PFM objectives would help to sharpen focus on

how PFM inuences government policy choices,

government actions, including service delivery,

and—ultimately—development results. Once the

complex trade-offs are apparent, PFM can work to

help policy makers address real-world priorities,

choices, and decisions (Schick, 2019).

This is difcult territory, as it begins to

depart from the concrete and “universalist”

assumptions of the “closed” model. Accordingly,

this chapter considers the potential for developing

a new generation of approaches to international

support for managing public nance that better

reect the real world. It addresses specically two

of the problems that were identied and discussed

in the introductory chapter. First, the ways in

which PFM matters for service delivery and for

sector outcomes are not well understood. “Service

delivery” is used in this chapter as an example of

a government action—a particularly important

action given the emphasis and priority attached to

public services by the international development

community. Second, the standard objectives and

conventions of PFM are not equipped to deal with

political choices and trade-offs or major policy

change, especially in the context of wide-ranging

and complex sets of government policy choices

and intended development results. (See Annex

B for a fuller discussion of the public policy

framework underpinning these concepts and their

interactions.)

Linking PFM to service delivery and

development results

There are clear linkages among levels of public

policy—namely, government policy choices,

government actions, and development results—

although the direction of causation is not easily

determined. Government policy choices concern

government intentions, as expressed through

statements and decisions about what it will

prioritize in principle and as revealed through its

actions. Public service delivery is one dimension

of government actions and outputs, concerning

specically the provision, commissioning,

or funding of collective goods and services.

Development results, as a subset of wider policy

impacts, are the outcomes related to government

choices and actions (Annex A; Annex B).

This chapter considers these issues in the

context of rethinking PFM as an “open” system.

Service delivery is considered rst as an example

of how PFM conventions relate to government

actions. Governments do much more than ensure

the delivery of services, but that particular task

clearly forms a crucial link between policy choices

and development results.

Understanding the link between PFM and

service delivery—as a means to improving

sector outcomes and other development

results—is a growing preoccupation for many

providers of development assistance. From this

agenda ows the general question of how PFM

conventions could be refreshed, harnessing better

evidence about the relevance and contribution

of specic PFM processes. Answering this will

require tools and techniques both to identify new

conventions and to manage the life cycle of existing

conventions. This effort should be accompanied

by a willingness to discard conventions that have

outlived their utility or their evidence base.

The dominance and the limits of PFM

conventions

The PFM conventions, discussed in Chapter

2, bring together lessons learned and the

tacit knowledge of practitioners to dene a

powerful and precise set of prescriptions. These

conventions are most readily observed in the wide

array—45 at current count—of PFM assessment

4. RETHINKING MANAGING PUBLIC FINANCE IN AN “OPEN” SYSTEM

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

frameworks that have been developed (PEFA

Secretariat, 2018), including most prominently

the PEFA methodology, TADAT, the Debt

Management Performance Assessment (DeMPA),

the Public Investment Management Assessment

(PIMA) tool, and the IMF’s Fiscal Transparency

Code (Annex F). Each framework is powered by

propositions about the “correct” conguration and

operational performance of PFM processes. For

example, the DeMPA framework asserts that:

to be predictable and transparent, government

should prepare an annual plan of aggregate

amount of domestic borrowing. The plan—

based on the annual budget cash ow

forecasts—should be made public. The

overall borrowing plan should preferably be

broken down on a monthly basis in an internal

document based on monthly cash ow

forecasts. The government should prepare

the market for the issuance by publishing a

borrowing calendar in advance for wholesale

instruments. (Panzer et al., 2015, p. 38)

This guidance about PFM processes is not

invented from thin air. It draws on the wisdom

and experience of generations of practitioners and

often from the observation of PFM systems in a

wide array of OECD countries. Multiplied across

the many dimensions of 45 separate frameworks,

this represents a substantial canon of conventional

wisdom about the PFM system.

This disciplinary approach of specifying

conventions concerning the desirable

performance of PFM processes has high

practical utility. As discussed earlier, the

approach has provided a conceptual map of

complex territory and is used to serve multiple

purposes. Assessment of conformity with the

conventions, through formal frameworks or

otherwise, provides a starting point for discussing

the PFM system in a particular country (PEFA

Secretariat, 2019). The use of convention-based

assessment frameworks can also satisfy an innate

tendency to want to emulate successful peers

(Lawson, 2012; Di Maggio & Powell, 1983).

These assessments are convenient for assessing

the strength and integrity of government systems

when deciding whether to channel external funds

through government budgets. External nance

providers, including donors and IFIs, can use the

independent assessments as “ready reckoners”

to anchor their judgments. Finally, having a

consistent set of generally applicable metrics over

a long time period permits comparative research

and analysis of trends and correlations.

The benets associated with PFM conventions

also mean, however, that they can become

xed and entrenched, without continuous

evolution based on emerging evidence. There

is a recognized distinction in organization theory

between “single-loop” and “double-loop” learning

within disciplines and organizations (Argyris,

1977) and a wider and closely related discussion

in the literature on learning within organizations

(Argyris & Schön, 1978, 1996; Senge, 1990).

Single-loop learning involves settling on a

conceptual framework and continuing to apply

it, rening the application of the framework but

not the framework itself. In double-loop learning,

feedback from the real world shapes the model

as well as the way in which it is applied. Public

management in general is stuck in a single

loop. The approaches described in the previous

chapter for learning better ways to help countries

conform more closely to the PFM conventions

are important—but ultimately this is still single-

loop learning. Continuously assessing whether the

conventions themselves could be improved would

represent double-loop learning.

The conventions need refreshing

Despite the impressive and valuable

disciplinary strength of PFM as a standalone

agenda in the development sphere, better PFM

is not a result that matters on its own; PFM

is part of the means not of the ends. Chapter

2 discussed the evolving objectives of PFM,

noting that it is important largely because of its

assumed contribution to government actions

and development results. Those include service

delivery and sector outcomes, both of which are

of particular interest in this report. The potential

of PFM to make a positive contribution to tangible

results in socioeconomic development has, over

the past decade, prompted a renewed focus by the

international community on trying to understand

the specic ways in which PFM may be relevant

to service delivery.

The knowledge underpinning the PFM

conventions is not certain and has a limited

empirical basis. The performance specications

referenced in PFM assessment frameworks

have not been proven empirically for their

contribution to development results and may not

even be widely observed in OECD countries. For

example, “if there were bullet proof evidence

that having competitive procurement generates

reliable and efcient provision of goods and

services, for instance, we would be safe in

only measuring compliance with competitive

procurement procedures. Unfortunately, we do not

have this evidence, and it must be said that many

OECD countries achieve relatively high levels

of functionality without necessarily complying

with these processes” (Andrews et al., 2014,

pp. 11–12). As noted in Chapter 2, the evidence

indicating that conformity with PFM conventions

is either necessary or sufcient to achieving the

three standard PFM objectives is limited.

Finding the links between PFM and

development results: the example of

service delivery

While it is clear that sound PFM relates to the

government’s effective provision of collective

goods and services, the important empirical

question is how.

Those collective goods and

services (often described in shorthand as “public

services,” “public goods,” or “service delivery”)

typically include—but are not limited to—

education, law and order, environmental protection,

national defense, roads and bridges, hospitals and

health care, welfare services, and public transport.

A distinction is made here between provision

through the budget and provision through other

instruments of government, namely subsidies

(although often technically considered a subset of

provision), production, and regulation (Hughes,

2018). Explaining the relationship between

PFM processes and service delivery results in

these areas is difcult. Sound PFM systems are

generally thought to be necessary for delivery, but

there is not a strong or extensive evidence base

for that contribution. The effects of other factors

such as the human resource management system

(for example, inadequate staff competencies and

perverse motivations) makes it clear that PFM

systems alone are also not sufcient for delivery

(Campos & Pradhan, 1996; PEFA Secretariat,

2016; Schick, 1998a; World Bank, 1998). While

it is widely accepted that PFM matters for service

delivery, the specic ways in which it matters are

not well established.

Research into the relationship between PFM

processes and service delivery offers tantalizing

glimpses but few certainties—even in the health

sector, where work is most well-advanced

(Annex H). Focusing on the delivery of efcient,

high-quality, accessible, and equitable public

services is rising fast as a priority for donors and

client governments. It is relatively straightforward

to identify the relevance of PFM conceptually and

anecdotally, but harder to show systematically the

relationships and dependencies. Emergent work

to investigate the links between PFM and service

delivery has looked at the issues in general and in

some specic country cases (Goryakin et al., 2017;

Krause, Sweet, Hedger & Chalise, 2013; Welham

et al., 2017; Welham, Krause & Hedger, 2013;

World Bank, 2016). The most intensive efforts

have focused on the health sector, where there is

burgeoning analytic work starting from both the

PFM and sector perspectives (Goryakin et al., 2017;

Krause et al., 2013; Welham et al., 2017; Welham

et al., 2013; World Bank, 2016; Barroy, Kabaniha,

Boudreaux, Cammack & Bain, 2019; Cashin et al.,

2017; Piatti-Fünfkirchen & Schneider, 2018). The

relationship between PFM and service delivery

has also been the focus of an increasing amount of

work on diagnostic frameworks in the health sector

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

(ODI, forthcoming). Yet, despite these initiatives,

knowledge remains partial and tentative.

Health care provides a stark example of the

challenges involved in determining the precise

ways in which PFM matters for service delivery

and, in turn, exactly how service delivery drives

sector outcomes. Welham et al. (2017) nd that

a stronger PFM system is associated with lower

under-ve and infant mortality, and higher life

expectancy at birth. Goryakin et al. (2017) show the

challenges in attributing these outcomes to PFM

systems and refer to Filmer and Pritchett (1999),

who nd that public spending accounts for less

than 1 percent of the variation in child mortality

rates, whereas 95 percent can be explained by

national income per capita, inequality in income

distribution, female education, and religious and

ethno-linguistic diversity. In such a situation, it is

not clear whether the problem is that the services

are not delivered well, that the services are not

delivered at all, or that the services are delivered

but are not relevant to the desired outcomes.

In all of these cases, little is known about the

contribution of the PFM system beyond some

general ideas.

The structure of PFM for service delivery

has to deal not only with sectors but also with

subnational governments of various types and

with varying levels of autonomy. Global agendas

such as the SDGs, the Addis Ababa Action

Agenda on nancing for development, the Paris

Agreement on climate change, and the New Urban

Agenda (Habitat III) emphasize the critical role of

subnational governments in service delivery and

nancing for sustainable development. Some of

these agendas specically argue that urban and

metropolitan governments are in a much better

position to manage and coordinate the nancing

of integrated territorial development and to deal

with some elements of climate change than sector

siloes of central government agencies. Multiple

development assistance interventions are already

specically targeting subnational PFM—probably

more than the number focused on PFM in sectors.

Accordingly, decentralization and levels of

government must be considered centrally in any

thinking on how PFM relates to service delivery.

Refreshing the conventions is necessary but

will be challenged

It is not a point of minor pedantry to suggest

a more dynamic approach to revising the

conventions and the frameworks in which

they are embedded. If the conventions stand as

guideposts indicating where discussions might

begin on strengthening PFM processes, then

incorporating or retaining less relevant content

distracts attention and precludes alternative and

better ideas. Revisions to the PEFA framework

in 2016 placed greater emphasis on transparency

and management of liabilities and scal risks, and

on internal control, and widened the scope of the

framework to include extrabudgetary units and

the management of social security funds. While

extensive consultations identied these areas as

priorities, the revisions were not as signicant

as some PFM experts and researchers thought

necessary (Krause et al., 2014). Others have

argued more recently that the PEFA framework

focuses excessively on formal procedures and on

form over function (Hadley & Miller, 2016) and

ignores strategic interactions in the budgeting

process (Andrews, 2007; Dabla-Norris et al.,

2010). Comparing the conventions in the PEFA

framework with assessments conducted by the

United Kingdom’s National Audit Ofce of the

multiyear spending reviews undertaken in the

United Kingdom in 2010 and 2015 (National

Audit Ofce, 2012, 2015, 2018) points to

some dimensions that are not covered by the

PEFA framework (Hood, King & Piotrowska,

forthcoming in 2019).

Adjusting the conventions is not a

straightforward task and will be challenged—

not least because it has implications for

the composition and use of standard PFM

assessment frameworks. For example, PEFA

asks whether a country’s PFM system conforms

to a set of standards that implicitly represent the

conventions. Stability in the conventions and,

hence, in the assessment frameworks allows

time series comparisons by providers of nancial

and technical assistance and by researchers

and analysts. To assess country progress, the

frameworks need to be relatively stable over

time and revisions to them need to prioritize

comparability between versions. Furthermore,

technical advisers in PFM and wider groups of

development assistance professionals are now

accustomed to treating the conventions embodied

in PFM assessment frameworks as universally

valid and applicable—across countries and across

government objectives.

These conventions have

become highly normative, and there are powerful

organizational and professional incentives that

perpetuate them (Scharfstein & Stein, 1990).

Despite these evident difculties, rethinking the

conventions will be important in ensuring the

continued relevance of PFM as a discipline. Two

approaches may hold promise in this area.

Some ways forward: updating and adapting

the conventions

More actively curating the life cycle of

PFM conventions

Proposal 4: Review the current conventions

embedded in standard PFM assessment

frameworks to test evidence of results and the

degree of applicability in advanced economies

The proposal is to make more explicit the

currently tacit criteria by which PFM

conventions are established and become

embedded in standardized PFM assessment

frameworks. Focusing rst on foundational PFM

processes remains the accepted wisdom of the

PFM discipline, and those “basic” elements have

not ceased to matter (Schick, 1998b). Accordingly,

this report proposes a set of criteria for judging

whether it is appropriate to treat any particular

PFM process and its operational performance as

a “basic,” or foundational, convention that might

then be relevant across countries (Table 3). The

three proposed criteria of rationality, universality,

and empirics echo in part the approach proposed

by Andrews (2008) for how to move “beyond

indicators without theory” in work on good

governance.

Along with establishing which particular

specications of PFM processes may be

suitable for inclusion in standard frameworks,

a corollary is to consider which established

PFM conventions might be “retired” from

the assessment frameworks. That would apply

if the evidence on their applicability turns out to

have been overstated or if there is concern about

excessive promotion in contexts where they are not

well suited. The same criteria would apply in both

instances: considering potential new conventions

and reviewing existing conventions. Among the

many PFM assessment frameworks in existence,

an obvious candidate for such a review would be

the PEFA framework because it covers the whole

PFM system and is among the best-known and

most widely used.

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University



Case

Criteria (✓ or X)

Implicaon

1.Raonality

Is it based on clear

reasoningorwell-

accepted theory?

2.Universality

Is it observed

in all advanced

economies?

3.Empirics

Is there

systemac

evidence on

results?

✓ ✓

Notrequired

Qualiesasafoundaonalelement

ofPFMinallsengs,suitable

foruseinstandardassessment

frameworks

✓

Qualiesforfurtherconsideraonas

arecommendedarrangementacross

countrysengs

✓

X X

Notrecommendedforadopon

withoutcarefulexperimentaonin

eachseng

Note:CaseAaddressesinstancesinwhichparcularspecicaonsofPFMprocessespassboththetestsofraonalityand

universality.CaseBreferstosituaonsinwhichPFMprocessspecicaonspassthersttestofraonalitybutnotthesecond

testofuniversality,andthuswouldneedtobebackedupbysystemicevidenceofresults.CaseCapplieswherethespecicaon

ofaPFMprocessisneitheruniversallyobservednorsupportedbysystemacevidenceofresults.

Source:Authors’formulaon.

The starting point for curating the PFM

conventions is to only consider particular PFM

process specications foundational or generally

valid if they satisfy two criteria. The rst is that

they are rational and based on accepted theory

or other logic. For example, budgets should be

comprehensive so that all claims are considered

together to avoid logrolling. The second is that

they are universal, in the sense that they are found

in all advanced economies. Universality is a proxy

for assuming that the particular PFM process

specications are achievable, practicable, and

strongly associated with successful socioeconomic

development—even if the exact causal link cannot

be ascertained. If particular specications of PFM

processes pass both these tests of rationality

and universality (Case A in Table 3), then it is

reasonable to treat them as PFM conventions

without requiring further evidence. Meeting these

two criteria would therefore be enough to justify

inclusion in standard PFM assessment frameworks

as a working convention.

Where a particular specication of a PFM

process is not universally observed, then

systematic evidence of results across a range

of relevant settings becomes important for

assessing suitability. If proposed PFM process

specications pass the rst test of rationality but

not the second test of universality, then to achieve

the status of recommended conventions they

must be backed up by systematic evidence of

results (Case B). Meeting those two tests would

suggest that the PFM process specication can

be treated as a recommended arrangement across

country settings. The challenge here is that the

external validity of empirical ndings—that is,

their generalizability to other settings—is always

limited.

In cases where the specication of a PFM

process is neither universally observed nor

supported by systematic evidence on results,

then much greater care would be needed with

regard to its application in practice (Case

C). MTEFs and performance-based budgeting

provide examples. To receive an “A” score on

indicator 16.2 on medium-term expenditure

ceilings, the PEFA assessment framework requires

that “aggregate and ministry-level expenditure

ceilings for the budget year and the two following

scal years are approved by government before

the rst budget circular is issued” and an “A” score

on indicator 16.4 on consistency of budgets with

the previous year’s estimates requires that “the

budget documents provide an explanation of all

changes to expenditure estimates between the last

medium-term budget and the current medium-term

budget at the ministry level” (PEFA Secretariat,

2016, p. 48). This specication would not qualify

automatically as a PFM convention because it is

not universally found in advanced economies.

Although there is some positive supporting

evidence (Vlaicu, Verhoeven, Grigoli & Mills,

2014), it is not conclusive across countries (World

Bank, 2012a). Pending clearer empirical evidence,

such PFM process specications would not be

promoted as conventions through standardized

assessment frameworks.

Identifying new PFM conventions that matter

for service delivery

Proposal 5: Use PFM assessment frameworks

to collect data and evidence “at the frontier”

to build insights about which aspects of PFM

operational performance matter for service

delivery

Potential future conventions could be found

through data collection “at the frontier”

by scoping out new ideas about how the

performance of PFM processes affects service

delivery, as a prominent example of government

actions. Almost all country-level assessments of

PFM performance are currently both formal and

ofcial. They are formal in the sense that they

rely on the application of one or more assessment

tools and accompanying guidelines about their

methodology and use. They are ofcial in the

sense that international organizations, donors,

or ofcial secretariats are typically responsible

for running, authorizing, validating, or assuring

the assessments and their results. These standard

assessments could be complemented by use of the

same tools—or components of them—for informal

or experimental inquiry with the aim of generating

knowledge rather than measuring performance.

A parallel and complementary step would be

to reappraise how service delivery is addressed

in the standard PFM objectives and to propose

revisions to make the relevant objective more t

for purpose. Table 2 set out an indicative proposal

for a potential reformulation of this objective to

focus more on operational efciency and service

quality: “Government should provide goods and

services at a cost that achieves ongoing efciency

gains and (to the extent appropriate) is competitive

with market prices. It should use modes of

delivery that optimize service quality and ensure

accessibility.” This proposed recasting enhances

the existing coverage of operational efciency

by bringing quality, accessibility, and mode of

delivery explicitly into scope. Once rigorously

developed based on strong theory and evidence,

the proposed new version of the objective

would be disseminated for wide discussion and

consultation.

The dimensions of PFM operational

performance that are most relevant to this

revised objective on service delivery are likely to

be found in the intermediate functions of well-

performing PFM systems. Whereas many of the

current conventions in standard PFM assessment

frameworks are “forms” (or de jure dimensions)

of PFM (Hadley & Miller, 2016; Krause et al.,

2014), there has not been a systematic attempt to

distinguish the conventions that are “functions”

with relevance to service delivery and sector

outcomes. Andrews et al. (2014) posit that a set

of functional specications for PFM processes

could be identied and tested with reference to

desirable intermediate results such as “prudent

scal decisions, credible budgets, reliable and

efcient resource ows and transactions, and

institutionalized accountability” (p. 6). They

break down these result areas into sixteen more

specic “functional concerns” of PFM that seem

to matter for service delivery. Those range from

“cash is provided to spending agencies when

agreed, in agreed amounts” to “concerns raised by

independent assurance exercises are transparently

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

discussed by citizens’ representatives, receive

timely follow-up and redress by the executive”

(Andrews et al., 2014, p. 6).

These intermediate result areas represent

attempts to nd further stepping stones in

the results chain, in this case moving from

PFM conventions toward service delivery

and development results. Not all of these result

areas are missing from the standard assessment

frameworks, but current PFM frameworks are

incomplete when it comes to some aspects of

functional performance (Andrews, 2007; Hadley

& Miller, 2016; Hood et al., forthcoming in 2019).

Aspects to integrate might include government

staff perceptions of budget credibility, the

reliability of resource ows and the accuracy of

transactions, sequencing of reforms, and external

assessments of accountability arrangements. Such

metrics would focus on ideas that are of interest

but have not achieved the status of conventions

accepted by the PFM discipline.

Existing PFM assessment approaches can be

accompanied by institutionalized collection of

data that are more speculative and draw on a

wider array of frameworks. This could involve

adapting or honing assessment frameworks or

other analytical tools such as public expenditure

reviews, public expenditure tracking surveys,

and service delivery surveys to learn more about

the relationships between PFM processes and

operational performance on the one hand and

service delivery on the other (Welham et al., 2017).

The tools currently under development by sector

experts and decentralization specialists provide

a rich menu of potential metrics. While some

preliminary work has been done in the health sector

to trace the contributory relationships through

the delivery chain from nancial allocations and

PFM processes downstream to results (World

Bank, 2016), there are few experiments that

compare variations in existing and potential PFM

conventions.

The mechanism for achieving this

institutionalized data collection could be a

virtual and open-access “beta” or “sandbox”

space for experimentation that would be

independent of the funding and afliation

pressure of major assistance providers. It would

be partly an observatory and partly a mechanism

to support inquiry, testing, piloting, and learning

about the effectiveness of PFM (Foster, Rinehart

& Springs, 2019).

There is currently no particular

institutional location for data and emergent ideas

on how PFM processes relate to service delivery.

The IMF, World Bank, PEFA Secretariat, and other

assistance providers already act as custodians of

most of the PFM assessment frameworks (Annex

F). Those entities, together with some research

institutions, already conduct long-term tracking

of country progress and trajectories. In expanding

this tracking capability, it would be essential to

involve specialists beyond the PFM discipline.

For example, people working in the sectors and

on subnational or decentralization issues could

collaborate with PFM specialists to develop

metrics and lessons, working backward from the

point of delivery. Government ofcials and service

users would also be important constituencies to

include in the task group.

Harnessing PFM for policy change

We now build on the proposed expansion of the

standard PFM objectives to consider how PFM

processes and technical PFM advice could

better inform and support policy change. The

remainder of this chapter considers how to harness

PFM and its disciplinary conventions to assist

governments in driving strategic reprioritization

in budget policies and scrutinizing the choices and

trade-offs among socioeconomic policy objectives.

Rethinking how PFM relates to public policy

Pressure is mounting for policy change

Policy and political pressures are driving

demand for fundamental changes in how public

nance is managed. A decade ago, Caiden (2010)

observed the looming demands of major policy

challenges:

In the face of global developments…

governments will be called on to expand their

investments in infrastructure, edu-cation,

research, energy, and technological innovation,

and to meet urgent challenges, such as nuclear

waste disposal, climate change, terrorism

and natural disasters. Many ‘big questions’

may not have been settled at all. Can existing

budget institutions and processes transcend

their current modes of conducting business to

cope with the demands that will be placed on

them? (p. 209).

Caiden (2010) further anticipated a resurgent

concern for equity and social justice in public

nance, arguing that: “budgets…are what people

and their governments choose to make of them.

They are instruments of economic and social

policy, so budgetary values may need to be

sacriced to more important societal values, social

stability, compassion for those who are unable to

care for themselves, justice, and equity” (p. 209).

Reprioritization of public resources and

redistribution of income and wealth have

recently gained political salience in both

advanced economies and lower-income

countries. Climate change mitigation,

infrastructure development, universal health

care, and reducing marginalization and inequality

have all assumed an even greater force and

resonance in public nance over the past decade,

following the global nancial crisis in the era of

the SDGs, the “leaving no one behind” agenda,

and the Paris Agreement on climate change. At

the forefront of this movement, one country has

shifted to a “well-being budget” (Government

of New Zealand, 2019) and others have made

statutory commitments to mitigate climate change

or have signicant political interests in doing so

(Prime Minister's Ofce, 2019; U.S. House of

Representatives, 2019). The IMF is increasingly

concerned about balancing economic prosperity

with adequate social sector spending and reduced

inequality (Lagarde, 2019).

The PFM discipline has remained detached

from policy questions

An emerging critique of the PFM discipline

in the development sector is that it has added

more inertia into budgetary institutions,

detaching PFM from evolving policy concerns.

“Grand challenges” such as eradicating poverty,

mitigating climate change, and expanding

public infrastructure require reprioritization and

reallocation of public resources on a scale without

much precedent outside of scal crisis episodes.

Rising inequality demands a fundamental

rebalancing of scal policy norms to consider the

distribution of income alongside aggregate scal

objectives. Yet neither of these two objectives

is well-supported by the PFM discipline’s

conventions and approaches.

Managing strategic reprioritization

Budgeting inherently constrains reallocation

Current PFM conventions struggle to support

the pace and scale of policy change and

reallocation needed to tackle grand challenges.

Those conventions are strongly associated

with incrementalism in budgeting (Wildavsky,

1992). Indeed, the budgetary process in essence

“recognizes the priority of old claims, stabilizing

government, giving citizens clear expectations

of continuity in services and entitlements, and

diminishing conict” (Caiden, 2010, p. 208).

Several factors converge to make incremental

adjustment the dominant paradigm over a

more “rationalist” approach in budgeting that

elevates information and analysis as the basis

for budgetary decision making (Fozzard, 2001).

These incrementalist traits have increased in

recent years, enhanced by developments such

as medium-term scal frameworks and baseline

budgeting that incorporate multiyear incentives

for policy stability into budgeting arrangements

(Schick, 2013).

Just as conformity with PFM conventions

contributes to incrementalism in budgeting, so

problem-based and adaptive approaches lean

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

toward incremental change in institutional

reform.

In time-honored tradition, decision

makers and their advisers “muddle through”

using a process of “disjointed incrementalism”

(Fozzard, 2001; Lindblom, 1959). The combined

effect is a bias toward continuity and stability in

budgeting and budgets. Even the most concerted

attempts at innovation in tools and processes—

such as the Planning Programming Budgeting

System, zero-based budgeting, Rationalisation des

Choix Budgétaires, and spending reviews—have

not succeeded in overcoming the inherent inertia

in budget policy making (Robinson, 2016).

Learning from past attempts at supporting

policy change

Adjusting the system of conventions to better

equip PFM processes to support major changes

in policy direction will not be an easy task. The

obvious precondition is strong political intention

at the highest level to reallocate resources of any

scale or signicance. Beyond episodes of crisis or

emergency, which can suspend the usual rules of

the game, the question is how PFM conventions

might be recongured to facilitate change instead

of continuity. Even if the desirable direction were

more apparent, the conventions embraced by

the PFM discipline are sticky and only slowly

responsive to new evidence and research ndings.

Attempts to learn lessons from past efforts

at reprioritization and reallocation of public

nance will need to look across country

contexts. There is a rich set of illustrations within

OECD settings of approaches to adapting PFM

processes to address particular policy concerns.

The experience of spending reviews has been a

focus of analysis, including studies of episodes

of budgetary reallocation outside of crisis periods

up to 2005 (Kraan & Kelly, 2005). More recently,

in the context of the global nancial crisis and

subsequent austerity programs, the OECD

Working Group of Senior Budget Ofcials has been

reviewing the experience of “strategic spending

reviews”

to address the prioritization of public

expenditure and the dimensions of inuence on

allocations (Catalano & Erbacci, 2018; Lau, 2011;

Robinson, 2013).

There are some lessons but few

convincing conclusions about what has worked

and what has not. The conclusion for the OECD

is that, although some countries became more

adept at prioritizing the allocation of new scal

space, “little attention was paid to mechanisms to

reprioritize (reallocate) baseline expenditure” and

there is now “widespread desire to build strong

institutions and processes capable reallocating

expenditure, including—but not limited to—

spending reviews” (Robinson, 2016, p. 31).

Some lessons may be learned from previous

attempts in lower-income countries although,

again, there are few successful and positive

examples. Initiatives include pro-poor budgeting

approaches in the 1990s (Carter, 2015),

attempts

at gender budgeting that emphasize policy-related

expenditure proposals (Elson, 1999),

proposals

on climate change mitigation through a more

fundamental review of budgetary and accounting

procedures as well as options for modeling

climate-related spending (Gilmore & St. Clair,

2018; Ricke, Moreno-Cruz, Schewe, Levermann

& Caldeira, 2016), and the examination of pension

and infrastructure liabilities (IMF, 2018).

Support for pro-poor budgeting initiatives is an

instructive case study. Reforms to PFM processes

introduced in lower-income countries during the

1990s and 2000s, when “pro-poor” expenditure

reforms were heavily promoted, often proved

insufcient to match countervailing incentives and

vested interests.

When institutional changes did

lead to increased spending, it was not sufcient

to drive improved results in most cases. Evidence

on the effectiveness of pro-poor budgeting

remains ambiguous (Simson, 2012; Wilhelm

& Fiestas, 2005; Williamson & Canagarajah,

2006),

in much the same way that spending

reviews in OECD countries have struggled

to drive successful strategic reprioritization.

International agencies eventually lost interest in

pro-poor budgeting approaches, as can be seen

from their disappearance from the discourse

on PFM. It may be the case, though, that these

efforts were simply not sustained or supported

sufciently well, instead becoming procedural

triggers for debt relief under the Highly Indebted

Poor Countries initiative.

Managing trade-offs among policy

objectives

Balancing stabilization and distributional

objectives in public finance

Tensions are increasing between policies to

achieve macroeconomic stabilization on the one

hand and more equitable income distribution

on the other. At a time of high and rising inequality

in income and wealth (Milanovic, 2016; van der

Weide & Milanovic, 2014), the negative impact

of scal consolidation linked to stabilization lies

behind many current socio-economic concerns.

Changing perceptions about social justice are

creating pressure on governments to implement

stronger and more active redistribution policies

(Mason & Kluegel, 2000; Munro, 2017). Evidence

shows that major episodes of scal consolidation

are associated with a persistent rise in income

inequality (Ostry, Loungani & Berg, 2019). There

is growing support for the argument that the

economic benets of scal consolidation must

be weighed more carefully against the short-run

social costs (Lagarde, 2019).

This notion has

prompted greater interest in PFM processes for

examining the likely distributional impact of scal

adjustments, and in institutional arrangements for

incorporating equity considerations and ensuring

that the burden of scal adjustment is spread

fairly and sustainably. The latest IMF review of

program conditionality between 2011 and 2017

found that the quality of scal adjustment under

IMF programs was inadequate and recommended

a greater focus on the quality of social spending

and the impact of program policies on poor and

vulnerable groups (IMF, 2019, p. 26).

Austerity policies implemented in the wake of

the global nancial crisis are an acute, but by no

means exceptional, example of how aggressive

short-term scal contraction can impose severe

welfare costs. Fiscal structural reforms in Greece

starting around 2010 offer an illustration of the

tensions between aggressive scal austerity,

wider social welfare, and political exigencies

(OECD, 2018).

It is now widely argued that

the bailout plans were based on overly optimistic

economic scenarios and unrealistic assessments

of what would be politically achievable goals

(Alesina, Favero & Giavazzi, 2019). Indeed, “the

May 2010 agreement committed Greece to an

adjustment that, while not unprecedented in its

size, was unprecedented in the adversity of the

macroeconomic context within which it was to be

implemented” (Ardagna & Caselli, 2014, p. 30).

The general lesson is that more and better analysis

of the effects of austerity policies on income

distribution is required across all cases (Rogoff,

2019).

These concerns and policy dilemmas are now

more routinely debated across country income

groups, from advanced economies to lower-

income countries. That is not to suggest an

equivalence in terms of the levels of hardship or

the obstacles to policy change, but to consider

how better analysis might help in considering

trade-offs. Although the example of the Greek

bailouts is the cause célèbre, scal policies

pursued since 2010 in the United Kingdom are

also under growing international scrutiny. There

is greater questioning of the choice to prioritize

deep and rapid scal retrenchment in pursuit of

macroeconomic stabilization over distributional

concerns (Alston, 2018, pp. 22–23).

While

clearly the choice is political, this emphasizes

the importance of using impact analysis more

effectively to inform and support both the

development and the implementation of scal

policy. In the United Kingdom case, it is argued

that “…the government should initiate an expert

assessment of the cumulative impact of tax and

spending decisions since 2010 and prioritize the

reversal of particularly regressive measures”

(Alston, 2018, p. 23).

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Balancing stabilization and allocation

objectives in public finance

Governments across the country income

spectrum are starting to rethink their

approaches to debt and decits. Debates about

the long-term scal sustainability of high levels

of public debt—and the balance between scal

and welfare costs—are now a live feature in

OECD countries and most conspicuously the

United States (Blanchard, 2019; “Economists

Are Rethinking Fiscal Policy,” 2019; Furman

& Summers, 2019). The same applies in lower-

income countries with regard to infrastructure

investment. The prescription of tight scal policy

faces a renewed test as China’s Belt and Road

Initiative reveals some incompatibility between

the scal stringency of conventional PFM and a

debt-nanced and infrastructure-led economic

growth model.

Increasing governments’ access

to borrowing from alternative concessional

sources and international capital markets further

compounds those tensions (Haque, Bogoev

& Smith, 2017; Presbitero, Ghura, Adedeji

& Njie, 2015). Continuing debates about the

merits and risks of international nancing from

capital markets or Chinese investment in African

infrastructure are a clear illustration (Chen, 2019).

That realpolitik could shift the nature of analysis

and debate away from efforts to dissuade from

borrowing and toward consideration of trade-offs.

New PFM conventions and new PFM objectives

Global policy and research initiatives to

address inequality are attempting to alter the

policy bias toward aggregate outcomes and to

shift political incentives toward distributional

action and trade-offs. The economics discipline

in international development has already started to

respond.

Norms in scal policy share the bias

toward aggregate effects such as scal discipline

and have given a powerful role to scal rules,

scal councils, multiyear frameworks, and accrual

accounting and budgeting. Stronger frameworks

and tools will be needed to assist governments

in evaluating distributional considerations and

examining trade-offs with other policy objectives.

Without taking any political or policy position

on the choices, the case is growing for better

and more multidimensional information and

analysis—allied with stronger participation and

legitimacy—to inform those choices.

This report has proposed expanding and

adapting the three standard objectives of PFM

to support a rebalancing of policy priorities

between scal and socioeconomic outcomes.

Expanding and adapting the PFM objectives

reveals long-standing normative conicts more

strongly and explicitly. “Budget policy involves a

number of distinct objectives, but these overlap in

practice, thereby complicating an efcient policy

design…. It is important to keep in mind that there

are three distinct policy objectives and policy

should try to minimize conicts among them”

(Musgrave & Musgrave, 1989, pp. 13–14). It is a

fundamental task of democratic politics to address

these conicts and attempt to mitigate or resolve

them.

The space for the PFM discipline to contribute

lies in developing processes and conventions for

more active consideration of potential spillover

effects and trade-offs in scal policies.

The

potential for conicts applies to both the revenue

and the expenditure side of the budget but, as

noted above, is often most pronounced and most

contentious in the balance between maintaining

a sustainable scal position and ensuring an

equitable distribution of income. Underlying PFM

processes are needed to produce the information

that can help to expose and scrutinize tensions

between prosperity and inclusion in scal policy

choices.



governance

Social justice and the strength of the social

contract have re-emerged as dominant

concerns in public policy. Participation and

accountability matter, both as intrinsic values and

as functional contributions to the achievement

of scal objectives and other policy outcomes

(Alt, 2019). Accountability and transparency

emphasize the nuts and bolts of democracy, equity,

and trust (de Renzio, 2019).

The connections

to rising concerns about fairness, social justice,

intergenerational equity, and environmental

sustainability are clear to see—although nding

the right response is challenging (Finger &

Gressani, 2014).

The PFM discipline can play a stronger role

in opening up PFM processes and conventions

to foster citizen inuence and participation

in scal governance. Closely related to policy

choices and effects are the processes needed to

permit civic participation in ways that can inform

policy decisions and establish their feasibility

and legitimacy. While mainstream advice now

recognizes transparency and participation as

signicant outcomes (IMF, 2017a), these values

seem to be subsidiary rather than foundational.

PFM must retain its classic job description

of supporting scal policy objectives, but not

at the expense of failing to address growing

economic inequality and the erosion of real

incomes, declining trust in government, and rising

intolerance and vulnerabilities to populism and

authoritarianism. The next generation of ideas and

tools for strengthening the voices of citizens in

scal governance must rise to this challenge.

Proponents of a more inclusive perspective have

yet to identify how local- and municipal-level

actions can meaningfully be scaled up to the

national level. It is not clear how to move beyond

small-scale local participatory budgeting processes

(Montambeault, 2015). Few governments have

successfully replicated this type of experience at

the national level or extended public engagement

beyond small infrastructure projects. Mainstream

advice positions transparency and participation

as distinctly modest contributions to this agenda

(IMF, 2017a). Technological advancement may

prove to be a positive driver or enabler of change,

but much of that potential has yet to be achieved

(Peixoto & Sifry, 2017). Some argue that moving

from the small-scale and local to the national level

requires the inclusion of rights-based indicators

that measure whether revenue and expenditure

policies meet people’s needs for food, health

care, housing, education, social protection, and a

healthy environment (Rudiger, 2018). These are

dauntingly high bars to clear (Ubaldi, 2013).

Two ideas explored below may help open up PFM

to supporting major policy change and analyzing

policy trade-offs.

Some ways forward: new conventions and

new analytical tools

The comparative advantage of the PFM

discipline lies in its focus on processes and

conventions. PFM processes create a framework

for managing public money and reinforcing scal

policy decisions. The resulting PFM conventions

have tended to emphasize arrangements that lock

in incrementalism and continuity, while scal

policy norms lock in a bias toward prioritizing

scal constraints or other scal objectives. These

issues are predominantly the preserve of scal

policy experts, but the PFM discipline and its

PFM conventions can play an important role.

Exploring how PFM can better support major

policy changes

Proposal 6: Investigate how PFM conventions

and processes could better support non-

incremental policy change through strategic

reprioritization in budget policies

Better understanding how PFM processes

could be used to support changes in policy

direction is a priority. Some investigation

of examples and lessons regarding how PFM

process and conventions have accommodated or

supported instances of major reallocation could

shed light on how innovations in PFM may be

helpful. Given the limited reference points or case

examples of successful strategic reprioritization,

a broader review might need to include episodes

of signicant spending increases (that is, the

allocation of additional public revenue rather than

reallocation within a xed envelope) or cases of

major subsidy reductions. The focus would be on

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

PFM dimensions (Kraan & Kelly, 2005), although

the dominant policy dimensions and political

factors would also need to be recognized and

understood.

The starting point would be to take stock of

the lessons learned through many previous

attempts to overcome incrementalism and

increase policy responsiveness in budgeting.

The review would identify the incentives toward

incrementalism, the historical track record of

attempts to move away from incrementalism in

OECD countries, and the lessons emanating from

past and current tools and initiatives. Examples

may include program budgeting, performance-

based budgeting, pro-poor budgeting, spending

reviews or cross-sectoral public expenditure

reviews, MTEFs, and country-specic approaches

such as Public Service Agreements in the United

Kingdom. Examining the success factors in cases

involving the politically difcult task of explicitly

transferring funds from a relatively low-priority

policy area to fund increased spending in an area

of higher priority will be critical.

Another way to approach this question is

to consider the t between particular policy

directions and the PFM processes and

conventions that underpin them. Judgment

will be needed to determine which processes

and conventions are versatile across multiple

policy objectives and which are more bespoke

elements that will support a particular policy

orientation. For example, there may be concurrent

but incompatible needs or pressures to constrain

spending and reduce the decit, to boost the volume

and ow of funds to infrastructure development

and service delivery, and to increase redistributive

transfers to marginalized con-stituencies facing

severe hardship and lack of opportunity. A strong

foundational set of PFM processes can help with

good information and analysis across any of

these priorities. However, different scal policy

orientations may indicate the need for different

processes to try to make those policies effective.

Fiscal rules and independent scal councils are

examples of such institutional innovations.

The purpose behind this proposal is to bring

the PFM discipline and thinking on the

budget process back into the political realm

in a way that functions across, or conforms

to, the specic realities of distinctive political

systems. One approach would be to study a

specic policy area, such as climate change,

to draw lessons on the ways in which current

PFM processes and conventions may impede

or facilitate grand policy adjustments. Drawing

wider lessons from efforts to achieve major sector

reprioritization in budget policy, concrete ideas,

and proposals could be piloted in countries in

real time, building on political demand for these

policy changes. Achieving this would require

assembling a group of policy makers (including

climate policy specialists and nance ministry

ofcials) and politicians that extends beyond the

PFM discipline. This multidisciplinary group

would lead the analysis and recommendations,

then the design, implementation, evaluation, and

dissemination of potential solutions. With the

growing emphasis by IFIs and other development

assistance providers on global public goods, the

venture would take advantage of the expanded

strategic space and demand.

It would be helpful if the work were not limited

to one country income group but conducted

instead as a collaborative initiative across the

income spectrum. Many of the policy challenges

and political pressures are common across

countries—whether as a result of the links to

global public goods or owing to the increasingly

convergent reference points and aspirations

among citizens (Kim, 2017). Many existing tools,

such as spending reviews or their equivalents,

are used across groups of countries. Political

challenges and obstacles apply in all contexts.

Consequently, some innovation in the types of

partnerships that are formed to investigate this

agenda would be valuable, reaching beyond the

usual organizational suspects and bringing in

former political leaders to help with insights about

better connecting technocratic PFM processes

with policy and political considerations.

Analyzing trade-offs among policy objectives

Proposal 7: Expand the toolkit of PFM to

analyze the trade-offs among policy objectives

and to consider the socioeconomic impact of

scal and budget policy choices

Demand is growing for the PFM discipline to

play a stronger contributing role in ensuring

that decision makers are better informed about

policy trade-offs and potential socioeconomic

impacts. The dominance of aggregate scal

concerns has arguably biased PFM toward giving

insufcient weight to wider socioeconomic values

and objectives. Yet aims such as reducing poverty,

improving sector outcomes, narrowing income

disparities, and strengthening citizen voice in

governance are increasingly on the radar in

discussions about PFM. There may be objections

from within the PFM discipline, and especially

from scal policy experts, about the risk of

overreach into policy matters. While some debate is

needed about whether this is an appropriate future

direction for the PFM discipline, it is notable that

leading players such as the IMF already appear to

be moving in this direction (Lagarde, 2019).

A useful rst step would be to launch a

reappraisal of the standard objectives in PFM

to make the case for their expansion and to

propose specic new formulations for debate.

Expanding the three long-standing PFM objectives

would make those objectives more relevant to

contemporary priorities and policy concerns.

It would also re-focus attention on the tensions

and trade-offs that exist among some of these

objectives, which had gradually become obscured

over the past two decades. Chapter 2 of this report

sets out an indicative proposal for the potential

reformulation of these objectives (Table 2).

Moving forward, the expanded objectives would be

developed more rigorously based on strong theory

and evidence. Most critically, the potential trade-

offs and tensions could be explained with some

concrete illustrations and current case examples,

especially concerning distributional incidence.

The revised proposed objectives would then be

put out for wide discussion and consultation.

There is a particular need to develop and

test new PFM conventions and supporting

processes that can be used to analyze the

distributional implications of policy choices,

as this is a signicant omission in the current

setup of conventions and frameworks. Several

tools and approaches could be tested, including

greater emphasis given to the revenue side of PFM.

Publication of socioeconomic impact statements

could be required or recommended when a

government proposes a specic scal course—for

example, requiring the assessment of impacts on

public services, income distribution, and poverty

levels. The government or an independent entity

could be required to produce assessments of the

distributional effects of scal and budget policy—

for example, impacts on distinct income quintiles

or on young or old populations in particular.

Governments could be required to publish a pre-

budget statement to encourage public debate and

analysis of policy options, thereby helping to

strengthen the role of the demand side in scal

governance.

Links could be made to the separate proposal to

introduce greater contestation of ideas as part of

the external technical advice provided to client

governments (Proposal 3). The main intention

would be to equip governments themselves with

better analytical tools and operating conventions

as part of their PFM system. These tools could also

be used by external actors to inform dialogue with

governments and technical advice. The core idea

would be to make explicit the trade-offs among

different objectives in managing public nance

as the basis for informed debate and government

decision making (Annex K).

Moving on from here

This chapter has explored some ideas for how to

engage with PFM as a more “open” system that

interacts uidly with public policy—covering

policy choices, government actions, and

development results. The proposals reconsider

Chapter 4: Rethinking Managing Public Finance in an “Open” System

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

the objectives of managing public nance and look

at the operational performance of PFM processes

from the perspective of their real-world impact.

They respond to two problems identied in the

introductory chapter of this report: rst, that the

ways in which PFM matters for service delivery

and development results are not well understood

and, second, that the standard objectives and

conventions of PFM are not equipped to deal with

political trade-offs or major policy change. The

nal four International Working Group proposals

are as follows:

(4) Review the current conventions embedded

in standard PFM assessment frameworks

to test evidence of results and the degree of

applicability in advanced economies;

(5) Use PFM assessment frameworks to collect

data and evidence “at the frontier” to build

insights about which aspects of PFM operational

performance matter for service delivery;

(6) Investigate how PFM conventions and

processes could better support non-incremental

policy change through strategic reprioritization

in budget policies; and

(7) Expand the toolkit of PFM to analyze the

trade-offs among policy objectives and to

consider the socioeconomic impact of scal and

budget policy choices.

The next and nal chapter summarizes the

proposals made throughout this report and

considers some of the potential implementation

challenges. It locates the discussion within the

strategic context of how external support for

institutional strengthening may be evolving and

looks at tactics to navigate the organizational

incentives that will affect progress on rethinking

international support for managing public nance.

An agenda for change: some

proposals for the PFM discipline

The purpose of this report is to provoke

debate and drive positive change. The report

began with four current problems identied by

the International Working Group on Managing

Public Finance based on extensive consultations

and literature review. Chapter 1 set out the guiding

parameters of the argument developed in this

report about how to rethink international support

for managing public nance in order to make it

more t for purpose in the 2020s.

Chapter 2 summarized the “state of PFM.”

It noted the distinctive strength of the PFM

“discipline” and the path that it has followed,

combining a strong set of conventions about

desirable practices with the growing adoption of

more problem-based and adaptive approaches. It

examined whether the standard PFM objectives

fully reect the range of contemporary policy

concerns and objectives. The chapter concluded

that current working assumptions in PFM represent

a “closed” model for diagnosing shortcomings and

developing proposals for action.

Chapter 3 examined the inuence and utility

of the “closed” model and proposed some ways

to build on its strengths without overstretching

the model. It recommended the development of

better data collection and knowledge-sharing

protocols to drive faster learning about how

technical advice can best support improvements

in PFM. It went on to recommend the introduction

of greater contestation among intervention options

and greater choice by governments as a means to

drive effectiveness.

Chapter 4 proposed some ways to open

up the model that underpins conventional

thinking about the objectives and operational

performance of PFM processes. This thinking

affects the diagnosis of PFM shortcomings and the

development of proposals for action. The chapter

acknowledged that this is difcult territory,

as it begins to depart from the concrete and

“universalist” assumptions of the “closed” model.

It considered the potential for developing a new

generation of approaches to international support

for managing public nance that better reect the

real world. The chapter recommended the more

active curation of PFM conventions based on

emerging evidence, along with experimentation

“at the frontier” to build insights about how

the PFM conventions relate to service delivery

and development results. It also recommended

investigating how PFM can better assist

governments in driving strategic reprioritization

in budget policies and in scrutinizing trade-offs

among socioeconomic policy objectives.

While the Working Group had the benet

of an independent perspective and extensive

consultations, not everyone will agree with

the perspective or the agenda set out in this

report. The aspiration is that the ideas developed

and discussed in the report will form the basis for

some practical action and experimentation—either

through the proposals put forward in response

to the identied problems (Table 4) or through

reactions to them. Improving the international

discourse on PFM and how it relates to advice,

money, and results will require action. This report

attempts to stimulate that action.

5. PUTTING IDEAS INTO PRACTICE

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Table 4: A recap on the problems and proposals

Problem idened in this report Working Group proposals

Proposals that build on the strengths of the exisng

“closed” model

Problem1:Thepaceoflearningabouthowexternal

assistancecanbeersupportimprovementsinPFMis

tooslow

(1)Assessoersoftechnicaladviceagainststandards

fortheprovisionofinformaonandevidencejusfying

theapproachandfortheplanneddisseminaonof

dataaboutimpact

(2)Establishpraconergroupstosharesensive

performanceinformaononPFMintervenonsona

voluntaryandcondenalbasistoidenfywaysof

improvingperformance

Problem2:Developmentagencies’preference

forfollowingacommonapproachhasreduced

opportuniestotestideasthroughcompeon

(3)Experimentwith“managedchoice”intheprovision

oftechnicaladvice,withacoalionoffundersand

clientgovernmentsandthecreaonofadedicated

fundingfacility

Proposals that envisage PFM as an “open” system

Problem3:ThewaysinwhichPFMmaersfor

servicedeliveryanddevelopmentresultsarenotwell

understood

(4)Reviewthecurrentconvenonsembeddedin

standardPFMassessmentframeworkstotestevidence

ofresultsandthedegreeofapplicabilityinadvanced

economies

(5)UsePFMassessmentframeworkstocollectdata

andevidence“atthefroner”tobuildinsightsabout

whichaspectsofPFMoperaonalperformancemaer

for service delivery

Problem4:Thestandardobjecvesandconvenonsof

PFMarenotequippedtodealwithpolicaltrade-os

ormajorpolicychange

(6)InvesgatehowPFMconvenonsandprocesses

couldbeersupportnon-incrementalpolicychange

throughstrategicrepriorizaoninbudgetpolicies

(7)ExpandthetoolkitofPFMtoanalyzethetrade-

osamongpolicyobjecvesandtoconsiderthe

socioeconomicimpactofscalandbudgetarypolicy

choices

Building a persuasive case for change: some initial ideas

Navigating development agency

incentives is critical

Another report, another set of recommenda-

tions. Why would any ofcial funder or

development agency want to take action to pursue

the ideas or implement the proposals in this

report? While the Working Group takes the view

that these recommendations will contribute to the

effectiveness of the PFM discipline in the medium

term, we acknowledge that they pose non-trivial

nancial and opportunity costs in the short term.

There are challenging time inconsistencies and

free-rider problems at play. Looked at from the

point of view of individual development agencies,

there is a question about why any one agency

would choose to invest in signicant new piloting,

research, and learning that would benet the eld

as a whole. Collective action among agencies may

be even more difcult to muster.

The Working Group recognizes the entrenched

and long-standing challenges of navigating

and inuencing development agency incentives

(Coyle et al, 2005; de Renzio et al, 2005). A good

place to start might be in thinking about how to

maintain a development focus on strengthening

public sector institutions. This agenda is a

major claimed comparative advantage of the

multilateral development system that provides

most international support for PFM. If agencies

recognize the strategic necessity to evolve—

for both external impact and institutional self-

interest—it may then be possible to mobilize

effort in support of ideas such as those set out in

this report.

The “institutions matter” argument may

be losing its force

There is a risk in not taking action. Professionals

in the PFM discipline, and in development agencies

more generally, are now used to an environment

in which institutional reform is accepted as the

point of entry to sustained improvements in public

sector performance. However, institutional reform

has been a relatively recent focus of development

assistance—dating largely from the mid-1990s

(Annex J). There is no reason to assume that it

will remain central to development assistance

efforts if it cannot make a direct and convincing

claim to supporting the evolving set of high-

priority development objectives. As this report

has highlighted, causal linkages between PFM

systems and results are currently more suggestive

than established. Combining the approaches of

conformity with conventions and of politically

aware diagnosis is promising, but it owes more

to recent development assistance traditions than

empirically based condence.

Development fashion could change again, with

a shift toward emphasizing policy outcomes

while leaving institutions to follow however

they can. Development assistance is diminishing

in political signicance as it declines in relative

nancial weight. Non-aid instruments are coming

to the forefront, including trade, remittances,

other nancial ows, and more general cultural

links. Similarly, classic development concerns

are waning as the number of lower-income

countries declines and more countries start to

transition from aid.

This change may well have

political consequences for the commitment to

support institutional reform and, more broadly,

the “governance agenda” of the past two decades.

PFM is a prominent aspect in that agenda and

is currently emphasized in external support for

lower-income and aid-dependent countries.

Predictions suggest that the focus on country-

level institutional strengthening may be

displaced in the future by a wider, more

dynamic set of priorities for development

cooperation. These arguments have been building

in force since the start of the decade as part of the

growing agenda on cross-border issues and global

public goods. One prominent thought leader in

global development has articulated this direction

of travel as follows:

Instead of today’s ODA there will be two

other major sources of ofcial outside funding

for developing countries. First will be lending

by multilateral…and national… development

nance institutions from their hard windows at

below market prices… Second will be transfers

from high-income countries to developing

countries in support of global public goods,

e.g. to protect forests, to subsidize clean energy

and compensate for fossil fuels not exploited,

to underwrite pandemic surveillance, and for

spending to undercut cross-border non-state

terrorism, piracy, drug and sex trafcking, etc.

Transfers in support of global public goods

will not necessarily go more to poorer than to

middle-income countries, as tends to be the

case today; they will go where they can be

most effectively deployed at the lowest cost.

(Birdsall, 2013)

The incentives could diminish for development

agencies to focus on complex institutional

reforms that cannot readily be shown to have

development impact or satisfy donor countries’

national interest. The risk is that agencies will

shift their strategic focus, instead redoubling

their efforts to tackle the symptoms of poverty,

where there will be quicker, more tangible, and

more certain results—in other words, politically

visible results (Barder, 2018). PFM may therefore

lose support on two fronts as development

cooperation focuses on global public goods such

as climate change mitigation and post-pandemic

reconstruction and infection control, and on direct

Chapter 5: Putting Ideas into Practice

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

development results in areas such as humanitarian

aid, health, nutrition, and sustainable water sources.

A call for proactive leadership

As a professional group, the PFM discipline

has made signicant and well-recognized

progress in analyzing and arguing the case

for PFM strengthening to be a central issue in

development. There has evidently been impressive

leadership and consensus building on an agenda,

its pertinence, and its conventions. Each of the 45

different PFM assessment tools (PEFA Secretariat,

2018) is associated with a distinct group of experts

who understand and own that framework. There

are also some well-published “new realist” thought

leaders who have been critical and inuential in

the task of developing more problem-based and

adaptive approaches (CID, 2014; Andrews, 2013;

Andrews et al., 2012; Blum et al., 2012; Booth,

2014; Booth & Unsworth, 2014; World Bank,

2000, 2012b). Bringing together the insights

and tools of these complementary groups of

thinkers and practitioners might produce powerful

leadership in rethinking how institution building

can support a changing development agenda. They

could act as a powerful guild for the development

and implementation of recommendations. The

coronavirus pandemic has started to catalyze some

thinking about the task for PFM (Gupta and Barry

2020; Gurazada et al., 2020; Saxena and Stone,

2020).

Development agencies will need to demonstrate

institutional-level leadership and some

willingness to direct nancial resources to

these ideas or new approaches. This is uncertain

territory, and some of the proposals are deliberately

speculative. Active experimentation and learning

will be required from the outset. Leaders in the

PFM discipline might take up these ideas, and

progress might be kick-started by funding for one

or more of the proposals set out in this report. It is

hoped that a small coalition—perhaps comprising

bilateral, multilateral, and foundation actors—

could come together to explore the agenda and

drive tangible progress. The Working Group itself

has no leverage or vested interest in any particular

way of proceeding. These ideas are submitted for

further debate, development, and testing in the

interest of contributing to improvements in PFM

and, ultimately, development results.

Moving on from here

This report contains a lot of words but not the

nal one. The Working Group has been privileged

with the time and opportunity to reect on a vital

area of international development activity. It has

benetted from extensive and generous advice,

as well as lessons and insights from experience

and from a large number of practitioners, policy

makers, and researchers. Although the Working

Group stands by the proposals and hopes they will

be picked up and taken up, they are ultimately

just examples of how to push PFM forward.

The mission now is to continue to learn more

from empirical observations, celebrate rather

than obscure political debate, and ensure the

continued contribution of a group of development

practitioners who have already contributed to

signicant results, with the aim of ensuring the

future relevance and effectiveness of the PFM

discipline within the overall development agenda.

CHAPTER REFERENCES

References for Chapter 1

Andrews, M. (2013). The Limits of Institutional Reform in Development. New York: Cambridge University Press.

Annen, K., & Moers, L. (2012). Donor Competition for Aid Impact, and Aid Fragmentation. Washington DC: IMF.

Barder, O. (2009). Beyond Planning: Markets and Networks for Better Aid. Washington DC: Center for Global

Development.

Bill & Melinda Gates Foundation. (2018). Goalkeepers Data Report. Seattle, Washington: Institute for Health

Metrics and Evaluation, University of Washington.

Booth, D. (2011). Aid effectiveness: bringing country ownership (and politics) back in. Working Paper 336. London:

Overseas Development Institute (ODI).

Bourguignon, F., & Platteau, J.-P. (2015). The Hard Challenge of Aid Coordination. World Development, 69, 86–97.

Coady, D., & Murgasova, Z. (2019). A Strategy for IMF Engagement on Social Spending. Washington DC: IMF.

Coats, D. R. (2019). Statement for the Record: Worldwide Threat Assessment of the US Intelligence Community.

Washington DC: Senate Select Committee on Intelligence.

Custer, S., DiLorenzo, M., Masaki, T., Sethi, T., & Harutyunyan, A. (2018). Listening to Leaders 2018: Is

Development Cooperation Tuned-in or Tone-Deaf? Williamsburg, VA: AidData, College of William &

Mary.

de Renzio, P. (2009). Taking Stock: What Do PEFA Assessments Tell Us About PFM Systems across Countries?

London: ODI.

Department of Foreign Affairs and Trade (DFAT), Australia. (2019). Framework for Engaging on Public Financial

Management. Canberra: DFAT.

Dijkstra, L., Poelman, H., & Rodríguez-Pose, A. (2018). The Geography of EU Discontent. Brussels: European

Union.

Easterly, W. (2002). The Cartel of Good Intentions: The Problem of Bureaucracy in Foreign Aid. The Journal of

Policy Reform, 5(4), 223–50.

Easterly, W. (2009). The Cartel of Good Intentions. Foreign Policy. Retrieved from https://foreignpolicy.

com/2009/11/11/the-cartel-of-good-intentions/

Elson, D. (1999). Gender Budget Initiative: Background Papers. London: Commonwealth Secretariat.

Fritz, V., Verhoeven, M., & Avenia, A. (2017). Political Economy of Public Financial Management Reforms:

Experiences and Implications for Dialogue and Operational Engagement (121436). Washington DC:

World Bank Group. Retrieved from http://hdl.handle.net/10986/28887

Gaspar, V., Amaglobeli, D., Garcia-Escribano, M., Prady, D., & Soto, M. (2019). Fiscal Policy and Development:

Human, Social, and Physical Investment for the SDGs. Washington DC: IMF.

Gilmore, E. A., & St. Clair, T. (2018). Budgeting for Climate Change: Obstacles and Opportunities at the US State

Level. Climate Policy, 18(6), 729–41.

Greenhill, R., Prizzon, A., & Rogerson, A. (2013). The Age of Choice: Developing Countries in the New Aid

Landscape. London: ODI.

Hudson Institute. (2016). The Index of Global Philanthropy and Remittances (2016). Washington DC: Hudson

Institute.

International Labour Organization (ILO). (2018). Global Wage Report 2018/19. Geneva: ILO.

International Monetary Fund (IMF). (2018a). 2018 Review of the Fund’s Capacity Development Strategy—Overview

Paper. Washington DC: IMF.

IMF. (2018b). World Economic Outlook (October 2018): Challenges to Steady Growth. Washington DC: IMF.

IMF. (2019). 2018 Review of Program Design and Conditionality (Policy Paper No. 19/012). Washington DC:

IMF.

IEG. (2018). Approach Paper: Public Finance for Development Evaluation. Washington DC: World Bank.

Inter-Agency Group on Economic and Financial Statistics. (2019). The Financial Crisis and Information Gaps: G20

Data Gaps Initiative (DGI-2). Washington DC: Financial Stability Board and IMF.

Joint World Bank/IMF/PEFA Public Expenditure Working Group. (2006). Supporting Better Country Public

Financial Management Systems: Towards a Strengthened Approach to Supporting PFM Reform –

Consultative Paper. In OECD. (2006). Harmonising Donor Practices for Effective Aid Delivery (Volume 2:

Budget Support, Sector Wide Approaches and Capacity Development in Public Financial Management).

Paris: OECD.

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Keating, M. (2001). Public Management Reform and Economic and Social Development. OECD Journal on

Budgeting, 1(2).

Kharas, H., McArthur, J. W., & Rasmussen, K. (2018). How Many People Will the World Leave Behind?

Washington DC: Brookings Institution.

Klein, M., & Harford, T. (2005). The Market for Aid. Washington DC: International Finance Corporation.

Koch, S., Leiderer, S., Faust, J., & Molenaers, N. (2017). The Rise and Demise of European Budget Support:

Political Economy of Collective European Union Donor Action. Development Policy Review, 35(4),

455–73.

Lagarde, C. (2019). Forging a Stronger Social Contract—the IMF’s Approach to Social Spending [Press release].

Retrieved from https://www.imf.org/en/News/Articles/2019/06/14/sp061419-md-social-spending

Manning, N., & Wetzel, D. (2010). Tales of the Unexpected: Re-Building Trust in Government. In O. Canuto & M.

Giugale (Eds.), The Day after Tomorrow: A Handbook on the Future of Economic Policy in the Developing

World (pp. 163–81). Washington DC: World Bank.

Milanovic, B. (2016). The Winners and Losers of Globalization. Cambridge, MA: Harvard University Press.

OECD. (2006). Harmonising Donor Practices for Effective Aid Delivery (Volume 2: Budget Support, Sector Wide

Approaches and Capacity Development in Public Financial Management). Paris: OECD.

OECD. (2018). Economic Survey of Greece. Paris: OECD.

PEFA Secretariat. (2018). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

Pritchett, L., Samji, S. & Hammer, J. (2013). It's All About MeE: Using Structured Experiential Learning ("E") to

Crawl the Design Space. Washington DC: Center for Global Development.

Rios-Avila, F. (2015). Decade of Declining Wages: From Bad to Worse. Baltimore: Levy Economics Institute.

Samman, E., Lucci, P., Hagen-Zanker, J., Bhatkal, T., Simunovic, A. T., Nicolai, S., Stuart, E., & Caron, C. (2018).

SDG Progress: Fragility, Crisis and Leaving No One Behind. London: ODI.

Schick, A. (2013). Reections on Two Decades of Public Financial Management Reforms. In M. Cangiano, T. R.

Curristine, & M. Lazare (Eds.), Public Financial Management and Its Emerging Architecture (pp. 21–78).

Washington DC: IMF.

Schick, A. (2019). An Introduction to Budgeting: Everywhere Budgeting Is the Same and Different. Budgeting and

Public Expenditures in OECD Countries 2019 (pp. 19–30). Paris: OECD.

Schmaljohann, M., & Prizzon, A. (2015). Age of Choice: How Developing Countries Are Managing the New Aid

Landscape. London: ODI.

Shanmugaratnam, T., Ahmed, S., Babacan, A., Belka, M., Frenkel, J. A., Issing, O., Ito, T., Lustig, N., Okonjo-

Iweala, N., Rajan, R. G., Saccomanni, F., Stern, N., Taylor, J. B., Trichet, J.-C., Velasco, A., & Min, Z.

(2018). Making the Global Financial System Work for All: Report of the G20 Eminent Persons Group on

Global Financial Governance. Washington DC: G20.

United Nations. (2019). Report of the Secretary-General on SDG Progress 2019. New York: United Nations.

van der Weide, R., & Milanovic, B. (2014). Inequality Is Bad for Growth of the Poor (but Not for That of the Rich).

Washington DC: World Bank.

Weingast, B. R., Shepsle, K. A., & Johnsen, C. (1981). The Political Economy of Benets and Costs: A Neoclassical

Approach to Distributive Politics. Journal of Political Economy, 89(4), 642–64.

Welham, B., Hart, T., Mustapha, S., & Hadley, S. (2017). Public Financial Management and Health Service

Delivery: Necessary, but Not Sufcient? London: ODI.

Wildavsky, A. (1992). The New Politics of the Budget Process. New York: Harper Collins.

World Bank. (2019a). Global Economic Prospects: Darkening Skies. Washington DC: World Bank.

World Bank. (2019b). Global Economic Prospects: Heightened Tensions, Subdued Investment. Washington DC:

World Bank.

References for Chapter 2

Acemoglu, D., & Robinson, J. A. (2012). Why Nations Fail: The Origins of Power, Prosperity and Poverty. New

York: Crown Publishers.

Alesina, A., Hausmann, R., Hommes, R., & Stein, E. (1999). Budget Institutions and Fiscal Performance in Latin

America. Working Paper Series 394. Washington DC: IADB. Retrieved from http://www.iadb.org/OCE/

pdf/394.pdf

Allen, R. (2018). Book Review: Building State Capability: Evidence, Analysis, Action. Governance(31), 401–03.

Allen, R., & Grigoli, F. (2012). Enhancing the Capability of Central Finance Agencies (Economic Premise No. 73).

Washington DC: World Bank.

Allen, R., Grigoli, F., & Howard, N. (2011). Enhancing the Capabilities of Central Finance Agencies: Synthesis

Report. Washington DC: World Bank.

Allen, R., Hemming, R., & Potter, B. (Eds.). (2013). The International Handbook of Public Financial Management.

London: Palgrave MacMillan.

Allen, R., & Krause, P. (2013). The Role, Responsibilities, Structure and Evolution of Central Finance Agencies.

In R. Allen, R. Hemming, & B. H. Potter (Eds.), The International Handbook of Public Financial

Management (pp. 98-115). Basingstoke, UK: Palgrave MacMillan.

AlphaBeta. (2018). The Landscape of PFM Reform in Developing Countries: Trends in Financing and

Implementation of PFM Reforms over 2002–16. Singapore: AlphaBeta.

Andrews, M. (2007). What Would an Ideal Public Finance Management System Look Like? In A. Shah (Ed.),

Budgeting and Budgetary Institutions (pp. 359–86). Washington DC: World Bank.

Andrews, M. (2008a). The Good Governance Agenda: Beyond Indicators without Theory Oxford Development

Studies, 36(4), 379–407.

Andrews, M. (2008b). Good Government Means Different Things in Different Countries. Cambridge, MA: Kennedy

School of Government. Retrieved from http://web.hks.harvard.edu/publications/getFile.aspx?Id=324

Andrews, M. (2013). The Limits of Institutional Reform in Development. New York: Cambridge University Press.

Andrews, M. (2014). Why Distributed End Users Often Limit Public Financial Management Reform Success.

Cambridge, MA: Center for International Development at Harvard University.

Andrews, M., Cangiano, M., Cole, N., de Renzio, P., Krause, P., & Seligmann, R. (2014). This Is PFM. Cambridge,

MA: Center for International Development at Harvard University.

Andrews, M., McNaught, T., & Samji, S. (2018). Opening Adaptation Windows onto Public Financial Management

Reform Gaps in Mozambique. CID Faculty Working Paper No. 341. Cambridge, MA: Center for

International Development at Harvard University.

Andrews, M., Pritchett, L., & Woolcock, M. (2012). Escaping Capability Traps through Problem-Driven Iterative

Adaptation (PDIA) (Working Paper 299). Washington DC: Center for Global Development.

Annett, A. (2002). Politics, Government Size, and Fiscal Adjustment in Industrial Countries. Washington DC: IMF.

Bain, K. A., Booth, D., & Wild, L. (2016). Doing Development Differently at the World Bank: Updating the

Plumbing to Fit the Architecture. London: ODI.

Bakir, C., & Jarvis, D. S. L. (2018). Institutional and Policy Change: Meta-Theory and Method. In C. Bakir & D.

S. L. Jarvis (Eds.), Institutional Entrepreneurship and Policy Change (pp. 1–38). New York: Palgrave

MacMillan.

Batley, R., & McLoughlin, C. (2015). The Politics of Public Services: A Service Characteristics Approach. World

Development, 74, 275–85.

Becher, T., & Trowler, P. R. (2001). Academic Tribes and Territories, Buckingham. Buckingham, UK: The Society

for Research into Higher Education and Open University Press.

Blum, J. R., Manning, N., & Srivastava, V. (2012). Public Sector Management Reform: Toward a Problem-

Solving Approach. Washington DC: World Bank. Retrieved from http://siteresources.worldbank.org/

EXTPREMNET/Resources/EP100.pdf

Booth, D. (2011). Working with the Grain and Swimming against the Tide: Barriers to Uptake of Research Findings

on Governance and Public Services in Low-Income Africa. London: ODI.

Booth, D. (2014). Aiding Institutional Reform in Developing Countries: Lessons from the Philippines on What

Works, What Doesn't, and Why. London: ODI.

Booth, D. (2015). Five Myths About Governance and Development. Retrieved from http://blogs.worldbank.org/

publicsphere/blog-post-month-ve-myths-about-governance-and-development

Booth, D., & Unsworth, S. (2014). Politically Smart, Locally Led Development. London: ODI.

Brinkerhoff, D. W., & Crosby, B. L. (2002). Managing Policy Reform. Bloomeld CT: Kumarian.

Brinkerhoff, D. W., & Ingle, M. D. (1989). Integrating Blueprint and Process: A Structured Flexibility Approach to

Development Management. Public Administration and Development, 9, 487–503.

Brooke, P. (2003). Study of Measures Used to Address Weaknesses in Public Financial Management Systems in the

Context of Policy Based Support. Final Report. Washington DC: PEFA Secretariat.

Campbell, C. (2001). Juggling Inputs, Outputs, and Outcomes in the Search for Policy Competence: Recent

Experience in Australia. Governance, 14(2), 253–82.

Campos, E., & Pradhan, S. (1996). Budgetary Institutions and Expenditure Outcomes (Policy Research Working

Paper No. 1646). Washington DC: World Bank.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Cangiano, M., Curristine, T., & Lazare, M. (2013a). Introduction. In M. Cangiano, T. R. Curristine, & M. Lazare

(Eds.), Public Financial Management and Its Emerging Architecture (pp. 1–20). Washington DC: IMF.

Cangiano, M., Curristine, T. R., & Lazare, M. (Eds.). (2013b). Public Financial Management and Its Emerging

Architecture. Washington DC: IMF.

Cangiano, M., Gelb, A., & Goodwin-Groen, R. (2019). Public Financial Management and the Digitalization of

Payments. Washington DC: Center for Global Development.

Case, K. E. (2008). Musgrave’s Vision of the Public Sector: The Complex Relationship between Individual, Society

and State in Public Good Theory. Journal of Economics and Finance, 32, 348–355.

Center for International Development (CID) at Harvard University, Building State Capability Program. (2014). The

DDD Manifesto. ‘Doing Development Differently’ workshop. Retrieved from http://buildingstatecapability.

com/the-ddd-manifesto/

Clarke, A., & Craft, J. (2019). The Twin Faces of Public Sector Design. Governance, 32(1), 5–21.

de Renzio, P. (2006). Aid, Budgets and Accountability: A Survey Article. Development Policy Review, 24(6),

627–45.

de Renzio, P. (2009). Approach & Methodology for the Evaluation of Donor Support to Public Finance Management

(PFM) Reform in Developing Countries (Part A - Assessing Budget Institutions and Budget Reforms in

Developing Countries: Overview of Theoretical Approaches and Empirical Evidence). Oxford, UK: Fiscus

Public Finance Consultants, Ltd.

de Renzio, P. (2013). Assessing and Comparing the Quality of Public Financial Management Systems: Theory,

History and Evidence. In R. Allen, R. Hemming, & B. H. Potter (Eds.), The International Handbook of

Public Financial Management (pp. 137–60). Basingstoke, UK: Palgrave MacMillan.

Derviş, K., Chandy, L., Kharas, H., Medler, A., & Unger, N. (2011). Global Development under Pressure.

Washington DC: Brookings Institution.

Diamond, J. (2013). Good Practice Note on Sequencing PFM Reforms. Washington DC: PEFA Secretariat.

Dom, R., & Miller, M. (2018). Reforming Tax Systems in the Developing World: What Can We Learn from the Past?

London: ODI.

Dressel, B., & Brumby, J. (2009). Enhancing the Capabilities of Central Finance Agencies: From Diagnosis to

Action. Washington DC: World Bank.

Drucker, P. (1974). Management. London: Butterworth Heinemann.

Dunn, W. N. (1988). Methods of the Second Type: Coping with the Wilderness of Conventional Policy Analysis.

Policy Studies Review, 7(4), 720–37.

Ear, S. (2009). Cambodia’s Patient Zero: The Political Economy of Foreign Aid and Avian Inuenza (Working Paper

No. 398). Stanford, CA: Stanford Center for International Development.

Ellerman, D. (2005). Helping People Help Themselves: From the World Bank to an Alternative Philosophy of

Development Assistance. Ann Arbor: University of Michigan Press.

Evans, P. (2004). Development as Institutional Change: The Pitfalls of Monocropping and the Potentials of

Deliberation. Studies in Comparative Institutional Development, 38(4), 30–52.

Evans, P., & Rauch, J. E. (1999). Bureaucracy and Growth: A Cross-National Analysis of the Effects of ‘Weberian’

State Structures on Economic Growth. American Sociological Review 64(5), 748–765.

Fritz, V., Fialho Lopes, A. P., Hedger, E., Tavakoli, H., & Krause, P. (2011). Public Financial Management Reforms

in Post-Conict Countries: Synthesis Report. Washington DC: World Bank.

Fritz, V., Levy, B., & Ort, R. (Eds.). (2014). Problem-Driven Political Economy Analysis. Washington DC: World

Bank.

Fritz, V., Lopes, A. P. F., Hedger, E., Tavakoli, H., & Krause, P. (2012). Public Financial Management Reforms in

Post-Conict Countries: Synthesis Report. Washington DC: World Bank.

Fritz, V., Sweet, S., & Verhoeven, M. (2014). Strengthening Public Financial Management: Exploring Drivers and

Effects. Washington DC: World Bank.

Fritz, V., Verhoeven, M., & Avenia, A. (2017). Political Economy of Public Financial Management Reforms:

Experiences and Implications for Dialogue and Operational Engagement (121436). Washington DC:

World Bank Group. Retrieved from http://hdl.handle.net/10986/28887

Gore, A. (1995). From Red Tape to Results: Creating a Government That Works Better and Costs Less. Washington

DC: White House.

Grindle, M. S. (2013). Public Sector Reform as Problem-Solving? Comment on the World Bank's Public Sector

Management Approach for 2011 to 2020. International Review of Administrative Science, 79(3), 398–405.

Guy, M. E. (2003). Ties That Bind: The Link between Public Administration and Political Science. The Journal of

Politics, 65(3), 641–55.

Hallerberg, M., & Yläoutinen, S. (2010). Political Power, Fiscal Institutions and Budgetary Outcomes in Central and

Eastern Europe. Journal of Public Policy, 30(1), 45–62.

Hausmann, R., Pritchett, L., & Rodrik, D. (2005). Growth Accelerations. Journal of Economic Growth, 10(4),

303–29.

Heifetz, R. A. (1994). Leadership without Easy Answers. Cambridge: Harvard University Press.

Hood, C. (1991). A Public Management for All Seasons? Public Administration, 69(Spring), 3–19.

Howlett, M. (2009). Governance Modes, Policy Regimes and Operational Plans: A Multi-Level Nested Model of

Policy Instrument Choice and Policy Design. Policy Sciences, 42(1), 73–89.

Hughes, O. (2018). Public Management and Administration: An Introduction (5th Edition). London: Palgrave.

IMF. (2010). Strategies for Fiscal Consolidation in the Post-Crisis World. Washington DC: IMF.

IMF. (2018a). 2018 Review of the Fund’s Capacity Development Strategy—Report of the External Advisory Group.

Washington DC: IMF.

IMF. (2018b). Fiscal Transparency Handbook. Washington DC: IMF.

IMF. (2018c). Macroeconomic Developments and Prospects in Low-Income Developing Countries. Washington

DC: IMF.

Jakobsen, M., & Jensen, R. (2014). Common Method Bias in Public Management Studies. International Public

Management Journal, 18(1), 3–30.

Keating, M., & Holmes, M. (1990). Australia's Budgetary and Financial Management Reforms. Governance, 3(2),

168–85.

Keefer, P. (2013). Organizing for Prosperity Collective Action, Political Parties and the Political Economy of

Development (Policy Research Working Paper 6583). Washington DC: World Bank.

Kettl, D. F. (2000). Public Administration at the Millennium: The State of the Field. Journal of Public

Administration Research and Theory, 10(1), 7–34.

Key, V. O. (1940). The Lack of a Budgetary Theory. American Political Science Review, 34(6), 1137–44.

Khemani, S. (2013). Buying Votes Vs. Supplying Public Services Political Incentives to Under-Invest in Pro-Poor

Policies (Policy Research Working Paper 6339). Washington DC: World Bank.

Kirkpatrick , D. L., & Kirkpatrick, J. D. (2009). Implementing the Four Levels: A Practical Guide for Effective

Evaluation of Training Programs. Oakland, California: Berrett-Koehler Publishers.

Krause, P., Hadley, S., Mustapha, S., & Welham, B. (2016). The Capabilities of Finance Ministries. London: ODI.

Krishnan, A. (2009). What Are Academic Disciplines? Southampton UK: ESRC National Centre for Research

Methods, University of Southampton.

Kristensen, J. K. (2019, February 28, 2019). [Email].

Lassen, D. D. (2010). Fiscal Consolidations in Advanced Industrialized Democracies: Economics, Politics, and

Governance. Copenhagen: University of Copenhagen.

Levy, B. (2014). Working with the Grain: Integrating Governance and Growth in Development Strategies. Oxford:

Oxford University Press.

Lindblom, C. (1959). The Science of Muddling Through. Public Administration Review, 19, 79–88.

Love, P., & Stockdale-Otárola, J. (Eds.). (2017). Debate the Issues: Complexity and Policy Making. Paris: OECD.

Manning, N. (2001). The Legacy of the New Public Management in Developing Countries. International Review of

Administrative Sciences, 67(2), 297–312.

Melchor, O. H. (2008). Managing Change in OECD Governments: An Introductory Framework. Paris: OECD.

Morgan, P. (2006). The Concept of Capacity. Maastricht, Netherlands: European Centre for Development Policy

Management.

Musgrave, R. A., & Musgrave, P. B. (1989). Public Finance in Theory and Practice (Fifth Edition). San Francisco:

McGraw Hill.

Mustapha, S., & Long, C. (2019). PEFA, Public Financial Management, and Good Governance. Washington DC:

World Bank.

Naidu, S., Rodrik, D., & Zucman, G. (2019). Economics for Inclusive Prosperity: An Introduction (Research Brief).

Economists for Inclusive Prosperity.

Nikitina, S. (2005). Cognition and Instruction. Pathways of Interdisciplinary Cognition, 23(3), 389–425.

North, D. C., Wallis, J. J., Webb, S. B., & Weingast, B. R. (2007). Limited Access Orders in the Developing World:

A New Approach to the Problems of Development, Policy Research Working Paper 4359. Washington DC:

World Bank. Retrieved from http://www-wds.worldbank.org/external/default/WDSContentServer/IW3P/IB

/2007/09/19/000158349_20070919115851/Rendered/PDF/WPS4359.pdf

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

OECD. (1995). Governance in Transition: Public Management Reforms in OECD Countries. Paris: OECD.

OECD. (2005). Modernising Government: The Way Forward. Paris: OECD.

OECD. (2010). Fiscal Consolidation: Requirements, Timing, Instruments and Institutional Arrangements. Paris:

OECD.

Orr, D. W. (1993). The Problem of Disciplines/the Discipline of Problems. Conservation Biology, 7(1), 10–12.

Paudyn, B. (2014). Credit Ratings and Sovereign Debt: The Political Economy of Creditworthiness through Risk and

Uncertainty. London: Palgrave Macmillan.

PEFA Secretariat. (2016). PEFA: Framework for Assessing Public Financial Management. Washington DC: PEFA

Secretariat. Retrieved from https://pefa.org/sites/default/les/PEFA%20Framework_English.pdf

PEFA Secretariat. (2018a). PEFA Handbook Volume III: Preparing the PEFA Report. Washington DC: PEFA.

PEFA Secretariat. (2018b). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

PEFA Secretariat. (2019). PEFA Handbook Volume IV: Using PEFA to Support Public Financial Management

Improvement. Washington DC: PEFA.

Plant, J. F., Stalebrink, O. J., & Vasavada, T. (2010). Public Values, Public Ofcial Associations, and

Professionalism: A Cross-National Analysis. The American Review of Public Administration, 40(6),

722–41.

Pollitt, C. (2008). Time, Policy, Management: Governing with the Past. New York: Oxford University Press.

Pollitt, C., & Bouckaert, G. (2011). Public Management Reform: A Comparative Analysis (Third ed.). Oxford, UK:

Oxford University Press.

Pollitt, C., & Bouckaert, G. (2018). Public Management Reform: A Comparative Analysis—Into the Age of Austerity.

Oxford: OUP.

Premchand, A. (1993). Public Expenditure Management. Washington DC: IMF.

Premchand, A. (2000). Control of Public Money: The Fiscal Machinery in Developing Countries. Oxford: Oxford

University Press.

Pressman, J. L., & Wildavsky, A. B. (1973). Implementation: How Great Expectations in Washington Are Dashed in

Oakland. Berkeley and Los Angeles: University of California Press.

Pritchett, L., Samji, S. & Hammer, J. (2013). It's All About MeE: Using Structured Experiential Learning ("E") to

Crawl the Design Space. Washington DC: Center for Global Development.

Pritchett, L., & Woolcock, M. (2004). Solutions When the Solution Is the Problem: Arraying the Disarray in

Development. World Development, 32(2), 191–212.

Raadschelders , J. C. N. (2018). Comparative Public Administration in a Globalizing World: A New Psychological

Challenge Next to the Traditional Scholarly Challenges Paper presented at the American Society for Public

Administration, March 9-13, 2018, Denver.

Ringold, D. (2012). Citizens and Service Delivery: Assessing the Use of Social Accountability Approaches in Human

Development. Washington DC: World Bank.

Roberts, A. (2018). The Aims of Public Administration: Reviving the Classical View Perspectives on Public

Management and Governance, 1(1), 73–85.

Rodrik, D. (2008). One Economics, Many Recipes: Globalization, Institutions, and Economic Growth. Princeton,

NJ: Princeton University Press.

Rossi, F. M., Caperchione, E., Cohen, S., & Brusca, I. (2019). Financial Management and Public Sector Accounting

in an Age of Reforms: Developments and Changes in Public Sector Financial Management. In E. Ongaro

(Ed.), Public Administration in Europe: The Contribution of EGPA (pp. 235–47). London: Palgrave

macMillan.

Sayre, W. S. (1958). Premises of Public Administration: Past and Emerging. Public Administration Review, 18(2),

102–05.

Schick, A. (1996). The Spirit of Reform: Managing the New Zealand State Sector in a Time of Change. Wellington,

New Zealand: State Services Commission and The Treasury.

Schick, A. (1998a). A Contemporary Approach to Public Expenditure Management (35116). Washington DC:

World Bank Institute.

Schick, A. (1998b). Why Most Developing Countries Shouldn't Try New Zealand's Reforms. The World Bank

Research Observer, 13(1), 123–31.

Scott, G., Bushnell, P., & Sallee, N. (1990). Reform of the Core Public Sector: New Zealand Experience.

Governance, 3(2), 138–67.

Sheely, R. (2014). Mobilization, Participatory Planning Institutions, and Elite Capture: Evidence from a Field

Experiment in Rural Kenya. World Development, 67, 251–66.

So, S., Woolcock, M., April, L., Hughes, C., & Smithers, N. (Eds.). (2018). Alternative Paths to Public Financial

Management and Public Sector Reform: Experiences from East Asia. Washington DC: World Bank.

Squire, S. J., & Beazley, I. (2016). Increasing Professionalism in Public Finance Management: A Case Study of the

United Kingdom. Washington DC: World Bank.

Stein, H. (2008). Beyond the World Bank Agenda. Chicago: University of Chicago Press.

TADAT Secretariat. (2015). TADAT Assessment Tool: Field Guide. Washington DC: IMF.

Thomas, M. A. (2010). What Do the Worldwide Governance Indicators Measure? The European Journal of

Development Research, 22(1), 31–54.

Unegbu, A. O. (2014). Theories of Accounting: Evolution & Developments, Income-Determination and Diversities

in Use. Research Journal of Finance and Accounting, 5(19), 1–15.

Vitale, J. E. (2013). An Essay on the Division between Craft-Based and Knowledge-Based Professions as an

Inhibitor of Interprofessional Healthcare Education and Practice, Part I. Health and Interprofessional

Practice, 1(3), 1–4.

von Hagen, J. (1998). Budgeting Institutions for Aggregate Fiscal Discipline (ZEI Working Paper, No. B 01). Bonn:

Rheinische Friedrich-Wilhelms-Universität Bonn, Zentrum für Europäische Integrationsforschung.

von Hagen, J. (2006). Political Economy of Fiscal Institutions. In B. Weingast & D. Whitman (Eds.), The Oxford

Handbook of Political Economy (pp. 464–78). Oxford: Oxford University Press.

von Hagen, J., & Harden, I. (1995). Budget Processes and Commitment to Fiscal Discipline. European Economic

Review, 39(3-4), 771–79.

Weber, M. (1978). Economy and Society (Vol. 2). Berkely, CA: University of California Press.

Wildavsky, A. (1992). The New Politics of the Budget Process. New York: Harper Collins.

Williamson, J. (1989). What Washington Means by Policy Reform. In J. Williamson (Ed.), Latin American

Readjustment: How Much Has Happened. Washington DC: Institute for International Economics.

Williamson, J. (2004). The Washington Consensus as Policy Prescription for Development (Lecture Delivered at the

World Bank on January 13, 2004). Washington DC: World Bank.

World Bank. (1997). World Development Report 1997: The State in a Changing World. New York: Oxford

University Press for the World Bank.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

World Bank. (2000). Reforming Public Institutions and Strengthening Governance: A World Bank Strategy.

Washington DC: World Bank.

World Bank. (2012a). Beyond the Annual Budget: Global Experience with Medium-Term Expenditure Frameworks.

Washington DC: World Bank.

World Bank. (2012b). The World Bank’s Approach to Public Sector Management 2011–2020: Better Results from

Public Sector Institutions. Washington DC: World Bank.

Wyplosz, C. (2011). Fiscal Discipline: Rules Rather Than Institutions. National Institute Economic Review, 217(1),

R19–R30.

Yanguas, P., & Hulme, D. (2014). Can Aid Bureaucracies Think Politically? The Administrative Challenges of

Political Economy Analysis in DFID and the World Bank. Manchester, UK: Effective States and Inclusive

Development Research Centre.

Zafarullah, H., & Huque, A. S. (2012). Managing Development in a Globalized World. London: Taylor and Francis.

References for Chapter 3

AlphaBeta. (2018). The Landscape of PFM Reform in Developing Countries: Trends in Financing and

Implementation of PFM Reforms over 2002–16. Singapore: AlphaBeta.

Anderson, R. (2005). Transport Performance and the Data Clubs Approach. Paper presented at the Where Does

Britain Rank? – International Public Services Rankings (13th December 2005), London.

Andrews, M. (2012). The Limits of Institutional Reform in Development. Cambridge: Cambridge University Press.

Andrews, M., & Manning, N. (2015). The EIP Peer-to-Peer Learning Guide. Paris: Effective Institutions Platform,

OECD.

Annen, K., & Moers, L. (2012). Donor Competition for Aid Impact, and Aid Fragmentation. Washington DC: IMF.

Barder, O. (2009). Beyond Planning: Markets and Networks for Better Aid. Washington DC: Center for Global

Development.

Barder, O., Gavas, M., Maxwell, S., & Johnson, D. (2010). Governance of the Aid System and the Role of the EU.

Paper presented at the Conference On Development Cooperation in Times Of Crisis and on Achieving the

MDGs, Madrid.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Bates, M. A., & Glennerster, R. (2017). The Generalizability Puzzle. Stanford Social Innovation Review, Summer

2017.

Berg, E. (1993). Rethinking Technical Cooperation: Reforms for Capacity Building in Africa. New York: UNDP

Regional Bureau for Africa and Development Alternatives Inc.

Bhattacharya, A., Kharas, H., Plant, M., & Prizzon, A. (2018). The New Global Agenda and the Future of the

Multilateral Development Bank System Washington DC and London: Brookings, CGD, and ODI.

Birdsall, N., & Savedoff, W. D. (2010). Cash on Delivery: A New Approach to Foreign Aid. Washington DC: Center

for Global Development.

Booth, D. (2013). Facilitating Development an Arm’s Length Approach to Aid. London: ODI.

Bourguignon, F., & Platteau, J.-P. (2015). The Hard Challenge of Aid Coordination. World Development, 69, 86–97.

Brooke, P. (2003). Study of Measures Used to Address Weaknesses in Public Financial Management Systems in the

Context of Policy Based Support. Washington DC: PEFA.

Brookings Institution & Center for Global Development (CGD). (2011). Avoidance of Project Implementation Units.

Retrieved from Quality of Ofcial Development Assistance (QuODA) Database at https://www.cgdev.org/

sites/all/cms_iframes/quoda_2014/bar.php?g=2&agency_group=none

Browne, S. (2012). Aid and Inuence, Do Donors Help or Hinder? Abingdon, Oxford: Earthscan Routledge.

CABRI. (2014). Towards a Greater Use of Country Systems in Africa: Recent Trends and New Approaches. Pretoria,

South Africa: Collaborative African Budget Reform Initiative.

Calvard, T. S. (2016). Big Data, Organizational Learning, and Sensemaking: Theorizing Interpretive Challenges

under Conditions of Dynamic Complexity. Management Learning, 47(1), 65–82.

Courty, P., & Marschke, G. (2004). An Empirical Investigation of Gaming Responses to Explicit Performance

Incentives. Journal of Labor Economics, 22(1), 23–56.

De Lay, S., Mills, L., Jadeja, K., & Lucas, B. (2015). Public Financial Management Evidence Mapping.

Birmingham, UK: GSDRC, University of Birmingham.

de Renzio, P., Andrews, M., & Mills, Z. (2010). Evaluation of Donor Support to Public Financial Management

(PFM) Reform in Developing Countries: Analytical Study of Quantitative Cross-Country Evidence.

Stockholm: SIDA.

de Renzio, P., & Rogerson, A. (2005). Power to Consumers? A Bottom-up Approach to Aid Reform. London: ODI.

Dendura, J., & Flynn, N. (2011). Towards a Common Approach to Risk (Paper Prepared for the EC). Brussels:

European Commission.

DFID. (2009). Guidance on Using the Revised Logical Framework London: DFID. Retrieved from http://www.dd.

gov.uk/Documents/publications1/how-to-guid-rev-log-fmwk.pdf

Diamond, J. (2012). Guidance Note on Sequencing PFM Reforms. Washington DC: EC, IMF and PEFA.

Diamond, J. (2013). Good Practice Note on Sequencing PFM Reforms. Washington DC: PEFA Secretariat.

Ellerman, D. (2005). Helping People Help Themselves: From the World Bank to an Alternative Philosophy of

Development Assistance. Ann Arbor: University of Michigan Press.

Eno, J., Glentworth, G., Hedger, E., & White, M. (2018). Public Finance, Accountability and Anti-Corruption in

Sierra Leone: Scoping Study for a New DFID Programme. London: DFID.

Evans, A. (2008). Civil Service Reform Thematic Paper Prepared for the Public Sector Reform Evaluation Project.

Washington DC: World Bank.

Fengler, W., & Kharas, H. (2010). Delivering Aid Differently: Lessons from the Field. Washington DC: Brookings

Institution Press.

Fritz, V., Hedger, E., & Lopes, A. P. F. (2011). Strengthening Public Financial Management in Postconict

Countries. Washington DC: World Bank.

Fritz, V., Sweet, S., & Verhoeven, M. (2014). Strengthening Public Financial Management: Exploring Drivers and

Effects. Washington DC: World Bank.

Fritz, V., Verhoeven, M., & Avenia, A. (2017). Political Economy of Public Financial Management Reforms:

Experiences and Implications for Dialogue and Operational Engagement (121436). Washington DC:

World Bank Group. Retrieved from http://hdl.handle.net/10986/28887

Glaister, S., & Anderson, R. (2005). Transport Performance and the Data Clubs Approach. Paper presented at the

ESRC International Public Service Rankings (December 2005), London.

Goodhart, C. A. E. (1975). Problems of Monetary Management: The UK Experience. Papers in Monetary

Economics, Reserve Bank of Australia.

Green, D. (2018). Can ‘Doing Development Differently’ Only Succeed If Aid Donors Stay Away from It? Retrieved

from http://oxfamblogs.org/fp2p/can-doing-development-differently-only-succeed-if-aid-donors-stay-away-

from-it/

Greenhill, R., Prizzon, A., & Rogerson, A. (2013). The Age of Choice: Developing Countries in the New Aid

Landscape. London: ODI.

Gulrajani, N. (2017). Bilateral Donors and the Age of the National Interest: What Prospects for Challenge by

Development Agencies? World Development, 96, 375–89.

HM Treasury (United Kingdom). (2018). The Green Book: Central Government Guidance on Appraisal and

Evaluation. London: HM Treasury.

Hadley, S., & Miller, M. (2016). PEFA: What Is It Good For? London: ODI.

Hanauer, L., & Morris, L. J. (2014). Chinese Engagement in Africa: Drivers, Reactions and Implications for U.S.

Policy. Santa Monica, CA: RAND National Defense Research Institute.

Haque, T. A., Knight, D. S., & Jayasuriya , D. S. (2012). Capacity Constraints and Public Financial Management in

Small Pacic Island Countries. Washington DC: World Bank.

Harris, D. (2011). Emerging Donors: Differentiation, Ownership and Harmonisation. The Networker(98), 8–11.

Honig, D. (2018). Navigation by Judgment: Why and When Top Down Management of Foreign Aid Doesn’t Work.

New York: Oxford University Press.

Hood, C. (2006). Gaming in Targetworld: The Targets Approach to Managing British Public Services. Public

Administration Review, 66(4), 515–21.

Hood, C., Dixon, R., & Wilson, D. (2009). ‘Managing by Numbers’: The Way to Make Public Services Better?

Retrieved from http://www.publicservices.ac.uk/wp-content/uploads/policy-brieng-nov2009.pdf

IEG. (2018). Approach Paper: Public Finance for Development Evaluation. Washington DC: World Bank.

Ingram, G., & Lord, K. M. (2019). Global Development Disrupted: Findings from a Survey of 93 Leaders.

Washington DC: Brookings Institution.

Inter-Agency Group on Economic and Financial Statistics. (2019). The Financial Crisis and Information Gaps: G20

Data Gaps Initiative (DGI-2). Washington DC: Financial Stability Board and IMF.

Khemani, S. (2017). Demand and Supply Curves in Political Markets: Understanding the Problem of Public Goods

and Why Governments Fail Them (Policy Research Working Paper 8213). Washington DC: World Bank.

Klein, M., & Harford, T. (2005). The Market for Aid. Washington DC: International Finance Corporation.

Koch, S., Leiderer, S., Faust, J., & Molenaers, N. (2017). The Rise and Demise of European Budget Support:

Political Economy of Collective European Union Donor Action. Development Policy Review, 35(4),

455–73.

Krause, P., Hadley, S., Hedger, E., de Renzio, P., Gill, S., & Andrews, M. (2014). Joint Response to the PEFA

Consultation: We Need a Stronger Summary Assessment. London: ODI.

Lauria, V., & Fumagalli, C. (2019). Brics, the Southern Model, and the Evolving Landscape of Development

Assistance: Toward a New Taxonomy. Public Administration and Development, 1–16.

Lawson, A. (2012). Evaluation of Public Financial Management Reform in Burkina Faso, Ghana and Malawi

2001–2010: Final Synthesis Report. Oxford, UK: Fiscus Public Finance Consultants, Ltd.

Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The Parable of Google Flu: Traps in Big Data Analysis.

Science, 343(6176), 1203–05.

Le Grand, J. (2007). The Other Invisible Hand: Delivering Public Services through Choice and Competition.

Princeton and Oxford: Princeton University Press.

Le Grand, J. (2010). Knights and Knaves Return: Public Service Motivation and the Delivery of Public Services.

International Public Management Journal, 13(1), 56–71.

Lubienski, C. (2009). Do Quasi-Markets Foster Innovation in Education?: A Comparative Perspective. Paris:

OECD.

Lum, T. (2009). China’s Assistance and Government – Sponsored Investment Activities in Africa, Latin America, and

Southeast Asia. Washington DC: Congressional Research Service.

Martens, B. (2005). Why Do Aid Agencies Exist? Development Policy Review, 23(6), 643–63.

McKechnie, A., & Davies, F. (2013). Localising Aid: Is It Worth the Risk? London: ODI.

Mills, L. (2018). Donor Support to Strengthen Public Financial Management in Partner Countries: Outcomes,

Experiences, and Ways Forward. Brighton, UK: Institute of Development Studies.

Morrison, J. S. (2013). Introduction. In J. S. Morrison (Ed.), Global Health Policy in the Second Obama Term.

Washington DC: Center for Strategic and International Studies.

Mosley, P. (1985). The Political Economy of Foreign Aid: A Model of the Market for a Public Good. Economic

Development and Cultural Change, 33(2), 373–93.

O’Keefe, J. (2007). Aid: From Consensus to Competition? Paper presented at the Brookings Blum Roundtable

(August 3, 2007), Washington DC.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

ODI/OECD. (2014). The New Development Finance Landscape: Developing Countries’ Perspective (Working Draft

Presented at the OECD Workshop on Development Finance on 25 June 2014). Paris: OECD.

OECD. (2017). Strengthening the Results Chain: Synthesis of Case Studies of Results-Based Management by

Providers. Paris: OECD DAC.

OECD DAC. (2003). Measuring Performance in Public Financial Management. Harmonising Donor Practices for

Effective Aid Delivery. Paris: OECD.

Olivié, I., & Pérez, A. (2016). Why Don’t Donor Countries Coordinate Their Aid? A Case Study of European Donors

in Morocco. Progress in Development Studies, 16(1), 52–64.

Orth, M., Schmitt, J., Krisch, F., & Oltsch, S. (2017). What We Know About the Effectiveness of Budget Support:

Evaluation Synthesis. Bonn, Germany: German Institute for Development Evaluation.

PEFA Secretariat. (2016). PEFA: Framework for Assessing Public Financial Management. Washington DC: PEFA

Secretariat. Retrieved from https://pefa.org/sites/default/les/PEFA%20Framework_English.pdf

PEMPAL Steering Committee. (2012). PEMPAL Strategy 2012–17. Washington DC: World Bank.

Peterson, S. (2010). Reforming Public Financial Management in Africa Cambridge Mass.: Harvard Kennedy

School.

Porter, D., Allen, B., & Thompson, G. (1991). Development in Practice – Paved with Good Intentions. London and

New York: Routledge.

Power, M., Scheytt, T., Soin, K., & Sahlin, K. (2009). Reputational Risk as a Logic of Organizing in Late Modernity.

Organization Studies, 30, 301–24.

Pretorius, C., & Pretorius, N. (2009). Review of Public Financial Management Reform Literature. London: DFID.

Pritchett, L., & Sandefur, J. (2013). Context Matters for Size: Why External Validity Claims and Development

Practice Do Not Mix. Journal of Globalization and Development, 4(2), 161–97.

Pritchett, L., Woolcock, M., & Andrews, M. (2010). Capability Traps? The Mechanisms of Persistent

Implementation Failure (Background Paper for the 2011 World Development Report on Development,

Conict and Fragile States). Washington DC: World Bank.

Prizzon, A. (2014). The Age of Choice: Papua New Guinea in the New Aid Landscape. London: ODI.

Radelet, S. (2015). The Great Surge: The Ascent of the Developing World. New York: Simon and Schuster.

Reinsberg, B. (2018). Blockchain Technology and the Governance of Foreign Aid. Journal of Institutional

Economics, 2018, 1–17.

Santiso, J., & Frot, E. (2010). Crushed Aid: Why Is Fragmentation a Problem for International Aid? VOX

CEPR Policy Portal. Retrieved from https://voxeu.org/article/crushed-aid-why-fragmentation-problem-

international-aid

Scharfstein, D., & Stein, J. (1990). Herd Behavior and Investment. American Economic Review, 80(June), 465–79.

Schick, A. (1998a). A Contemporary Approach to Public Expenditure Management (35116). Washington DC:

World Bank Institute.

Schick, A. (1998b). Why Most Developing Countries Should Not Try New Zealand’s Reforms. World Bank

Research Observer (International), 13, 23–31.

Schick, A. (2019). An Introduction to Budgeting: Everywhere Budgeting Is the Same and Different. In Budgeting

and Public Expenditures in OECD Countries 2019 (pp. 19–30). Paris: OECD.

Schmaljohann, M., & Prizzon, A. (2014). The Age of Choice: Fiji and Vanuatu in the New Aid Landscape. London:

ODI.

Scott, Z. (2011). Evaluation of Public Sector Governance Reforms 2001–2011: Literature Review Oxford, UK:

Oxford Policy Management.

Simson, R., Sharma, N., & Aziz, I. (2011). A Guide to Public Financial Management Literature: For Practitioners

in Developing Countries. London: ODI.

Stevens, M. (2004). Institutional and Incentive Issues in Public Financial Management Reform in Poor Countries.

Washington DC: World Bank.

Szulanski, G. (1996). Exploring Internal Stickiness: Impediments to the Transfer of Best Practice within the Firm.

Strategic Management Journal, 17(Special Issue: Knowledge and the Firm), 27–43.

Team Technologies. (2005). The Log Frame Handbook. Washington DC: World Bank.

Teskey, G., & Tyrrel, L. (2017). Thinking and Working Politically in Large, Multi-Sector Facilities: Lessons to Date.

Canberra ACT, Australia: Abt Associates.

United States Agency for International Development (USAID). (2014). Guide to Public Financial Management.

Washington DC: USAID Ofce of Economic Policy, Capacity Building Division.

Valters, C. (2015). Theories of Change: Time for a Radical Approach to Learning in Development. London: ODI.

Valters, C., & Whitty, B. (2017). The Politics of the Results Agenda in DFID: 1997–2017. London: ODI.

Van Dooren, W. (2006). Performance Measurement in the Flemish Public Sector: A Supply and Demand Approach.

Leuven, Belgium: Faculty of Social Sciences.

Van Dooren, W., Manning, N., Malinska, J., Kraan, D.-J., Sterck, M., & Bouckaert, G. (2006). Issues in Output

Measurement for "Government at a Glance": OECD Gov Technical Paper 2. Paris: OECD.

Vogel, I. (2012). Review of the Use of ‘Theory of Change’ in International Development. London: DFID.

Welham, B., Hart, T., Mustapha, S., & Hadley, S. (2017). Public Financial Management and Health Service

Delivery: Necessary, but Not Sufcient? London: ODI.

Wildy, H. (2004). The Data Club: Helping Schools Use Accountability Data. Yearbook of the National Society for

the Study of Education, 103(2), 155–68.

Wilson, D. (2010). Targets, Choice and Voice: Accountability in Public Services. London: 2020 Public Services

Trust.

Woods, N. (2008). Whose Aid? Whose Inuence? China, Emerging Donors and the Silent Revolution in

Development Assistance. International Affairs, 84(6), 1205–21.

Woolcock, M. (2013). Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions.

Cambridge, MA: Harvard Kennedy School.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

World Bank. (2002). Thailand's Hurdle Approach to Budget Reform. Washington DC: World Bank.

World Bank (2010). Safeguards and Sustainability Policies in a Changing World. IEG, World Bank, Washington

DC.

World Bank. (2013). Program Results Framework: Public Expenditure Management Network in Asia (PEMNA),

P129651. Washington DC: World Bank.

References for Chapter 4

Ababou, A. (2019). Building PFM Capabilities Programme: Year 2 in Retrospect. Retrieved from https://www.cabri-

sbo.org/en/blog/2019/building-pfm-capabilities-programme-year-2-in-retrospect

Abiad, A., Furceri, D., & Topalova, P. (2015). The Macroeconomic Effects of Public Investment: Evidence from

Advanced Economies. Washington DC: IMF.

Agénor, P.-R., & Moreno-Dodson, B. (2006). Public Infrastructure and Growth: New Channels and Policy

Implications. Washington DC: World Bank.

Alesina, A., Favero, C., & Giavazzi, F. (2019). Austerity : When It Works and When It Doesn't. Princeton NJ:

Princeton University Press.

Allen, R. (2019, March 11, 2019). [Email].

Alston, P. (2018). Statement on Visit to the United Kingdom by Professor Philip Alston (United Nations Special

Rapporteur on Extreme Poverty and Human Rights). Retrieved from https://www.ohchr.org/en/

NewsEvents/Pages/DisplayNews.aspx?NewsID=23881&LangID=E

Alt, J. E. (2019). Twenty Years of Transparency Research (Keynote Speech at “Public Sector Economics 2018 –

Fiscal Openness: Transparency, Participation and Accountability in Fiscal Policies” in Zagreb on October

26, 2018). Cambridge, MA: Institute of Public Finance, International Budget Partnership and Friedrich-

Ebert-Stiftung.

Andrews, M. (2007). What Would an Ideal Public Finance Management System Look Like? In A. Shah (Ed.),

Budgeting and Budgetary Institutions (pp. 359–86). Washington DC: World Bank.

Andrews, M. (2008). The Good Governance Agenda: Beyond Indicators without Theory. Oxford Development

Studies, 36(4), 379–407.

Andrews, M., Cangiano, M., Cole, N., de Renzio, P., Krause, P., & Seligmann, R. (2014). This Is PFM. Cambridge,

MA: Center for International Development at Harvard University.

Ardagna, S., & Caselli, F. (2014). The Political Economy of the Greek Debt Crisis: A Tale of Two Bailouts.

American Economic Journal: Macroeconomics, 6(4), 291–323.

Argyris, C. (1977). Double Loop Learning in Organizations. Harvard Business Review(September 1977).

Argyris, C., & Schön, D. (1978). Organizational Learning: A Theory of Action Perspective. Reading, MA: Addison

Wesley.

Argyris, C., & Schön, D. (1996). Organizational Learning II: Theory, Method and Practice. Reading, MA: Addison

Wesley.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Atkinson, T., Grice, J., Simkins, A., Freitas, L. D., Hemingway, J., King, B., Lee, P., Lyon, M., Mai, N., Mehmi, S.,

Pritchard, A., Snelling, J., Tuke, A., Watson, L., & Fletcher-Cooke, G. (2005). Measurement of Government

Output and Productivity for the National Accounts. Basingstoke: Palgrave.

Barroy, H., Kabaniha, G., Boudreaux, C., Cammack, T., & Bain, N. (2019). Leveraging Public Financial

Management for Better Health in Africa: Key Bottlenecks and Opportunities for Reform. Geneva: World

Health Organization.

Blanchard, O. (2019). Public Debt: Fiscal and Welfare Costs in a Time of Low Interest Rates (Policy Brief 19-2).

Washington DC: Peterson Institute for International Economics.

Caiden, N. (2010). Challenges Confronting Contemporary Public Budgeting: Retrospectives/Prospectives from

Allen Schick. Public Administration Review, 70(2), 203–10.

Campos, E., & Pradhan, S. (1996). Budgetary Institutions and Expenditure Outcomes (Policy Research Working

Paper No. 1646). Washington DC: World Bank.

Cangiano, M. (2017). Of Rating and Ranking in the Comparison of Fiscal Institutions. Journal of Comparative

Policy Analysis: Research and Practice, 19(4), 377–89.

Carter, B. (2015). Pro Pro-Poor National Budgets. Birmingham, UK: GSDRC, University of Birmingham.

Cashin, C., Bloom, D., Sparkes, S., Barroy, H., Kutzin, J., & O'Dougherty, S. (2017). Aligning Public Financial

Management and Health Financing: Sustaining Progress toward Universal Health Coverage. Geneva:

WHO.

Catalano, G., & Erbacci, A. (2018). A Theoretical Framework for Spending Review Policies at a Time of

Widespread Recession. OECD Journal on Budgeting, 2017(2).

Chen, Y. (2019). Ethiopia and Kenya Are Struggling to Manage Debt for Their Chinese-Built Railways. Quartz

Africa. Retrieved from https://qz.com/africa/1634659/ethiopia-kenya-struggle-with-chinese-debt-over-sgr-

railways/

Clements, B. J., de Mooij, R. A., Gupta, S., & Keen, M. (Eds.). (2015). Inequality and Fiscal Policy. Washington

DC: IMF.

Collier, P., & Hoefer, A. (2005). Resource Rents, Governance, and Conict. Journal of Conict Resolution, 49(4),

625–33.

Dabla-Norris, E., Allen, R., Zanna, L.-F., Prakash, T., Kvintradze, E., Lledo, V., Yackovlev, I., & Gollwitzer, S.

(2010). Budget Institutions and Fiscal Performance in Low-Income Countries. Washington DC: IMF.

de Renzio, P. (2019). Putting the ‘Public’ Back into Public Finance: How Better Budgets Can Save Democracy.

Di Maggio, P. J., & Powell, W. W. (1983). The Iron Cage Revisited: Institutional Isomorphism and Collective

Rationality in Organizational Fields. American Sociological Review, 48(April), 147–60.

Economists Are Rethinking Fiscal Policy. (2019). The Economist (May 16, 2019).

Elson, D. (1999). Gender Budget Initiative: Background Papers. London: Commonwealth Secretariat.

European Commission. (2013). A Vision for Public Services (Draft Version). Brussels: European Commission, DG

Connect. Retrieved from http://ec.europa.eu/digital-agenda/en/news/vision-public-services

European Commission. (2019). Financial Assistance to Greece: Information on the Enhanced Surveillance

Framework for Greece. Retrieved from https://ec.europa.eu/info/business-economy-euro/economic-and-

scal-policy-coordination/eu-nancial-assistance/which-eu-countries-have-received-assistance/nancial-

assistance-greece_en

Filmer, D., & Pritchett, L. (1999). The Impact of Public Spending on Health: Does Money Matter? Social Science

and Medicine, 49(10), 1309–23.

Finger, H., & Gressani, D. (2014). Toward New Horizons: Arab Economic Transformation Amid Political

Transitions. Washington DC: IMF.

Foster, A. K., Rinehart, A. K., & Springs, G. R. (2019). Piloting the Purchase of Research Data Sets as Collections:

Navigating the Unknowns. Libraries and the Academy, 19(1), 315–28.

Fozzard, A. (2001). The Basic Budgeting Problem: Approaches to Resource Allocation in the Public Sector and

Their Implications for Pro-Poor Budgeting (Working Paper 147). London: ODI.

Furman, J., & Summers, L. (2019). Who’s Afraid of Budget Decits? How Washington Should End Its Debt

Obsession. Foreign Affairs, March/April 2019.

Gilmore, E. A., & St. Clair, T. (2018). Budgeting for Climate Change: Obstacles and Opportunities at the US State

Level. Climate Policy, 18(6), 729–41.

Goryakin, Y., Revill, P., Mirelman, A., Sweeney, R., Ochalek, J., & Suhrcke, M. (2017). Public Financial

Management and Health Service Delivery: A Literature Review. London: ODI.

Government of New Zealand. (2019). The Wellbeing Budget. Wellington, New Zealand: Department of the Prime

Minister and Cabinet.

Hadley, S., & Miller, M. (2016). PEFA: What Is It Good For? London: ODI.

Haque, T., Bogoev, J., & Smith, G. (2017). Push and Pull: Emerging Risks in Frontier Economy Access to

International Capital Markets. Washington DC: Macroeconomics and Fiscal Management (MFM) Global

Practice, World Bank.

Hood, C., King, M., & Piotrowska, B. (2019 (forthcoming)). Evaluating the Efcacy of Expenditure Control over

Time: A New Standard Approach? Oxford: Blavatnik School of Government, Oxford, Nufeld and the

ESRC History of UK Public Expenditure Control Project.

Howell, T. J. (2013). IMF Chief: U.S. Dance with the Debt Limit Is ‘Very, Very Concerning.’, October 13, 2013.

Retrieved from https://www.washingtontimes.com/news/2013/oct/13/imf-chief-us-dance-debt-limit-very-

very-concerning/.

Hughes, O. (2018). Public Management and Administration: An Introduction (5th Edition). London: Palgrave.

Hughes, R. (2019, March 11, 2019). [Email].

IMF. (2010). Imf Survey Online: Interview with Olivier Blanchard. Retrieved from https://www.imf.org/en/News/

Articles/2015/09/28/04/53/sonew090910a

IMF. (2017a). Fiscal Monitor: Tackling Inequality. Washington DC: IMF.

IMF. (2017b). Gender Budgeting in G7 Countries. Washington DC: IMF.

IMF. (2018). Fiscal Monitor: Managing Public Wealth. Washington DC: IMF.

IMF. (2019). 2018 Review of Program Design and Conditionality (Policy Paper No. 19/012). Washington DC: IMF.

Kim, J. (2017). Growing Aspirations of People around the World—How Do We Meet Them? Retrieved from https://

olc.worldbank.org/about-olc/growing-aspirations-people-around-world-how-do-we-meet-them

Kraan, D.-J., & Kelly, J. (2005). Reallocation: The Role of Budgetary Institutions. Paris: OECD.

Krause, P., Hadley, S., Hedger, E., de Renzio, P., Gill, S., & Andrews, M. (2014). Joint Response to the PEFA

Consultation: We Need a Stronger Summary Assessment. London: ODI.

Krause, P., Sweet, S., Hedger, E., & Chalise, B. (2013). Operational Risk Assessment of Public Financial

Management Reform in Nepal: A Review of Challenges and Opportunities. Washington DC: World Bank.

Lagarde, C. (2019). Forging a Stronger Social Contract—The IMF’s Approach to Social Spending [Press release].

Retrieved from https://www.imf.org/en/News/Articles/2019/06/14/sp061419-md-social-spending

Lau, E. (2011). Typology and Implementation of Spending Reviews – Discussion Paper (OECD Senior Budget

Ofcials Network on Performance and Results – 7th Annual Meeting, 9-10 November 2011). Paper

presented at the Working Party of Senior Budget Ofcials, Paris.

Lawson, A. (2012). Evaluation of Public Financial Management Reform in Burkina Faso, Ghana and Malawi

2001–2010: Final Synthesis Report. Oxford, UK: Fiscus Public Finance Consultants, Ltd.

Lin, J. Y., & Wang, Y. (2017). Development Beyond Aid: Utilizing Comparative Advantage in the Belt and Road

Initiative to Achieve Win-Win. Journal of Infrastructure, Policy and Development, 1(2), 149–67.

Lindblom, C. (1959). The Science of Muddling Through. Public Administration Review, 19, 79–88.

Mason, D., & Kluegel, J. (Eds.). (2000). Marketing Democracy: Changing Opinion About Inequality and Politics in

East Central Europe. Lanham, MD: Rowman & Littleeld.

Milanovic, B. (2016). The Winners and Losers of Globalization. Cambridge, MA: Harvard University Press.

Montambeault, F. (2015). The Politics of Local Participatory Democracy in Latin America : Institutions, Actors, and

Interactions. Stanford, CA: Stanford University Press.

Moore, M. (2013). Revenue Reform and Statebuilding in Anglophone Africa Brighton, U.K.: Institute of

Development Studies

Munro, N. (2017). Predictors of Support for State Social Welfare Provision in Russia and China. Europe-Asia

Studies, 69(1), 53–75.

Musgrave, R. A., & Musgrave, P. B. (1989). Public Finance in Theory and Practice (Fifth Edition). San Francisco:

McGraw Hill.

Naidu, S., Rodrik, D., & Zucman, G. (2019). Economics for Inclusive Prosperity: An Introduction (Research Brief).

Economists for Inclusive Prosperity.

National Audit Ofce. (2012). Managing Budgeting in Government. London: NAO.

National Audit Ofce. (2015). Spending Review. London: NAO.

National Audit Ofce. (2018). Improving Government’s Planning and Spending Framework. London: NAO.

O'Hagan, A., & Klatzer, E. (Eds.). (2018). Gender Budgeting in Europe: Developments and Challenges. New York:

Palgrave Macmillan US.

ODI. (forthcoming). Review of Public Financial Management Diagnostics for the Health Sector. London: ODI and

UNICEF.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

OECD. (2005). Modernising Government: The Way Forward. Paris: OECD.

OECD. (2018). Economic Survey of Greece. Paris: OECD.

Ostry, J. D., Loungani, P., & Berg, A. (2019). Confronting Inequality: How Societies Can Choose Inclusive Growth

New York: Columbia University Press.

Panzer, J., Thomas, M. R., Peuker, A. R., Razlog, L., Prasad, A., Anderson, P. R. D., & Li, Y. (2015). Debt

Management Performance Assessment (DeMPA) Methodology. Washington DC: World Bank.

PEFA Secretariat. (2016). PEFA: Framework for Assessing Public Financial Management. Washington DC: PEFA

Secretariat. Retrieved from https://pefa.org/sites/default/les/PEFA%20Framework_English.pdf

PEFA Secretariat. (2018). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

PEFA Secretariat. (2019). PEFA Handbook Volume IV: Using PEFA to Support Public Financial Management

Improvement. Washington DC: PEFA.

Peixoto, T., & Sifry, M. L. (2017). Civic Tech in the Global South : Assessing Technology for the Public Good.

Washington DC: World Bank and Personal Democracy Press.

Piatti-Fünfkirchen, M., & Schneider, P. (2018). From Stumbling Block to Enabler: The Role of Public Financial

Management in Health Service Delivery in Tanzania and Zambia. Health Systems & Reform, 4(4), 336–45.

Presbitero, Andrea F., Ghura, D., Adedeji, O. S., & Njie, L. (2015). International Sovereign Bonds by Emerging

Markets and Developing Economies: Drivers of Issuance and Spreads. Washington DC: IMF.

Prime Minister's Ofce. (2019). PM Theresa May: We Will End UK Contribution to Climate Change by 2050

[Press release]. Retrieved from https://www.gov.uk/government/news/pm-theresa-may-we-will-end-uk-

contribution-to-climate-change-by-2050

Ricke, K. L., Moreno-Cruz, J. B., Schewe, J., Levermann, A., & Caldeira, K. (2016). Policy Thresholds in

Mitigation. Nature Geoscience, 9, 5–6.

Roberts, J. (2003). Poverty Reduction Outcomes in Education and Health: Public Expenditure and Aid. London:

ODI.

Robinson, M. (2013). Spending Reviews (Paper for the Annual Meeting of OECD Senior Budget Ofcials, 3-4 June

2013). Paper presented at the Working Party of Senior Budget Ofcials, Paris.

Robinson, M. (2016). Budget Reform before and after the Global Financial Crisis. OECD Journal on Budgeting,

2016(1), 29–63.

Rogoff, K. (2019). The Austerity Chronicles. Retrieved from https://www.project-syndicate.org/onpoint/the-

austerity-chronicles-by-kenneth-rogoff-2019-04?barrier=accesspaylog

Ross, M. L. (1999). The Political Economy of the Resource Curse. World Politics, 51, 297–322.

Rudiger, A. (2018). Fiscal Futures: Scenario Thinking for Fiscal Transparency and Accountability. Washington DC:

Carnegie Endowment for International Peace, the International Budget Partnership and the Transparency

and Accountability Initiative.

Scharfstein, D., & Stein, J. (1990). Herd Behavior and Investment. American Economic Review, 80(June), 465–79.

Schick, A. (1998a). A Contemporary Approach to Public Expenditure Management (35116). Washington DC:

World Bank Institute.

Schick, A. (1998b). Why Most Developing Countries Should Not Try New Zealand's Reforms. World Bank Research

Observer (International), 13, 23–31.

Schick, A. (2013). Reections on Two Decades of Public Financial Management Reforms. In M. Cangiano, T. R.

Curristine, & M. Lazare (Eds.), Public Financial Management and Its Emerging Architecture (pp. 21–78).

Washington DC: IMF.

Schumpeter, J. A. (1918). Die Krise Des Steuerstaates. Berlin: Graz, Leuschner & Lubensky.

Senge, P. (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. London: Random House.

Simson, R. (2012). Following the Money: Examining the Evidence on ‘Pro-Poor’ Budgeting. London: ODI.

Tilly, C. (1992). Coercion, Capital and European States, AD 990–1992. Oxford: Blackwell Publishing.

Ubaldi, B. (2013). Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives

(OECD Working Papers on Public Governance, No. 22). Paris: OECD.

United States House of Representatives (116th Congress). (2019). “Recognizing the Duty of the Federal

Government to Create a Green New Deal.” H.Res. 109 (February 7).

van der Weide, R., & Milanovic, B. (2014). Inequality Is Bad for Growth of the Poor (but Not for That of the Rich).

Washington DC: World Bank.

Vlaicu, R., Verhoeven, M., Grigoli, F., & Mills, Z. (2014). Multiyear Budgets and Fiscal Performance: Panel Data

Evidence. Journal of Public Economics, 111(C), 79–95.

Welham, B., Hart, T., Mustapha, S., & Hadley, S. (2017). Public Financial Management and Health Service

Delivery: Necessary, but Not Sufcient? London: ODI.

Welham, B., Krause, P., & Hedger, E. (2013). Linking PFM Dimensions to Development Priorities. London: ODI.

Wildavsky, A. (1992). The New Politics of the Budget Process. New York: Harper Collins.

Wilhelm, V., & Fiestas, I. (2005). Exploring the Link between Public Spending and Poverty Reduction: Lessons from

the 90s. Washington DC: World Bank Institute.

Williamson, T., & Canagarajah, S. (2006). Pro-Poor Public Spending Reform : Uganda's Virtual Poverty Fund

(PREM Notes No. 108). Washington DC: World Bank.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

World Bank. (2012a). Beyond the Annual Budget: Global Experience with Medium-Term Expenditure Frameworks.

Washington DC: World Bank.

World Bank. (2012b). The World Bank’s Approach to Public Sector Management 2011–2020: Better Results from

Public Sector Institutions. Washington DC: World Bank.

World Bank. (2014a). Climate Change Public Expenditure and Institutional Review Sourcebook. Washington DC:

World Bank.

World Bank. (2014b). Moving toward Climate Budgeting: Policy Note. Washington DC: World Bank.

World Bank. (2016). PFM in Health – Conceptual Framework (Unpublished Draft Report). Washington DC.

References for Chapter 5

Andrews, M. (2013). The Limits of Institutional Reform in Development. New York: Cambridge University Press.

Andrews, M., Pritchett, L., & Woolcock, M. (2012). Escaping Capability Traps through Problem-Driven Iterative

Adaptation (PDIA) (Working Paper 299). Washington DC: Center for Global Development.

Barder, O. (2018). Aid in the National Interest: When is Development Cooperation Win-Win? Retrieved from

https://www.cgdev.org/blog/aid-national-interest-when-development-cooperation-win-win

Birdsall, N. (2013). The Future of Aid: 2030 ODA No More. In A. Sumner & T. Kirk (Eds.), The Donors’ Dilemma:

Emergence, Convergence and the Future of Foreign Aid. Chichester, UK: Wiley.

Blum, J. R., Manning, N., & Srivastava, V. (2012). Public Sector Management Reform: Toward a Problem-

Solving Approach. Washington DC: World Bank. Retrieved from http://siteresources.worldbank.org/

EXTPREMNET/Resources/EP100.pdf

Booth, D. (2014). Aiding Institutional Reform in Developing Countries: Lessons from the Philippines on What

Works, What Doesn't, and Why. London: ODI.

Booth, D., & Unsworth, S. (2014). Politically Smart, Locally Led Development. London: ODI.

PEFA Secretariat. (2018). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

Gupta, S. and Barroy, H. (2020). The COVID-19 Crisis and Budgetary Space for Health in Developing Countries.

Retrieved from https://www.cgdev.org/blog/covid-19-crisis-and-budgetary-space-health-developing-

countries

Gurazada, S., Kristensen, J., Sjoblom, M., Piatti, M. and Farooq, K. (2020). Getting government nancial

management systems COVID-19 ready. Retrieved from https://blogs.worldbank.org/governance/getting-

government-nancial-management-systems-covid-19-ready

Saxena, S. and Stone, S. (2020). Preparing Public Financial Management Systems to Meet Covid-19 Challenges.

Retrieved from https://blog-pfm.imf.org/pfmblog/2020/03/preparing-public-nancial-management-

systems-to-meet-covid-19-challenges.html

World Bank. (2000). Reforming Public Institutions and Strengthening Governance: A World Bank Strategy.

Washington DC: World Bank.

World Bank. (2012b). The World Bank’s Approach to Public Sector Management 2011–2020: Better Results from

Public Sector Institutions. Washington DC: World Bank.

Chapter References

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Annex A: Key concepts and terms

used in the report

This annex denes a number of the operative

terms and concepts used in this report. There are

very few settled denitions in the PFM literature,

and the intention is to explain and clarify the usage

in this report—not to join wider debates about

denitions or preferred usage.

For ease of explanation, four categories are used

here: public policy, scal policy, public nancial

management, and external assistance. Underlined

terms are those dened in Table 5 below.

1. Public policy: Public policy is a three-

level framework encompassing government

choices, actions, and results. Government

policy choices cover stated and revealed

intentions and objectives. Government

actions and outputs are policy choices being

put into action, including service delivery

(or the provision of collective goods and

services). Development results (including

sector outcomes) are the policy impacts that

stem from government choices and actions,

or lack thereof (see also Annex B).

2. Fiscal policy: Fiscal policy covers

government public nance policies. It is

characterized by scal policy norms that

inuence strategy and choices. The norms

are often expressed or observed through

scal rules of different types (such as

numerical limits on budgetary aggregates).

Choices about scal rules also feed back into

scal policy norms.

3. Public nancial management: The standard

PFM objectives dene the purpose of PFM.

The PFM system in a country consists of all

of its separate PFM processes and how they

t or work together. The PFM conventions

are the orthodoxy about how each of the

individual PFM processes should look or

perform. The PFM assessment frameworks

measure the performance of individual

PFM processes against the prevailing

PFM conventions. The PFM discipline

is the professional guild of experts and

practitioners who work in the eld of PFM.

4. External assistance: The intervention logic

explains the design assumptions about how/

why the intervention is expected to work.

Development assistance modalities are the

mechanisms or instruments through which

external support is provided. Problem-

based and adaptive approaches are a way

of attempting to ensure that the intervention

logic is not prescriptive and that the

assistance modalities are not rigid.

ANNEXES

Table 5: Terms of art used in the report

Term of art Denion Examples (if applicable)

A. Public policy (see also Annex B)

Public policy “Anythingagovernmentchoosestodoornotto

do”(Dye,1972,p.2).

“Publicpolicy-makinginvolvesafundamental

choice[thatisdeliberateandconscious]onthe

partofgovernmentstodosomethingortodo

nothingwithrespecttoaproblem”(Howle&

Cashore,2014,p.18).

Representedbyathree-levelframework

comprising:governmentintenonsorpolicy

choices,governmentaconsandoutputs,and

policyimpacts(or,morenarrowly,development

results).(Seealsoseparatedenions.)

Government

policy

choices

“Decisionsmadebypolicians,civilservants,

orothersgrantedauthoritythataredirected

towardsusingpublicpowertoaectthelivesof

cizens”(Peters,2018,p.4).

Agovernment’sintenonsanddecisions.These

includebothpolicychoicesexplicitlystatedby

government,andthoserevealedimplicitlyby

governmentacons.

Improvedprosperity,well-being,equity,and

sustainability—asexpressed,forexample,

throughspeciccommitmentsonpublic

servicesandpublicmanagement,on

regulaonofeconomicbehavior,andon

responsestoachangingmacroenvironment.

Government

acons

“Policychoicesbeingputintoacon”(Peters,

2018,p.4).

Governmentaconscovereconomicinstruments

(includingpublicservicesandnancial

incenves),regulatoryinstruments,informaon

instruments,andadministravemeasures.

Provisionofpublicservices,cashtransfers,

regulaon,informaoncampaigns,

budgetsystemreforms,humanresource

managementreforms,andsoon.

Service

delivery

Servicedeliveryencompassestherangeof

collecvegoodsandserviceswithinthesphere

ofgovernmentacon.Thesegoodsandservices

areeitherprovidedbygovernmentthroughthe

budget,orfallundertheremitofgovernment

producon,subsidy,orregulaon(Hughes,2018).

Servicedeliveryisasubsetofgovernmentacons

takentoimplementitspublicpolicyintenons,

typicallywithanemphasisongovernment

provisionanditsoutputs.

Collecvegoodsandservicesinclude

educaon,lawandorder,environmental

protecon,naonaldefense,roadsand

bridges,hospitalsandhealthcare,welfare

services,publictransport,andsoon.

Development

results

“Theeectsthatpolicychoicesandpolicy

outputshaveoncizens”(Peters,2018,p.4).

Abroadcategorycoveringtheimpactsof

governmentpolicychoicesandacons(see

separatedenions).Forthepurposesof

thisreport,itisrestrictedtothesubsetof

policyimpactswithwhichtheinternaonal

developmentcommunityistypicallymost

concernedandwhicharetreatedasdesirable

developmentoutcomes.

Themostgenerallyusedandaccepted

contemporaryformulaonofdevelopment

resultsisthe2030AgendaforSustainable

Development:

• “Endingpovertyandotherdeprivaons

mustgohand-in-handwithstrategiesthat

improvehealthandeducaon,reduce

inequality,andspureconomicgrowth—all

whiletacklingclimatechangeandworking

topreserveouroceansandforests”

(UnitedNaons,2015).

Annex A: Key concepts and terms used in the report

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Term of art Denion Examples (if applicable)

Adisnconmaybedrawnbetween

intermediateresultssuchastheoutputs

fromservicedeliverybygovernment,andthe

outcomesinsectorstowhichthatservicedelivery

hascontributed.

Developmentresultsextendbeyondafocuson

sectorstocovergovernanceissues,including

scalandinstuonalsustainability(Kaufmann,

Kraay&Mastruzzi,2010).

B. Fiscal policy

Fiscal policy Measuresemployedbygovernmentstostabilize

theeconomy,specicallybymanipulangthe

levelsandallocaonsoftaxesandgovernment

expenditures.

Fiscal policy

norms

Theorthodoxiesthatgovern,determine,or

constrainpublicnancepolicyaconsand

parameters.Theyarealsothenormave

assumponsthatinformandinuenceexternal

ideasandadviceaboutsuitablepolicydirecons

andstrategy.

Theyareoenformallycodiedas“scalrules”

thatfocusonscaldisciplineandsustainability.

Fiscal rules “Ascalruleisalong-lasngconstraintonscal

policythroughnumericallimitsonbudgetary

aggregates”(Lledó,Yoon,Fang,Mbaye&Kim,

2017,p.8).

Typesofscalrulesincludebudgetbalance

rules,debtrules,expenditurerules,andrevenue

rules—applyingtothecentralorgeneral

governmentorthepublicsector(Eyraud,Debrun,

Hodge,Lledo&Pallo,2018;Lledóetal.,2017).

Examplesofscalrules:

• “EUMemberStatesareconstrainedbythe

rulesoftheStabilityandGrowthPact.This

includesarequirementtokeepborrowing

below3%ofGDP.

• Structuralborrowing(borrowingadjusted

fortheupsanddownsoftheeconomy)

oftheGermanfederalgovernmentis

cappedat0.35%ofGDPinthecountry’s

constuon.Germanyalsohastargetsfor

theLänders(states)”(Keep&Ward,2019,

pp.3–4).

C. Public nancial management

Standard

PFM

objecves

Thestated“desirableoutcomes”ofaPFM

system,reecnganormaveorientaonto

parcularobjecvesthathavebecomethe

orthodoxyoverthepast20–25years(Campos

&Pradhan,1996;Schick,1998;WorldBank,

1998).Theseareineecta“holytrinity”ofPFM

objecves:scaldiscipline,allocaveeciency,

andoperaonaleciency.

ThestandardPFMobjecvesmaybe

describedasfollows:

• “Aggregatescaldisciplinerequires

eecvecontrolofthetotalbudgetand

managementofscalrisks.

• Strategicallocaonofresources involves

planningandexecungthebudgetin

linewithgovernmentprioriesaimedat

achievingpolicyobjecves.

• Ecientservicedeliveryrequiresusing

budgetedrevenuestoachievethebest

levelsofpublicserviceswithinavailable

resources”(PEFASecretariat,2016,p.2).

Term of art Denion Examples (if applicable)

PFM system TheamalgamofallPFMprocessesand

proceduresinaparcularjurisdicon.

Ineect,thesearetheinstuonalarrangements

forPFMandhowitoperatesinagivencountry

(orsubnaonaldomain).“PFMisnowseenasan

‘umbrella’denion,coveringasetofsystems

aimedatproducinginformaon,processes,and

rules…”(Cangianoetal.,2013,pp.1–2).

PFM

processes

The technical elements or components of the

PFMsystem.

Thesearenotinandofthemselvesnormave

proposions,unlikethePFMconvenons(see

separatedenion).Theyaretechnicalbuilding

blocksaboutwhichdesignchoicescanbemade.

Budgetcalendar,scalstrategypaper,

nancialstatements,auditreport,budget

hearings,cash-owforecast,payrollrecords,

procurementmethod,andsoon.

PFM

assessment

frameworks

(or tools)

The“toolsavailableforassessingPFMsystems

asabasisfordevelopingandmonitoringreform

plansandthusstrengtheningPFMprocesses”

(PEFASecretariat,2018,p.1).

Theseframeworksandtoolsnotonlycapture

the PFM processes that comprise the overall

PFMsystem,theyalsoreecttheprevailing

“convenons”aboutthedesirableoperaonal

performanceofthePFMsystem(seeseparate

denion).

PEFA,TADAT,DeMPA,PIMA,OpenBudget

Index,FiscalTransparencyCode,andothers

(AnnexF).

PFM

convenons

Thenormaveassumponsaboutdesirable

“operaonalperformanceofkeyelementsofthe

PFMsystem”basedon“internaonalstandards

andgoodpraccesoncrucialaspectsofPFM,

asidenedbyexperiencedpraconers”and

asexpressedinthestandardPFMassessment

frameworksandtools(PEFASecretariat,2016,pp.

1–4).

Theframeworkstypicallyincludegraduated

performancescores,sotheconvenons

representatargengofthehighestscores.

Thereisanormaveassumponthatastronger

andmorecapablePFMsystemisoneinwhich

processesandproceduresfulllrequirementsat

thetopendofthemeasuredscales.

Generallyspeaking,thePFMconvenonscanbe

seeninthedescriponsofkeyelementsofthe

PFMsystemthatgarnerthehighestscoreinthe

variousassessmentframeworks.

Systemforallocangtransfers(PEFA

dimension7.1)

• AnAscore=“Thehorizontalallocaonof

alltransferstosubnaonalgovernments

fromcentralgovernmentisdetermined

bytransparent,rule-basedsystems”(PEFA

Secretariat,2016,p.26).

Performanceevaluaonforservicedelivery

(PEFAdimension8.4)

• AnAscore=“Independentevaluaons

oftheeciencyandeecvenessof

servicedeliveryhavebeencarriedout

andpublishedformost ministries at least

oncewithinthelastthreeyears”(PEFA

Secretariat,2016,p.28).

Countriesshouldhave:

• “Systemsforpublishinganannualplanof

aggregatedomescborrowingbasedon

theannualbudgetcashowforecasts

• Systemsforpublishingaborrowing

calendarinadvanceforwholesale

instruments”(Panzeretal.,2015,p.38).

Annex A: Key concepts and terms used in the report

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Term of art Denion Examples (if applicable)

PFM

discipline

Aprofessionaldisciplinecombiningacademic

subjectmaerwithcareerpathsthatinclude

opportuniesforadvancementinworkand

reputaon.

A“sociologyofcareerincenves,normsand

socializaonpaerns”(Naiduet.al,2019,p.1).

D. External assistance

Intervenon

logic

(intervenon

design)

Severaloverlappingquesonsaregenerally

consideredexplicitlyor,moreoen,implicitly,in

thedesignofexternalintervenons:

• Thedegreetowhichitisassumedthatthe

speciedinput(training,newprocedures)will

leadinalinearfashiontotheintendedoutput

andoutcomeassetoutinaresultschain

(OECD,2017),logframe(DFID,2009;Team

Technologies,2005)orequivalent.

• Whetherthetheoryofchangesuggests

uncertaintyabouttheconsequencesofany

intervenonandtheneedforexibilityand

parcipaonbylocalactors(Valters,2015)and

henceemphasizestheneedforiteraonand

adaptaon.

• Whetherincenvestoachieveaspecied

outcomecouldbecombinedwithexibility

concerningthechoiceofinput(Birdsall&

Savedo,2010).Examplesofintervenontypes

thatfollowapureoutcome-basedlogicinclude

“cashondelivery”aidanddevelopmentimpact

bonds.

• Quesonsaboutthetechnicalsequencingof

theproposedPFMcontent(Diamond,2013).

“TheNaonalAssemblyofZambia

component operates on the demand side

ofPFM.Itseekstoimprovetheprocesses

ofaccountabilityandtransparencyby

strengtheningtheabilityandcapacityofthe

legislaturetoholdtheExecuveresponsible

foritsplansandacons.Strengtheningthe

demandforbeerPFM,includinggreater

accountabilityandtransparency,isseenas

aneecvemechanismforencouraging

improvedperformance”(DFCConsorum,

2016,p.67).

Development

assistance

modalies

(or

instruments)

Theformalstructuresofintervenonthrough

whichPFMideasandknowledgearedeployedor

deliveredtoclients/beneciaries(byinternaonal

organizaons,donorgovernments,consulng

rms,non-protorganizaons,individual

advisers,andsoon).

Thiscoversdesign,implementaon,and

evaluaonstagesofsupport,asapplicable.

Dierentcomponentsorinstrumentsmaybe

bundledtogetheraspartofasingleintervenon.

Technicalassistancemissionsorprojects,

investmentprojects,nancialsupport

throughgovernmentsystems(including

generalbudgetsupportandsectorbudget

support),pooledfunds,policyorstructural

reformcondions,results-basedfunding

(suchascashondeliveryandpaymentfor

results),andsoon.

Term of art Denion Examples (if applicable)

Problem-

based and

adapve

approaches

Techniquesforinvesgaon,dialogue,and

intervenon,which:

• Emphasizethelocally-speciccausesand

purposesofagivenapparentdysfuncon

(Andrewsetal.,2012)

• Assertthatinstuonalreformsareaddressing

“adapve”ratherthan“technical”problems

(Heifetz,1994,p.121)

• Emphasizetrialanderrorthrough“guided

incrementalexperimentaon”(Pritcheetal.,

2013,p.48).

“Problem-DrivenIteraveAdaptaon”

(Andrewsetal.,2012).

Flexibleprocesses(Brinkerho&Crosby,

2002;Brinkerho&Ingle,1989).

“Tryingout”newapproachestobudgeng,

accounng,procurement,andsoon,in

nanceministriesforalimitednumberof

agenciesforalimitedperiodofmetotest

howthingswork(Kristensen,2019;CID,

2014).

References

Andrews, M., Pritchett, L., & Woolcock, M. (2012). Escaping Capability Traps through Problem-Driven Iterative

Adaptation (PDIA) (Working Paper 299). Cambridge MA: Harvard Kennedy School.

Birdsall, N., & Savedoff, W. D. (2010). Cash on Delivery: A New Approach to Foreign Aid. Washington DC: Center

for Global Development.

Brinkerhoff, D. W., & Crosby, B. L. (2002). Managing Policy Reform. Bloomeld CT: Kumarian.

Brinkerhoff, D. W., & Ingle, M. D. (1989). Integrating Blueprint and Process: A Structured Flexibility Approach to

Development Management. Public Administration and Development, 9, 487–503.

Campos, E., & Pradhan, S. (1996). Budgetary Institutions and Expenditure Outcomes (Policy Research Working

Paper No. 1646). Washington DC: World Bank.

Cangiano, M., Curristine, T., & Lazare, M. (2013). Introduction. In M. Cangiano, T. R. Curristine, & M. Lazare

(Eds.), Public Financial Management and Its Emerging Architecture (pp. 1–20). Washington DC: IMF.

Center for International Development (CID) at Harvard University, Building State Capability Program. (2014). The

DDD Manifesto. ‘Doing Development Differently’ workshop. Retrieved from http://buildingstatecapability.

com/the-ddd-manifesto/

DFID. (2009). Guidance on Using the Revised Logical Framework London: DFID. Retrieved from http://www.dd.

gov.uk/Documents/publications1/how-to-guid-rev-log-fmwk.pdf

DFC Consortium. (2016). EU-Funded Programme in Zambia: Mid-Term Evaluation of the PFM Support

Programme. Brussels: EU.

Diamond, J. (2013). Good Practice Note on Sequencing PFM Reforms. Washington DC: PEFA Secretariat.

Dye, Thomas R. (1972). Understanding Public Policy. Englewood Cliffs: Prentice-Hall.

Eyraud, L., Debrun, X., Hodge, A., Lledó, V. D., & Pattillo, C. A. (2018). Second-Generation Fiscal Rules :

Balancing Simplicity, Flexibility, and Enforceability. Washington DC: IMF.

Government of New Zealand. (2019). The Wellbeing Budget. Wellington, New Zealand: Department of the Prime

Minister and Cabinet.

Heifetz, R. A. (1994). Leadership without Easy Answers. Cambridge: Harvard University Press.

Howlett, M., & Cashore, B. (2014). Conceptualizing Public Policy. In I. Engeli & Allison C.R. (eds), Comparative

Policy Studies. Research Methods Series. London: Palgrave Macmillan.

Hughes, O. (2018). Public Management and Administration: An Introduction (5th Edition). London: Palgrave.

Kaufmann, D., Kraay, A., & Mastruzzi, M. (2010). The Worldwide Governance Indicators: Methodology and

Analytical Issues. Washington DC: World Bank.

Keep, M., & Ward, M. (2019). Balanced Budget Rule. London: House of Commons Library.

Kristensen, J. K. (2019, February 28, 2019). [Email].

Lledó, V., Yoon, S., Fang, X., Mbaye, S., & Kim, Y. (2017). Fiscal Rules at a Glance. Washington DC: IMF.

Naidu, S., Rodrik, D., & Zucman, G. (2019). Economics for Inclusive Prosperity: An Introduction (Research Brief).

Economists for Inclusive Prosperity.

New Economics Foundation. (2012). Measuring Wellbeing: A guide for practitioners. London: New Economics

Foundation.

Annex A: Key concepts and terms used in the report

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

OECD. (2017). Strengthening the Results Chain: Synthesis of Case Studies of Results-Based Management by

Providers. Paris: OECD DAC.

Panzer, J., Thomas, M. R., Peuker, A. R., Razlog, L., Prasad, A., Anderson, P. R. D., & Li, Y. (2015). Debt

Management Performance Assessment (DeMPA) Methodology. Washington DC: World Bank.

PEFA Secretariat. (2016). PEFA: Framework for Assessing Public Financial Management. Washington DC: PEFA

Secretariat. Retrieved from https://pefa.org/sites/default/les/PEFA%20Framework_English.pdf

PEFA Secretariat. (2018). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

Peters, B. G. (2018). American Public Policy: Promise and Performance. New York: Sage.

Pritchett, L., Samji, S. & Hammer, J. (2013). It's All About MeE: Using Structured Experiential Learning ("E") to

Crawl the Design Space. Washington DC: Center for Global Development.

Schick, A. (1998). A Contemporary Approach to Public Expenditure Management (35116). Washington DC: World

Bank Institute.

Team Technologies. (2005). The Log Frame Handbook. Washington DC: World Bank.

United Nations. 2015. Sustainable Development Goals. Retrieved from https://sustainabledevelopment.un.org/sdgs

Valters, C. (2015). Theories of Change: Time for a Radical Approach to Learning in Development. London: ODI.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

Annex B: A framework for public

policy—government policy

choices, government actions, and

development results

A central proposition of the Working Group is

the need to ensure that there is an explicit and

dynamic connection between PFM and public

policy. While this report offers a reappraisal of the

scope and objectives of PFM, we do not attempt a

comparable assessment of public policy as a topic.

Instead we invoke well-established denitions and

frameworks from the public policy literature to

organize the arguments of the report.

This annex provides a description and explanation

of the framework for public policy that is adopted

by the Working Group. In particular, it highlights

some of central concepts used in this report—

namely, “policy choices,” “service delivery,”

and “development results”—and it locates those

within the framework. Recognizing that there is

no consensus on the terminology and concepts

discussed here, our purpose is not to suggest any

sort of general resolution, but rather to provide

a readers’ guide to the particular usage in the

present report. Annex A on key terms, and Annex

H on results chains, provide additional and

complementary background.

Public policy as choices, actions, and

results

Articulating the chain of connections between

intentions and outcomes, beyond broad

generalities, is challenging. To start with, there are

few settled denitions of what constitutes public

policy. Peters (2018) captures the ambiguity well:

“Samuel Johnson once commented that patriotism

is the last refuge of fools and scoundrels. To some

degree, public policy has become just such a

refuge for some academic disciplines” (p. 4).

As a starting point, it is necessary to note that

public policy is not merely a list of government

intentions. “Policy is both the plan and the ensuing

course of action” (Biggs & Helms, 2007, p. 7).

Accordingly, Peters (2018, p. 4) identies three

levels of public policy:

• Government policy choices, comprising

“decisions made by politicians, civil servants

or others granted authority that are directed

towards using public power to affect the lives of

citizens.”

• Government policy outputs or “policy choices

being put into action.”

• Policy impacts or “the effects that policy choices

and policy outputs have on citizens.”

We adopt these three levels as our framework in the

report. We re-label them slightly as “government

policy choices,” “government actions,” and

“development results.” One important departure

from Peters (2018), however, is that we focus on

development results as a narrower category than

policy impacts—covering the subset of policy

impacts with which the international development

community is typically most concerned and which

are captured in global normative frameworks such

as the SDGs.

Even with an agreed disaggregation and taxonomy,

there remains substantive difculty in interpreting

the relationships between the three levels of

public policy. These links between government

policy choices (in the sense of what governments

say—or even don’t say—they want to do),

government actions, and the observed policy

impacts (incorporating development results) are

not automatic, linear, or unidirectional. Moreover,

the literature on policy reform in developing

countries largely avoids this question by focusing

instead on normative policy proposals or on the

political economy of adopting those proposals

(Jones, 2009).

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Figure 4 depicts the full framework for public

policy—government policy choices (A),

government actions (B), and development results

(C)—used in the Working Group report, along

with some illustrative content. We briey explain

each of the “public policy” levels in turn, agging

some pointers and caveats about the interactions

between the three dimensions.

Figure 4: Links and interactions between the dimensions of public policy

Government policy choices (A)

Government policy intentions to underpin improved prosperity, well-being, equity, and sustainability, such as:

• Regulation of economic behavior (enabling environment policies, industrial strategy, and so on)

• Managing responses to changing macro environment (fiscal policy, monetary policy, investment policy, structural reforms,

and so on)

• Priorities in public services (sector policies) and administrative arrangements (institutional design)

Policy as explicitly stated by government Policy as revealed by government actions

Government actions (B)

Economicinstruments,parallycoveredby: Regulatoryinstruments

(“measures

undertaken…to

inuencepeopleby

meansofformulated

rulesanddirecves”

(Vedung,2007,p.31))

Informaon

instruments

(persuasionwithout

“materialresourcesor

obligatorydirecves”

(Vedung,2007,p.31))

Administrave

measures(“budgeng,

managementand

accountability

approaches”(Manning

andLau,2015,p.42),

suchasintroduconof

anMTEFornewhiring

procedures)

Publicservices(or

“servicedelivery”)

(servicesfunded

throughgovernment

spending,either

commissioned or

directlyprovided)

Financialincenves

(taxexpenditures,

cashtransfers,loans

andloanguarantees,

governmentinsurance

andgrants,andso

on)

Development results (within wider policy impacts) (C)

Sectorandcross-cungoutcomes,andcizensasfacon,parally

measuredby:

Governanceandinstuonaloutcomes,parally

measuredby:

• Fiscalandinstuonalsustainability

• Voiceandaccountability

• Policalstabilityandabsenceofviolence

• Regulatoryquality

• Ruleoflaw

• Controlofcorrupon

Sector-specic

outcomes

(forexample,progress

onindicatorsofSDGs

onhealth,educaon,

sanitaon,hunger,

climate)

Cross-cungoutcomes

(compositemeasures

ofcizenwell-being,

equity,prosperity,

sustainability,andso

on)

Cizensasfacon

(compositemeasures

ofcizensasfacon

withpublicservices,

governmentintegrity,

andsoon)

Source:Authors’compilaon,drawingprincipallyonKaufmannetal.(2010),ManningandLau(2015),Peters(2018),Vedung

(2007),andWorldBank(2012b).

Government acons reveal

implicit policy choices (α)

Government acons lead to policy impacts,

and ideally to development results

Policy changes or policy iniaves

lead to government acons

Policy impacts are the

primary signal of eecve

government acons (β)

Government policy choices (A)

Government policy choices are a compound of

government’s stated intentions and objectives

on the one hand (Evans & Manning, 2003) and,

on the other, its policy as revealed de facto by

its actions (Glismann & Horn, 1988; Skrbinjek,

Šušteršic & Lesjak, 2018, pp.168-186; Stolper,

1972). The rst level in Figure 4 shows these

policy choices (A), which will underpin objectives

such as improved prosperity, well-being, equity,

and sustainability—and which may be expressed

through specic commitments or proposals

about priorities in public services and public

management, regulation of economic behavior,

and responses to a changing macro environment.

Policy choices can be seen both in what a

government says it is going to do and in what it

actually does. The feedback loop (α) in Figure 4

shows that interaction. For example, a government

that states it will improve maternal health care

for rural villages, and then does it, is showing its

stated intentions and its actions. A government

that allows teacher absenteeism to go unchecked

may be revealing that its policy includes deferring

to a powerful voting bloc (Keefer & Vishwanath,

2003). Administrative measures, such as reforms

in PFM or human resources management, may

also create a feedback loop from government

actions to government policy choices. They may

affect or reveal policy choices, even if those effects

are unintended. For example, the introduction of

an MTEF or a change in senior staff employment

contracts may serve, respectively, to introduce

inertia into policy change (Schick, 2013) or to

increase loyalty to political principals (Pollitt,

2015).

Added to the challenge of discerning a

government’s policy choices, the stated policy

objectives may be closer to a political façade than

a genuine government commitment to provide the

resources and authority needed for the stated policy

to be implemented in a meaningful way (Rakner,

Mukubvu, Ngwira, Smiddy & Schneider, 2004;

Sabatier & Weible, 2014). Many policy statements

are concerned with political positioning and

agenda setting rather than with forming a reliable

indication of a government’s intentions. Examples

range from the Armenian “economic program”

introduced to parliament by the Prime Minister

in February 2019, to the annual “US Economic

Report of the President,” to the (typically) annual

“Queen’s Speech” to Parliament in the United

Kingdom (and its equivalent, the “Speech from the

Throne,” in Canada), to the National Development

Plans that have been prepared and published by

134 countries. In those National Development

Plans the costing of policy choices and associated

nancing plan are often “the weakest element”

(Chimhowua, Hulme & Munroc, 2019, p. 83).

More specic and, arguably, reliable government

policy commitments are set out in budget

documentation—as proposed by the executive

and approved by the legislature. Shapiro (2001)

notes that the “budget is the most important

economic policy instrument for governments.

It reects a government’s social and economic

policy priorities more than any other document,

translating policies, political commitments, and

goals into decisions on where funds should be

spent and how funds should be collected” (pp.

6–7).

In a development cooperation setting, there is

the additional complication that there may not be

alignment between the preferences and priorities

of a government on the one hand and development

assistance providers on the other. “Country

ownership” is often assumed or invoked without

sufciently careful consideration or foundation

(Booth, 2011). Government incentives to provide

private goods may conict with donors’ concerns

for the provision of broad-based public services

in sectors such as health and education (Keefer &

Khemani, 2003; Khemani, 2017). These tensions

or mismatches in preference give further weight to

the argument that government policy choices are

often revealed by what gets done as much as by

what was promised.

Annex B: A framework for public policy

Government policy choices, government actions, and development results

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Government actions (B)

Government actions are “policy instruments…

[which are] concrete and specied operational

forms of intervention by public authorities”

(Bemelmans-Videc, 2007, p. xi). Such actions

may or may not be taken as a result of government

policy and it is those actions, not the intentions,

that may (or, again, may not) contribute to

development results. The second level in Figure

4 shows these government actions (B), which

consist of economic instruments (including public

services and nancial incentives), regulatory

instruments, and information instruments.

shown in the framework, administrative measures

could be considered an additional category of

government actions in response to policy changes

(Manning & Lau, 2015).

Using economic instruments—comprising

either public services or nancial incentives—to

implement government policy choices involves

“the handing out or the taking away of material

resources, be they in cash or in kind” which “make

it cheaper or more expensive in terms of money,

time, effort or other valuables to pursue certain

actions” (Vedung, 2007, p. 10).

Public services involve budgetary actions by

the government related to collective goods and

services (Hughes, 2018), especially in sectors—

such as education, law and order, environmental

protection, national defense, roads and bridges,

hospitals and health care, welfare services, and

public transport. Government may provide these

public goods directly through its own agencies

and staff, or it may commission them through

non-state actors,

or it may only fund them

through transfer of resources (Biggs & Helms,

2007). Table 6 presents a more detailed analysis

of the options available to government concerning

public services.

In the context of development cooperation in

lower-income countries, the language of “service

delivery” is used regularly and has become a

prominent focus for many development assistance

providers. Figure 5 shows the rising number

of references to service delivery in relation to

development assistance in publications between

1960 and 2005. In line with this emphasis and

trend, this report examines more specically the

relationship between PFM and service delivery,

as one critical element of government actions

through its economic instruments.

Figure 5: Ngram plot of references in literature published in English containing

the terms “service delivery” and “development assistance”

Year of publication

Relative frequency of “service delivery” and

“development assistance” in published literature

A complication arises when assessing government

actions concerning public services. To make

a credible claim that service delivery has

“improved,” there must have been a signicant

downstream contribution to development results,

especially in terms of related sector or cross-

cutting outcomes. The quality of an output, or

service, is ultimately a measure of “the attributable

incremental contribution of the service to the

outcome” (Atkinson et al., 2005, p. 42).

In other

words, policy impacts are the primary signal of

effective government actions. The feedback loop

(β) in Figure 4 shows this interaction.

As an alternative to public services, a government

can use nancial incentives to inuence individuals

and rms to adopt particular behaviors. These

incentives may consist of tax expenditures, cash

transfers, loans and loan guarantees, government

insurance and grants, and so on. This instrument,

in essence, involves a specic cash-based offering

(or levy) from government that leaves individuals

and rms with options about their behaviors—

albeit with potential penalties for making the

wrong choice.

Alongside economic instruments, there are other

actions that governments can take in relation

to their policy choices. Either they can mandate

actions to be taken or not taken by individuals

and rms through regulation, or they can provide

information with the aim of changing views on

why particular actions should be preferred. In

contrast to funded or taxable options implemented

through economic instruments, these actions

do not involve the ow of funds as the primary

mechanism.

Regulatory instruments are “measures

undertaken…to inuence people by means of

formulated rules and directives” (Vedung, 2007,

p. 31). These can, of course, have a public service

intention if they affect how market and non-prot

providers deliver services that specic groups are

able to receive (Biggs & Helms, 2007).

The information instrument “covers attempts

at inuencing people through the transfer of

knowledge, the communication of reasoned

argument, and persuasion,” in which “the

absence of obligation makes information different

from regulation which, by denition, contains

mandatory rules of conduct” (Vedung, 2007, pp.

11–13). It does not involve “material resources or

obligatory directives” (Vedung, 2007, p. 31).

Development results (C)

Development results are the subset of potential

policy impacts that are of greatest concern to

the international development community. They

represent those policy impacts that are considered

to be critical for sustainable economic and social

development. The third level in Figure 4 shows

these development results (C), which include

sector outcomes, cross-cutting outcomes, citizen

satisfaction, and governance and institutional

outcomes.

Development results at the sector level can

include progress toward desired outcomes in

health, education, sanitation, hunger, climate,

and so on. Cross-cutting outcomes may include

progress in areas such as citizen well-being,

equity, prosperity, and sustainability. These may

be viewed as composite objectives that draw on

outcomes across sectors. Related to these sector

and cross-cutting outcomes is the level of public

condence in and satisfaction with the quality

of public services, the degree of government

integrity, and so on. Development results can also

include, broadly, governance and institutional

outcomes as a distinct category. The scope of such

a category would include scal and institutional

sustainability, voice and accountability, political

stability and absence of violence, regulatory

quality, rule of law, and control of corruption

(Kaufmann et al., 2010).

We lay no claim to an authoritative typology of

development results here. In a domestic public

policy setting, conguring the framework for

capturing and assessing policy impacts rests with

government and civil society. At a global level, the

SDGs provide the most comprehensive and widely

accepted results framework. But the SDGs will

Annex B: A framework for public policy

Government policy choices, government actions, and development results

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

not necessarily align tightly with the way govern-

ments track the impacts of their policy choices and

actions. For example, governments often include

among their objectives strengthened security

forces, but consideration of policy impacts in the

defense and security sector is unlikely to feature

so strongly in the results frameworks used by

development assistance providers.

Peters (2018) offers a useful reminder that policy

impacts are “the effects that policy choices and

policy outputs have on citizens” (p. 4). There

can be a tension between what the development

community regards as policy priorities and what

an individual government considers it has to do

to manage domestic constituencies and real-

world pressures. The framework for public policy

presented in this annex does not suggest that the

latter should be made subject to the former, or

vice versa. It simply notes that, in considering the

development of public policy, both considerations

are prominent.

Table 6: Government actions for public services range from direct provision

to regulation

How transacons are recorded in the

system of naonal accounts (SNA)

Examples of agencies

responsible for a service

Government acons to ensure that services are

provided in line with government policy decisions

Administraveunitsingovernment Directproduconofservices

by ministries and departments

incentralgovernment

Directprovisionofservices,for

exampleteachinginstateschools

Direct

provision

Allnon-marketnon-protinstuons

thatarecontrolledandmorethan50

percentnancedbygovernmentunits

Schools,hospitals,andsimilar

instuonsthatarefunded

andcontrolledlargelyby

centralgovernmentbutnot

ownedbygovernment

Commissioningorfundingof

services,forexamplestatefunding

ofstudentplacesinreligiousschools

thatarealsorequiredtofollow

naonalcurriculaandstandards

Marketproducers,controlledby

government,sellinggoodsorservices

ataneconomicallysignicantprice

(“publicenterprises”),including

publicnancial(quasi-)corporaons

andpublicnon-nancial(quasi-)

corporaons,asdenedbyS.11and

S.12intheSNA

Publiclyownedbanks,publicly

ownedharbors,airports

Commissioningorfundingof

services,forexamplestatefundingof

sociallysignicantferryrouteswith

requirementsconcerningfrequency

Marketproducerswhoseindirect

publicfundingcomprisesmorethan

50percentoftotalrevenue,including

non-protandfor-protinstuons,

asdenedbyS.11,S.12intheSNA

For-protornon-protprivate

hospitalsaccessibletopublicly

insuredclients

Regulaonandcommissioningor

funding,forexampleregulaon

ofhospitalsconcerningprovision

ofcaretouninsuredpaentsand

reimbursementmechanisms

Non-protinstuonsserving

households,nancedmorethan

50percentbygovernment,butnot

controlledbygovernment,including

non-protinstuonsserving

households,asdenedbyS.15inthe

SNA

Schools,hospitals,andother

instuonsthatarefunded

largelybygovernmentbut

neitherownednorcontrolled

bygovernment

Regulaonandcommissioningor

funding,forexampleeducaon

vouchersandregulaonofschools

that can accept them

Privateenterpriseswithadisncve

andstatutorilyprivilegedmarket

posion,includingprivatesector

ulieslicensedtooperateinlimited

markets(suchaswater,energy,

sewage,wastedisposal,andpostal

services,butnottelecommunicaons)

andlegalmonopolies,asdenedby

S.11intheSNA

Energycompanies,localpublic

transportcompanies,naonal

train company

Regulaon,forexamplerequiring

minimumservicestandards

Marketproducersnotnancedor

controlledbygovernment

Private enterprises Regulaon,forexampleconcerning

transparency in sales of services

Non-protinstuonsserving

households,notnancedorcontrolled

bygovernment

Charies Regulaon,forexampleconcerning

equityintreatmentofapplicantsfor

assistance Regulaon

Note:SomeclassicaonsinthistableweredevelopedbyPilichowskiandTurkisch(2008).Thistableomitssocialsecurityfunds

ateachlevelofgovernment—forexample,ahealthfund,unemploymentfund,orpensionfund—whichareincludedintheSNA.

Annex B: A framework for public policy

Government policy choices, government actions, and development results

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

References

Atkinson, T., Grice, J., Simkins, A., Freitas, L. d., Hemingway, J., King, B., Lee, P., Lyon, M., Mai, N., Mehmi, S.,

Pritchard, A., Snelling, J., Tuke, A., Watson, L., & Fletcher-Cooke, G. (2005). Measurement of Government

Output and Productivity for the National Accounts. Basingstoke: Palgrave.

Bemelmans-Videc, M.-L. (2007). Policy Instrument Choice and Evaluation. In M.-L. Bemelmans-Videc, R. C. Rist,

& E. Vedung (Eds.), Carrots, Sticks and Sermons: Policy Instruments and Their Evaluation (pp. ix–xxiii).

New Brunswick: Transaction Publishers.

Biggs, S., & Helms, L. B. (2007). The Practice of American Public Policymaking. Abingdon, UK: Routledge.

Booth, D. (2011). Aid Effectiveness: Bringing Country Ownership (and Politics) Back In. London: ODI.

Chimhowua, A. O., Hulme, D., & Munroc, L. T. (2019). The ‘New’ National Development Planning and Global

Development Goals: Processes and Partnerships. World Development, 120(August 2019), 76–89.

Evans, G., & Manning, N. (2003). Helping Governments Keep Their Promises: Making Ministers and Governments

More Reliable through Improved Policy Management. Washington DC: World Bank.

Glismann, H. H., & Horn, E.-J. (1988). International Arms Trade: Revealed Political Preferences or Cartel

Behaviour? Kiel, Germany: Institut für Weltwirtschaft.

Hughes, O. (2018). Public Management and Administration: An Introduction (5th Edition). London: Palgrave.

Jones, H. (2009). Policy-Making as Discourse: A Review of Recent Knowledge-to-Policy Literature. London: ODI.

Kaufmann, D., Kraay, A., & Mastruzzi, M. (2010). The Worldwide Governance Indicators: Methodology and

Analytical Issues. Washington DC: World Bank.

Keefer, P., & Khemani, S. (2003). Democracy, Public Expenditures, and the Poor. Washington DC: World Bank.

Keefer, P. E., & Vishwanath, A. N. T. (2003). The Political Economy of Decentralization in Pakistan. Washington

DC: World Bank.

Khemani, S. (2017). Political Economy of Reform. Washington DC: World Bank.

Manning, N. & Lau, E. (2015). “Public Management Reforms across OECD Countries.” In T. Bovaird and E. Löfer

(eds.) Public Management and Governance, London, Routledge, pp. 39–54.

OECD. (1997). Measuring Public Employment in OECD Countries: Sources, Methods and Results. Paris: OECD.

Peters, B. G. (2018). American Public Policy: Promise and Performance. New York: Sage.

Pilichowski, E., & Turkisch, E. (2008). Employment in Government in the Perspective of the Production Costs

of Goods and Services in the Public Domain. Paris: OECD. Retrieved from http://www.oecd.org/

dataoecd/33/51/40088862.pdf

Pollitt, C. (2015). “Public Servants to Public Managers: The European Story: Transformation or Torture?” 11th

Transatlantic Dialogue (TAD) 3-5 June. Boston, M.

Rakner, L., Mukubvu, L., Ngwira, N., Smiddy, K., & Schneider, A. (2004). The Budget as Theatre – the Formal and

Informal Institutional Makings of the Budget Process in Malawi. Bergen: Chr. Michelsen Institute.

Sabatier, P. A., & Weible, C. M. (2014). Theories of the Policy Process. New York: Avalon Publishing.

Schick, A. (2013). Reections on Two Decades of Public Financial Management Reforms. In M. Cangiano, T. R.

Curristine, & M. Lazare (Eds.), Public Financial Management and Its Emerging Architecture (pp. 21–78).

Washington DC: IMF.

Shapiro, I. (Ed.) (2001). A Guide to Budget Work for NGOs. Washington DC: International Budget Project, Center

on Budget and Policy Priorities.

Skrbinjek, V., Šušteršic, J., & Lesjak, D. (2018). Political Preferences and Public Funding of Tertiary Education

During the Economic Crisis. Tertiary Education and Management, 24(2).

Stolper, W. F. (1972). Budget, Economic Policy, and Economic Performance in Underdeveloped Countries.

Michigan: Center for Research on Economic Development, University of Michigan.

Sturgess, G. L. (2018). Public Service Commissioning: Origins, Inuences, and Characteristics. Policy Design and

Practice, 1(3), 155–68. doi:10.1080/25741292.2018.1473100

Vedung, E. (2007). Policy Instruments: Typologies and Theories. In M.-L. Bedelmans-Videc, R. C. Rist, & E.

Vedung (Eds.), Carrots, Sticks and Sermons: Policy Instruments and Their Evaluation (pp. 1–38). New

Brunswick: Transaction Publishers.

World Bank. (2012b). The World Bank’s Approach to Public Sector Management 2011–2020: Better Results from

Public Sector Institutions. Washington DC: World Bank.



management reforms in context

Technical reforms in how money, people, and

organizations are managed within the public

sector—and how risk is shared among the various

actors who are responsible for producing public

services—have been common in advanced and

developing economies over the last three decades.

The selections at a particular time in a particular

place might vary, but the overall menu (set out

below) is likely to be familiar to many public

managers.

Introducing new institutional options for managing

budgetary practices and compensation and careers

are the most well-known. Both of these areas have

been associated closely with the development of

performance metrics, generally for informational

purposes but with growing use of spending

reviews that draw on performance evaluations to

create scal space for higher-priority spending or

for decit reduction.

In many advanced economies, changes to

budgetary procedures since 2008 have also been

driven by scal consolidation in the wake of

economic downturn. Many recent developments

in the establishment of scal rules (for example,

numerical limits on budgetary aggregates),

bolstered by the creation of independent scal

institutions that provide independent analysis of

scal policy and long-term scal sustainability

and by the monitoring of compliance with scal

rules, were driven by consolidation experiences

(IMF, 2010; Lassen, 2010; OECD, 2010).

Table 7: Recent public sector reforms

Public sector domain Common reform measures and innovaons

Instuonal changes for budgeng and public nancial management

Budget pracces and

procedures

•Input-orientedlineitembudget—incrementalorwithsomeperformance

informaon

•Budgetproceduresandmingbasedaroundperformancerepornginthe

1990s

•Consideraonofaccrualsbudgeng

• Use of scal rules (expenditure, budget balance, debt, and revenue) and

establishment of independent scal instuons

• Medium-term perspecve in budget preparaon, top-down budgeng, and

budget exibility (for example, the carryover of unused funds)

• Fiscal transparency

Accounng • Cash-based

•Double-entrybookkeeping

• Accrual accounng with extended cost calculaon supported by performance

measurement system

Audit •Tradionalnancialandcomplianceaudit

• Value-for-money focus with elements of performance audit and evaluaon

Instuonal changes for human resource management in the public sector

Workforce size and

composion

•Incenvesforworkforcereducon

• Strategic workforce planning

• Greater use of lateral entry, parcularly for senior sta

Compensaon and careers •Movestowardaposion-basedsystem(oreventowardtheuseofageneral

laborlaw)

• Decentralizaon of the employer and personnel management funcons

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Public sector domain Common reform measures and innovaons

• More explicit codes of conduct and codes of ethics

• Limited introducon of performance-related pay

• Moves toward dened-contribuon pension schemes

• Creaon of senior civil service or equivalent

Instuonal changes for regulaon, service delivery, and intergovernmental relaons

Regulatory management • Growth in independent regulators

• Creaon of central bodies for regulatory management

Intergovernmental

decentralizaon

• Somefunconalandscaldecentralizaontosubnaonalgovernments

Organizaonal diversicaon • Unbundling: creaon of “arm’s-length” agencies

• Delegaon of managerial authority within central ministries and departments

Instuonal changes for sharing risks

Outsourcing and public

private partnerships

• Increasedoutsourcing

• E-procurement

• Use of public private partnerships at the expense of tradional infrastructure

procurement

Instuonal changes for empowering cizens

Market-based mechanisms • Market-based approaches to delivering public services (including the

introducon of user charges and some use of vouchers)

• Market-based approaches to delivering internal government services

• Market-based approaches to seng regulatory standards or prices

Open government • “Passive” openness: provision of informaon about services and entlements

through charters; freedom of informaon legislaon

• “Acve” openness: open government data; open services; open policy making,

cizen engagement, and user parcipaon

• Extension of oces of ombudsman

E-government • Integrated service delivery, in parcular through one-stop shops, based on

aligned infrastructure and legal frameworks

• Shared back-oce services

• Applicaons, data storage, and compung power through cloud compung

rather than publicly owned systems

• Mobile government

Note:Itemsinitalicsarethosethathavebecomemoreprominentwithinthelastthreedecades.

Sources:ThisisdevelopedfromManning&Lau(2015)anddrawsonJoumard,Kongsrud,Nam&Price(2004);OECD(2005a,

2005b,2007,2009,2011,2013);andPolli&Bouckaert(2011).

References

IMF. (2010). Strategies for Fiscal Consolidation in the Post-Crisis World. Washington DC: IMF.

Joumard, I., Kongsrud, P. M., Nam, Y.-S., & Price, R. (2004). Enhancing the Cost Effectiveness of Public Spending:

Experience in OECD Countries. OECD Economic Studies, 2003/2(37), 109–61.

Lassen, D. D. (2010). Fiscal Consolidations in Advanced Industrialized Democracies: Economics, Politics, and

Governance. Copenhagen: University of Copenhagen.

Manning, N., & Lau, E. (2015). Public Management Reforms across OECD Countries. In T. Bovaird & E. Löfer

(Eds.), Public Management and Governance (pp. 39–54). London: Routledge.

OECD. (2005a). Modernising Government: The Way Forward. Paris: OECD.

OECD. (2005b). The OECD Human Resources Working Party: A Summary Retrospective and an Agenda for Action.

Paris: OECD.

OECD. (2007). Towards Better Measurement of Government (OECD Working Papers on Public Governance,

2007/1). Paris: OECD. Retrieved from http://www.oecd.org/dataoecd/11/61/38134037.pdf

OECD. (2009). Government at a Glance. Paris: OECD.

OECD. (2010). Fiscal Consolidation: Requirements, Timing, Instruments and Institutional Arrangements. Paris:

OECD.

OECD. (2011). Government at a Glance. Paris: OECD.

OECD. (2013). Government at a Glance. Paris: OECD.

Pollitt, C., & Bouckaert, G. (2011). Public Management Reform: A Comparative Analysis (Third ed.). Oxford, UK:

Oxford University Press.

Annex C: Public nancial management reforms in context

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Annex D: PFM focus within World

Bank sector activities

World Bank project data allows analysis of

activities by Global Practice (the World Bank’s

15 Global Practices map roughly to the “sector,”

or the part of the economy that the activity is

supporting)

and by theme (the specic purposes

of the support).

Theme taxonomies were changed

in 2017, so longer time series data are available

only before that date. Examining the proportion of

activities within a particular Global Practice that

contain signicant PFM thematic content

gives a

snapshot of the changing signicance of PFM as

a body of knowledge and area of concern within

that Global Practice.

The activities shown in the gures below

encompass all lending instruments (except those

that were closed before implementation) by year

of approval, including development policy loans,

investments, Program for Results, non-project

technical assistance, and other advisory services

and analytics.



Health, Nutrition, and Population Global Practice

Note:Theacviesincludedcontainatleast30%PFMthemacfocus.

Source:Authors’calculaonsfromWorldBankprojectdata.



Education Global Practice

Note:Theacviesincludedcontainatleast30%PFMthemacfocus.

Source:Authors’calculaonsfromWorldBankprojectdata.



Governance Global Practice

Note:Theacviesincludedcontainatleast30%PFMthemacfocus.

Source:Authors’calculaonsfromWorldBankprojectdata.

References

World Bank. (2016a). Sector Taxonomy and Denitions. Washington DC: World Bank.

World Bank. (2016b). Theme Taxonomy and Denitions. Washington DC: World Bank.

Annex D: PFM focus within World Bank sector activities

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Annex E: The quality of data on



Determining the size or impact of mainstream

donor inuence on PFM reform in client countries

remains a daunting challenge, constrained by

data weaknesses and denitional difculties

(AlphaBeta, 2018). Current measures of

development nancing for PFM stem largely

from OECD DAC data on ofcial development

assistance (ODA).

These data can be somewhat

problematic, but it is likely that accompanying

data on other ofcial ows (OOF) are even more

poorly captured (Hynes & Scott, 2013).

ODA captures ofcial aid from most major

providers, whether they are formal members of the

OECD’s DAC (such as the United States, United

Kingdom, France, and Germany) or have a reporting

relationship with DAC (including Russia, Turkey,

the United Arab Emirates, Kuwait, and Saudi

Arabia). Other donors do not currently report to

DAC (including China, Brazil, India, and Qatar).

ODA funds are channeled largely via multilateral

and regional development banks as well as

bilateral foreign affairs, nance, and development

ministries and agencies. Under the OECD’s

classication, Public Finance Management

(code 15111) is classied as part of the subgroup

Government and Civil Society (code 151). PFM

includes donors’ efforts to assist countries in scal

policy design and implementation, together with

more narrowly dened PFM reforms such as

revenue collection, expenditure management, and

public debt management. Currently available data

track annual nancial ows from DAC countries,

multilateral agencies, and regional development

banks from 2002 to 2016.

OOF data are recorded separately, recognizing

a proportion of loans at market or near-market

interest rates and bilateral export credits. OOFs

are largely loans for general development or scal

reforms, or other similar activities. Reported OOF

disbursements for PFM are widely thought to be

somewhat inated, with a signicant proportion

supporting scal policy reforms. It is estimated

that the proportion of OOF spent on PFM lies,

in reality, between 5 and 20 percent of what was

reported.

Other nancial ows are not captured in ODA or

OOF totals:

• The IMF has noted that, in recent years, China

has become the largest single trading partner

for Africa and a key investor and provider of

aid. Moreover, China has become the largest

nancier of Africa’s infrastructure, accounting

for around one-third of total nancing. Chinese

banks have provided around US$ 132 billion in

nancing to Africa and Latin American countries

since 2003 (Bräutigam & Gallagher, 2014;

Evans, 2018; Gallagher, 2016; IMF, 2013; Lin &

Wang, 2017).

• Besides China, new government funders of

development activities are emerging, with so-

called “non-traditional” development actors

(such as India and Turkey) growing in their

indirect inuence on the eld of PFM and on

PFM reform advice in lower-income countries.

Over the past 10 years, these (non-China)

emerging economies have become major

sources of international development nance,

infrastructure investment, and outward foreign

direct investment.

• Some funding is provided by large-scale private

philanthropy, and this can be construed as quasi-

ofcial support. The Bill & Melinda Gates

Foundation, for example, now reports to DAC

and is increasingly deemed to hold de facto

ofcial status.

Total non-ODA, non-OOF nancing that is

provided for PFM is likely to be small (AlphaBeta,

2018). The picture may change, however, with

the development of measures of “total ofcial

support for sustainable development” (TOSSD)

as a complement to ODA. This new measure

may provide a clearer indication of the resources

invested to achieve sustainable development. It

is expected to encompass all nancing provided

by ofcial bilateral and multilateral institutions,

regardless of the level of concessionality involved

or the instrument used—whether grants, loans,

or other equity—and to include contributions to

global public goods. In sum, while current ODA

totals measure the grant equivalent of resource

ows, registering donor “effort” in making

resources available, the new measure will consider

the full face value of the ow (gross amounts),

thereby providing a fuller picture of resources

available to developing countries (Benn & Ibanez,

2018). As yet, we do not have estimates of the

likely differences that might be picked up by

the use of the TOSSD methodology (Delalande,

Rühmann, Bejraoui & Benn, 2019).

References

AlphaBeta. (2018). The Landscape of PFM Reform in Developing Countries: Trends in Financing and

Implementation of PFM Reforms over 2002–16. Singapore: AlphaBeta.

Benn, J., & Ibanez, M. B. (2018). Total Ofcial Support for Sustainable Development (TOSSD): TOSSD Task Force

Issues Paper. Paris: OECD.

Bräutigam, D., & Gallagher, K. P. (2014). Bartering Globalization: China's Commodity-Backed Finance in Africa

and Latin America. Global Policy, 5(3), 346–52.

Delalande, G., Rühmann, F., Bejraoui, A., & Benn, J. (2019). Nigeria’s Perspective on TOSSD. Paris: OECD DAC.

Evans, M. (2018). Chinese Lending for African Infrastructure Trebles, Led by Policy Banks. Retrieved from https://

www.bakermckenzie.com/en/newsroom/2018/07/chinese-lending-for-african-infrastructure

Gallagher, K. P. (2016). The China Triangle: Latin America's China Boom and the Fate of the Washington

Consensus. Oxford: Oxford University Press.

Hudson Institute. (2016). The Index of Global Philanthropy and Remittances (2016). Washington DC: Hudson

Institute.

Hynes, W., & Scott, S. (2013). The Evolution of Ofcial Development Assistance: Achievements, Criticisms and a

Way Forward. Paris: OECD.

IMF. (2013). Regional Economic Outlook: Sub-Saharan Africa – Keeping the Pace. Washington DC: IMF.

Lin, J.Y., & Wang, Y. (2017). Development Beyond Aid: Utilizing Comparative Advantage in the Belt and Road

Initiative to Achieve Win-Win. Journal of Infrastructure, Policy and Development 1(2): 149–67.

Annex E: The quality of data on development nancing for PFM

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Annex F: Mapping of PFM

assessment frameworks

The 2005 Paris Declaration on Aid Effectiveness

committed donors to harmonizing diagnostic

reviews and assessment frameworks for PFM to

lessen the burden on governments. The Public

Expenditure and Financial Accountability (PEFA)

program led the eld, but assessments have

subsequently proliferated, with the World Bank’s

Debt Management Performance Assessment

(DeMPA), the IMF’s Tax Administration

Diagnostic Assessment Tool (TADAT), and the

Public Investment Management Assessment tool

(PIMA) among the most well-known. A recent

review (PEFA Secretariat, 2018b) identied

45 distinct frameworks for assessing PFM

performance (Table 8), some of which cover the

entire government PFM system, while others were

developed exclusively for assessing subnational

governments, for evaluating specic elements of

the PFM system, or for development partners to

assess duciary risk.

These tools make a distinction between measures

for “benchmarking against good practices,

professional standards or thematic principles”

and “database tool(s) for comparison across

governments” (PEFA Secretariat, 2018b, p. 10).

The former seem to be standards, particularly

when they entail a non-narrative scoring system.

There is some uncertainty about terminology.

It seems that these should be referred to as

assessment tools rather than diagnostic tools,

since they assess against a framework rather

than identifying some underlying pathology. Yet

they are called diagnostic tools in much of the

literature. This report uses the terms “assessment

frameworks” and “assessment tools.”

Table 8: Mapping of PFM assessment tools

Custodian Tool Technical Coverage Instuonal Coverage

A. Broad PFM diagnoscs: diagnosc or analycal tools covering the whole of the PFM system

PEFA PEFAFrameworkforAssessingPublic

FinancialManagement

All of PFM Naonal

Subnaonal

IMF FiscalTransparencyEvaluaon(FTE) Transparency in all of PFM Naonal

OECD RecommendaonoftheCouncilon

BudgetaryGovernance(RCBG)

All of PFM Naonal

OECD InternaonalBudgetPracces&

ProceduresDatabase(IBPPD)

All of PFM Naonal

OECD SIGMAPrinciplesofPublic

Administraon(PPA)

All of PFM

Publicadministraon

broadly

Naonal

Internaonal

Budget

Partnership

OpenBudgetSurvey(OBS) Transparency in all of PFM Naonal

GlobalIniave

for Fiscal

Transparency

Principles on Fiscal Transparency

(PFT)

Transparency in all of PFM Naonal

WorldBank PublicExpenditureReview(PER) Expenditurepolicy

Expenditure

administraon

Naonal

Subnaonal

Sector

EuropeanUnion ECFINOperaonalAssessment(OA) All of PFM Naonal

Custodian Tool Technical Coverage Instuonal Coverage

WorldBank RapidAssessmentsandAconPlans

toImproveDeliveryinSubnaonal

Government(RAAP-ID)

All of PFM

Developmentoutcomes

andoutputs

Subnaonal

WorldBank MiGesonInstuonalCapacity

Diagnosc

All of PFM

Publicadministraon

broadly

Subnaonal

Councilof

Europe

BenchmarkingFiscal

Decentralisaon(BFD)

Fiscaldecentralizaon Subnaonal

B. Diagnosc tools that focus on individual PFM elements, instuons, or sub-systems

IMF TaxAdministraonDiagnosc

AssessmentTool(TADAT)

Revenueadministraon Naonal

WorldBank IntegratedAssessmentModelfor

TaxAdministraon(IAMTAX)

Revenueadministraon Naonal

OECD TaxAdministraonComparave

InformaonSeriesonOECDand

otherAdvancedandEmerging

Economies(TACIS)

Revenueadministraon Naonal

USAID CollecngTaxesDatabase(CTD) Revenueadministraon Naonal

EuropeanUnion FiscalBlueprints(EU-FB) Revenueadministraon Naonal

WorldBank HandbookforTaxSimplicaon

(HTS)

Revenueadministraon Naonal

IMF RevenueAdministraonFiscal

InformaonToolkit(RA-FIT)

Revenueadministraon Naonal

IMF RevenueAdministraonGap

AnalysisProgram(RA-GAP)

Revenueadministraon Naonal

Sector

WorldBank CustomsAssessmentTradeToolkit

(CATT)

Revenueadministraon

Tradefacilitaon

Naonal

EITI ExtracveIndustriesTransparency

Iniave(EITI)

Naonalresource

Revenueadministraon

Expendituremanagement

Naonal

Subnaonal

WorldBank StateOwnedEnterpriseCorporate

GovernanceandRiskToolkit(SoE

Toolkit)

Publiccorporaons Naonal

Sector

Singleinstuon

WorldBank DebtManagementPerformance

Assessment(DeMPA)

Debtmanagement Naonal

IMF PublicInvestmentManagement

Assessment(PIMA)

Investmentmanagement Naonal

WorldBank DiagnoscFrameworkforAssessing

PublicInvestmentManagement

(DF-PIM)

Investmentmanagement Naonal

Subnaonal

OECD MethodologyforAssessing

ProcurementSystems(MAPS)

Procurement Naonal

WorldBank PublicExpenditureTrackingSurveys

(PETS)

Expendituremanagement Naonal

Subnaonal

Sector

Annex F: Mapping of PFM assessment frameworks

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

Custodian Tool Technical Coverage Instuonal Coverage

WorldBank DiagnoscFrameworktoAssessthe

CapacityofaGovernmentFinancial

ManagementInformaonSystemas

aBudgetManagementTool(DF-

FMIS)

Managementinformaon

systems

Naonal

Internaonal

Records

Management

Trust

IntegrangRecordsManagementin

FMISDesign(IRM-FMIS)

Recordsmanagement Naonal

Subnaonal

Singleinstuon

WorldBank GapAnalysisFrameworkfor

ComparingPublicSectorAccounng

andAudingtoInternaonal

Standards(AA-GAP)

Accounng

Auding

Naonal

WorldBank ReportontheObservanceof

StandardsandCodesAccounng

andAuding(AA-ROSC)covering

accounngandauding,corporate

governance,andinsolvencyand

creditorrights

Publiccorporaons Naonal

Instuteof

InternalAuditors

InternalAuditCapabilityModel

(IA-CM)

Internalaudit Naonal

Singleinstuon

Internaonal

Organizaonof

SupremeAudit

Instuons

SupremeAuditInstuon

PerformanceMeasurement

Framework(SAI-PMF)

Externalaudit Naonal

African

Organisaonof

English-Speaking

SupremeAudit

Instuons

InstuonalCapacityBuilding

Framework(ICBF)

Externalaudit Naonal

Chartered

Instute

ofPublic

Finance and

Accountancy

FinancialManagementModel

(FMM)

All of PFM Singleinstuon

C. Tools used by development partners to make decisions on duciary risk

ADB AssessingandManagingGovernance

RisksinADBOperaons(AMGR)

All of PFM Naonal

Subnaonal

Sector

ADB FinancialManagementAssessment

(FMA)

All of PFM Singleinstuon

IADB ToolforDeterminingtheLevel

ofDevelopmentandUseofPFM

Systems(GUS)

All of PFM Naonal

Subnaonal

WorldBank AssessmentoftheUseofCountry

FinancialManagementSystemsin

Bank-Financed Investment Projects

Cashandexpenditure

management

Naonal

Subnaonal

100

Custodian Tool Technical Coverage Instuonal Coverage

EuropeanUnion BudgetSupportGuidelines(EU-BSG) All of PFM Naonal

Subnaonal

Sector

DFID FiduciaryRiskAssessment(FRA) All of PFM Naonal

Subnaonal

Federal Ministry

for Economic

Cooperaonand

Development,

Germany(BMZ)

BudgetSupportintheFrameworkof

Programme-OrientedJointFinancing

(BMZ-BSG)

All of PFM Naonal

Danish

Development

Cooperaon

(DANIDA)

GuidelinesforRiskManagement

(GRM)

All of PFM Naonal

USAID PFMRiskAssessmentFramework

(PFMRAF)

All of PFM Naonal

Subnaonal

Sector

Source:AdaptedfromPEFASecretariat(2018).

References

PEFA Secretariat. (2018). Stocktake of PFM Diagnostic Tools 2016. Washington DC: PEFA.

Annex F: Mapping of PFM assessment frameworks

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

101

Annex G: Relating the institutional

and economic perspectives on

PFM—a thought experiment

Among the most difcult and elusive questions

explored by the Working Group was the one that may

sound as if it should be the most straightforward:

how does PFM relate to public nance? Part

of the challenge is that PFM has established its

own standard objectives or “desirable budgetary

outcomes” (Campos & Pradhan, 1996; Schick,

1998; World Bank, 1998), which exist alongside,

and somewhat independently of, the classic public

nance policy objectives (Musgrave & Musgrave,

1989). This observation prompted the Working

Group to ask how the two sets of objectives might

relate to one another—whether incidentally or

intentionally.

It is striking that very few papers in the canon of

PFM literature since the mid-1990s even reference

Musgrave and Musgrave (1989). Notable

exceptions are Fozzard’s analysis of approaches to

resource allocation in budget policy (2001) and de

Renzio’s review dening budget institutions and

their desirable features (2009). To the extent that

it was a conscious choice over the past 25 years

to keep PFM and public nance policy separate

(Premchand, 1996), has that dichotomy become

too entrenched? Is it time for a reappraisal?

Conventional thinking on PFM since the late 1990s

has concentrated on identifying and strengthening

the elements of a PFM system that will contribute

to budget credibility and, thereafter, to the goals of

aggregate scal discipline, allocative efciency,

and operational efciency.

The total amount of money a government

spends should be closely aligned to what

is affordable over the medium term and, in

turn, with the annual budget; spending should

be appropriately allocated to match policy

priorities; and the spending should produce

intended results at least cost. (World Bank,

1998, p. 3)

These PFM objectives are often described as

“desirable budgetary outcomes” (Campos &

Pradhan, 1996), although the labelling as outcomes

is misleading. They are arguably more like the

means to an end than the end itself. Drawing on

literature about the administrative capacities of

the state, they might better be considered as a

set of institutional capabilities through which

the PFM system can support wider public policy

objectives (Krause et al., 2016; Lodge & Wegrich,

2014). What is certain is that these capabilities

(or standard PFM objectives, as described in this

report) have been highly inuential in producing

and reinforcing a strong set of conventions for how

the underpinning PFM system should look. This

leads to a further question: do the recommended

PFM capabilities enable governments to make

informed choices across all policy objectives,

or do they tilt the policy playing eld? In other

words, is there a policy bias inherent in the three

PFM capabilities?

Although the classic formulation of public nance

policy objectives—allocation, distribution, and

stabilization (Musgrave & Musgrave, 1989)—

predates the three PFM capabilities—aggregate

scal discipline, allocative efciency, and

operational efciency—it does not appear to have

driven their development and specication. The t

between the two sets is ambiguous and imperfect

(de Renzio, 2009).

Table 9 below conducts a thought experiment

about some of these relationships. It asks in which

ways the three PFM capabilities may be relevant

to the three public nance objectives and suggests

the relative strengths of those connections. What

emerges is a moderate-to-strong orientation of

standard PFM capabilities toward stabilization

and allocation, but only a weak-to-moderate

orientation toward distribution. (The analysis

assumes that operational efciency does not

have a strong implicit focus on the quality and

accessibility of service delivery.)

This focus on objectives compatible with

stabilization is not surprising given the prime

102

placement of aggregate scal discipline in the

trinity of PFM capabilities and its conventional

primacy in the hierarchy. The emergence of this

formulation in the late 1990s, at a time when

macroeconomic stabilization was high on the scal

policy agenda and when debt forgiveness under

the Heavily Indebted Poor Countries initiative was

prominent, makes it more evident why aggregate

scal discipline was emphasized (Premchand,

1996). To a secondary and subordinate extent,

expenditure allocation and efciency (allocative

and operational efciency) were on the agenda

because of concern about the use of scal space

following debt relief and in the presence of

program/budget support. Distributional concerns

such as inequality were less in focus for scal

policy and appear to receive less emphasis among

the PFM capabilities.

What is revealed most clearly is that the

PFM capabilities as laid out by Campos and

Pradhan (1996) appear to lack an emphasis on

distributional issues, whereas the public nance

functions described by Musgrave and Musgrave

(1989) appear not to focus on the choice of

instrument (or mode of delivery/intervention

by government). There seems to have been no

intention by the original proponents of the three

PFM capabilities to relate them to the three

objectives of public nance. This exercise lends

weight to the argument that some reappraisal and

debate may be needed. To borrow from Premchand

(1996), the “hierophanies” of PFM may need to

be reconsidered.

Annex G: Relating the institutional and economic perspectives on PFM

A thought experiment

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

103

Table 9: Capabilities of an effective PFM system and their relationship to



Public nance objecves (Musgrave & Musgrave, 1989)

Allocaon: “Theprovision

forsocialgoods,orthe

processbywhichtotal

resourceuseisdivided

betweenprivateand

socialgoodsandby

whichthemixofsocial

goodsischosen.This

provision may be termed

the allocaon funcon of

budgetpolicy”(p.6).

Distribuon: “Adjustment

ofthedistribuonof

incomeandwealthto

ensureconformancewith

whatsocietyconsiders

a‘fair’or‘just’stateof

distribuon,herereferred

to as the distribuon

funcon”(p.6).

Stabilizaon: “Theuse

ofbudgetpolicyasa

meansofmaintaining

highemployment,a

reasonabledegreeof

pricelevelstability,and

an appropriate rate of

economicgrowth,with

allowancesforeectson

trade and on the balance

ofpayments.Werefer

toalltheseobjecvesas

the stabilizaon funcon”

(p.6).

Instuonal

capabilies of

the PFM system

(Schick, 1998;

Campos &

Pradhan, 1996;

World Bank,

1998; PEFA

Secretariat,

2016)

Aggregate Fiscal

Discipline: “Budgettotals

shouldbetheresult

ofexplicit,enforced

decisions;theyshould

not merely accommodate

spendingdemands.These

totalsshouldbeset

beforeindividualspending

decisionsaremade,and

shouldbesustainableover

themedium-termand

beyond”(Schick,1998,

p.2).

Fiscal discipline is relevant

forallocaonbecauseit

determinesandxesthe

overalllevelsofpublic

revenueandexpenditure,

thuseecvelysengthe

levelofsocialgoodsinthe

economy.

Moderaterelaonship

Fiscal discipline is relevant

fordistribuonbecauseit

ensuresthattheagreed

levelsofpublicrevenue

andexpenditurearenot

breached,thusmakingit

feasiblefordistribuonal

incidence implied by the

budgettobeimplemented

inpracce.

Weakrelaonship

Fiscal discipline is relevant

forstabilizaonbecause

itensuresaggregate

controloverrevenueand

expenditurelevels,thus

keepingdebtanddecit

levelswithinagreedpolicy

thresholdsandsupporng

broader macroeconomic

management.

Strong relaonship

Allocave Eciency:

“Expendituresshouldbe

basedongovernment

prioriesandon

eecvenessofpublic

programs.Thebudget

systemshouldspur

reallocaonfromlesserto

higherprioriesandfrom

lesstomoreeecve

programs”(Schick,1998,

p.2).

Allocaveeciencyis

relevantforallocaon

becauseitdetermines

themixofsocialgoods,

basedonpolical

prioriesandprogram

eecveness.

Strongrelaonship

Allocaveeciencyis

relevantfordistribuon

becausedecisions

aboutthecomposion

ofpublicexpenditure

across sectors and

publicprogramswill

determinedirectly,or

aectindirectly,the

distribuonalincidence

ofthebudget.

Weakrelaonship

Allocaveeciencyis

relevantforstabilizaon

becausedecisionsabout

thespreadofpublic

expenditureacross

sectorsandprograms

mayaectthebalances

ofconsumpon/

investment,wage

bill/goods,imports/

exports,andsoon,

withknock-oneects

for macroeconomic

variables.

Moderaterelaonship

Operaonal Eciency:

“Agenciesshould

producegoodsand

services at a cost that

achievesongoing

eciencygainsand(to

theextentappropriate)

iscompevewith

marketprices”(Schick,

1998,p.2)

Operaonaleciencyis

relevantforallocaon

becauseeciency

gains(orlosses)

andthussavings(or

overspending)within

or across years may

leadtochangesin

thecomposionof

spending,andthusthe

mixofsocialgoods.

Weakrelaonship

Operaonaleciencyis

relevantfordistribuon

becausethemixof

inputsinprovidingsocial

goodsmayaectpublic

sector employment and

thusincomes,andthe

degreeofexibilitymay

aectwagelevels.

Weakrelaonship

Operaonaleciencyis

relevantforstabilizaon

becauseunitcosts

andoutputeciency

may have an indirect

eectonprices,

andoutsourcingof

produconorprovision

to the private sector may

haveaneectonthe

economy.

Weakrelaonship

104

References

Campos, E., & Pradhan, S. (1996). Budgetary Institutions and Expenditure Outcomes (Policy Research Working

Paper No. 1646). Washington DC: World Bank.

de Renzio, P. (2009). Approach & Methodology for the Evaluation of Donor Support to Public Finance Management

(PFM) Reform in Developing Countries (Part A - Assessing Budget Institutions and Budget Reforms in

Developing Countries: Overview of Theoretical Approaches and Empirical Evidence). Oxford, UK: Fiscus

Public Finance Consultants, Ltd.

Fozzard, A. (2001). The Basic Budgeting Problem: Approaches to Resource Allocation in the Public Sector and

Their Implications for Pro-Poor Budgeting (Working Paper 147). London: ODI.

Krause, P., Hadley, S., Mustapha, S., & Welham, B. (2016). The Capabilities of Finance Ministries. London: ODI.

Lodge, M., & Wegrich, K. (2014). The Problem-Solving Capacity of the Modern State – Governance Challenges

and Administrative Capacities. Oxford: Oxford University Press.

Musgrave, R. A., & Musgrave, P. B. (1989). Public Finance in Theory and Practice (Fifth Edition). San Francisco:

McGraw Hill.

PEFA Secretariat. (2016). PEFA: Framework for Assessing Public Financial Management. Washington DC: PEFA

Secretariat.

Retrieved from https://pefa.org/sites/default/les/PEFA%20Framework_English.pdf

Premchand, A. (1996). Issues and New Directions in Public Expenditure Management (Working Paper 96/123).

Washington DC: IMF.

Schick, A. (1998). A Contemporary Approach to Public Expenditure Management (35116). Washington DC: World

Bank Institute.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

Annex G: Relating the institutional and economic perspectives on PFM

A thought experiment

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

105

Annex H: A partial summary of

PFM research

In looking for cause and effect in moving between

“PFM interventions” and “results that matter,” it

is useful to review the causal evidence that: (i)

interventions as the unit of analysis have led to

improved conformity with basic conventions in

PFM; and (ii) they have led, along whatever route,

to improvements in nal results that matter. We

can also consider: (iii) whether the norms and

standards as the unit of analysis are themselves

associated with better results.

Given the high “causal density” (Woolcock, 2013,

p. 8) of the territory, with many factors in addition

to planned PFM development interventions

possibly leading to change, we can also look at

observed improvements and ask what caused

them. The questions here are: (iv) whether there is

a credible case that improvements at the national

level as the unit of analysis are attributable to

development interventions; (v) whether any

improved results are attributable to improvements

in conformity with the PFM conventions; and (vi)

whether improved conformity is itself attributable

to donor interventions (Figure 9).

Figure 9: Causal chains and attributions to be considered

Table 10 offers a partial summary of the literature reviewed for this report, with some representative research

summaries shown.

Table 10: A partial summary of key research

Element of the results chain Summary of evidence

(i)Didintervenonsleadto

improvedconformitywiththe

convenons?

Limited evidence

Thereissomebroadevidencethatbudgetsupportcanimprovethe

conformityofnancialmanagementandbudgetprocesseswithstandard

convenons(Orthetal.,2017,forexample).Amuchearlierstudyfound

thatimprovementsinPFM(thenmeasuredusingtheCountryPolicyand

InstuonalAssessment,orCPIA)hadtakenplaceinthemajorityof

countriesthatobtainedprojectsinthoseareas(IEG,2008).

Mostrecently,however,Fritzetal.(2017)foundthat,despiteeortsmadeto

strengthenPFMsystemsindevelopingcountriesovermanyyears,foralllow-

andmiddle-incomecountriescollecvely,boththePEFAindicatorsandthe

CPIA-13indicatorshowedonlysmallaggregateimprovementoverme.

TheobservaonmadebydeRenzioetal.(2010)remainsaconfounding

factor:thereisapossibilitythatdonorsbackwinnersormaybelosers,

suggesngthatstudyresultscouldincorporateunknowndegreesofbiasin

eitherdirecon.

106

Element of the results chain Summary of evidence

(ii)Didintervenonslead,along

whateverroute,toimprovements

innalscalresultsthatmaer?

Some evidence—parcularly concerning parcipaon and consultaon

DeLayetal.(2015)mappedintervenonstoresults,ndingthat

intervenonscouldimprovethepolicyrelevanceofresourceallocaon,

transparency,andscruny.Theyfoundthatreformsthatimproved

parcipaonandconsultaonhadthegreatestimpact.

(iii)Didconformitywiththe

convenonsleadtobeer

results?

Some evidence—parcularly concerning openness and transparency

Studiesthatfocusontheintroduconofspecicinstuonalarrangements

andpraccesshowthatthesehavesome,althoughratherlimited,impact

onresults(see,forexample,Goryakinetal.,2017;Mills,2018).Welhamet

al.(2017)ndthatbeerPFMisassociatedwithlowervariancesinoverall

budgetexecuonratesandthatabeerPFMsystemyieldslowerunder-ve

andinfantmortality,andhigherlifeexpectancyatbirth.

InreviewingcasestudiesoffragilestatesundergoingPFMreform,Hedger,

Krause,andTavakoli(2012)foundthatitwasnotclearifandwhenprogress

onPFMreformwouldcontributetothestate’sparcipaoninservice

delivery(WorldBank,2012).

Theimposionofscalrules(numericallimitsonbudgetaryaggregates),

drivenbyrecentexperiencesofscalconsolidaon(IMF,2010;Lassen,

2010;OECD,2010),hasatbestamarginalimpactonactualbehaviors.The

WorldBank(2012)wasmoreposivewithregardtotheimpactofMTEFson

scaldiscipline,butnotedthattheimpactofMTEFadoponontechnical

eciencyislesspronounced.

InanalysisbyMustaphaandLong(2019),countriesthatcrediblyenforced

penalesfornon-compliancecollectedmoretaxes.Knack,Biletska,and

Kacker(2017)foundthatstrongerprocurementproceduresledtofewer

kickbacks.deRenzioandWehner(2017)foundthatscalopennesshada

benecialimpact.

However,MustaphaandLong(2019)andFritz,Sweet,andVerhoeven(2014)

foundnostascallysignicantrelaonshipbetweenaPEFA-basedmeasure

ofoverallPFMqualityandthescalbalanceinbothnon-fragileandfragile

states,consistentwithAlesinaetal.(1999)andvonHagenandHarden

(1995).

(iv)Isthereacrediblecasethat

improvedresultsatthenaonal

orsectorlevelareaributableto

developmentintervenons?

No evidence either way

Thereisevidenceofgeneraldevelopmentprogress,butthisisnotevidently

aributabletoPFM(see,forexample,Radelet,2015).Thesameappliesto

majorsectorimprovements,asinthehealthsector(Morrison,2013).

(v)Areanyimprovements

inresultsaributableto

improvementsinconformitywith

convenons?

Lile evidence that improvements are aributable to PFM

Arecentreviewofhealthservicedeliveryimprovementsandthedegree

towhichtheseweredrivenbyPFMreforms,seeminglyassessedlargelyin

termsofconformitywithacceptedPFMconvenons,foundthatthereis

lileevidenceofacausalrelaonshipandtheresultsareoenconicng

(Welhametal.,2017).

Annex H: A partial summary of PFM research

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

107

Element of the results chain Summary of evidence

(vi)Isimprovedconformitywith

thePFMconvenonsaributable

todonorintervenons?

No evidence either way

Thisisdiculttoanswer,asthereislimitedevidenceofimprovedconformity

withPFMconvenons(see,forexample,Vani,2012).deRenzioetal.(2010)

foundamixedpictureinSub-SaharanAfrica,althoughtheGermanSociety

forInternaonalCooperaon(GIZ,2015)foundaslightlymoreposivetrend.

Ronsholt(2012)foundasomewhatposivepictureinexaminingchangesin

PEFAscoresovermein32countriesthathadbeenthesubjectofrepeat

PEFAassessments.

Lawson(2012)lookedatcountrieswherebudgetinstuonshadimproved,

boththosewith“highdonoreort”(BurkinaFaso,Tanzania,andZambia)

andthosewith“lowdonoreort”(Ethiopia,Ghana,andMali).Theywere

abletosaylileabouthowthedonorintervenonsmighthavecausedthe

instuonalimprovements,otherthanthroughrecognizedgeneralieslike

strongpolicalcommitment,reformdesignsandimplementaonmodels

thatweresuitedtothecontext,andstronggovernment-ledcoordinaon

arrangements.

Fritzetal.(2017)analyzedthecausesofcross-countryvariaonsinthe

qualityofPFMsystems,ndingthesetobeassociatedmostsignicantly

androbustlywithcountryincomepercapita(posively)andtheshareof

revenuesobtainedfromnaturalresources(negavely).Thesendingsare

consistentwithdeRenzioetal.(2010)concerningtheweakcorrelaon

betweenPFMsystemsandODAsupportforPFMreform.Totheextent

thatthereisanyimprovementinPEFAscores,deRenzioetal.(2010)

showedthatmostcountryvariaoninPFMqualityisduetoeconomicand

instuonalfactors.

Note:Theunitsofanalysisareitalicized.

References

Alesina, A., Hausmann, R., Hommes, R., & Stein, E. (1999). Budget Institutions and Fiscal Performance in Latin

America. Working Paper Series 394. Washington DC: IADB. Retrieved from http://www.iadb.org/OCE/

pdf/394.pdf

De Lay, S., Mills, L., Jadeja, K., & Lucas, B. (2015). Public Financial Management Evidence Mapping.

Birmingham, UK: GSDRC, University of Birmingham.

de Renzio, P., Andrews, M., & Mills, Z. (2010). Evaluation of Donor Support to Public Financial Management

(PFM) Reform in Developing Countries Analytical Study of Quantitative Cross-Country Evidence (Final

Report). Stockholm: Swedish International Development Cooperation Agency.

de Renzio, P., & Wehner, J. (2017). The Impacts of Fiscal Openness. The World Bank Research Observer(32),

185–210.

Fritz, V., Sweet, S., & Verhoeven, M. (2014). Strengthening Public Financial Management: Exploring Drivers and

Effects. Washington DC: World Bank.

Fritz, V., Verhoeven, M., & Avenia, A. (2017). Political Economy of Public Financial Management Reforms:

Experiences and Implications for Dialogue and Operational Engagement (121436). Washington DC:

World Bank Group. Retrieved from http://hdl.handle.net/10986/28887

GIZ. (2015). Good Financial Governance in Africa: Developments 2007–2014. Pretoria, South Africa: Deutsche

Gesellschaft für Internationale Zusammenarbei.

Goryakin, Y., Revill, P., Mirelman, A., Sweeney, R., Ochalek, J., & Suhrcke, M. (2017). Public Financial

Management and Health Service Delivery: A Literature Review. London: ODI.

Hedger, E., Krause, P., & Tavakoli, H.. (2012). Public Financial Management Reform in Fragile States. London:

ODI.

IMF. (2010). Strategies for Fiscal Consolidation in the Post-Crisis World. Washington DC: IMF.

108

IEG. (2008). Public Sector Reform: What Works and Why? Washington DC: World Bank.

Knack, S., Biletska, N., & Kacker, K. (2017). Deterring Kickbacks and Encouraging Entry in Public Procurement

Markets. Washington DC: World Bank.

Lassen, D. D. (2010). Fiscal Consolidations in Advanced Industrialized Democracies: Economics, Politics, and

Governance. Copenhagen: University of Copenhagen.

Lawson, A. (2012). Evaluation of Public Financial Management Reform in Burkina Faso, Ghana and Malawi

2001–2010: Final Synthesis Report. Oxford, UK: Fiscus Public Finance Consultants, Ltd.

Mills, L. (2018). Donor Support to Strengthen Public Financial Management in Partner Countries: Outcomes,

Experiences, and Ways Forward. Brighton, UK: Institute of Development Studies.

Morrison, J. S. (2013). Introduction. In J. S. Morrison (Ed.), Global Health Policy in the Second Obama Term.

Washington DC: Center for Strategic and International Studies.

Mustapha, S., & Long, C. (2019). PEFA, Public Financial Management, and Good Governance. Washington DC:

World Bank.

OECD. (2010). Fiscal Consolidation: Requirements, Timing, Instruments and Institutional Arrangements. Paris:

OECD.

Orth, M., Schmitt, J., Krisch, F., & Oltsch, S. (2017). What We Know About the Effectiveness of Budget Support:

Evaluation Synthesis. Bonn, Germany: German Institute for Development Evaluation.

Radelet, S. (2015). The Great Surge: The Ascent of the Developing World. New York: Simon and Schuster.

Ronsholt, F. E. (2012). Are Public Financial Management Systems Improving in Low and Middle Income

Countries? A Preliminary Analysis Based on Data from PEFA Assessments in 32 Countries. Washington

DC: PEFA.

Vani, S. (2012). Has Global PFM Improved in the Last Decade? Retrieved from https://blog-pfm.imf.org/

pfmblog/2012/09/has-global-pfm-improved-in-the-last-decade.html

von Hagen, J., & Harden, I. (1995). Budget Processes and Commitment to Fiscal Discipline. European Economic

Review, 39(3-4), 771–79.

Welham, B., Hart, T., Mustapha, S., & Hadley, S. (2017). Public Financial Management and Health Service

Delivery: Necessary, but Not Sufcient? London: ODI.

Woolcock, M. (2013). Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions.

Cambridge, MA: Harvard Kennedy School.

World Bank. (2012). Beyond the Annual Budget: Global Experience with Medium-Term Expenditure Frameworks.

Washington DC: World Bank.

Annex H: A partial summary of PFM research

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

109

Annex I: The elusive hope of

incorporating context

Problems in providing effective technical

assistance (or technical cooperation or capacity

building) are perplexingly persistent. Observations

made in 2001 are not vastly different from those

that might be made today:

In contrast to the historical process of

countries purchasing skills and know-how

from suppliers, the relationship now appears

very different...

• Control: Demand was manufactured, not

self-generated. The capacity gaps came

to be perceived and funded by donor

country governments, not the recipients,

and control of the transfer arrangements

shifted accordingly. The recipient countries

owned neither the gaps nor the resources

put up to ll them. For as long as there has

been aid, there have been concerns about

country ownership.

• Public sector bias: The reversal of control

was facilitated by the fact that aid was in

the public funding domain of the donors.

This had various consequences:

o Technical cooperation became the

preserve of bureaucratic minds and

processes in the donor countries, with

an often limited comprehension of the

character and context of the recipient

countries;

o Each donor bureaucracy devised its

own set of procedures and practices for

administering [technical cooperation],

resulting in a proliferation of

administrative paperwork;

o “Projectization” became a feature.

Technical cooperation was conceived

in time- and money-bound segments, a

conguration which best suited public

spending patterns.

• Accountability: Aid had key stakeholders

in the donor countries, but beneciaries

elsewhere, setting up a potential tension

between two sets of objectives.

o The stakeholders had expectations

of how aid should “perform”, and

the criteria of success were often

commercial, political or in other ways

strategically signicant, rather than

developmental. (Morgan, 2001, as

cited in Browne, 2002, pp. 19–20)

These observations themselves showed that

the hope that the United Nations Development

Programme’s (UNDP) 1993 report on rethinking

technical cooperation (Berg, 1993) would

represent a “breakthrough in critical thinking” had

not been fullled (Table 11).

110

Table 11: Adaptive Management, Doing Development Differently, and Thinking

and Working Politically vs. Berg’s ideas from the early 1990s

Theme Andrews

(2012)

Teskey & Tyrrel

(2017)

Green

(2018)

Berg

(1993)

Bigger view Mustbeableto

developmulple

theoriesofchange

or theories of

acon(inparcular,

policalacon)for

every problem

Systemsthinking:

constantinteracon

and feedback

mechanisms

betweeneconomic,

social,andpolical

factors that can

variouslyenableand

inhibitchange

Focusonthewider

publicsector

environmentwithin

whichnewsystems

aremeanttowork

Learn as you go Problem-driven

learning

Purposivemuddling

Embedded

monitoringand

learning

Uncertaintyand

learningbydoing/

failing;natureofthe

problemnotknown

in advance

Local soluons Welcomelocal

hybridinstuonsif

theyfuncon

Mobilizelocalagents

intocommuniesof

change

Priorizetacit

knowledge,focuson

recruingstawith

policalknowledge

and“insider”

networksspecicto

the reform problem

at hand

Highnumbers

ofnaonalsta

inprogram

management

posions

Giveprioritytolocal

knowledge:“always

givetheworkback

togovernment—it’s

acoachingrole,not

usdoingthework”

(asquotedinGreen,

2018)

Priorizecultural

sensivity

over technical

competence

Greaterfocuson

training/capacity

buildingand

coachingandon

usinglocalsta

Flexibility in

allocaon of

resources and in

denion of objec

ves

Ability to make

acvityandbudget

modicaons;

adapngthe

programinrealme

Delegaonof

discreonover

budget,acvity

decisions,and

localnetworks/

relaonshipsto

programmanagers

Oppositeofdonors’

preference for

control,logframes,

riskminimizaon,

andpredictable,

tangibleresults

Delegaontolocal

sta

Move beyond

projectstoprograms

Emphasize

measurable outputs

Some things just

have to be done

Don’t be ashamed of

lling gaps if that is

what is needed

Annex I: The elusive hope of incorporating context

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

111

Theme Andrews

(2012)

Teskey & Tyrrel

(2017)

Green

(2018)

Berg

(1993)

Change the donor

architecture

Expose/createa

market for technical

assistance;charge

fullprice(even

ifgovernmentis

reimbursedby

donors in some

way)andunbundle

projects/programs

sothatgovernments

can pick and choose

whattheywant

Note: Italicizedphrasesarethosethataredierentfromornotfoundintheotherideas.

References

Andrews, M. (2012). The Limits of Institutional Reform in Development. Cambridge: Cambridge University Press.

Baser, H., & Morgan., P. (2001). The Pooling of Technical Assistance. The Hague: Netherlands Ministry of Foreign

Affairs.

Berg, E. (1993). Rethinking Technical Cooperation: Reforms for Capacity Building in Africa. New York: UNDP

Regional Bureau for Africa and Development Alternatives Inc.

Browne, S. (Ed.) (2002). Developing Capacity through Technical Cooperation: Country Experiences. London and

Sterling, Virginia: UNDP and Earthscan Publications Ltd.

Green, D. (2018). Can ‘Doing Development Differently’ Only Succeed If Aid Donors Stay Away from It? .

Retrieved from http://oxfamblogs.org/fp2p/can-doing-development-differently-only-succeed-if-aid-donors-

stay-away-from-it/

Morgan, P. (2001). “Technical Assistance.” Background paper for the UNDP.

Teskey, G., & Tyrrel, L. (2017). Thinking and Working Politically in Large, Multi-Sector Facilities: Lessons to Date.

Canberra ACT, Australia: Abt Associates.

112

Annex J: Evolution in theories of

externally supported change in

the public sector

Historically, the eld of PFM advice has been

prone to advancing a series of relatively distinct

models for achieving change—each of which has

appeared to be clear and self-standing until the

next one arrived (Pretorius & Pretorius, 2009).

Table 12: Stylized theories of growth and of externally supported change in

public management

Decade Broad theories of economic

growth

Theories about public sector reform

Theories of externally supported

change

Underlying ideas

1960s Developmentpromoon

moreaboutcapitalows,less

aboutgovernmentpoliciesand

instuons.Emphasisoncapital

accumulaon,technological

adopon,andimportsubstuon

understateguidance.

Theoriesofpublicsectorinstuonal

reformwereimplicit,comprising

embeddedandunstatedassumpons

aboutllingcapacitygaps(knowledge,

ability,technicalexperse),parcularly

innewlyindependentcountries.

Gapllingincapital

and in capacity

1970s Capitalowsnotenough;turnto

macropolicies.

1980s Macronotenough;turntoa

morecomprehensivepackage,

includingscaldiscipline,

broadeningofthetaxbase,and

outwardorientaonthrough

freeingandenablingmarkets.

EmergenceoftheWashington

Consensus(Williamson,1989).

Theoriesofpublicsectormanagement

reformbegintoemergemoreexplicitly,

withagreaterfocusonincenvesinside

thepublicsectorthatcanbechanged

throughthetransferofbestpracces

(Levy,2010).Broaderincenvesofstate

actors,includingpolicians,excluded

fromdebate.

Reformcontentbegins

todominate(asin,this

reformisuniversally

therightthingtodo).

1990s Donorsseehighgrowthin

mediocrepolicyenvironments,

slowgrowthingoodpolicy

environments.Policiesnot

enough;turnto“instuons”

within“newgrowththeory”

(WorldBank,2005),thoughthe

noonofinstuonsisbroadly

andambiguouslyarculated.

Highnoonof“scalmachinery”

perspecve(Premchand,2000),with

itsemphasisonlinkingexpenditures,

atleastconceptually,withoutcomes

(WorldBank,1998).“Basicsrst”makes

anappearance(Schick,1998).

2000s Begintoadd“policalincenves

topursuedevelopment”into

themix.Demiseoftheoriginal

WashingtonConsensus;some

arguethatinadequateaenon

waspaidtooriginalinstuonal

concernsoftheWashington

Consensus(Singhetal.,2005),

othersthatthelistofinstuonal

Theoriesbecomemoreexplicitabout

matchingreformcontenttothecountry

instuonalcontext.Goodtdevelops

asaconcern(WorldBank,2000).

Contextualconsideraonsbroadened

toincludepolicalactors’incenvesto

implementreform.Increasingconcern

withshapingintervenonstomeet

policalrealies;nibblingawayat

Broadcountrycontexts

increasinglyseenas

primaryissue;reform

contenttobejudgedin

termsofitssuitability

forthecontext.

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

113

Decade Broad theories of economic

growth

Theories about public sector reform

Theories of externally supported

change

Underlying ideas

concerns needed to be

augmented(Rodrik,2006).

policalconstraintsatthemargin(as

intransparencyiniaves(UNDESA,

2001)).Theconceptofcontextalso

deepenedtoextendtosequencing

theories,suchasthePFMplaorm

approach(Brooke,2003)andthe

“StrengthenedApproach”(JointWorld

Bank/IMF/PEFAPublicExpenditure

WorkingGroup,2006).

2005

and

beyond

Connuingslowprogresssuggests

needforexperimentaon/

diversity.Recognionthatthere

aremulplepaths.“Another

mistake…inthe1990shasbeen

thetranslaonofgeneralpolicy

principlesintoauniquesetof

acons”(WorldBank,2005,

p.11).Overall,argumentfor

movingawayfromformulaic

policymakingtofocusonbinding

constraint(s).

Theoriesstarttoembraceactors’

subjecveunderstandingofthereform

processmoredirectly,sincetheproblem

thatistobesolvedinpublicsector

managementreformsisnotprimarily

“technical”but“adapve”(Heifetz,

1994).Noonsofchangespaceemerge

(Andrews,2013).

The process of

understandingthe

problemhasnow

becometherst-order

task.Contextremains

crical,andreform

contentiswhatever

can be contained

withinthespacethatis

foundtobeavailable.

2010

and

beyond

Socialmobilizaonisassociated

withchangedpolicalincenves

thatconstraingrowth,although

thereremainsuncertaintyabout

whichwaythecausalityruns

(Fukuyama,2010;Rodrik,2011).

Infrastructurebeginstotakea

largerplaceinpolicydiscussions.

Theoriesstarttosuggestthat

intervenonrequiresaner-grained

viewofthecontext(Kelsall,Booth,

Cammack&Golooba-Mutebi,2011).

Goodgovernancecansomemesbe

demanded,andchangingpolical

incenvestosupplygoodgovernance

maymeanimprovingcizencapacityto

demandgoodgovernance.

The process of

understandingthe

problem remains

preeminent,but

contextconsideraons

have become more

complex.Reform

contentistheresidual.

Source:ThistabledrawsonWorldBank(2005,2012).

References

Andrews, M. (2013). The Limits of Institutional Reform in Development. New York: Cambridge University Press.

Birdsall, N., de la Torre, A., & Valencia Caicedo, F. (2010). The Washington Consensus: Assessing a Damaged

Brand. Washington DC: Center for Global Development.

Brooke, P. (2003). Study of Measures Used to Address Weaknesses in Public Financial Management Systems in the

Context of Policy Based Support. Washington DC: PEFA.

Fukuyama, F. (2010). The Origins of Political Order. New York: Farrar, Straus & Giroux.

Heifetz, R. A. (1994). Leadership without Easy Answers. Cambridge, MA: Harvard University Press.

Joint World Bank/IMF/PEFA Public Expenditure Working Group. (2006). Supporting Better Country Public

Financial Management Systems: Towards a Strengthened Approach to Supporting PFM Reform

Harmonising Donor Practices for Effective Aid Delivery (Volume 2) (pp. 75–81). Paris: OECD.

Kelsall, T., Booth, D., Cammack, D., & Golooba-Mutebi, F. (2011). Developmental Patrimonialism? Questioning

the Orthodoxy on Political Governance and Economic Progress in Africa. London: ODI.

Levy, B. (2010). The Case for Principled Agnosticism. Journal of Democracy, 21(4), 27–34.

Premchand, A. (2000). Control of Public Money: The Fiscal Machinery in Developing Countries. Oxford: Oxford

University Press.

114

Pretorius, C., & Pretorius, N. (2009). Review of Public Financial Management Reform Literature. London: DFID.

Rodrik, D. (2006). Goodbye Washington Consensus, Hello Washington Confusion? A Review of the World Bank’s

Economic Growth in the 1990s: Learning from a Decade of Reform. Journal of Economic Literature,

XLIV(December 2006), 973–87.

Rodrik, D. (2011). The Poverty of Dictatorship. Project Syndicate. Retrieved from http://www.project-syndicate.

org/commentary/rodrik53/English

Schick, A. (1998). Why Most Developing Countries Should Not Try New Zealand's Reforms. World Bank Research

Observer (International), 13, 23-31.

Singh, A., Belaisch, A., Collyns, C., De Masi, P., Krieger, R., Meredith, G., & Rennhack, R. (2005). Stabilization

and Reform in Latin America: A Macroeconomic Perspective on the Experience since the Early 1990s.

Washington DC: IMF.

UNDESA. (2001). Globalization and New Challenges of Public Finance: Financial Management, Transparency and

Accountability Paper presented at the Meeting of the Group of Experts (28–30 November 2001), Rome.

Williamson, J. (1989). What Washington Means by Policy Reform. In J. Williamson (Ed.), Latin American

Readjustment: How Much Has Happened. Washington DC: Institute for International Economics.

World Bank. (1998). Public Expenditure Management Handbook. Washington DC: World Bank.

World Bank. (2000). Reforming Public Institutions and Strengthening Governance: A World Bank Strategy.

Washington DC: World Bank.

World Bank. (2005). Economic Growth in the 1990s: Learning from a Decade of Reform. Washington DC: World

Bank.

World Bank. (2012). The World Bank’s Approach to Public Sector Management 2011–2020: Better Results from

Public Sector Institutions. Washington DC: World Bank.

Annex J: Evolution in theories of externally supported change in the public sector

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

115

Annex K: Putting external

contestation of policy ideas into

practice

Institutionalized and disinterested contestation of

external policy ideas and advice will require effort,

as well as a change among and within development

assistance providers. Client governments could

draw more actively on multiple sources of advice

to inform their own independent positions and

choices. That would require structured contestation

among providers. At the same time, international

agencies and other providers of external advice

could introduce greater contestation of ideas as

part of their internal processes. They would need

to become more accustomed to their advice being

contested by client governments and compared to

alternative ideas and options.

Actions by client governments to

promote contestation

Several precedents can help in devising models

that support disinterested contestation of ideas

among multiple experts. They include the Elders

group of wise persons on international affairs,

the “Brain Trust” used by Franklin Roosevelt for

the rst and second New Deal in the 1930s, the

G20 Eminent Persons Group on Global Financial

Governance in 2017–18, the “Balcerowicz Plan”

and commission of experts on economic reforms

in Poland in 1989, the Independent Challenge

Group for the United Kingdom’s 2010 spending

review, the “Brain Trust” of advisers to Shinzo

Abe on economic policy from 2012 in Japan,

the “Wise Men” of economic forecasting in the

United Kingdom in the early 1990s, the “Iwakura

Mission” by Japan to the United States and Europe

in the 1870s, and the “Tswalu Dialogues” run by

the Brenthurst Foundation.

None of these examples is advocated as a template

for the contestation model proposed here—nor

are they without critique and criticism—but they

illustrate some possible variants. They all exhibit

a notable difference in character compared to

standard ofcial mechanisms such as the IMF’s

Article IV reviews and technical assistance

missions, the OECD DAC’s peer reviews, and

the European Commission’s Structural Reform

Support Service missions. In particular, they

have the inherent characteristics of a plurality

of perspectives beyond those provided by

ofcial agencies and the studied absence of any

ofcial positions or inuences. The plurality of

perspectives and views contrasts with the model

of individual high-level experts appointed as

independent external advisers to governments.

A shift is already underway from the consultant-

centric and supply-driven model of advice to one

in which multiple advisers present and debate

ideas and the government then decides which

approach to pursue. China has used this model

for external scal advice on its domestic reforms,

although the incentives at play for such a major

economic actor that does not require external

funding for advice are somewhat different than

those encountered in some other emerging and

lower-income economies. Emulating this type

of approach would involve a mechanism for

independent funding of external advisers and for

the government to set the agenda and control the

process, separate from the role and inuence of

development assistance or nance providers and

their downstream implementation consultants.

There are some relevant examples of the provision

of independent research, evidence, advice,

and ideas in lower-income settings to provide

alternative options for governments. One is the

role played by the International Growth Centre

on enterprise mapping and industrial policy in

Ethiopia. Other examples, which have been

deployed in Ethiopia and Rwanda, include third-

party funding for independent advice on tax policy

and external review of progress on Integrated

Financial Management Information Systems

reforms by a group of independent advisers. The

former was pre-funded, on-demand advice from

the United Kingdom’s Institute for Fiscal Studies,

and the latter was a mid-course review function

to examine and provide advice on challenges

around reform sustainability. In a similar mold,

116

the G7 CONNEX Initiative provides on-demand

independent advisory support to governments

of low-income countries in the negotiation of

complex commercial contracts in the extractive

sector. While institutionalized consideration and

contestation of alternative ideas is still novel,

these examples show that independent advice and

a “second opinion” approach are gaining ground

in lower-income countries.

There will inevitably be questions as to whether

structured contestation of ideas in a process led

by government is applicable and appropriate

in all country settings—especially in fragile

states. Ethiopia and Rwanda may well be on

track to following China’s approach of creating

mechanisms for institutionalized consideration

of alternative ideas and perspectives. There are

legitimate concerns, however, that some other

governments may lack capacity to orchestrate and

exploit this type of contestation and debate, or that

in practice the donors and IFIs that are funders

or nanciers of government will hold too much

passive inuence and thus crowd out real choice.

There is no clear indicator or proxy for the

conditions that are required in order for contestation

of ideas to be feasible and productive. It may be

that access to alternative and more diverse sources

of external nance, such as international capital

markets borrowing on competitive terms, signals

government capacity and downgrades donor

inuence (Haque et al., 2017). It may be that

strategic capability in government to articulate

priorities and make choices (as distinct from

technical capacity to run systems and processes)

is the important factor (Fritz et al., 2011; Krause,

2012). It may be that governments need to reach

the point where PFM shifts from a technocratic

concern of the nance ministry into a broader policy

concern for ministers of spending departments, so

that it is about government as a whole and not just

the nance ministry. Under these circumstances,

the salience of results from public spending could

override tradition in asserting the importance of

particular institutional arrangements for PFM.

The approach taken for the present report is

to avoid country categorization and instead to

consider how mechanisms can accommodate

risks as well as benets. It seems an unreasonably

constraining and short-term–and even conde-

scending–view to argue that some governments

cannot cope with options and choice, or that they

need not be fully informed about the spillovers

and trade-offs associated with particular choices.

Drawing on positive examples of putting external

contestation of policy ideas into practice, from

the full range of country settings, is an important

starting point. As Booth and Unsworth (2014)

point out, contestation is often part of the process

of working with complex coalitions during reform.

Actions within international

agencies to promote contestation

More debate and contestation of ideas is also

needed within international agencies to challenge

potential policy biases and reveal potential

spillovers and trade-offs. The difculty is to

devise mechanisms and provide models that

encourage group deliberation among people who

may disagree but who have a common interest

in developmental outcomes (World Bank, 2014,

p. 183). It is a well-established nding that

“contestatory modes of communication” are

helpful for “more fully exploring all sides of an

issue, for uncovering shared information, and for

reducing conrmatory bias” (Bächtiger & Gerber,

2014, p. 116) through a process of “collaboration

engineering” (Kolfschoten, de Vreede & Briggs,

2010). Within donor agencies there are frequent

rejoinders to strengthen the peer review process,

“institutionalizing teams that review plans in an

explicitly argumentative manner” or creating

teams that are institutionally distinct from the

proponents to create space for more candor and

critique (World Bank, 2014, p. 184). In addition,

exhausting available research evidence is

necessary, if far from sufcient (Ravallion, 2011).

Behavior change by the international agencies that

provide nancing and policy advice is difcult to

Annex K: Putting external contestation of policy ideas into practice

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

117

effect. The need for change and the obstacles to

change apply both to these agencies’ approaches

to external engagement with client countries and

to their internal ways of working in areas such

as peer review. The External Advisory Group to

a recent IMF review of capacity development

noted that: “…even when all parties agree that

change would, in principle, be a good thing, [it] is

difcult. Paradoxically perhaps, internally strong

and broadly effective organizations, like the Fund,

can suffer from status quo bias and inertia. Many

of the recommendations require that people in

the Fund do things differently than in the past

and … have a more concrete plan for how [to]

… overcome the powerful inertia of “business as

usual.” (IMF, 2018a, p. 5).

References

Abb, P., Köllner, P., & Maslow, S. (2016). The Role of External Policy Advice in China’s and Japan’s Economic

Policies since the Global Financial Crisis. Paper presented at the Interpretive Policy Analysis 11th

International Conference University of Hull, United Kingdom, 5 July 2016.

Alexander, D., & Lewis, J. M. (Eds.). (2015). Making Public Policy Decisions: Expertise, Skills and Experience.

Abingdon, UK: Routledge.

Bächtiger, A., & Gerber, M. (2014). ‘Gentlemanly Conversation’ or Vigorous Contestation?: An Exploratory

Analysis of Communication Modes in a Transnational Deliberative Poll. In K. Grönlund, A. Bächtiger, &

M. Setälä (Eds.), Deliberative Mini-Publics: Involving Citizens in the Democratic Process (pp. 115-134).

Colchester UK: ECPR Press, University of Essex.

Booth, D., & Unsworth, S. (2014). Politically Smart, Locally Led Development. London: ODI.

Fritz, V., Fialho Lopes, A. P., Hedger, E., Tavakoli, H., & Krause, P. (2011). Public Financial Management Reforms

in Post-Conict Countries: Synthesis Report. Washington DC: World Bank.

Haque, T., Bogoev, J., & Smith, G. (2017). Push and Pull : Emerging Risks in Frontier Economy Access to

International Capital Markets. Washington DC: Macroeconomics and Fiscal Management (MFM) Global

Practice, World Bank.

IMF. (2018). 2018 Review of the Fund’s Capacity Development Strategy—Report of the External Advisory Group.

Washington DC: IMF.

Kolfschoten, G.L., de Vreede, G.-J., & Briggs , R.O. (2010). Collaboration Engineering. In D. M. Kilgour & C.

Eden (Eds.), Handbook of Croup Decision and Negotiation, Advances in Group Decision and Negotiation

(pp. 339-358). London: Springer.

Krause, P. (2012). Public Financial Management Reforms in Fragile States: The Case of West Bank and Gaza.

London: ODI.

Ravallion, M. (2011). Development Impact Calls for Knowledgeable Development Practitioners. Retrieved from

http://blogs.worldbank.org/impactevaluations/development-impact-calls-for-knowledgeable-development-

practitioners

World Bank. (2014). World Development Report 2015: Mind, Society, and Behavior. Washington DC: World Bank.

118

Endnotes

Chapter 1

The Australian Department for Foreign Affairs and Trade (DFAT), in its note urging development staff to address

PFM issues centrally, observes that:

Poor PFM can prevent governments from achieving their policy objectives, create uncertainty for the

private sector, and hamper growth and poverty reduction efforts. Unsustainable or volatile spending

may cause debt blowouts that limit a government’s capacity to pursue its objectives, and make it hard

for the private sector to plan and invest for the future. Inefcient resource allocation means funds

may not go to the top priority policies and most effective programs. Inefcient spending wastes

scarce public resources and can limit the quality and coverage of public services. Dysfunctional

PFM, including low levels of transparency and accountability, also raises the risk of corruption.

(DFAT, 2019, p. 3)

DFAT suggests that the relationship must hold true, but the problem in proving the connection more specically

lies in the challenges of evaluating impact (Keating, 2001). In a similar vein, the IMF in its recent review of

capacity building for public nancial management argues that:

Institutions are the foundation that supports sustainable economic development. They facilitate access

to economic opportunities, play a key role in the efcient allocation of resources, and in rewarding

human and capital investment, protect property rights and the rule of law, and help build resilience to

shocks and crises. (IMF, 2018a, p. 9)

Experiences with pro-poor budgeting suggest that there is a dilemma about how starkly to reveal the trade-

offs entailed: too stark, and vested interests would ensure that technical warfare over models and assumptions

would engulf the budgeting process; too nuanced, and the impact would be lost. The question is how to make

distributive impacts a standard feature of policy making and reporting. A useful start could be to add impact

statements to the budget and government accounts, but the challenge is to nd a stronger and more universal

version of the arrangements attempted during the pro-poor budgeting era. Gender budgeting proposals of a

few years ago emphasized policy-related expenditure proposals (see, for example, Elson, 1999). More recent

proposals for climate change and mitigation approaches advise a more fundamental review of budgetary and

accounting procedures as well as options for modeling climate-related spending, suggesting that there are lessons

to be learned from recent experiences in examining pension and infrastructure liabilities (Gilmore & St. Clair, 2018).

In September 2019, heads of state attended the High-Level Political Forum to review progress on the SDGs. They

were confronted with the facts that only 35 percent of low- and middle-income countries are “on track” to achieve

SDG targets related to meeting basic needs that are essential for a decent quality of life (such as proper nutrition,

access to sanitation, and gender equality of opportunity) and that only 18 percent of fragile states are “on track”

(Bill & Melinda Gates Foundation, 2018; IMF, 2018b; Kharas, McArthur & Rasmussen, 2018; Samman et al.,

2018). The meeting concluded that the world is not on track to achieve the target of bringing the population living

in extreme poverty under 3 percent of the world population by 2030 (United Nations, 2019). Financing of the

SDGs is, unsurprisingly, also off track. For emerging market economies, the average additional annual spending

required in 2030 to reach key SDGs stands at 4 percentage points of gross domestic product (GDP), compared

to 15 percentage points of GDP for the average low-income developing country (Gaspar, Amaglobeli, Garcia-

Escribano, Prady & Soto, 2019).

At times of rapid growth, new challenges can be addressed through distribution, rather than redistribution, of

the benets of growth. At times of slower growth, this strategy is ineffective, since resources have to be taken

away from existing commitments and constituencies—a much higher political bar to clear. This paper comes at

a time of weakening in global growth. Trade tensions are high, developing economies are experiencing nancial

stress, and risks to global growth have increased. Emerging markets (most prominently Argentina, Brazil, China,

Mexico, South Africa, and Turkey) and developing economies face some of the greatest risks. The slowdown is

most marked in Africa and East Asia. While per-capita GDP continues to increase, the growth slowdown signals a

major challenge for those who manage public nances as they seek to respond to expectations for public services,

particularly from the more politically connected middle classes. Deterioration in services will place poor people

under particular pressure. Austerity measures imposed by countries to offset budget decits could prove to be

politically difcult to maintain and may lead to destabilizing protests, as in the case of the July 2018 disruptions

in Haiti (Coats, 2019; World Bank, 2019a).

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

119

The trend toward refocusing on ends more than institutional means, and on policy objectives more than on the

stability of the scal machinery, is a reminder that PFM can have only a partial inuence alongside broader

economic, social, and political factors (Weingast, Shepsle & Johnsen, 1981).

As the IMF acknowledges in a review of its own program design and conditionality between 2011 and 2017,

“there was limited focus on the quality of social spending, and on social protection and inequality, more generally”

(IMF, 2019, p. 24).

Chapter 2

Any allusion to history in public management risks the accusation that it has failed to acknowledge key historical

developments. To avoid this criticism, tribute should be paid here to the writings of Max Weber who noted that a

modern bureaucracy acts, in principle, as a lasting, impartial, rule-abiding, and non-partisan executor of laws and

regulations, which are devised by the political leadership (Weber, 1978). Weber observed (but did not prescribe)

that these traits were more likely to be found in hierarchical organizational structures staffed by career ofcials

operating within stable and comprehensive sets of rules. Pritchett and Woolcock (2004) remind us that, while

Weber made these observations in Western Europe, Western Europe did not invent rational bureaucracy, as shown

by the long history of bureaucracy in China and India).

This thinking opened up contentious debates following some initial enthusiasm for exibility and internal trading

arrangements branded as “new public management” (Hood, 1991), quickly giving way to skepticism about the

feasibility of such dramatic moves (Manning, 2001; Schick, 1998b).

Drawing on evidence from Brazil, Peru, Indonesia, Madagascar, and Cambodia, Ringold (2012) points out that

PFM and other institutional weaknesses act as a block on social accountability.

For social accountability mechanisms to inuence service delivery, changes are needed in the behavior

and the content of the work of frontline service providers. If the incentive framework under which

providers operate isolates or protects them…social accountability mechanisms, such as scorecards

or hotlines, are unlikely to work well. Social accountability is not just a ‘demand-side’ concern. The

mechanisms…cannot be considered separately from the broader incentive framework under which

providers operate. (Ringold, 2012, pp. 94–95)

There is similar empirical evidence from Kenya (Sheely, 2014).

Detailed survey evidence and robust research in the Philippines have shown that vote buying is robustly correlated

with lower levels of services that are within local control, that are pro-poor, and that are not targeted at particular

groups. Thus, for example, expenditures on local health services, which the rich tend not to use, diminish in the

presence of vote-buying (Khemani, 2013). Non-programmatic parties do not have a broad membership base that

is able to discipline party leaders. Again, research has shown that, under these circumstances, party leaders are

less likely to focus public expenditures on:

the more proximate determinants of development: education and the accumulation of human capital;

infrastructure; security, including the protection of property and contract rights; public health; or the

proper regulation of market failures, ranging from the environmental to the nancial. (Keefer, 2013,

p. 2)

The starkest example of PFM constraints to effective service delivery are cash budgeting and cash-rationing

practices. There are, of course, factors outside of PFM that are also essential for effective service delivery—not

least oversight and accountability, managerial attitudes, and staff engagement with the tasks at hand.

See IMF World Economic Outlook (WEO) denitions and classications, in particular (IMF, 2018c).

120

Fiduciary risk tools include:

ADB AssessingandManagingGovernanceRisksinADBOperaons

ADB FinancialManagementAssessment

IADB ToolforDeterminingtheLevelofDevelopmentandUseofPFMSystems

WorldBank Assessment of the Use of Country Financial Management Systems in Bank

Financed investment Projects

EU BudgetSupportGuidelines

DFID FiduciaryRiskAssessment

BMZ Strategy181:BudgetSupportintheFrameworkofProgramme-OrientedJoint

Financing

Danida GuidelinesforRiskManagement

USAID PublicFinancialManagementRiskAssessmentFramework

Source: Developed from PEFA Secretariat (2018b).

Donors often accompany these assessments with a parallel, but strictly unrelated, concern about fungibility:

did the aid provided for one donor-approved purpose free up government nancing for a different purpose that

the donor might consider unproductive? Donors are frequently subject to intense scrutiny from politicians who

are skeptical about the use of taxpayers’ money to nance activities in other countries and prompt to condemn

spending on activities they do not support. When donors doubt the capacity of the government to reallocate

nancing in line with development priorities, they often turn to distinctive project instruments that seek to limit

fungibility as well as to provide duciary assurance.

The reform areas captured in Figure 3 are dened as follows:

Forecasting and strategic budgeting – Macro and revenue forecasting, program-based budgeting, zero-based

budgeting, medium-term expenditure frameworks, participatory budgeting, program impact studies, gender

budgeting

PFM legislation – Introduce budget circulars, committees for budget debates, create budget ofce in parliament

to help assess proposals, increased legislative oversight, scal decentralization, enact scal rules, budget

classication systems, debt management frameworks, budget cycle changes, public-private partnership legal

framework

Treasury Single Accounts – Consolidate governments’ cash resources to ensure that revenue is stored in one

place. In countries with fragmented government banking arrangements, the establishment of a Treasury Single

Account should receive priority in the PFM reform agenda to increase overall traceability and accountability of

funds

Core Financial Management Information Systems (FMIS) infrastructure – Refers to an ecosystem of information

technology (IT) infrastructure that supports the daily functions of public entities by tracking and summarizing

nancial events and data. These systems automate core activities such as budget preparation (e.g. compiling

of expenditure plans from different departments), nancial reports generation, and improved ease of access to

nancial data. Reforms include capex on hardware and software, digitization of (including historical) data, and

capacity building within the civil service to use the IT systems prociently

Expenditure side reforms – More efcient and transparent practices (independence of procurement bureaus,

promote transparent bidding through electronic and competitive procurement, standardized forms), effectiveness

of service delivery, improved subsidy targeting mechanisms such as identication means (e.g. Aadhaar in India),

payroll reforms including human resource management systems

Revenue side reforms – Digital tax collection, increasing taxpayer convenience, remittances management,

increased taxpayer buy-in, effectiveness of revenue collection agencies, promote and strengthen independence

of collection body

Accounting and reporting – Adopt accrual accounting standards, professionalizing the accounting function,

standardizing charts of accounts, applying international standards to classication of fund stocks and ows

Internal audit and control – Programs to support Ofce of the Auditor-General, capacity building (e.g. analysis

of public expenditure reports), strengthen internal controls and monitoring standards, procure and use of software

to improve productivity, increased ability to prosecute, internal control and compliance

External audit and monitoring – Publish civilian budgets, maintain updated scal portals to increase transparency,

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

121

PEFA assessments, public reports to raise awareness on budget matters, public investment information systems,

funding non-governmental organizations (NGOs) in audit and monitoring

(AlphaBeta, 2018)

The sector classication of OECD DAC Purpose Codes is dened as follows: “The sector of destination of a

contribution should be selected by answering the question ‘which specic area of the recipient’s economic or social

structure is the transfer intended to foster’” (see https://www.oecd.org/dac/stats/purposecodessectorclassication.

htm). In this classication of development nance statistics, “public nance management” (purpose code 15111)

is treated as separate from “domestic revenue mobilisation” (purpose code 15114), “local government nance”

(purpose code 15128), and “public procurement” (purpose code 15125). AlphaBeta (2018) follows the OECD

DAC classication and includes only data under the 15111 “public nance management” purpose code in its

analysis of external nance.

10.

“Men [sic] of the sociological tribe rarely visit the lands of the physicists and have little idea of what they do over

there. If the sociologists were to step into the building occupied by the English department, they would encounter

the cold stares if not the slingshots of the hostile natives” (Becher & Trowler, 2001, p. 45).

“Despite decades of talk about ‘interdisciplinary courses’…there is a strong belief that such talk is just talk. Those

thought to be sober mostly presume that real scholarship means getting on with the advancement of knowledge

organized exclusively by disciplines and sub-disciplines. It does not seem to matter that some knowledge may

not contribute to an intelligible whole…” (Orr, 1993, p. 10).

11.

“…academic discipline can be seen as a form of specic and rigorous scientic training that will turn out

practitioners who have been ‘disciplined by their discipline’ for their own good. In addition, ‘discipline’ also

means policing certain behaviours or ways of thinking. Individuals who have deviated from their ‘discipline’ can

be brought back in line or excluded” (Krishnan, 2009, p. 8).

12.

The existence of disciplines within development can have a downside. Levy (2014) identies the somewhat

shameful conicts within the World Bank when considering governance reforms between, on the one hand,

“governance advisers who saw the strengthening of ‘country systems’ within the public sector as key to

effectiveness” and, on the other, “protagonists who gave priority to more bottom-up, community-based

approaches” and champions of “vertical global programs” which, the governance advisers felt, undercut country

ownership and “added new difculties to the challenge of strengthening country systems” (p. 5). In a zero-sum

competition for prominence and project nance, the “wins” for one side can mean that the knowledge of the other

side is lost.

13.

“…in ve of the eight cases (Afghanistan, DR Congo, Liberia, Sierra Leone, and West Bank and Gaza), the

appointees formerly worked for international nancial institutions for a period of time. From a government’s

perspective, such appointments serve a dual purpose. A minister with an international background may be

expected to serve as a champion of PFM reforms while at the same time being expected to liaise effectively

with the international community and to ensure external assistance ows. In Afghanistan and the DR Congo, in

particular, such appointments were made during early post-conict years when relations with the international

(aid) community needed to be established” (Fritz et al., 2011, p. 6).

14.

Sixty-eight self-identied public nancial management specialists working in development completed an online

survey.

15.

Commonly recognized as:

1. Budget preparation and approval phase with strategic decisions made that translate policy objectives into

nancing in the context of revenue forecasts with budgets then prepared, generally for submission to the

legislature

2. Budget execution phase in which governments implement, or not, the actions necessary and agreed for

nancing to achieve their policy objectives, with revenues collected and resources distributed to spending

agencies with cash liquidity and debt managed. Internal control and audit procedures ensure compliance with

rules and manage risks, the maintenance of records, and reports on the execution of the budget, all with the

aim of monitoring and controlling compliance with budget appropriations.

3. The external audit and policy review phase completes the cycle. Supreme audit institutions are responsible

for public external scrutiny of the government’s annual nancial statements or equivalent, examining whether

government nancial activities were carried out in compliance with the original budget law, and respecting

all other rules and procedures. Sometimes, although perhaps more often aspired to than achieved, this phase

includes an ex-post analysis of impact of government policy programs.

122

16.

“Many European countries have started changing their accounting systems by adopting a set of standards largely

inspired by International Public Sector Accounting Standards (IPSAS) issued by the IPSAS Board (IPSASB). As

the IPSASB has no legitimate power, the adoption of the IPSAS in local and central governments is completely

voluntary…. In the European Union (EU), since 2012 Eurostat is working on a project for the harmonization of

[public sector accounting] of member states and decided for the development of a European PSA standards set,

which will consider IPSAS as a reference” (Rossi, Caperchione, Cohen & Brusca, 2019, p. 237).

17.

These are set by the Committee of Sponsoring Organizations of the Treadway Commission.

18.

Professional PFM bodies and their standards and basis for membership.

Professional body Explicit standards Implicit norms Membership

CharteredInstuteofPublic

FinanceandAccountancy

Individual

CommieeofSponsoring

Organizaonsofthe

TreadwayCommission

Standards for internal

controls

ExtracveIndustries

TransparencyIniave

EITI Standard Naonallevel

IMF Fiscal Transparency Code

IMF RevenueAdministraonGap

AnalysisProgram

InstuteofInternalAuditors InternaonalProfessional

PraccesFramework

Individual

InternaonalAccounng

StandardsBoardwithin

InternaonalFederaonof

Accountants

InternaonalPublicSector

AccounngStandards

Individual

InternaonalOrganizaonof

SupremeAuditInstuons

InternaonalStandardsof

SupremeAuditInstuons

Agencylevel

WorldBank CustomsAssessmentTrade

Toolkit

WorldBank ReportonObservanceOf

StandardsandCodes–

AccounngandAuding

Source: Developed from PEFA Secretariat (2018b).

The depth of knowledge that these bodies are drawing on is reected to some degree by the size of their

membership. For example, the Institute of Internal Auditors, a professional body based in the United States,

serves 185,000 internal auditors in nearly 190 countries (Allen, 2018).

19.

Since their separation from political scientists a century or so ago, public administration and management specialists

start with the problem that they do not draw on a distinct academic discipline (Guy, 2003; Raadschelders 2018).

The study of public administration has multiple theories and many generalizations, but these have not coalesced

into a distinctive body of knowledge recognized for its ability to predict the future course of organizational or

societal behavior. Any broad progress is limited by the confounding effect of context and deeply embedded

cultural traditions within the public sector, which prove difcult to pin down.

For example, the Germanic tradition can be stated as an ideal type, but the structure and functioning of German

and Dutch governments are very different. Historical occupation by France is generally seen as the test for what

constitutes a Napoleonic country, but Italy and the Netherlands are hardly alike.

The challenge of tradition is compounded by the cultural dominance of the Anglosphere; American and British

textbooks are translated into Chinese, Korean, and Spanish, but the reverse is not (as) true. Blurring the conceptual

boundaries even further, the language used to distinguish key components of the public sector can confuse. The

position of civil servant in the Netherlands and China is legally dened; in the United Kingdom it is sociologically

dened as someone who works in Whitehall or for the government; and in France and Germany the distinction

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

123

between higher- (fonctionnaire, Beamte) and lower- (employee, Angestellte) level civil servants is both legally

and sociologically dened (Raadschelders 2018).

Although never stellar, the prestige of the public administration eld has waned in the last half century,

overshadowed by economics and widely seen as craft-based rather than knowledge-based work, drawing on

practical skills required for work in the public sector (Kettl, 2000; Vitale, 2013).

20.

While Evans and Rauch (1999) show a causal link between the quality of public administration and economic

growth, for example, examination of growth accelerations such as those in China after the late 1970s and in

South Korea from the early 1960s do not reveal any signicant public sector management reforms that preceded

them (Hausmann, Pritchett & Rodrik, 2005). Current evidence points to those that protect the returns on private

investment, in particular property rights and the rule of law, but beyond that, it is not clear which institutions are

causally related to economic development and, in fact, the direction of causation might be the reverse, with richer

societies demanding better governance structures (Booth, 2015).

21.

For example, “a TADAT assessment report will not…include recommendations for specic reforms….Rather,

the report will—in highlighting weaknesses of administration—provide the basis for discussions about reform

goals, implementation strategies, and prioritization and sequencing of interventions” (TADAT Secretariat, 2015,

p. 6).

22.

For example, “the PEFA methodology draws on PFM international standards and good practices as identied by

experienced practitioners and academics…” (PEFA Secretariat, 2018a, p. 3).

23.

The use of convention compliance as the approach for understanding PFM arrangements has arguably been

strengthened by two other factors. First, as credit rating agencies have broadened their scope and increased their

activities, they have found indicators such as those elaborated by the PEFA program useful in justifying their

claim that many countries need oversight because of their institutional weaknesses (Paudyn, 2014). Second, in

contexts of low staff capacity, the existence of a set of standards facilitates the efcient development of staff

competencies (Kirkpatrick & Kirkpatrick, 2009).

There is an evolving “ecology” of diagnostic tools as well. For example, as China provided support to the belief

that economic development required large investments in infrastructure, public investment management became

more prominent in the tools. As domestic resource mobilization became recognized as a key constraint, revenue

gap analysis emerged.

24.

“The inuence of the ‘good governance’ debate saw a convergence in thinking about tax administration in much

the same way as the previous period led to convergence in tax policy advice. Today, most developing countries

have not only rationalised their tax systems by replacing trade taxes by indirect taxes but have also invested

in a standard set of reforms with a view to modernising their administrations. This is best illustrated in the

establishment of semi-autonomous revenue authorities (SARAs), which can be dened as tax administrations

which operate with a degree of autonomy from their nance ministries. First established in Jamaica, the model has

spread to Latin America, Africa, Asia and recently Greece. In line with [New Institutional Economics] thinking,

ring-fencing the administration from political interference supposedly signals a credible commitment to a less

discretionary tax collection process, which in turn should boost compliance… However, the effect of SARAs is

subject to debate.

The creation of these new authorities coincided and sometimes facilitated the modernisation of specic tax

administration processes and brought them into line with best practices from NPM. Reforms focused on increasing

efciency by reducing collection costs and increasing compliance by approaching taxpayers as customers. …(E)

xamples of some of the key reforms:

• Increasing the reliance on information technology.

• Introducing unique taxpayer identication numbers.

• Re-organising the collection system from one organised around different taxes, to one organised around

individual taxpayers.

• Segmenting taxpayers by, for example, establishing large taxpayer ofces.

• Separating back- and front-ofce functions to reduce the scope for bribery.

• Creating ‘one-stop shops’.

• Simplifying procedures” (Dom & Miller, 2018, p. 21).

25.

Campos and Pradhan (1996) contrasted the emphasis on contracting and output specication in New Zealand

with the emphasis on delegation within hard constraints in Australia in the 1990s.

26.

A term coined by Richard Batley, Emeritus Professor of Development Administration, University of Birmingham.

27.

The difculties of obtaining data about the internal workings of the public sector have been noted in the social

124

science research literature for many years (Jakobsen & Jensen, 2014). New budgetary procedures can be proposed

and agreed, but implementing a new accounting framework requires changing the hard-to-observe behavior of

thousands of public servants, many of whom can continue old habits while claiming to have introduced the policy

wholeheartedly.

28.

“In technical situations, where the authority has the expertise to dene and solve the problem…let the experts

do their job. Adaptive situations, however, tend to demand a more participative mode of operating to shift

responsibility to the primary stakeholders” (Heifetz, 1994, p.121). As Booth (2014) rather memorably puts it

in reference to some complex institutional reforms in the Philippines, “this was a guerrilla operation, not a war

of xed positions” (p. ix). Similar arguments have been made for OECD countries (Pollitt & Bouckaert, 2011).

Melchor (2008) and the OECD (2005) both observe and welcome the prevalence of incremental adaptation in

reform. We can see this in practice when nance ministries “try out” a new approach to budgeting, accounting,

procurement, and the like for a limited number of agencies for a limited period of time to test how things work

(Kristensen, 2019). This is consistent with the observed experiences of PFM reform in East Asia (So, Woolcock,

April, Hughes & Smithers, 2018).

Incrementalist and adaptive approaches are often attributed to the changing thinking in economics (Rodrik,

2008), but they have a history in public administration: the need for reform processes that allow iteration and

adaptation has long been identied (Brinkerhoff & Crosby, 2002; Brinkerhoff & Ingle, 1989); Evans (2004) has

warned against institutional “monocropping” when “deliberation” is more appropriate; and Ellerman (2005) set

out a radical critique of donor agencies’ tendencies to know best.

Much of the academic and research discussion of this comes under the broad heading of “design thinking”

(Clarke & Craft, 2019; Howlett, 2009).

29.

Ear (2009) provides a fascinating case study of donors confronted by bureaucratic politics and rivalries that they

did not recognize or understand.

30.

Andrews, McNaught, and Samji (2018) provide a good example of how PEFA scores were used as the basis for

dialogue with nancial management staff across government in Mozambique.

31.

Williamson’s observations were that there was a consensus among IFIs in Washington concerning the need

for: (1) scal discipline; (2) a redirection of public expenditure; (3) tax reform; (4) nancial liberalization; (5)

adoption of a single, competitive exchange rate; (6) trade liberalization; (7) elimination of barriers to foreign

direct investment; (8) privatization of state-owned enterprises; (9) deregulation of market entry and competition;

and (10) protection of property rights (Williamson, 1989).

32.

There is arguably some difference in the formulations of the second objective (allocative efciency) among Schick

(1998a) (“expenditures should be based on government priorities and on effectiveness of public programs. The

budget system should spur reallocation from lesser to higher priorities and from less to more effective programs”);

World Bank (1998) (“allocation of resources in accordance with strategic priorities”); and PEFA Secretariat

(2016) (“strategic allocation of resources involves planning and executing the budget in line with government

priorities aimed at achieving policy objectives”). Whereas Schick includes an emphasis on performance in the

second objective (that is, with regard to the effectiveness of public programs), the World Bank appears to address

that dimension as part of the third objective on operational efciency (“efcient and effective use of resources

in the implementation of strategic priorities”). The PEFA Secretariat takes a more ambiguous middle path. In

practice, these differences of emphasis across objectives and their formulation are not signicant when all three

objectives are treated interdependently. Nonetheless, they do reveal the important interpretive inuence of the

leading experts and organizations in the international PFM eld.

33.

Cangiano et al. (2013a) argue that:

It is important to dispel a potential misunderstanding of the meaning of the rst PFM key objective, maintaining a

sustainable scal position. This objective should not be seen as, and certainly does not imply, a bias toward scal

tightening. It is a way to achieve legitimate macroeconomic objectives, most notably inclusive and balanced

growth, in an orderly fashion without prejudging the state of public nances, which may, in turn, have negative

repercussions on macroeconomic stability and growth. (p. 2)

Chapter 3

The literature points clearly to the important effects of country-specic factors on PFM reform outcomes (Evans,

2008; Peterson, 2010; Pretorius & Pretorius, 2009, p. x; Scott, 2011). The constraints posed by human capital

limitations and economic endowments on PFM reform prospects have been explored in both case-specic and

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

125

cross-national analysis (Fritz, Sweet & Verhoeven, 2014; Haque, Knight & Jayasuriya 2012). Context is about

both the intricate pressures that cause actors to adapt, perhaps even distort, budget practices to everyday realities

(Stevens, 2004) and the features of the broader political system in which stakeholders engaged in PFM reforms

are embedded. The overwhelming importance of country context in determining which reform approaches and

measures will be successful, and in which ways, is a statement about both the primacy of country factors and the

limits of external ones, most notably donor inuence.

Two kinds of reactions can be distinguished. One entails the manipulation of the measures or criteria that are

selected. In this case, the operations remain the same but the representation of these operations is deliberately

skewed by selecting indicators that give preferred results. This results in a loss of data quality. The alternative is

to alter the output itself. This usually results in a loss of output quality. A combination of both is also possible:

Source: Van Dooren (2006), cited in Van Dooren et al. (2006).

There is an extensive literature on the challenge of gaming, exemplied by Charles Goodhart’s “Law,” which he

developed following his analysis of the consequences of the Government of the United Kingdom relying solely

on money supply targets in the 1970s: “Any observed regularity will tend to collapse once pressure is placed on

it for control purposes” (Goodhart, 1975). Risks and remedies are set out in Courty and Marschke (2004) and Van

Dooren et al. (2006).

Knowledge transfer and the genuine replication of practices across settings are hard to achieve in an objective

sense, irrespective of the possibility of an underlying incentive to game the assessments. Evidence from the

corporate sector about “internal stickiness” is relevant and instructive here: “the major barriers to internal

knowledge transfer are shown to be knowledge-related factors such as the recipient's lack of absorptive capacity,

causal ambiguity, and an arduous relationship between the source and the recipient” (Szulanski, 1996, p. 28).

Especially for those dimensions of the assessment frameworks that focus on “functional” reforms more than

merely “formal” ones (Andrews, 2012), the likelihood of gaming may be further diminished.

De Lay et al. (2015) provide a more comprehensive treatment of the research difculties.

The sequencing debate is based around the premise that some basic disciplines (typically around managing PFM

inputs and human resource management) should be entrenched prior to more advanced public sector management

reforms, including arrangements for measuring and managing outputs and/or performance. This case was most

prominently articulated in Schick (1998b) and the associated mantra of “look before you leapfrog.” This was

followed by the World Bank’s Public Expenditure Management Handbook (World Bank, 1998), which stressed

the importance of getting the basics right rst:

• Control inputs before seeking to control outputs

• Account for cash before moving to accrual accounting

• Operate a reliable budget for inputs before moving to budgeting for results

• Make a comprehensive budget and reliable accounting system before trying an integrated nancial

management system

• Get a proper budgeting and accounting function before strengthening the auditing function

• Do reliable nancial auditing before trying performance auditing.

This logic was operationalized most clearly in PFM in the platform approach proposed by Brooke (2003) and

in the “hurdle approach” to PFM reform in Thailand (World Bank, 2002), where competence in a set of internal

ministry processes were prerequisites for enhanced autonomy, including budget planning, output costing,

procurement management, budget and funds control, nancial and performance reporting, asset management,

and internal audit. A recent and comprehensive summary of assumptions concerning what comes rst is set out

in Diamond (2012).

See https://pefa.org/news/enhanced-standards-pefa-reports-launch-new-pefa-check-criteria.

126

A 2018 survey of leaders of development agencies found “a fragmented development ecosystem and an ever-

expanding cast of players… During this upheaval, development leaders are innovating, harnessing technology

in exciting ways, using data to drive decision-making, and empowering partners on the front lines. They are

painfully aware that not all of their organizations are likely to accomplish their goals or continue to exist in their

current form. Overall, the survey results paint a picture of a global development sector rife with experimentation

and transition” (Ingram & Lord, 2019, p. 1). “Among issues seen as neglected or insufciently resourced, climate

change tops the list, followed by youth, state fragility, and governance” (Ingram & Lord, 2019, pp. 5–6).

See, for illustration, Eno, Glentworth, Hedger & White (2018).

Current and past examples include the Bill & Melinda Gates Foundation, the Tony Blair Institute for Global

Change (formerly the Africa Governance Initiative), the CABRI network of African budget ofcials, the

“Building State Capability” program at Harvard Kennedy School, the “Budget Strengthening Initiative” and

Fellowship Scheme at the Overseas Development Institute, the International Growth Centre, the African Centre

for Economic Transformation, the Harvard Institute for International Development, the Duke University Center

for International Development, the National Institute of Public Finance and Policy in India, the Development

Policy Centre at Australian National University, the Scott Fellows scheme at the Center for Global Development,

the United Kingdom’s Institute for Fiscal Studies, the Korea Institute of Public Finance, the Center for

International Development at Korea Development Institute, the Center for Excellence in Finance, the Oxford

Policy Fellowship, and the Blavatnik School of Government at Oxford University.

10.

“…unless developing countries are offered genuine choice about which aid agencies they want to work with,

the effectiveness of aid in reducing poverty will decline and the rhetoric about recipient country ownership will

remain empty” (de Renzio & Rogerson, 2005, p. 1). Moreover, Browne (2012) sees open choice and competition

among different development models as key to major transformations in each of the last three centuries (Peter the

Great’s modernization of Russia in the early 18th century, Japan’s opening up to economic and institutional advice

from outside in the mid-19th century, and China’s request for modernization through capital investment and skill

development from the League of Nations in the early 20th century). In these cases, the countries “decided on their

best course of action and purchased overseas the skills that they lacked… The client country paid and the jobs got

done the way the client wanted” (Browne, 2012, pp. 23–24).

11.

The emergence of competition is generally accompanied by attempts to suppress it. “It is peculiar that an

abundance of suppliers is criticised in the ‘aid market’, when economics underline the virtue of competition

almost everywhere. However in the world of aid, the presence of many donors does not imply competition among

them, but more often superposition of costs and administrative procedures” (Santiso & Frot, 2010). Interagency

competition in development assistance is evidently present but publicly decried.

12.

“Financing a separate facility with like-minded donors to compete directly with current players offering PFM

advice and implementation support. One example of such a facility is the Global Fund for Women, where BMGF

is a key contributor. The Global Fund for Women has group of core sponsors that appear to have little or no

control over the decision making of the fund which is governed by a board of directors represented by personnel

from diverse geographies and backgrounds. The board is supported by an extensive list of global experts that

provide due diligence, outreach, and analysis which guide the fund’s activities” (AlphaBeta, 2018, p. 5).

13.

This is the “expertise-and-policy-advice tail that wags the development dog” (Browne, 2012, p. 163).

14.

Countries “value exibility and the use of country systems, speed of delivery, and alignment to their national

strategies. When considering the nancial terms for debt resources, a minimum grant element of 35% of the

nominal value of the loan (the IMF benchmark for low-income countries) would be the prevailing criterion for

the Ministries of Finance in Ghana and Senegal when seeking project-type nance. However, both countries

chose to pay signicantly more for Eurobond issues and syndicated regional loans offering much larger volume

and exibility. Timor-Leste sets the return on its offshore reserves as a ceiling on borrowing rates” (ODI/OECD,

2014, p. 8).

15.

“Viewed as a quasi-market, the aid industry is now maturing.… [T]he consumers – in this case, the aid recipients

– are beginning to have a modest inuence on the pricing and product behavior of suppliers (aid donors). An

interesting trend to track in the coming years will be whether concessional aid suppliers begin to compete with

one another on the basis of brand and other non-price attributes” (O’Keefe, 2007, p. 10).

16.

“For example, a market for technical assistance similar to ‘eBay’ would enable developing countries to set out

their requirements, enabling a bigger variety of providers to bid according to the terms of reference. Service

providers would [be] able to earn (or lose) [their] reputation based on feedback from previous clients provided

through the online system. Reports and analysis from such technical assistance would all be shared online”

(Barder, 2009, p. 35).

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

127

17.

Seemingly, Barder (2009) has choice in mind when he argues for what he terms a “collaborative market.” “Aid

funders could make more use of market incentives to encourage innovation and efcient delivery of services,

and to close down ineffective or unnecessary programmes and organisations.…Networks hold the promise of

increasing and improving information and…reducing transactions costs, to help to close the feedback loop

between beneciaries and funders, and providing a platform for collaboration. Using markets and networks

together more effectively may permit decentralized and uncoordinated decision-making to produce collectively

benecial outcomes in transparent and regulated markets” (Barder, 2009, p. 37).

18.

It is currently popular in development circles to argue that the linear logic of logframes and so on, with their

implication of an inevitable chain of cause and effect, from input to output to outcome (DFID, 2009; Team

Technologies, 2005), should be replaced or at least supplemented by a “theory of change” that allows for

uncertainty and unpredictable contingencies, reecting a disorderly “real world” picture, with the possibility that

different paths might lead to change (Valters, 2015).

In reality, there is immense confusion about the meaning of theories of change, partly because they originate

from two perspectives: an evaluation perspective, seeking to dene how inputs are linked to outcomes, and a

“social action” perspective, emphasizing participation and continuing reection as action proceeds (Vogel, 2012).

The result is that a theory of change can be anything between, on the one hand, a rebranded logframe and, on

the other hand, an action perspective that suggests that there cannot be certainty about the consequences of any

intervention and that small steps, informed by careful observation and participation, are the only way forward.

While the Working Group is agnostic about whether current thinking offers practical ways forward, it endorses

two ideas implicit in current debates about theories of change. First, it considers that the linear logic of logframes

is more appropriate for building bridges than for reforming institutions. Local context matters and the responses

of the actors cannot be fully predicted in advance (Porter, Allen & Thompson, 1991). Whatever the terminology,

the design of an intervention must allow for contingency and must answer the question of why the relevant

actors will make the changes assumed within a reform program. The design of programs must be “autonomy-

respecting” (Ellerman, 2005). Simply asserting that a different way of doing things will be better is a weak

lever, and compulsion in development cooperation has proved to be both infeasible and ineffective. Second,

approaches to reforming institutional arrangements must always rest on a “theory” of some sort about why

facilitating particular managerial or institutional changes will lead eventually to the desired improvement through

the results chain. “All development activities – projects, programs, policy reform, technical assistance, training

workshops, capacity building, research and evaluation – therefore operate on the basis of a theory of change, even

if this theory is only implicit and never articulated” (Pritchett, Woolcock & Andrews, 2010, p. 1). The challenge is

to make that theory more explicit so that it produces predictions that are capable of being falsied. Observations,

insights, and lessons may then feed back into rened theory.

19.

Lawson notes that many of the institutional reform case histories he analyzed raised examples of policy space

constraints:

• “In Malawi, Procurement reforms and Internal Audit reforms were both premised on the use of ‘best practise’

models in which responsibilities would be decentralised. This would have required new skills to be developed

and embedded in virtually all government institutions. These models were fundamentally inappropriate to

a context of signicant shortages of trained staff within the civil service, combined with the persistence of

hierarchical modes of working, limited accountability and a culture of frequent disregard of rules. Simpler

reform models were needed, less demanding of organisational and human resource capacities.

• The BPEMS reform in Ghana was based on a technology-driven model, which gave insufcient attention to

change management issues, and to the assessment of capacity constraints and training needs. It was also based

on an ambitious ‘big bang reform’ approach rather than a more gradual, ‘incrementalist’ approach. Again this

was a ‘best practise’ model, which was inappropriate to the context.

• The MTEF reform in Ghana presented a different type of policy space constraint, in that the consultants

leading its introduction selected and promoted an idiosyncratic model, which was never likely to achieve

the desired outcomes of an MTEF. The reform design was idiosyncratic rstly, in placing emphasis on the

‘bottom-up’ elements of an MTEF (programme-based budgeting, detailed costing of plans, integration of aid-

nanced projects) and barely mentioning the upstream elements (macro-scal framework, strategic policy

and expenditure review processes) that are conventionally seen as the initial priorities in establishing a robust

MTEF process. Secondly, it adopted an activity-based budgeting approach to the development of multi-year

budgets, which generated voluminous, highly detailed documents, which made it very difcult to discern the

strategies and priorities underlying MDA budgets, thus effectively defeating the purpose of the exercise.

128

• In Burkina Faso, the problem was essentially a failure to sequence work on the medium term expenditure

framework and programme budgets in an adequate manner. It involved the attempted introduction of programme

budgets before there was an aggregate medium term scal framework in place, when many sector policy and

strategy frameworks were unclear and when the processes for tracking performance during budget execution at

the programme level were yet to be designed.” (Lawson, 2012, p. 61)

20.

Robust peer review must involve teams that are institutionally distinct from the proponents, in order to create

space for more candor and critique (World Bank, 2014, p. 184).

21.

The problem of “domain narcissism” (a phrase coined by Professor Richard Marcy, Assistant Professor of

Organizational Behavior at the University of Victoria School of Public Administration)—in which one discipline

seeks dominance over others, employing a narrowing of perception and rejection of incompatible information

from other disciplines, and with an associated degree of emotionalism—is well-recognized (see, for example,

Alexander & Lewis, 2015).

Chapter 4

The same is true of all “upstream” public sector management arrangements since “sustained improvements in

education, health, and other sectors downstream often depend on institutional reforms upstream, at the center of

government” (World Bank, 2012b, p. 18).

The connection between service delivery and outcome is about quality. In a seminal review of output measurement

for the United Kingdom government, Tony Atkinson and others pointed out that the quality of an output, or

service, is really a measure of “the attributable incremental contribution of the service to the outcome” (Atkinson

et al., 2005, p. 42).

As a group of leading PFM experts noted in their evidence submission to the last PEFA review process:

The PEFA framework was not designed to identify why indicators were scoring poorly and can only

suggest crudely which broad areas…might require attention – and more specic diagnostics. Yet in

developing countries, PEFA assessments are sometimes used formulaically to develop a detailed

PFM reform plan or to justify a wide range of highly ambitious interventions. One example is the

2009 Maldives’ Action Plan for PFM Reforms, though there are many others such as Sierra Leone’s

PFM Reform Strategy and Tajikistan’s PFM reform programme. (Krause et al., 2014, p. 1).

For example, “the next generation of indicators will have to minimize the use of normative-based best practices

and help devise better metrics, with shifted emphasis on contextual functionality and actual implementation

rather than on announcement effect so that substance can prevail over form” (Cangiano, 2017, p. 380).

Although stronger or more intentional variants could be considered, there would be risks around mandate,

legitimacy, and incentives. For example, the National Institute for Health Care and Excellence in the United

Kingdom (https://www.nice.org.uk/).

“While the rationalist approach elevates information and analysis as the basis for budgetary decision making,

seeking to achieve an optimal allocation of resources, the incrementalists demonstrate that political considerations,

institutional role play and bounded rationality provide a more accurate depiction of budgetary behaviour”

(Fozzard, 2001, p. 33). In addition, Fozzard (2001) writes:

The nature of the budget process also promotes incremental decision-making. Allocation decisions

tend to be made sequentially – between Ministries of Finance and individual spending agencies

and then by reviews of agency proposals within collegiate bodies such as Cabinet and in legislative

committees – rather than by assessing trade-offs between alternative claimants. Furthermore,

budgeting is repetitive, with the same allocation issues being addressed in successive budgeting

exercises. In this way, participants arrive at a tacit agreement regarding each claimants ‘fair

share’, constituting a ‘convergence of expectation on roughly how much an agency is to receive in

comparison to others’ (Wildavsky, 1992: 87). Once the fair share is established, bargaining about

allocational decisions is restricted to the distribution of additional resources rather than redistribution

the budget base. This reduces the risk of cutthroat competition between rival claimants for resources

and means that allocations do not have to be revised every year. Instead, the process is ‘premised on

agreement on the size, scope and distribution of expenditure. Conict was conned to the margins, a

little more here, a little less there’ (Wildavsky, 1992: 73). Without this stability, incrementalists argue,

the budgeting process would descend into chaos” (p. 30).

The argument here is that, with baselines, the starting point for producing next year’s budget is this year’s

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

129

budget plus built-in increases such (for example, an increased number of pensioners or ination adjustments

for benets). Policy changes are dened as cutbacks when they reduce spending below the baseline, even when

spending is above current levels. MTEFs extend the baseline to the medium term, incorporating only agreed

spending increases and, only rarely, spending decreases, as the basis for future budgets (Schick, 2013).

The Building PFM Capabilities program was developed by CABRI, in collaboration with the Building State

Capability Program at Harvard’s Center for International Development. The program uses the Problem Driven

Iterative Adaptation approach to support PFM reform and was successfully completed by seven African countries

in 2017 (Ababou, 2019).

There are diverse examples of how the design of the PFM system is overtly adjusted to reect the policy goals

and decisions of the government:

• The government may decide on a scal rule to contain public spending within a xed ceiling of GDP, or to limit

the annual growth of expenditure. The PFM challenge would then be to design the appropriate rule and develop

a monitoring mechanism to ensure that it is enforced.

• The government may announce a new policy to achieve greater gender equality. The budget system may then

need to be redesigned to include a gender budget statement, appropriate provisions in the annual budget circular,

a framework for conducting gender impact assessments and gender audits, an extension of the performance

budgeting system, and so on (IMF, 2017b).

• The government may decide that it wants to give a degree of preference to national contractors and suppliers in

the procurement process. To implement this policy, the law may need to be changed and adjustments made to

tendering procedures.

• The government may decide to boost tax collection by establishing a large taxpayer unit, providing special

incentives for large taxpayers to comply with the statute, and streamlining collection procedures.

All of these examples may also require changes to the primary PFM law or other PFM-related laws or regulations,

as well as changes to business processes, manuals, and instructions for nancial managers; changes to IT systems;

and so on (Allen, 2019).

There are also “covert” connections, such as the discontinuation of Public Service Agreements in the United

Kingdom to facilitate austerity measures. The introduction of Public Service Agreements for public expenditure

was devised as a means to justify and defend spending increases in the Spending Review. They were discontinued

as a way to make spending cuts easier under austerity. The recent emphasis in the United Kingdom on the

management of scal risks and of the government balance sheet (asset management) was an institutional response

to a policy emphasis on (continuing) cuts and austerity, as well as a response to large outsourcing failures and

greater post-crisis surveillance (Hughes, 2019).

10.

“When evaluations are commissioned ex ante with a specic objective to identify budgetary savings across

government, however, they require some specic characteristics in order to full this role. These kinds of

evaluations, or spending reviews, can be used to reduce the decit and/or to make scal space for higher priority

programmes either through restructuring or cutting activities. While spending reviews can also look at programme

effectiveness, i.e. how to improve programme performance and design, their distinguishing feature is the focus

on identifying and extracting savings through the budget process” (Lau, 2011).

11.

The four dimensions proposed are political, social, organizational, and process (Catalano & Erbacci, 2018).

12.

Experiences with pro-poor budgeting suggest that there is a dilemma about how starkly to reveal the trade-offs

entailed: too stark and vested interests would ensure that technical warfare over models and assumptions would

engulf the budgeting process, too nuanced and the impact would be lost. The question is how to make distributive

impacts a standard feature of policy making and reporting. A gentle start would be to add impact statements to

the budget and government accounts—but the challenge is to nd a stronger and more universal version of the

arrangements attempted during the pro-poor budgeting era.

13.

Elson (1999) gives a avor of this in her suggestions concerning the adjustments needed for effective gender

budgeting:

1. Gender-aware policy appraisal

This analytical approach involves scrutinizing the policies of different portfolios and programs by

paying attention to the implicit and explicit gender issues involved. It questions the assumption that

policies are ‘gender neutral’ in their effects and asks instead: In what ways are the policies and their

associated resource allocations likely to reduce or increase gender inequalities?

130

2. Gender-disaggregated beneciary assessments

This research technique is used to ask actual or potential beneciaries the extent to which government

policies and programs match their priorities.

3. Gender-disaggregated public expenditure incidence analysis. This research technique compares

public expenditure for a given program, usually with data from household surveys, to reveal

distribution of expenditure between women and men, girls and boys.

4. Gender-disaggregated tax incidence analysis

This research technique examines both direct and indirect taxes in order to calculate how much

taxation is paid by different individuals or households.

5. Gender-disaggregated analysis of the impact of the budget on time-use.

This looks at the relationship between the national budget and the way time is used in households.

This ensures that the time spent by women in unpaid work is accounted for in policy analysis.

6. Gender-aware medium-term economic policy framework

This attempts to incorporate gender into the economic models on which medium-term economic

frameworks are based.

7. Gender-aware budget statement

This process may utilize any of the above tools. It requires a high degree of commitment and

coordination throughout the public sector as ministries or departments undertake an assessment of

the gender impact of line budgets. Elson (1999), cited in O'Hagan & Klatzer (2018, p. 24).

14.

“Reforms in public expenditure management and in donor practice, combined in the pursuit of poverty reduction

strategies, offer some grounds for hope of increased effectiveness in pro-poor social sector expenditure

programmes. However, the implementation of reforms will be a slow process and resource reallocation to favour

high impact programmes is likely to meet institutional and stakeholder resistance” (Roberts, 2003, p. xi).

15.

A comparable initiative in climate budgeting, led by the UNDP and the World Bank in the late 2000s and early

2010s, revealed similar challenges and was also inconclusive on the evidence (World Bank, 2014a, 2014b).

16.

There has been renewed interest in OECD countries—especially following the global nancial crisis—in evidence

and lessons on spending reviews to identify savings and reallocate them to higher priorities, as well as in the

development of a framework to capture dimensions that support “cutback management” in public expenditure

(Catalano & Erbacci, 2018; Lau, 2011; Robinson, 2013).

17.

“The notion that scal consolidations can be expansionary (i.e., raise output and employment), championed,

among others, in the academic world by Harvard economist Alberto Alesina or in the policy world by former

European Central Bank President Jean-Claude Trichet, has been seriously debunked. Instead, the short-run costs

must be balanced against the potential longer-term benets that consolidation can confer” (Ostry et al., 2019,

p. 78). Clements, de Mooij, Gupta & Keen (2015) “do not suggest that countries should not undertake scal

consolidation…however…the benets of scal adjustments should be weighed against their likely distributional

impact… History shows that scal plans succeed when they permit “some exibility while credibly preserving

the medium term consolidation objectives”… (and) “equity considerations suggest that a larger share of the

adjustment burden could be borne by the rich, which could be achieved through revenue measures targeted at the

higher income segments of the population” (Clements et al., 2015, p. 156). “…the overall effect of redistribution

is pro-growth, with the possible exception of extremely large redistributions. There is no negative direct effect,

and the resulting lower inequality seems to be associated with longer growth spells” (Ostry et al., 2019, p. 105).

18.

Fiscal biases play out very differently in advanced and low-income economies. In advanced economies, when

the economy stagnates, built-in or discretionary scal policies enable an increase in decit spending. When low-

income countries stagnate (often because of a decline in the market price of critical commodities), the opposite

occurs: the government is compelled to tighten its belt, thereby aggravating economic misery. The double

standard that allows rich countries to use decit nancing and compels poor countries to austerity is not itself due

to PFM, but it is critical that PFM institutions be mobilized to counter this bias.

19.

See also European Union surveillance reports for a review of scal structural reforms (European Commission,

2019). There is much discussion of the tension between the approach adopted by the troika in Greece and the

softer advocacy of IFIs. Ostry et al. (2019) write: “…on scal policy, the IMF’s then-chief economist Olivier

Blanchard said ‘what is needed in many advanced economies is a credible medium-term scal consolidation, not

a scal noose today’ (IMF, 2010). In October 2013, the IMF’s managing director Christine Lagarde applauded

the decision by the U.S. Congress to raise the country’s debt ceiling. On the pace of U.S. scal consolidation,

Lagarde advised: ‘we say slow down because the point is not to contract the economy by slashing spending

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

131

brutally now as recovery is picking up’ (Howell, 2013). For the euro area, the IMF advocated that ‘those with

scal space should use it to support investment’” (p. 108).

20.

“Austerity could easily have spared the poor, if the political will had existed to do so. Resources were available to

the Treasury at the last budget that could have transformed the situation of millions of people living in poverty…”

(Alston, 2018, pp. 22–23).

21.

The primary argument for increased public spending on infrastructure is that it will promote growth through the

impact of improved transport, water supply and sanitation, and information and communications technology on

the productivity of private enterprise and the rate of return on capital, particularly when stocks of infrastructure

assets are starting from a relatively low base. Reliable power grids and well-maintained roads reduce the need for

the private sector to spend on maintenance of its own stock of physical capital and have a direct impact on health

and education outcomes. This previously theoretical claim is now backed up by some growing empirical evidence

and has been given momentum by China’s “Belt and Road” infrastructure, trade, and investment initiative, which

has attracted the participation of more than 70 countries since its launch in 2013. The initiative has benetted

from China’s framing of its engagement with the developing world as one of “non-interference” in implied

contrast to the ideologically driven traditional donor model promoting democratic governance and human rights

(Abiad, Furceri & Topalova, 2015; Agénor & Moreno-Dodson, 2006; Lin & Wang, 2017).

22.

Questions about inclusion and broader well-being are now being more actively researched and debated in the

context of economic prosperity. The case is growing for “inclusive prosperity,” under which “the ‘inclusive’

modier demands both that we consider the whole distribution of outcomes, not simply the average, and that we

consider prosperity broadly, including non-pecuniary sources of well-being, from health to climate change to

political rights” (Naidu et al., 2019, p. 1).

23.

Musgrave and Musgrave (1989) write:

As noted before, budget policy involves a number of distinct objectives, but these overlap in practice,

thereby complicating an efcient policy design, i.e., a design which does justice to its diverse goals.

Suppose rst that the public wishes an increased supply of public services. Increased taxes are needed

to pay for these, which leads in turn, to the question of how they should be distributed. Depending on

what taxes are used, taxation may well change the distribution of income that remains available for

private use. Hence some voters may favor (reject) the proposed change in public services because they

like (dislike) the associated change in distribution rather than because they like (or dislike) the public

service. Ideally, the two issues would be separated: Society would provide for what is considered a

fair state of distribution and then adjust the nancing of public services in line with the benets which

taxpayers derive there-from. Because this two-step procedure is difcult to accomplish, decisions on

the provision of public services tend to be mixed with and distorted by distributional considerations.

Similar reasoning also applies in the reverse direction, when the supply of public services and hence

taxes are reduced.

Next suppose that society wishes to shift distribution in the direction of greater (lesser) equality. Such

a shift may be accomplished by using progressive (regressive) taxes to nance transfers to lower

(higher) incomes. But it may also be done by increasing (reducing) the supply of public services of

particular value to low (high) income groups. This, however, interferes with the pattern of public

services which consumers want to obtain at a given distribution of income. Once more, one policy

objective may be implemented such that it interferes with another.

Finally, consider the role of scal policy in stabilization. Suppose that a more (less) expansionary

policy is needed. This may be accomplished by raising (lowering) outlays on public services or by

reducing (raising) the level of taxation. In the former case the allocation objective of scal policy is

interfered with, whereas in the latter it is not. However, in the latter case there is the further question

of how changes in the level of taxation are to be implemented. For stabilization measure to be neutral

regarding both allocation and distribution goals, proportional changes in the level of taxes might offer

the appropriate solution.

As we will see in the course of this study, there are many exceptions which call for qualication of

the simple rules just given. Nevertheless, it is important to keep in mind that there are three distinct

policy objectives and policy should try to minimize conicts among them. (pp. 13–14).

24.

Concerning the taxation-accountability relationship, Welham et al. (2013) argue that some of the political science

accounts of how states came about in different historical contexts have been picked up by the development policy

community and employed to understand contemporary states. They note that the most prominent account suggests

132

that today’s states grew out of a mutually reinforcing process of tending to the needs of warfare, which led to

ever-rising taxes, which in turn eventually prompted increasing accountability to taxpayers (Tilly, 1992). The

observation that the nature of a state’s tax regime strongly inuences governance goes back to Schumpeter (1918),

cited in Welham et al. (2013), and has been applied by Moore (2013) to contemporary developing countries. The

detrimental effect of not needing to tax one’s own citizens, mainly in the context of natural resource wealth, has

been much debated in economics and political science (Collier & Hoefer, 2005; Ross, 1999).

25.

PFM and scal policy responses to the decit-nanced spending hikes in the aftermath of the Arab Spring risk

adding to the very social pressures that led to the unrest in the “Arab Countries in Transition” (Finger & Gressani,

2014).

26.

An open government in the OECD is understood as one under which businesses, civil society organizations,

and citizens have increased their power to know what has been decided (transparency), to obtain their legitimate

service entitlement (accessibility), and to be heard (consultation and participation) (OECD, 2005). Freedom of

information legislation has been important in establishing rights of access to information and has been adopted

by over 90 percent of OECD member countries. More “active” forms of openness involve publishing government

datasets (Open Data), giving citizens greater opportunities to engage in the design and delivery of services (Open

Services), and engaging with citizens in policy making (European Commission, 2013).

The G8 Open Data Charter implicitly recognizes that making government data more open has an impact

on improved governance and public sector innovation only if those data can be reused by others outside of

government to interact with the public sector and create new services. For this to happen, datasets need to be open

(unless there is a compelling reason against this), shareable, open-format, and machine-readable to allow mass

processing (Ubaldi, 2013).

Chapter 5

To take the United Kingdom as an example, “…if development were only about aid, then someone would be

bound to ask whether we need a Cabinet-level minister to manage aid to twenty or thirty mostly small low-

income countries” (Maxwell, 2013).

Annex B

This typology of instruments available to governments follows the seminal classication by Vedung (2007),

which distinguishes among economic means (“carrots”), regulations (“sticks”), and information (“sermons”).

Amid many denitional uncertainties, this paper follows Sturgess (2018) in using “commissioning” to mean the

assessment of a need and the design of a response in addition to its funding.

OECD (1997) gives the example of a health service spending money on outputs that evidence-based medicine

suggests are useless—for example, most tonsillectomies. So even if lots of tonsillectomies are being performed

and we see movement toward the social objective of improved public health, since no causality can be attributed

to the surgeries, the quality of tonsillectomies as an output is zero.

Vedung (2007) addresses the question of what is the difference between a tax (economic policy action) and

a regulation (administered action), since both involve a “measure of authority or obligatory force” (p. 10).

Taking the example of tobacco purchasing, his explanation is that “a prohibition forbids the action proper—

tobacco purchasing—while the obligation…applies to the payment of the tax when you purchase cigars, not the

purchasing itself.” In other words, the economic policy action allows certain behaviors but might make them

more expensive, while the administered action can ban it outright under certain circumstances.

Annex D

Sectors are:

• High-level grouping of economic activities based on the on the types of goods and services produced;

• Used to indicate which part of the economy is supported by the World Bank intervention; and

• Mutually exclusive.

Sectors are not:

• Reections of the World Bank’s administrative structure;

• Used to indicate methods of delivery of World Bank support or ways of doing business; or

Endnotes

“Advice, Money, Results – Rethinking International Support for Managing Public Finance”

International Working Group Report, New York University

133

• Measures of outcomes (such as corporate priorities).

(World Bank, 2016a, p. i).

Themes are:

• Goals and objectives of World Bank-supported activities;

• Policy areas; and/or

• Used to capture World Bank support to the Sustainable Development Goals.

Themes are not:

• Methods or instruments of delivery of World Bank support or ways of doing business; or

• Mutually exclusive.

(World Bank, 2016b, p. i).

More specically, the themes included in the charts are:

• Debt Management and Fiscal Sustainability

• Macroeconomic Management

• Municipal Finance

• Public Expenditure, Financial Management, and Procurement

• Tax Policy and Administration

Annex E

ODA refers to nancial ows (both grants and loans) to countries on the DAC list of ODA recipients and

multilateral institutions: (a) that are provided by ofcial agencies, including state and local governments or

by government agencies; and (b) for which each transaction is administered with the promotion of economic

development and welfare as the main objective, is concessional, and conveys a grant element of at least 25

percent (discount rate of 10 percent) (AlphaBeta, 2018, p. 7).

Data prior to 2002 are incomplete and reportedly reect only 70 percent of donors’ ODA reporting.

The Hudson Institute indicates that the total of private ows (US$ 801 billion in 2014) are more than ve times

greater than ofcial ows (US$ 147 billion, including US$ 137 billion from DAC donors and US$ 10 billion from

11 non-DAC donors over the same period). However, these totals hide the obvious point that the majority of these

private ows are comprised of foreign direct investment and remittances. Nevertheless, private philanthropy was

US$ 64.4 billion for that year, of which US$ 63.7 billion was from DAC donors (Hudson Institute, 2016).

Annex I

This is a reference to the consensus that Williamson identied as emerging de facto, with its emphasis on a

paradigmatic shift in favor of macroeconomic stabilization and market-based development—not the more

ideological version that subsequently formed the basis for much controversy (Birdsall, de la Torre & Valencia

Caicedo, 2010).

Annex K

See, for example:

• The Elders (https://www.theelders.org/)

• “Brain Trust,” such as that of Franklin Roosevelt in the United States in the 1930s (https://www.nytimes.

com/1932/11/20/archives/the-cabinet-mr-roosevelt-already-has-the-group-of-advisers-who.html)

• Economic policy “Brain Trust” to Shinzo Abe in Japan in 2012 (Abb, Köllner, and Maslow, 2016).

• G20 Eminent Persons Group on Global Financial Governance (https://www.globalnancialgovernance.org/

about-g20-epg/)

• Independent Challenge Group for the 2010 Spending Review in the UK (https://webarchive.nationalarchives.

gov.uk/20130102173537/http://www.hm-treasury.gov.uk/spend_icg_members.htm)

• “Wise Men” of economic forecasting in 1990s in the United Kingdom (https://www.independent.co.uk/news/uk/

the-not-quite-so-magnicent-seven-the-treasury-plans-to-set-up-a-panel-of-outside-advisers-robert-1563383.

html)

• External Advisory Group for the Review of the IMF’s Capacity Development Strategy (https://www.imf.org/~/

media/Files/Publications/PP/2018/pp1002182018reviewfunds-cdstrategyextadvisory.ashx)

134

• “Iwakura Mission” by Japan to the United States and Europe in the 1870s (https://en.wikipedia.org/wiki/

Iwakura_Mission)

• “Tswalu Dialogues” on economic development run by the Oppenheimer family and the Brenthurst Foundation

(http://www.thebrenthurstfoundation.org/what-we-do/)

• “Balcerowicz Plan” and commission of experts to advise the Polish nance minister in 1989 (https://

en.wikipedia.org/wiki/Balcerowicz_Plan)

See OECD DAC peer reviews of members (http://www.oecd.org/dac/peer-reviews/);

IMF Article IV reviews (https://www.imf.org/external/about/econsurv.htm); and

EC Structural Reform Support Service (https://ec.europa.eu/info/departments/structural-reform-support-service_

en).

Individual examples include Joseph Stiglitz advising the Papandreou government in Greece during the Eurozone

crisis, Jeffrey Sachs advising the Polish government from 1989 during the economic transition, Paul Collier in

Uganda in the 1990s, and Steven Radelet in Liberia in 1990s and 2000s. Independent agencies include AGI/TB-

IGC, CABRI, HKS-BSC, HIID, ODI, IGC, Duke, ACET, NIPFP, ANU-DPC, Fiscus Ltd, and IFS.

Based on the personal experience of one of the principal authors of the present paper, who participated in such an

engagement in China.

For further discussion of the incentives at play in lending operations and provision of policy advice between the

Government of China and the World Bank, as one example of a multilateral development nancing and advisory

organization, see discussion by the panellists at a CGD event in April 2019 (https://www.cgdev.org/event/future-

world-bank-under-next-president).

International Growth Centre research and advice in Ethiopia on economic policy (https://www.theigc.org/impact/

mapping-industrialisation-africa/) and (https://www.theigc.org/publication/an-enterprise-map-of-ethiopia-by-

john-sutton/).

G7 CONNEX Initiative and CONNEX Support Unit (http://connex-unit.org/)

Endnotes