One Size Does Not Fit All: Towards More Nuanced Evaluation of Policy Effectiveness?

By Chris Klein

Concerns over climate change, pollution reduction and the secure availability of fossil fuels has led an increasing number of industrialized countries to support public financing of renewable energies (RES-E). In politics, any such policy intervention is usually accompanied by intense debate about its normative desirability: either you are for a policy or you are against it. Unfortunately, this binary reasoning appears to have crept into scholarship as well; empirical studies that have attempted to evaluate the effectiveness and efficiency of RES-E policies across different jurisdictions have usually relied on binary indicator (dummy) variables to represent RES-E policies within their statistical models. These dummy variables account only for whether or not a country or state has a certain form of RES-E policy in place; however, they say nothing about the quality of the policy, the specific characteristics of the country in question, or any changes that may occur over time in the incentives that the policy provides. In policy – as in most parts of our lives – the devil is very much in the details.

The Feed-in Tariffs vs. Renewable Portfolio Standards debate

The basic premise of renewable energy promotion policies is that they create demand for climate-friendly technologies that otherwise would not exist, at least at desired levels, under current market conditions. The two most common approaches to stimulating demand for RES-E are quotas, such as the Renewable Portfolio Standards (RPS) that are common in the U.S., and Feed-in Tariffs (FITs), which have been adopted by most European countries. Quotas are command and control instruments that regulate the quantity of RES-E generation, requiring utilities to generate a certain portion of their electricity from renewable sources. FITs regulate the price of RES-E generation, increasing the payments received by RES-E producers for each unit of electricity produced.

The European Union ended an internal debate about the superiority of either approach in 2008, concluding that FITs were the superior instrument for stimulating investment into RES-E generation capacity, largely due to the high level of security that FIT schemes provide to investors. This conclusion was largely based on the analysis of a handful of success stories, namely Denmark, Spain and Germany, whereas perception of the alleged “failure” of RPS policies was based on the rather ineffective policy interventions of the United Kingdom and various U.S. states. What was lacking from this debate, however, was a truly rigorous econometric analysis of these policies, across borders and over time.

The Road Ahead

But rejoice, there is a light on the horizon. In 2009, Yin and Powers developed a measure for the strength of state-level Renewable Portfolio Standards in U.S. states to determine the effectiveness of such policies in incentivizing investment in new RES-E generation capacity. This new indicator explicitly takes into account the heterogeneity in coverage and existing capacity, thus capturing the size of the new incentives created by these policies more accurately than previous studies. Among Yin and Powers’ key findings: some seemingly aggressive RPS policies actually provide only weak incentives, while other seemingly moderate RPS policies are in fact relatively ambitious. Most importantly, their paper highlights the importance of assessing each policy within its political, geographical, institutional and economic context.

Following in Yin and Powers’ footsteps, Steffen Jenner, Felix Groba and Joe Indvik presented their research on a new indicator for the strength of European FIT legislations at the 30th U.S. Association of Energy Economists conference in Washington, D.C. on October 12. The new indicator captures variability in tariff size, contract duration, digression rate, wholesale electricity price and electricity generation cost to estimate the return on investment (ROI) provided by each specific FIT regime. In their forthcoming paper, they employ panel data for 1998-2008 to assess the effectiveness of FIT policies in promoting solar photovoltaic (PV) and onshore wind power development in EU countries and find positive effects of FIT policies. More specifically, they argue that the design of each policy is a more important determinant for RES-E generation than the enactment of policy alone.

Jenner, Groba and Indvik’s paper has not yet been accepted by a peer-reviewed journal, and whether their indicator will be validated by the academic community remains to be seen. Nonetheless, their attempt to make FIT policies quantifiable, and thus comparable, across different jurisdictions and times is an important step towards a more rigorous quantitative evaluation of European FIT legislation, just as Yin and Powers’ contribution was for the evaluation of U.S. state-level RPS regulations. Admittedly, the proper construction of variables in policy evaluation research may not look like a giant leap forward in the long-running campaign to build a greener planet; however, in times where we don’t know exactly where the next giant leap is going to come from, these smaller steps are absolutely crucial.

 

 

+ posts

Established in 1995, the Georgetown Public Policy Review is the McCourt School of Public Policy’s nonpartisan, graduate student-run publication. Our mission is to provide an outlet for innovative new thinkers and established policymakers to offer perspectives on the politics and policies that shape our nation and our world.

2 thoughts on “One Size Does Not Fit All: Towards More Nuanced Evaluation of Policy Effectiveness?

  1. Nice review of the literature. Innovation in policy measurement is really crucial and complicated. Its so much easier just to ignore the complexity and see if you can get signficance from dummy variables.

    Now I’ll go back to coding Dodd Frank regulatory announcements as 1’s or 0’s. Sigh.

  2. Excellent post, Chris! We do indeed hope that our FIT strength indicator will carve out a place in academic discourse. It has been well-received so far, but peer-reviewed publication is still pending. I absolutely agree that acknowledging that not all policies are created equal is crucial—in academic, policy, and public discourse alike.

Comments are closed.