Introduction to Instrumental Variables, Part One

MIT's Josh Angrist introduces one of econometrics most powerful tools: instrumental variables.

Instrumental variables (IV, for those in the know), allow masters of econometrics to draw convincing causal conclusions when a treatment of interest is incompletely or imperfectly randomized.

For example, arguments over American school quality often run hot, boiling over with selection bias. See a school with strong graduation rates and enticing test scores? Is that a good school or just an ordinary school fortuitously located in a good neighborhood?

Lotteries that randomize offers of a school seat at in-demand schools should unravel the school quality conundrum. But lotteries only offer seats. Families are free to accept or go elsewhere and these choices are far from random.

IV provides a path to causal conclusions even in the face of this sort of incomplete randomization.
 

In this video, we cover the following:

  • Incomplete random assignment
  • IV terminology: first stage, second stage, instrument, reduced form
  • Three key IV assumptions: substantial first stage, independence assumption, exclusion restriction

Teacher Resources

Transcript

The path from cause to effect is dark and dangerous. But the weapons of econometrics are strong. Attack with fierce and flexible instrumental variables when nature blesses you with fortuitous random assignment.

Randomized trials are the surest path to ceteris paribus comparisons. Alas, this powerful tool is often unavailable. But sometimes, randomization happens by accident. That's when we turn to instrumental variables -- IV for short. Today's lesson is the first of two on IV. Our first IV lesson begins with a story of schools. 

Charter schools are public schools freed from daily district oversight and teacher union contracts. The question of whether charters boost achievement is one of the most important in the history of American education reform. The most popular charter schools have more applicants than seats, so the luck of a lottery draw decides who's offered a seat. A lot is at stake for the students vying for their chance, and waiting for the lottery results brings up lots of emotions as was captured in the award-winning documentary "Waiting For Superman." 

Do charters really provide a better education? Critics most definitely say no, arguing that charters enroll better students to begin with, smarter or more motivated, so differences in later outcomes reflects selection bias.

Wait, this one seems easy. In a lottery, winners are chosen randomly, so just compare winners and losers. 

On the right track, Kamal, but charter lotteries don't force kids into or out of a particular school -- they randomize offers of a charter seat. Some kids get lucky. Some kids don't. If we just wanted to know the effect that charter school offers, we could treat this as a randomized trial. But we're interested in the effects of charter school attendance, not offers. And not everyone who is offered, accepts. IV turns the effect of being offered a charter seat into the effect of actually attending a charter school. 

Let's look at an example, a charter school from the Knowledge Is Power Program, or KIPP for short. This KIPP school is in Lynn -- a faded industrial town on the coast of Massachusetts. The school has more applicants than seats and therefore picks its students using a lottery. 

From 2005 to 2008, 371 fourth and fifth graders put their names in the KIPP Lynn lottery, 253 students won a seat at KIPP, 118 students lost. A year later, lottery winners had much higher math scores than lottery losers. But remember, we're not trying to figure out whether winning a lottery makes you better at math. We want to know if attending KIPP makes you better at math. 

Of the 253 lottery winners, only 199 actually went to KIPP. The others chose a traditional public school. Similarly, of the 118 lottery losers, a few actually ended up at KIPP. They got an offer later. So what was the effect on test scores of actually attending KIPP? 


Why can't we just measure their math scores?

Great question. Who would you compare them to? 

Those who didn't attend.

Is attendance random?

No. - Selection bias. 

Correct. 

The KIPP offers are random so we can be confident of ceteris paribus, but attendance is not random. The choice to accept the offer might be due to characteristics that are related to math performance -- say, for example, that dedicated parents are more likely to accept the offer. Their kids are also more likely to do better in math, regardless of school. 

IV converts the offer effect into the effect of KIPP attendance, adjusting for the fact that some winners go elsewhere and some losers manage to attend KIPP anyway. Essentially, IV takes an incomplete randomization and makes the appropriate adjustments. How? IV describes a chain reaction. Why do offers affect achievement? Probably because they affect charter attendance, and charter attendance improves math scores. 

The first link in the chain called the first stage is the effect of the lottery on charter attendance. The second stage is the link between attending a charter and an outcome variable -- in this case, math scores. The instrumental variable, or "instrument" for short, is the variable that initiates the chain reaction. The effect of the instrument on the outcome is called the reduced form. This chain reaction can be represented mathematically. We multiply the first stage, the effect of winning on attendance, by the second stage, the effect of attendance on scores. And we get the reduced form, the effect of winning the lottery on scores. The reduced form and first stage are observable and easy to compute. However, the effect of attendance on achievement is not directly observed. This is the causal effect we're trying to determine. Given some important assumptions we'll discuss shortly, we can find the effect of KIPP attendance by dividing the reduced form by the first stage. This will become more clear as we work through an example. 

A quick note on measurement. We measure achievement using standard deviations, often denoted by the Greek letter sigma (σ). One σ is a huge move from around the bottom 15% to the middle of most achievement distributions. Even a ¼ or ½ σ difference is big.

Now we're ready to plug some numbers into the equation we introduced earlier. 

First up, what's the effect of winning the lottery on math scores? KIPP applicants' math scores are a third of a standard deviation below the state average in the year before they apply to KIPP. But a year later, lottery winners score right at the state average, while the lottery losers are still well behind with an average score around -0.36 σ. The effect of winning the lottery on scores is the difference between the winners' scores and the losers' scores. Take the winners' average math scores, subtract the losers' average math scores, and you will have 0.36 σ. 

Next up: what's the effect of winning the lottery on attendance? In other words, if you win the lottery, how much more likely are you to attend KIPP than if you lose? 

First, what percentage of lottery winners attend KIPP? Divide the number of winners who attended KIPP by the total number of lottery winners -- that's 78%. To find the percentage of lottery losers who attended KIPP, we divide the number of losers who attended KIPP by the total number of lottery losers -- that's 4%. Subtract 4 from 78, and we find that winning the lottery makes you 74% more likely to attend KIPP. 

Now we can find what we're really after -- the effect of attendance on scores, by dividing 0.36 by 0.74. Attending KIPP raises math scores by 0.48 standard deviations on average. That's an awesome achievement gain, equal to moving from about the bottom third to the middle of the achievement distribution.

These estimates are for kids opting in to the KIPP lottery, whose enrollment status is changed by winning. That's not necessarily a random sample of all children in Lynn. So we can't assume we'd see the same effect for other types of students. But this effect on keen for KIPP kids is likely to be a good indicator of the consequences of adding additional charter seats.

IV eliminates selection bias, but like all of our tools, the solution builds on a set of assumptions not to be taken for granted. 

First, there must be a substantial first stage -- that is the instrumental variable, winning or losing the lottery, must really change the variable whose effect we're interested in -- here, KIPP attendance. In this case, the first stage is not really in doubt. Winning the lottery makes KIPP attendance much more likely. Not all IV stories are like that. 

Second, the instrument must be as good as randomly assigned, meaning lottery winners and losers have similar characteristics. This is the independence assumption. Of course, KIPP lottery wins really are randomly assigned. Still, we should check for balance and confirm that winners and losers have similar family backgrounds, similar aptitudes and so on. In essence, we're checking to ensure KIPP lotteries are fair with no group of applicants suspiciously likely to win. 

Finally, we require the instrument change outcomes solely through the variable of interest -- in this case, attending KIPP. This assumption is called the exclusion restriction.

IV only works if you can satisfy these three assumptions. 
I don't understand the exclusion restriction. How could winning the lottery affect math scores other than by attending KIPP?

Great question. Suppose lottery winners are just thrilled to win, and this happiness motivates them to study more and learn more math, regardless of where they go to school. This would violate the exclusion restriction because the motivational effect of winning is a second channel whereby lotteries might affect test scores. While it's hard to rule this out entirely, there's no evidence of any alternative channels in the KIPP study.

IV solves the problem of selection bias in scenarios like the KIPP lottery where treatment offers are random but some of those offered opt out. This sort of intentional yet incomplete random assignment is surprisingly common. Even randomized clinical trials have this feature. IV solves the problem of non-random take-up in lotteries or clinical research. But lotteries are not the only source of compelling instruments. Many causal questions can be addressed by naturally occurring as good as randomly assigned variation. 

Here's a causal question for you: Do women who have children early in their careers suffer a substantial earnings penalty as a result? After all, women earn less than men. We could, of course, simply compare the earnings of women with more and fewer children. But such comparisons are fraught with selection bias. If only we could randomly assign babies to different households. Yeah, right, sounds pretty fanciful. 

Our next IV story -- fantastic and not fanciful -- illustrates an amazing, naturally occurring instrument for family size.
 

Subtitles

Verified Available Languages
  • English
  • Spanish
  • Chinese


Thanks to our awesome community of subtitle contributors, individual videos in this course might have additional languages. More info below on how to see which languages are available (and how to contribute more!).

How to turn on captions and select a language:

  1. Click the settings icon (⚙) at the bottom of the video screen.
  2. Click Subtitles/CC.
  3. Select a language.
     


 

Contribute Translations!

Join the team and help us provide world-class economics education to everyone, everywhere for free! You can also reach out to us at support@mru.org for more info.


Submit subtitles

 

 

Accessibility

We aim to make our content accessible to users around the world with varying needs and circumstances.

Currently we provide:


Are we missing something? Please let us know at support@mru.org

Download

Creative Commons

Creative Commons License

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
The third party material as seen in this video is subject to third party copyright and is used here pursuant
to the fair use doctrine as stipulated in Section 107 of the Copyright Act. We grant no rights and make no
warranties with regard to the third party material depicted in the video and your use of this video may
require additional clearances and licenses. We advise consulting with clearance counsel before relying
on the fair use doctrine.