Wednesday, November 02, 2005

excerpted from my causalist manifesto

Chris! I spent my whole walk to work this morning thinking about the question of whether restoring voting rights would decrease the likelihood of an ex-felon returning to crime after being released. This, despite not actually reading your paper, and despite not knowing anything about the larger literature in the area. And now, despite all this ignorance, I'm going to write a long post about it!

Readers who are not Chris should keep in mind, I have had two separate institutional occasions to familiarize myself with the Uggen oeuvre, and I have enormous respect for the work he's done and what he's trying to do. I'm sure he's thought about most/all the same issues as I have and in more depth to boot. So this isn't a directed criticism at him or at others at all. Indeed, in writing down my thoughts, I considered changing the example to something entirely different just to make plain that I wasn't pretending to offer informed commentary on the work of Chris and others in this area, but then I decided not to, partly because this rumination has already taken up enough of my morning.

Instead, as is usual for this blog, it's All About Me. Specifically, it's about me thinking through why I am such a causal pessimist. So, allow me a morning stroll through how I think about things in terms of imagining the question of whether voting rights reduce recidivism in stylized terms mostly innocent of knowledge of the actually available data and evidence.

Consider: Frankie and Johnny are both ex-felons. In terms of the measures we have on them, Frankie and Johnny are identical. Frankie votes, Johnny doesn't. A year later, Frankie hasn't had any further trouble with the law, while Johnny knocked over a liquor store and is back in jail. If Johnny had voted instead of Frankie, would their positions be reversed?

Aggregate to: Frankie and Johnny are just two members of a large sample of ex-felons. In the whole sample, though, everybody can be divided into pairs like Frankie and Johnny, where they are alike on all other pre-election-day measures, except that the Frankies voted and the Johnnies did not. A year later, 20% of the Frankies are back in jail, as opposed to 30% of the Johnnies. If the Johnnies had voted and the Frankies had not, would these percentages be reversed? (Or, more fairly, if the Johnnies had the reintegrative experience manifested in voted and the Frankies had not, would the percentages be reversed?)

In a nutshell: the whole inference that there is something causal about voting turns on the idea that Frankie and Johnny really were otherwise alike, just that one voted and the other didn't. (If you know from the start that the Frankies and Johnnies are different, we can engage in strategies of "covariance adjustment" to attempt to account for these differences, but this results in an even weaker inference than the hypothetical matched-pairs scenario, so if you are already a causal pessimist in the matched-pairs scenario, you are necessarily even more of a pessimist when covariance adjustment is required.)

The problem: If they were so alike, why did Frankie vote and Johnny not? I mean, isn't Frankie practically screaming, "Yo! Don't treat me like I'm identical to Johnny! He spent election day watching TV, and I went out and voted!" Fundamentally, it doesn't matter how many different ways (as manifested in variables) that you have showing how Frankie and Johnny are alike. Perversely even, to some extent, more measures can be misleading, to whatever extent they can foster the illusion that Frankie and Johnny really were alike, except that one voted and the other didn't. So long as the decision to vote is highly nonrandom and says something about the person who chooses to vote, Frankie and Johnny are revealing an important difference between the two of them by the very fact that one voted and the other didn't.

A digression: In a different-but-analogous scenario, this is the closest I've come to lapsing into profanity in my graduate methods class. We were looking at a paper on neighborhood effects, where the whole inference turned on the idea that two kids entering junior high were otherwise identical (or adjustedly identical), except that one lived in a bad neighboorhood and the other lived in a good neighborhood. To which I said, animatedly, "If they are so freaking alike, why do they live in different neighborhoods?" And then, even more loudly and slowly: "If they are so freaking alike, why do they live in different neighborhoods?"

The situation would be more promising if: You had two groups of Frankies and Johnnies who were otherwise identical, and the Frankies could vote and the Johnnies couldn't. If you have reason to think the Johnnies would have voted in the same proportion if only they could vote--which, if they were really otherwise identical, they would--then a lower recidivism rate for the Frankies vs. the Johnnies is possibly compelling. (It depends on how well one can make the "otherwise identical" case, especially since, for example, if the Frankies live in one state and the Johnnies live in another, you have to start worrying about the ways in which the state affects their probability of re-imprisonment, etc., etc.)

Barring that, the situation is not so bleak if: You had two groups of Frankies and Johnnies who were otherwise identical, except the Frankies happened to be in a circumstance that exerted an effect on their probability of voting (or, really, their average level of electoral-related civic integration) without at all affecting their probability of recidivism. And so, the Hunt For Exogenous Variation is on. Say the Frankies were released in the year preceding various presidential elections, and the Johnnies were released in the year after various presidential elections. It might be compelling to see that the Frankies had lower recidivism rates than the Johnnies. (One problem here is that elections are also known affect the probability of recidivism, as when mayors increase policing in their own election years to appeal to crime-antsy voters.)

Additionally, there may be the opportunity to Disaggregate And Conquer: Frankies and Johnnies are otherwise identical. Imagine that there are Class A felonies and Class B felonies. Frankies can only vote if they have committed a Class A felony, not if they've committed a Class B felony. Johnnies cannot vote regardless. We compare the recidivism rates for Class A and Class B felons. Whatever the relative difference in recidivism rates for Class A felons vs. Class B felons among the Johnnies, the relative rate among the Frankies should be even lower. In other words, if recidivism rates were 30% for both Class A and Class B felons among the Johnnies, then we might be persuaded that voting matters if the rates among the Frankies were 20% for the Class A felons and 30% for the Class B felons.

And, all the while, the question has actually shifted: The larger animating question for policy is, "Does restoring voting rights to felons affect the recidivism rate?" That question depends actually on comparisons across states, so instead of Frankie and Johnny you have Frankansas and Johnnisota. You can imagine how hard that is, i.e., "If the two states are so freaking alike, why does one allow ex-felons to vote and the other doesn't?" So you pretty much have to change the question. The story of Frankie and Johnny begins with the mutation of the question to something like: "Does voting reduce the probability of recidivism?" or perhaps more elaborately "Does the sense of civic reintegration made possible by and manifested essentially in the act of voting reduce the probability of recidivism?" But it could be that restoring the voting rights to felons decreases recidivism by decreasing stigma or increasing reintegration regardless of whether ex-felons take advantage of the franchise by actually voting. This is especially scary because we could get results that show that voting doesn't matter for recidivism when restoring the right to vote (the real matter of policy) does matter. In other words, problems of causal inference can force us into addressing causal questions that are different from the ultimate questions of policy interest, and the questions can have discrepant answers.

A final irony of all this being: For my own beliefs about whether ex-felons should be allowed to vote, the answer to the question about recidivism doesn't matter on way or the other. I would probably even support restoring the right to vote to ex-felons if it increased their probability of recidivism. But, of course, there are people out there who might be persuaded to agree with the policy change if it was shown that restoring voting rights reduced recidivism rates. And what I ultimately care about is the policy change, not whether the reason the policy change gets enacted coincides with my own beliefs about the reasons the policy change is desirable. So I'm invested in the answer even though it's irrelevant to my own position.

11 comments:

Anonymous said...

Wednesday 12:01 AM?

Anonymous said...

I've decided that you're either a genius or someone who never sleeps. Perhaps both.

jeremy said...

Well, I'm certainly no genius, and sooner you disabuse yourself of that, the better for all of us. But I do have all kinds of trouble sleeping; it is, as I often say, the bane of my existence.

Allen said...

I have two questions:

1) Are Frankie and Johnnie also lovers?

2) Do your students constantly misread "causal" as "casual", or is that just something that happens at places like Southeast Missouri State, and not at Hahvahd?

3) Why is most of your post about whether the act of voting affects the probability of recidivism, when it appears that the issue is whether giving people the right to vote affects the probability of recidivism? Isn't the section titled "And, all the while, the question has actually shifted" actually the only part that addresses the question itself?

Allen said...

Well, three questions, then, if you're going to get all quantitative about it.

jeremy said...

Allen: The causal/casual mixup is everywhere, especially if you try to raise awareness of causal inference problems by trying to sponsor "causal Friday" events.

As for the matter of question-shifting, it seems to me that if one demonstrated that voting was truly causally associated with lower recidivism, then it would seem a likely (but not certain) bet that extending voting rights would be associated with lower recidivism. The inference doesn't go the other way, though.

Allen said...

I agree fully that if voting reduces recidivism, then giving ex-felons the right to vote would reduce recidivism. It just doesn't seem obvious to me at all that this is supposed to be the reasoning behind giving ex-felons the right to vote. It seems a lot more likely, if there is any causal connection, that it would be due to the "civic pride and acceptance" effect of saying "you're now a full member of society again" by giving them the right to vote. Even if they don't actually exercise it, just like large numbers of law-abiding citizens.

christopher uggen said...

jeremy, thanks for the kind words and attention! i'll respond in some detail when i post or send the new paper (i feel worse about not writing it yet than you do about not reading it yet). a couple quick notes, though:

1. you don't say much about timing or harnessing variation within-persons in voting and crime. a standard approach in the employment and crime literature is to estimate a fixed effects model that shows reduced crime during periods of employment. my guess is that you wouldn't buy that employment (or marriage, or...) is causal in this scenario. what if voting in the last biennial election is a strong time-varying covariate -- net of the fixed effect that should be absorbing stable differences across persons?

2. i think part of the reason for pessimism on this front relates to theory. i'll have to do a better job establishing the plausibility of a voting-crime connection. tocqueville, mill and expressivist theories of voting offer some help(virtue through civic participation), as does life course criminology and classic deviance work on insiders and outsiders. in any case, lacking a tight conceptual argument, it is hard to see the mechanism for such a connection.

3. as i noted yesterday, i'd like to manipulate the opportunity for civic reintegration experimentally with a "practical citizenship" prison intervention. prisoners assigned to treatment status would do some sort of inside/outside voting and service for community-based organizations or politicians that interest them. those assigned to the treatment status should be more likely to vote upon release (my intermediate outcome) and we could learn whether they'd be less likely to recidivate. i know that this wouldn't give the treatment effect of the physical act of voting, but i think it would give the treatment effect of a general "civic reintegration" manipulation with random assignment helping to assure comparability of t and c prior to the intervention. even with randomization, i could still use covariate adjustment for time-varying work and family effects (presuming that the experiment also affected these processes) and incomplete randomization.

of course, i'm interested in the effects of voting (that is, "voting per se") and i realize that i might be a lone pollyanna on the possibilities of "civic reintegration." ultimately, though, i want to identify ways to reduce crime and increase public safety. helping former criminals become stakeholders in the citizenry might offer a low-risk, low-cost means for attaining this goal. if the idea bombs with the application of successively more stringent tests, i'll keep looking (though civic reintegration seems pretty conservative compared to some ideas i'm kicking around now).

Brayden said...

Isn't this problem of finding causality in statistics the reason that bringing a good theory to the table is so important. In my mind, you begin with a strong theory (as I assume Chris has done) that has empirical implications. If you find evidence that a statistical relationship exists between two variables you theorized would have a relationship, you have some evidence to support the causal proposition of your theory. You haven't proven causality, which seems to be Jeremy's main concern, but you have provided support that the causal relationship may exist, assuming you've controlled for a number of other important predictors.

I don't think this goes far enough to establish causality, but it's an important start to linking theory with statistical evidence. I've wondered why in sociology we don't use two-stage least squares regression more. It would seem to get at some of the endogeneity/exogeneity issues that concern you Jeremy.

john said...

The act of voting belies civic reintegration or a sense of civic responsibility in an ex-felon, but as a measure of that responsibility, it seems largely unsatisfactory. There are definitely exogeneity issues relating to whether or not people vote, so recidivism relating to the right to vote is probably a more credible statistic relationship.

That's my best interpretation of the problem and situation, but I'm just a layman, so my grasp on this problem could be poor.

jeremy said...

Chris: I appreciate your comments, as always. With regard to your point #1, I have many thoughts on fixed-effects estimators with regard to two-sided selection processes like marriage and employment (I don't know to what extent, when the literature talks about these things, they talk about the inferential problems posed by two-sided selection). Maybe I'll get a chance to post sprawlingly on that, but of course blogging is what it is in terms of our ability to actually get the chance to articulate what we want to articulate.

As to your point #2, from my standpoint--and again I'm thinking about all this from an innocent outsider's purview--it's not the lack of plausibility of the voting-crime connection, necessarily, but more that once there are specific ideas about what the voting-crime connections is caused by, then suddenly it's easier to pin down the implications of what we would expect to see in observational data.

As to #3, I think this is a great idea, and a much more clearly defined treatment effect than voting is anyway, especially since it seems from the way you talk about it like it's not really voting per se anyway but the civic integration manifested in the participation in voting.

Brayden: I just see 2SLS as an estimation strategy for a class of solutions that rely upon a search for sources of exogenous variation in the causal variable, which is what I was trying to get at by talking about presidential year elections.