Scaling Text and Machine Learning

"An Experimental Approach to Measuring Ideological Positions in Political Text". (Online Appendix).

Though powerful as general tools, automated measures of position-taking in text often perform poorly when models of speech are difficult to develop or theoretically contested. Rather than model text, I develop an experimental approach to measure perceptions of partisanship in speech, with an application to 2008 Congressional advertisements. I randomly assign ads to subjects recruited in a large-N survey, and ask them to 'guess' the party of featured candidates, with ads scored as their average party inference. These party perception scores are empirically synonymous with a liberal-conservative dimension, and highly reliable across samples and experimental conditions. Party identity has little impact on guesses, indicating the inferential task significantly mutes partisan bias. For validation, I assess which words influence guessing, and whether ad-scores correspond to expectations about how candidates target voters. Importantly, this experimental approach can augment or validate automated text analysis, and generalize to study speech across many other contexts.

Representation, Elections and Campaigns

I develop and test a theory of issue distancing in congressional elections.  Rather than emphasize a personal vote or defend 'out-of-step' positions, polarized candidates portray their records as relatively centrist. They do so by prioritizing counter-stereotypical issues and policy frames in positive ads, cultivating moderation without flip-flopping or splitting supporters, while attacking opponents as partisan extremists. For evidence, I analyze issue statements in 23,577 House and Senate ads aired from 1968 to 2014, using a novel crowdsourcing approach that places messages on a left-right scale.  Linking scores to roll calls, I show positive messages are increasingly issue-focused, deviate substantially from legislative records, and target district opinion, with negative messages tracking partisan-driven polarization.  I then explore competition and information mechanisms in issue distancing. Overall, this study provides evidence for a Downsian campaign logic that may drive shortfalls in voter knowledge, and weaken constituent control of polarized politicians.

"Seeing Spots: An Experimental Examination of Voter Appetite for Partisan and Negative Campaign Ads" (with Alexander G. Theodoridis). (Online Appendix), Political Behavior 40:4, December 2018. (Replication data here.)

We utilize a novel experimental design to assess voter selectivity to political advertising. We randomly expose respondents to comparable positive or negative ads aired by Democratic or Republican candidates from the 2012 Presidential race and the 2013 Virginia Gubernatorial contest. The experiment closely mirrors real consumption of campaign information by allowing subjects to skip ads after five seconds, re-watch and share ads with friends. Using these measures of ad-seeking behavior, we find little evidence that negativity influences self-exposure to election advertising. We find partisans disproportionately tune out ads aired by their party’s opponents, though this behavior is asymmetric: Republican-identifiers are more consistent screeners of partisan ads than Democrats. The results advance our understanding of selectivity, showing that party source, and not ad tone, interacts with partisanship to mediate campaign exposure. The findings have important implications about the role self-exposure to information plays in campaigns and elections in a post-broadcast era.

"Gerrymandering Incumbency: Does Non-Partisan Redistricting Increase Electoral Competition?" (with Brian T. Hamel and Aaron M. Goldzimer), (Online Appendix), Journal of Politics 80:3, July 2018. (Replication data here.)

Many political advocacy groups, journalists, and scholars view decennial redistricting as a major force in weakening the representational link between voters and officeholders by helping insulate legislative incumbents from electoral defeat. Motivated by this concern, reformers in a number of states have proposed giving control over redistricting to ‘politically-neutral’ independent commissions. Freed from partisan and electoral pressures, independent redistrictors would be expected to draw districts without giving favor to parties or their incumbents. In this study, we analyze two novel datasets of alternative redistricting plans, to evaluate whether maps drawn by independent commissions are more electorally competitive than those produced by party-controlled legislatures, compared to the proposals that could have been adopted. We find that the redistricting process on the margin, helps sustain the electoral security of incumbents. Yet, counter to reformers’ expectations, we find that independent redistrictors produce virtually the same degree of insulation as plans devised in legislatures or by politician commissions. Overall, our results suggest caution in overhauling state redistricting institutions to increase electoral competition: independent commissions may not be as politically-neutral as theorized.

"Cause or Effect? Turnout in Hispanic Majority-Minority Districts" (with Jasjeet Sekhon and Rocio Titiunik). (Online Appendix), Political Analysis 24:3, Summer 2016. (Replication data here). 

Legislative redistricting alters the political and electoral context for some voters but not others, thus offering a potentially promising research design to study many questions of interest in political science. We apply this design to study the effect that descriptive representation has on co-ethnic political engagement, focusing on Hispanic participation following California’s 2000 redistricting cycle. We show that when redistrictors draw legislative boundaries in California’s 1990, 2000 and 2010 apportionment cycles, they systematically sort higher participating Hispanic voters into majority-Hispanic (MH) jurisdictions represented by co-ethnic candidates, biasing subsequent comparisons of Hispanic participation across districts. Similar sorting occurs during redistricting in Florida and Texas, though here the pattern is reversed, with less participating Hispanic voters redistricted to MH districts. Our study highlights important heterogeneity in redistricting largely unknown or under-appreciated in previous research. Ignoring this selection problem could significantly bias estimates of the effect of Hispanic representation, either positively or negatively. After we correct for these biases using a hierarchical genetic matching algorithm, we find that, in California, being moved to a district with an Hispanic incumbent has little impact on Hispanic participation in our data.

"Mediating the Electoral Connection: The Information Effects of Voter Signals on Legislative Behavior" (with John Brooks). (Online Appendix), Journal of Politics 78:3, July 2016. (Replication data here).

We develop and assess an elite-information account of representation. Accordingly, politicians face uncertainty about voter opinion, and use previous vote-margins to 
gauge future electoral outcomes. Losses in vote support elicit ideological moderation given new information about electorates. To test this account, we use rain around Election Day as a natural experiment in voting in U.S. House races from 1956 to 2008. We find each additional inch of rainfall exogenously dampens Democratic vote-margins by 1.4 to 2.0 percentage points, and shifts incumbents rightward in their roll-call positions in subsequent Congresses. We find responsiveness mainly in competitive districts with the greatest risk of defeat, and by Democrats rather than Republicans, suggesting asymmetry in party representation. Overall, we highlight the importance of elite information uncertainty as a mechanism driving the electoral connection, and show that idiosyncratic electoral effects can meaningfully impact legislative behavior.

Participation, Partisanship and Voter Competence

"Blind Guessing? Voter Competence About Partisan Messaging". (Online Appendix).

Considerable research uncovers fundamental limits to voter competence about parties and candidates, which may undermine their ability to hold incumbents accountable. In this study, I develop novel inference experiments to assess whether these limits extend to voters as 'blind guessers', unable to discern differences in electoral messages aired by Democrats and Republicans. The experiments ask respondents to infer candidates' party from their campaign statements, allowing an analysis of individual and ad-level factors that influence accurate party inferences. I find voters, including the least informed, possess modest competence about partisanship in campaign messaging. Accuracy varies by political knowledge, though only for negative, and not positive ads. I find a major role for elite strategy driving the latter - candidates communicate counter-typical issue statements in promotion ads, diminishing voter ability to correctly attribute party. The study underscores that strategic or `crafted talk' can be an important elite source of voter ignorance even for the most politically informed, raising broader implications for theories of voter competence and blind retrospection in elections.

"What Goes with Red and Blue? Mapping Partisan and Ideological Associations in the Minds of Voters" (with Stephen Goggin and Alex Theodoridis). (Online Appendix), Political Behavior, Forthcoming. (Replication data here.)

To what extent do voters grasp "what goes with what'" among key political objects as they attempt to understand the choices they face at the ballot box? Is recognition of these associations limited to only the most informed citizens?  We design a novel conjoint classification experiment that minimizes partisan boosting and allows for the relative comparison of attribute effect when mapping voter associative networks, the cluster of attributes linked to parties and ideological labels. We ask respondents to 'guess' the party or ideology of hypothetical candidates with fully randomized issue priorities and biographical details.  There is remarkable agreement among both high- and low-knowledge voters in linking issues to each party and ideology, suggesting this minimalist form of associative competence is more widely held in the mass public than perhaps previously thought.  We find less agreement about biographical traits which appear to pose greater information challenges for voters. Notably, nearly identical issue priorities and traits are associated with party and ideology, indicating these two dimensions are largely fused in the minds of today's American voters. 

"Hookworm Eradication as a Natural Experiment for Schooling and Voting in the American South". (Online Appendix), Political Behavior 40:2 June 2018. (Replication data here.)

Educational attainment is robustly associated with greater political participation, yet the causal nature of this finding remains contested. To assess this relationship, I leverage a natural experiment in the Rockefeller Sanitary Commission's (RSC) anti-hookworm campaign, which exogenously expanded primary and secondary education in the early-20th century American South. I evaluate two RSC hookworm interventions: exposure to the campaign and proportion treated.  I use genetic matching to control for observable factors that influenced the haphazard dispensing of treatment, and develop new matching methods for continuous campaign interventions. I also use a variety of methods to assess the robustness of the results to a number of alternative accounts. Throughout, I find a consistent positive effect of education on participation, suggesting additional evidence for a causal interpretation of the ‘education effect’.

"Who Matches? Propensity Scores and Bias in the Causal Effects of Education on Participation" (with Sara Chatfield), Journal of Politics 73:3, July 2011. (Replication data here.)

In a recent study, Kam and Palmer (2008) employ propensity score matching to assess whether college attendance causes participation after reducing selection bias due to pre-adult factors. After matching the authors find no correlation, upending a major pillar in political science. However, we argue that this study has serious flaws and should not be the basis for rejecting the traditional view of an "education effect" on participation. We match on 766,642 propensity scores and use genetic matching to recover better matches with lower covariate imbalances. We consistently find positive effects as covariate balance improves, though no matching approach yields unbiased results. We demonstrate that selection is a serious concern in studying the participatory effects of college attendance and that balance in the covariates and robustness to sensitivity diagnostics should be the ultimate guide for conducting matching analyses.

"Untangling the Education Effect: Moving Educational Interventions into the Experimental Frontier" (with Sara Chatfield), Resources, Engagement, and Recruitment: New Advances in the Study of Civic Volunteerism, Casey A. Klofstad (Ed.), Temple University Press, 2016.

Untangling the precise education effect has proven to be challenging. Directly manipulating educational outcomes through experimentation is usually infeasible and unethical. And there is a serious concern that non-random pressures to seek out more education remain problematic in observational studies, and perhaps even those utilizing natural experiments. Our aim in this chapter is to outline a roadmap for future research on the political returns to schooling, given the limits to experimentation in educational attainment. We think developing such a guide will be helpful for political scientists as the field moves increasingly into the experimental frontier. This is especially so given a possible future where scholars have exhausted new sources of exogenous natural variation, are unable to experimentally manipulate years of education, and yet remain skeptical of much of the observational findings about education.

"Mobilizing a Generation? The Many Behavioral Effects of the Vietnam Draft Lottery".

A generation of men were randomly assigned lottery numbers to determine draft status during the Vietnam War, offering researchers a natural experiment to assess the behavioral effects of differential exposure to the risk of military service. However, conventional analysis of these effects may be biased by out-of-sample attrition influenced by draft risk. Additionally, the effects of the lottery may be unconventional or heterogeneous - low numbers could elicit an 'empathetic' effect on draft-age men ineligible due to prior military service. In this study, I use the Youth Parent Socialization Panel Study (YPSPS) data to assess whether the Vietnam draft lottery influenced attrition behavior in later waves of the survey, and whether these and other effects depend heterogeneously on pre-1969 military service. I find that high draft risk decreases attrition for draft-eligible men, but has no impact on the ineligible. I then impute missing data over multiple waves to reevaluate the impact of the lottery on participation and attitudes accounting for sample attrition, and validate these imputations through a variety of tests. Overall, I find that either the behavioral effects of the draft uncovered in prior studies are substantially overstated, or that the mere assignment of low draft numbers demobilized men insulated from draft risk, and pushed their attitudes and partisanship in a conservative direction. 

Causal Inference and Estimation

"Estimating Causal Effects Using Coarsened Treatments as Instruments".

Researchers often estimate causal effects in experimental or observational studies after coarsening continuous measures of treatments. In the statistical matching context, in particular, non-discrete interventions are frequently discretized to facilitate pair-stratification using traditional matching approaches for binary treatments. A well-known issue in studying coarsened interventions is that any coarsening induces measurement error that attenuates estimates, while inflating estimator standard errors. While this bias is known, there is yet no standard correction for it. This research note illustrates the error-in-variables structure underlying the use of discrete transformations of non-discrete (or dose) interventions. It also recommends the use of the standard IV estimator to recover an unbiased estimate of the uncoarsened treatment effect. Particular attention is given to the problem of matching with a continuous intervention, which motivates simulations.

"Maximum Entropy Imputation".

In this study, I develop a novel methodological approach, maximum entropy imputation (MEI), to correct for attrition bias when estimating causal effects in experiments. MEI reweights observations to construct a synthetic sample from the group of non-attritioners that near-perfectly resemble those who actually attrition in their observable characteristics. I introduce and discuss the assumptions of MEI for identifying causal effects with sample attrition. I then present the results from a series of validation tests and simulations that assess the performance of the imputation approach. Finally, I employ MEI to replicate a number of behavioral findings in political science and epidemiology.

"A Genetic Matching Approach to Estimating Treatment Effects Using Non-Binary Interventions".

I develop a new evolutionary approach to maximizing covariate balance when matching using continuous or non-binary interventions. The aim in the approach is to match units to minimize distances on covariates, while maximizing differences on the continuous treatments, so that there are ’high’ and ’low’ dose units in each matched pair. Balance is maximized on covariates across the high and low doses, us- ing weights to heighten differences on some covariates over others during matching. An evolutionary algorithm is used to identify the weights that optimally minimize covariate differences over successive generations. Through simulations and empirical applications, I show that continuous genetic matching generally performs much better that alternative parametric and matching estimation approaches.