American Journal of Political Science 61.4 (2017): 786-803. (With J. Hemker)
A growing experimental literature uses response rates to fictional requests to measure discrimination against ethnic minorities. This paper argues that restricting attention to response rates can lead to faulty inferences about substantive discrimination depending on how response dummies are correlated with other response characteristics. We illustrate the relevance of this problem by means of a conjoint experiment among all German welfare offices, in which we randomly varied five traits and designed requests to allow for a substantive coding of response quality. We find that response rates are statistically indistinguishable across treatment conditions. However, putative non-Germans receive responses of significantly lower quality, potentially deterring them from applying for benefits. We also find observational evidence suggesting that discrimination is more pronounced in welfare offices run by local governments than in those embedded in the national bureaucracy. We discuss implications for the study of equality in the public sphere.