Item response rates frequently serve as indicators of data quality and potential nonresponse bias. However, key variables from surveys, such as total household income or net worth, are often composite variables constructed from several underlying components. Because such composite variables do not have clearly identifiable response rates, inference on the data quality of these key measures is more difficult. This article proposes three new methods for aggregating data on response rates across questions to create a measure of item response for composite variables. To compare the three methods and illustrate how they can be used (both individually and collectively) to investigate data quality, I analyze item response for net worth in the Survey of Income and Program Participation (SIPP) and the Survey of Consumer Finances (SCF). These new measures provide detailed information about net worth estimates that would be difficult to assess without an item response aggregation method. Overall, these new item response rate methods provide a new way of describing data quality for key measures in surveys and for analyzing changes in data quality over time.
Bollinger, C.R., B.T. Hirsch, C.M. Hokayem, and J.P. Ziliak. 2015. “Trouble in the Tails? What We Know about Earnings Nonresponse Thirty Years after Lillard, Smith, and Welch.” University of Kentucky Center for Poverty Research Discussion Paper Series, no. 120. Available at: https://uknowledge.uky.edu/ukcpr_papers/120/. (accessed April 2019).
Bound, J. and A.B. Krueger. 1991. “The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?” Journal of Labor Economics 9: 1–24. Doi: http://dx.doi.org/10.1086/298256.
Groves, R.M. and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias a Meta-Analysis.” Public Opinion Quarterly 72(2): 167–189. Doi: https://doi.org/10.1093/poq/nfn011.
Hokayem, C., T. Raghunathan, and J. Rothbaum. 2017. “Ignorable Nonresponse? Improved Imputation and Administrative Data in the CPS ASEC.” Paper presented at the Association for Public Policy Analysis and Management Fall Research Conference, November 2–4, 2017. Available at: https://appam.confex.com/appam/2017/webprogram/Paper23281.html. (accessed April 2019).
Hsu, J.W. M.D. Schmeiser, C. Haggerty, and S. Nelson. 2017. “The Effect of Large Monetary Incentives on Survey Completion: Evidence from a Randomized Experiment with the Survey of Consumer Finances.” Public Opinion Quarterly 81(3): 736–747. Doi: https://doi.org/10.1093/poq/nfx006.
Lillard, Lee, James P. Smith, and Finis Welch. 1986. “What do we really know about wages? The importance of nonreporting and census imputation.” Journal of Political Economy 94, no. 3, Part 1: 489–506. Doi: https://doi.org/10.1086/261386.
National Research Council. 2009. Reengineering the Survey of Income and Program Participation. Constance F. Citro and John Karl Scholz, eds. Committee on National Statistics, Division of Behavioral and Social Sciences Education. Washington, DC: The National Academies Press.
Watson, D., L.A. Clark, and A. Tellegen. 1998. “Development and Validation of Brief Measures of Positive and Negative Affect: the PANAS scales.” Journal of Personality and Social Psychology 54(6): 1063–1070. Doi: http://doi.apa.org/journals/psp/54/6/1063.html.
Westra, A., M. Sundukchi, and T. Mattingly. 2015. “Designing a Multipurpose Longitudinal Incentives Experiment for the Survey of Income and Program Participation.” In Proceedings of the 2015 Federal Committee on Statistical Methodology (FCSM) Research Conference, December 1–3, 2015. Washington, DC: Federal Committee on Statistical Methodology. Available at: https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2016/03/E3_Westra_2015FCSM.pdf. (accessed April 2019).