Can I Just Check...? Effects of Edit Check Questions on Measurement Error and Survey Estimates

Open access

Abstract

Household income is difficult to measure, since it requires the collection of information about all potential income sources for each member of a household.Weassess the effects of two types of edit check questions on measurement error and survey estimates: within-wave edit checks use responses to questions earlier in the same interview to query apparent inconsistencies in responses; dependent interviewing uses responses from prior interviews to query apparent inconsistencies over time.Weuse data from three waves of the British Household Panel Survey (BHPS) to assess the effects of edit checks on estimates, and data from an experimental study carried out in the context of the BHPS, where survey responses were linked to individual administrative records, to assess the effects on measurement error. The findings suggest that interviewing methods without edit checks underestimate non-labour household income in the lower tail of the income distribution. The effects on estimates derived from total household income, such as poverty rates or transition rates into and out of poverty, are small.

AAPOR (2011). Standard Definitions: Final Dispositions of Case Codes and Outcome Reports for Surveys, (7th edition). Lenexa, KS: American Association for Public Opinion Research. Available at: http://www.aapor.org/AM/Template.cfm?Section¼Standard_Definitions2&Template¼/CM/ContentDisplay.cfm&ContentID¼3156 (accessed October 30, 2013).

Bound, J., Brown, C., and Mathiowetz, N. (2001). Measurement Error in Survey Data. In Handbook of Econometrics Vol. 5, J.J. Heckman and E. Leamer (eds). Amsterdam: Elsevier, 3705-3843.

Brewer, M., Muriel, A., Philips, D., and Sibieta, L. (2009). Poverty and Inequality in the UK: 2009. The institute for fiscal studies, London. Available at: http://www.ifs.org.uk/ comms/c109.pdf (accessed October 30, 2013).

Böheim, R. and Jenkins, S.P. (2006). A Comparison of Current and Annual Measures of Income in the British Household Panel Study. Journal of Official Statistics, 22, 733-758.

Hale, A. and Michaud, S. (1995). Dependent Interviewing: Impact on Recall and on Labour Market Transitions. SLID Research Paper Series No. 95-06. Ottawa: Statistics Canada.

Jenkins, S.P., Cappellari, L., Lynn, P., Ja¨ckle, A., and Sala, E. (2006). Patterns of Consent: Evidence from a General Household Survey. Journal of the Royal Statistical Society Series A, 169, 701-722. DOI: http://www.dx.doi.org/10.1111/j.1467-985X.2006.00417.x

Jenkins, S.P., Lynn, P., Ja¨ckle, A., and Sala, E. (2008). The Feasibility of Linking Household Survey and Administrative Record Data: New Evidence from Britain. International Journal of Social Research Methodology, 11, 29-43. DOI: http://www.dx.doi.org/10.1080/13645570701401602

Jäckle, A. (2008). Measurement Error and Data Collection Methods: Effects on Estimates from Event History Data. ISER Working Paper 2008-13. Colchester: University of Essex. Available at: http://www.iser.essex.ac.uk/publications/working-papers/iser/ 2008-13.pdf (accessed October 30, 2013).

Jäckle, A. (2009). Dependent Interviewing: A Framework and Application to Current Research. In Methodology of Longitudinal Surveys, P. Lynn (ed.). Chichester: Wiley, 92-112.

Jäckle, H., Laurie, H., and Uhrig, S.C.N. (2007). The Introduction of Dependent Interviewing on the British Household Panel Survey. ISER Working Paper 2007-7. Colchester: University of Essex. Available at: http://www.iser.essex.ac.uk/publications/working-papers/iser/2007-07.pdf (accessed October 30, 2013).

Jäckle, A., Sala, E., Jenkins, S.P., and Lynn, P. (2004). Validation of Survey Data and Employment: the ISMIE Experience. ISER Working Paper 2004-14. Colcheste: University of Essex. Available at: http://www.iser.essex.ac.uk/publications/working-papers/iser/2004-14 (accessed October 30, 2013).

Lynn, P., Jäckle, A., Jenkins, S.P., and Sala, E. (2006). The Effects of Dependent Interviewing on Responses to Questions on Income Sources. Journal of Official Statistics, 22, 357-384.

Lynn, P., Jäckle, A., Jenkins, S.P., Sala, E. (2012). The Impact of Interviewing Method on Measurement Error in Panel Survey Measures of Benefit Receipt: Evidence from a Validation Study. Journal of the Royal Statistical Society, Series A, 175, 289-308. DOI:http://www.dx.doi.org/10.1111/j.1467-985X.2011.00717.x

Lugtig, P. and Jäckle, A. (2011). Can I just check: : : ? Effects of edit check questions on measurement error and survey estimates, ISER Working Paper 2011-23. Colchester: University of Essex. Available at: http://www.iser.essex.ac.uk/publications/working-papers/iser/2011-23 (accessed October 30, 2013).

Mathiowetz, N.A. and McGonagle, K.A. (2000). An Assessment of the Current State of Dependent Interviewing in Household Surveys. Journal of Official Statistics, 16, 401-418.

Moore, J., Bates, N., Pascale, J., and Okon, A. (2009). Tackling Seam Bias through Questionnaire Design. In Methodology of Longitudinal Surveys, P. Lynn (ed.). Chichester: Wiley, 72-92.

Pascale, J., Roemer, M.I., and Resnick, M. (2009). Medicaid Underreporting in the CPS. Results from a Record Check Study. Public Opinion Quarterly, 73, 497-520. DOI: http://www.dx.doi.org/10.1093/poq/nfp028

Pennell, S.G. (1993). Cross-Sectional Imputation and Longitudinal Editing Procedures in the Survey of Income and Program Participation. Washington, DC: US Bureau of the Census. Available at: http://www.census.gov/sipp/workpapr/wp9314.pdf (accessed October 30, 2013).

Sala, E., Uhrig, S.C.N., and Lynn, P. (2009). It is Time Computers Do Clever Things! The Impact of Dependent Interviewing on Interviewer Burden. Field Methods, 23, 3-23. DOI: http://www.dx.doi.org/10.1177/1525822X10384087

Taylor, M.F., Brice, J., Buck, N., and Prentice-Lane, E. (2009). British Household Panel Survey User Manual Volume A: Introduction, Technical Report and Appendices. Colchester: University of Essex. Available at: http://www.iser.essex.ac.uk/bhps/documentation/pdf_versions/volumes/bhpsvola.pdf (accessed October 30, 2013).

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2017: 0.662
5-year IMPACT FACTOR: 1.113

CiteScore 2017: 0.74

SCImago Journal Rank (SJR) 2017: 1.158
Source Normalized Impact per Paper (SNIP) 2017: 0.860

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 68 68 28
PDF Downloads 12 12 6