Adaptive Intervention Methodology for Reduction of Respondent Contact Burden in the American Community Survey

Open access

Abstract

The notion of respondent contact burden in sample surveys is defined, and a multi-stage process to develop policies for curtailing nonresponse follow-up is described with the goal of reducing this burden on prospective survey respondents. The method depends on contact history paradata containing information about contact attempts both for respondents and for sampled nonrespondents. By analysis of past data, policies to stop case follow-up based on control variables measured in paradata can be developed by calculating propensities to respond for paradata-defined subgroups of sampled cases. Competing policies can be assessed by comparing outcomes (lost interviews, numbers of contacts, patterns of reluctant participation, or refusal to participate) as if these stopping policies had been followed in past data. Finally, embedded survey experiments may be used to assess contact-burden reduction policies when these are implemented in the field. The multi-stage method described here abstracts the stages followed in a series of research studies aimed at reducing contact burden in the Computer Assisted Telephone Interview (CATI) and Computer Assisted Personal Interview (CAPI) modes of the American Community Survey (ACS), which culminated in implementation of policy changes in the ACS.

Bates, N., J. Dahlhamer, P. Phipps, A. Safir, and L. Tan. 2010. “Assessing Contact History Paradata Quality Across Several Federal Surveys.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Vancouver, BC, July 31-August 5, 2010. Alexandria, VA: American Statistical Association. 91–105. Available at: http://ww2.amstat.org/sections/SRMS/Proceedings/y2010/Files/306005_55654.pdf (accessed February 2017).

Bates, N., J. Dahlhamer, and E. Singer. 2008. “Privacy Concerns, Too Busy, or Just Not Interested: Using Doorstep Concerns to Predict Survey Nonresponse.” Journal of Official Statistics 24: 591–612.

Biemer, P.P., P. Chen, and K. Wang. 2013. “Using Level-of-Effort Paradata in Non-Response Adjustments with Application to Field Surveys.” Journal of the Royal Statistical Society: Series A 176: 147–168. Doi: http://dx.doi.org/10.1111/j.1467-985X.2012.01058.x.

Bothwell, L. and S. Podolsky. 2016. “The Emergence of the Randomized, Controlled Trial.” New England Journal of Medicine 375: 501–504. Doi http://dx.doi.org/10.1056/NEJMp1604635.

Bradburn, N. 1978. “Respondent Burden.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, San Diego, California, August 14–17, 1978. Alexandria, VA: American Statistical Association. 35–40. Available at: http://ww2.amstat.org/sections/SRMS/Proceedings/papers/1978_007.pdf (accessed February 2017).

U.S. Census Bureau. 2014. Design and Methodology: American Community Survey. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/programs-surveys/acs/methodology/design-and-methodology.html (accessed February 2017).

Dyer, W. 2004. “Contact History Instrument.” In IBUC Proceedings, International Blaise Users Conference, Qubec, Canada, September 22–24, 2004. International Blaise Users Group. 35–61. Available at: www.blaiseusers.org/2004/papers/03.pdf (accessed February 2017).

Ferguson, T. 1967. Mathematical Statistics: A Decision Theoretic Approach. Academic Press.

Fricker, S., T. Yan, and S. Tsai. 2014. “Response Burden: What Predicts It and Who is Burdened Out?” In AAPOR Proceedings, American Association for Public Opinion Research, Anaheim, California, May 15–18, 2014. Oakbrook Terrace, IL: American Association for Public Opinion Research. 4568–4577. Available at: https://www.bls.gov/osmr/pdf/st140170.pdf (accessed February 2017).

Griffin, D. 2013. “Effect of Changing Call Parameters in the American Community Survey’s Computer Assisted Telephone Interviewing Operation.” American Community Survey Research and Evaluation Report Memorandum Series ACS13-RER-17. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2013/acs/2013_Griffin_03.pdf (accessed February 2017).

Griffin, D. 2014. “Reducing Respondent Burden in the American Community Survey’s Computer Assisted Personal Visit Interviewing Operation – Phase 2 Results.” American Community Survey Research and Evaluation Report Memorandum Series ACS14-RER-07. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2014/acs/2014_Griffin_01.pdf (accessed February 2017).

Griffin, D. and T. Hughes. 2013. “Analysis of Alternative Call Parameters in the American Community Survey’s Computer Assisted Telephone Interviewing.” American Community Survey Research and Evaluation Report Memorandum Series ACS13-RER-11. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2013/acs/2013_Griffin_02.pdf (accessed February 2017).

Griffin, D. and D. Nelson. 2014. “Reducing Respondent Burden in the American Community Survey’s Computer Assisted Personal Visit Interviewing Operation – Phase 1 Results (Part 2).” American Community Survey Research and Evaluation Report Memorandum Series ACS14-RER-22. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2014/acs/2014_Griffin_02.pdf (accessed February 2017).

Griffin, D., E. Slud, and C. Erdman. 2015. “Reducing Respondent Burden in the American Community Survey’s Computer Assisted Personal Visit Interviewing Operations – Phase 3 Results.” American Community Survey Research and Evaluation Report Memorandum Series ACS14-RER-28-R1. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2015/acs/2015_Griffin_01.pdf (accessed February 2017).

Groves, R. and S. Heeringa. 2006. “Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs.” Journal of the Royal Statistics Society: Series A 169: 439–457. Doi: http://dx.doi.org/10.1111/j.1467-985X.2006.00423.x.

Groves, R.M., J. Wagner, and E. Peytcheva. 2007. “Use of Interviewer Judgments about Attributes of Selected Respondents in Post-Survey Adjustment for Unit Nonresponse: An Illustration with the National Survey of Family Growth.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Salt Lake City, Utah, July 29-August 2, 2007. Alexandria, VA: American Statistical Association. 3428–3431. Available at: http://ww2.amstat.org/sections/SRMS/Proceedings/y2007/Files/JSM2007-000782.pdf (accessed February 2017).

Hedlin, D., Dale, T., Haraldsen, G., and Jones, J. eds. 2005. Developing Methods for Assessing Perceived Response Burden. Research Report. Stockholm: Statistics Sweden, Oslo: Statistics Norway, and London: Office for National Statistics. Available at: http://ec.europa.eu/eurostat/documents/64157/4374310/10-DEVELOPING-METHODS-FOR-ASSESSING-PERCEIVED-RESPONSE-BURDEN.pdf/1900efc8-1a07-4482-b3c9-be88ee71df3b (accessed February 2017).

Hughes, T., E. Slud, R. Ashmead, and R. Walsh. 2016. “Results of a Field Pilot to Reduce Respondent Contact Burden in the American Community Survey’s Computer Assisted Personal Interviewing Operation.” American Community Survey Research and Evaluation Report Memorandum Series #ACS16-RER-07. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2016/acs/2016_Hughes_01.pdf (accessed February 2017).

Klein, J. and M. Moeschberger. 2003. Survival Analysis: Techniques for Censored and Truncated Data, 2nd ed. Springer-Verlag.

Kreuter, F., K. Olson, K.J. Wagner, T. Yan, T. Ezzati-Rice, C. Casas-Cordero, M. Lemay, A. Peytchev, R. Groves, and T. Raghunathan. 2010. “Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-response: Examples from Multiple Surveys.” Journal of the Royal Statistical Society: Series A 173: 389–407. Doi: http://dx.doi.org/10.1111/j.1467-985X.2009.00621.x.

Luiten, A. and B. Schouten. 2013. “Tailored Fieldwork Design to Increase Representative Household Survey Response: An Experiment in the Survey of Consumer Satisfaction.” Journal of the Royal Statistical Society: Series A 176: 169–189. Doi: http://dx.doi.org/10.1111/j.1467-985X.2012.01080.x.

Maitland, A., C. Casas-Cordero, and F. Kreuter. 2009. “An Evaluation of Nonresponse Bias using Paradata from a Health Survey.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Washington, DC, August 1–6, 2009. Alexandria, VA: American Statistical Association. 370–378. Available at: http://ww2.amstat.org/sections/SRMS/Proceedings/y2009/Files/303004.pdf (accessed February 2017).

Mills, G. 2016. “Simulated Effects of Changing Calling Parameters and Workload Size on Computer Assisted Telephone Interview Productivity in the American Community Survey.” American Community Survey Research and Evaluation Report Memorandum Series #ACS16-RER-22. Washington, DC: U.S. Census Bureau. Available at: https://census.gov/content/dam/Census/library/working-papers/2016/acs/2016_Mills_02.pdf (accessed August 2017).

Olson, K. and R. Groves. 2012. “An Examination of Within-Person Variation in Response Propensity over the Data Collection Field Period.” Journal of Official Statistics 28: 29–51.

Schouten, B., F. Cobben, and J. Bethlehem. 2009. “Indicators for the Representativeness of Survey Response.” Survey Methodology 35: 101–113.

Sharp, L. and J. Frankel. 1983. “Respondent Burden: A Test of Some Common Assumptions.” Public Opinion Quarterly 47: 36–53. Doi: https://doi.org/10.1086/268765.

Slud, E. 1998. “Predictive Models for Decennial Census Household Response.” In JSM Proceedings, Survey Research Methods Section, American Statistical Association, Dallas, Texas, August 9–13, 1998. Alexandria, VA: American Statistical Association. 272–277. Available at: http://ww2.amstat.org/sections/SRMS/Proceedings/papers/1998_043.pdf (accessed February 2017).

Slud, E. 1999. “Analysis of 1990 Decennial Census Checkin-Time Data.” In FCSM Proceedings Federal Committee on Statistical Methodology, Washington, DC, November 15–17, 1999. Available at: https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/05/IX-B_Slud_FCSM1999.pdf (accessed August 2017).

Slud, E. and C. Erdman. 2013. “Adaptive Curtailment of Survey Follow-up Based on Contact History Data.” In FCSM Proceedings Federal Committee on Statistical Methodology, Washington, DC, November 4–6, 2013. 1–11. Available at: https://s3.amazonaws.com/sitesusa/wpcontent/uploads/sites/242/2014/05/B1_Slud_2013 FCSM.pdf (accessed August 2017).

Virgile, M. 2015. “Measurement Error in American Community Survey Paradata and 2014 Redesign of the Contact History Instrument.” Center for Statistical Research and Methodology Report Series (Survey Methodology #RSM2016-01). Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/srd/papers/pdf/RSM2016-01.pdf (accessed February 2017).

Wagner, J. and T. Raghunathan. 2010. “A New Stopping Rule for Surveys.” Statistics in Medicine 29: 1014–1024. Doi: http://dx.doi.org/10.1002/sim.3834.

Zelenak, M.F. and M. Davis. 2013. “Impact of Multiple Contacts by Computer-Assisted Telephone Interview and Computer-Assisted Personal Interview on Final Interview Outcome in the American Community Survey.” American Community Survey Research and Evaluation Report Memorandum Series #ACS13-RER-08. Washington, DC: U.S. Census Bureau. Available at: www.census.gov/content/dam/Census/library/working-papers/2013/acs/2013_Zelenak_01.pdf (accessed February 2017).

Zelenak, M.F. 2014. “Reducing Respondent Burden in the American Community Survey’s Computer Assisted Personal Visit Interviewing Operation – Phase 1 Results (Part 1).” American Community Survey Research and Evaluation Report Memorandum Series #ACS14-RER-06. Washington, DC: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2014/acs/2014_Zelenak_01.pdf (accessed February 2017).

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2018: 0,837
5-year IMPACT FACTOR: 0,934

CiteScore 2018: 1.04

SCImago Journal Rank (SJR) 2018: 0.963
Source Normalized Impact per Paper (SNIP) 2018: 1.020

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 506 396 180
PDF Downloads 198 153 22