An Analysis of Interviewer Travel and Field Outcomes in Two Field Surveys

Open access

Abstract

In this article, we investigate the relationship between interviewer travel behavior and field outcomes, such as contact rates, response rates, and contact attempts in two studies, the National Survey of Family Growth and the Health and Retirement Study. Using call record paradata that have been aggregated to interviewer-day levels, we examine two important cost drivers as measures of interviewer travel behavior: the distance that interviewers travel to segments and the number of segments visited on an interviewer-day. We explore several predictors of these measures of travel - the geographic size of the sampled areas, measures of urbanicity, and other sample and interviewer characteristics. We also explore the relationship between travel and field outcomes, such as the number of contact attempts made and response rates.We find that the number of segments that are visited on each interviewer-day has a strong association with field outcomes, but the number of miles travelled does not. These findings suggest that survey organizations should routinely monitor the number of segments that interviewers visit, and that more direct measurement of interviewer travel behavior is needed.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Biemer P.P. P. Chen and K. Wang. 2013. “Using Level-of-Effort Paradata in Non-Response Adjustments with Application to Field Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 176(1): 147-168. Doi: http://dx.doi.org/10.1111/j.1467-985X.2012.01058.x.

  • Bienias J.L. E.M. Sweet and C.H. Alexander. 1990. A Model for Simulating Interviewer Travel Costs for Different Cluster Sizes. Proceedings of the Survey Research Methods Section American Statistical Association 20-27. Anaheim CA. Available at: https://ww2.amstat.org/sections/srms/Proceedings/papers/1990_004.pdf (accessed October 14 2017).

  • Blom A.G. 2012. “Explaining Cross-Country Differences in Survey Contact Rates: Application of Decomposition Methods.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 175(1): 217-242. Doi: http://dx.doi.org/10.1111/j.1467-985X.2011.01006.x.

  • Blom A.G. E.D. de Leeuw and J.J. Hox. 2011. “Interviewer Effects on Nonresponse in the European Social Survey.” Journal of Official Statistics 27(2): 359-377.

  • Bruce A. and J.G. Robinson. 2006. “Tract-Level Planning Database with Census 2000 Data.” US Department of Commerce US Census Bureau: Washington DC 226. Available at: https://www.census.gov/2010census/partners/pdf/TractLevelCensus-2000Apr_2_09.pdf (accessed October 14 2017).

  • Campanelli P. P. Sturgis and S. Purdon. 1997. Can You Hear Me Knocking?: Investigation into the Impact of Interviewers on Survey Response Rates National Centre for Social Research London.

  • Chen B.-C. 2012. “Simulating NHIS Field Operations.” Proceedings of the 2012 Research Conference Federal Committee on Statistical Methodology. Office of Management and Budget. Washington DC. Available at: https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/05/Chen_2012FCSM_II-A.pdf (accessed October 14 2017).

  • Cochran W.G. 1977. Sampling Techniques (Third ed.). New York: John Wiley and Sons. Durrant G.B. and F. Steele. 2009. “Multilevel Modelling of Refusal and Non-Contact in Household Surveys: Evidence from Six UK Government Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 172(2): 361-381. Doi: http://dx.doi.org/10.1111/j.1467-985X.2008.00565.x.

  • Groves R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70(5): 646-675. Doi: http://dx.doi.org/10.1093/poq/nfl0.

  • Groves R.M. and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72(2): 167-189.

  • Hansen M.H. W.N. Hurwitz and W.G. Madow. 1953. Sample Survey Methods and Theory. New York: Wiley.

  • Health and Retirement Study. 2008. Sample Evolution: 1992-1998. Available at: http://hrsonline.isr.umich.edu/sitedocs/surveydesign.pdf (accessed February 6 2014).

  • Heeringa S.G. and J.H. Connor. 1995. Technical Description of the Health and Retirement Survey Sample Design. Available at: http://hrsonline.isr.umich.edu/sitedocs/userg/HRSSAMP.pdf (accessed February 6 2014).

  • Judkins D. J. Waksberg and D. Northrup. 1990. Cost Functions for NHIS and Implications for Survey Design. Proceedings of the American Statistical Association Survey Research Methods Section 34-43. Anaheim CA.

  • Kalsbeek W.D. O.M. Mendoza and D.V. Budescu. 1983. “Cost Models for Optimum Allocation in Multi-Stage Sampling.” Survey Methodology 9(2): 154-177.

  • Kalsbeek W.D. S.L. Botman J.T. Massey and P.-W. Liu. 1994. “Cost-Efficiency and the Number of Allowable Call Attempts in the National Health Interview Survey.” Journal of Official Statistics 10(2): 133-152.

  • Kish L. 1965. Survey Sampling. New York: John Wiley & Sons Inc.

  • Lepkowski J.M. W.D. Mosher K.E. Davis R.M. Groves and J. Van Hoewyk. 2010. The 2006-2010 National Survey of Family Growth: Sample design and analysis of a continuous survey. Washington DC: National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/series/sr_02/sr02_150.pdf (accessed October 14 2017).

  • Mayer C.S. 1968. “A Computer System for Controlling Interviewer Costs.” Journal of Marketing Research 5: 312-318.

  • Montaquila J.M. J.M. Brick M.C. Hagedorn C. Kennedy and S. Keeter. 2008. “Aspects of Nonrepsonse Bias in RDD Telephone Surveys.” In Advances in Telephone Survey Methodology edited by J.M. Lepkowski C. Tucker J.M. Brick E.D. de Leeuw L. Japec P.J. Lavrakas M.W. Link and R.L. Sangster 561-586. Hoboken NJ: John Wiley & Sons.

  • Morton-Williams J. 1993. Interviewer Approaches. Hants England: Aldershot.

  • Olson K. and A. Peytchev. 2007. “Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes.” Public Opinion Quarterly 71(2): 273-286.

  • Olson K. and J. Wagner. 2015. “A Feasibility Test of Using Smartphones to Collect GPS Information in Face-to-Face Surveys.” Survey Research Methods 9(1): 1-13. Doi: http://dx.doi.org/10.18148/srm/2015.v9i1.6036.

  • O’Muircheartaigh C. and P. Campanelli. 1999. “A Multilevel Exploration of the Role of Interviewers in Survey Non-Response.” Journal of the Royal Statistical Society Series A 162: 437-446. Doi: http://dx.doi.org/10.1111/1467-985X.00147.

  • Peachman J. 1992. Design and Methodology of 1991/92 Household Interview Survey. Paper presented at the Australasian Transport Research Forum 17th Conference 149-161. Canberra Australia. Available at: http://atrf.info/papers/1992/1992_Peachman.pdf (accessed October 14 2017).

  • Pickery J. and G. Loosveldt. 2002. “A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse.” Quality and Quantity 36: 427-437. Doi: https://doi.org/10.1023/A:1020905911108.

  • Purdon S. P. Campanelli and P. Sturgis. 1999. “Interviewers’ Calling Strategies on Faceto-Face Interview Surveys.” Journal of Official Statistics 15(2): 199-219.

  • Raudenbush S.W. and A.S. Bryk. 2002. Hierarchical linear models : applications and data analysis methods. Thousand Oaks Sage Publications.

  • Schouten B. F. Cobben P. Lundquist and J. Wagner. 2016. “Does More Balanced Survey Response Imply Less Non-Response Bias?” Journal of the Royal Statistical Society: Series A (Statistics in Society) 179(3): 727-748. Doi: http://dx.doi.org/10.1111/rssa.12152.

  • Stoop I. J. Billiet A. Koch and R. Fitzgerald. 2010. Improving Survey Response: Lessons Learned from the European Social Survey. West Sussex UK: Wiley.

  • Stokes S.L. and P.A. Jones. 1989. Evaluation of the Interviewer Quality Control Procedure for the Post-Enumeration Survey. 1989 Proceedings of the Survey Research Methods Section. American Statistical Association. Alexandria VA 696-698.

  • Stroup W. 2011. Living with Generalized Linear Models. Paper 349-2011 SAS Global Forum 2011. Available at: http://support.sas.com/resources/papers/proceedings11/349-2011.pdf (accessed October 14 2017).

  • Sudman S. 1965-66. “Time Allocation in Survey Interviewing and in Other Field Occupations.” Public Opinion Quarterly 29: 638-648.

  • Sudman S. 1967. Reducing the Cost of Surveys. Chicago: Aldine.

  • Sudman S. 1978. “Optimum Cluster Designs Within a Primary Unit Using Combined Telephone Screening and Face-to-Face Interviewing.” Journal of the American Statistical Association 73: 300-304. Doi: http://dx.doi.org/10.2307/2286656.

  • United States Census Bureau. 2013. “Metropolitan and Micropolitan Statistical Areas Main.” Available at: http://www.census.gov/population/metro/. Last Revised: May 6 2013; (accessed January 15 2015).

  • Wagner J. K. Olson and M. Edgar. 2013. Using GPS and Other Data to Assess Errors in Level-of-Effort Data in Field Surveys. Paper presented at the Joint Statistical Meetings August 2013. Montreal Canada.

  • Wang K. and P. Biemer. 2010. The Accuracy of Interview Paradata: Results from a Field Investigation. Paper presented at the Annual Conference of the American Association for Public Opinion Research Chicago IL. Available at: https://www.rti.org/sites/default/files/resources/aapor10_biemer_pres.pdf (accessed October 14 2017).

  • Weeks M.F. R.A. Kulka J.T. Lessler and R.W. Whitmore. 1983. “Personal versus Telephone Surveys for Collecting Household Health Data at the Local Level.” American Journal of Public Health 73: 1389-1394.

  • West B.T. and K. Olson. 2010. “How Much of Interviewer Variance is Really Nonresponse Error Variance?” Public Opinion Quarterly 74(5): 1004-1026. Doi: http://dx.doi.org/10.1093/poq/nfq061.

  • West B.T. K.B. Welch and A.T. Galecki. 2015. Linear Mixed Models: A Practical Guide Using Statistical Software (Second ed.). Boca Raton FL: CRC Press.

Search
Journal information
Impact Factor


IMPACT FACTOR 2018: 0,837
5-year IMPACT FACTOR: 0,934

CiteScore 2018: 1.04

SCImago Journal Rank (SJR) 2018: 0.963
Source Normalized Impact per Paper (SNIP) 2018: 1.020

Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 715 490 12
PDF Downloads 270 180 1