An Analysis of Interviewer Travel and Field Outcomes in Two Field Surveys

Open access

Abstract

In this article, we investigate the relationship between interviewer travel behavior and field outcomes, such as contact rates, response rates, and contact attempts in two studies, the National Survey of Family Growth and the Health and Retirement Study. Using call record paradata that have been aggregated to interviewer-day levels, we examine two important cost drivers as measures of interviewer travel behavior: the distance that interviewers travel to segments and the number of segments visited on an interviewer-day. We explore several predictors of these measures of travel - the geographic size of the sampled areas, measures of urbanicity, and other sample and interviewer characteristics. We also explore the relationship between travel and field outcomes, such as the number of contact attempts made and response rates.We find that the number of segments that are visited on each interviewer-day has a strong association with field outcomes, but the number of miles travelled does not. These findings suggest that survey organizations should routinely monitor the number of segments that interviewers visit, and that more direct measurement of interviewer travel behavior is needed.

Biemer, P.P., P. Chen, and K. Wang. 2013. “Using Level-of-Effort Paradata in Non-Response Adjustments with Application to Field Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 176(1): 147-168. Doi: http://dx.doi.org/10.1111/j.1467-985X.2012.01058.x.

Bienias, J.L., E.M. Sweet, and C.H. Alexander. 1990. A Model for Simulating Interviewer Travel Costs for Different Cluster Sizes. Proceedings of the Survey Research Methods Section, American Statistical Association, 20-27. Anaheim, CA. Available at: https://ww2.amstat.org/sections/srms/Proceedings/papers/1990_004.pdf (accessed October 14, 2017).

Blom, A.G. 2012. “Explaining Cross-Country Differences in Survey Contact Rates: Application of Decomposition Methods.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 175(1): 217-242. Doi: http://dx.doi.org/10.1111/j.1467-985X.2011.01006.x.

Blom, A.G., E.D. de Leeuw, and J.J. Hox. 2011. “Interviewer Effects on Nonresponse in the European Social Survey.” Journal of Official Statistics 27(2): 359-377.

Bruce, A. and J.G. Robinson. 2006. “Tract-Level Planning Database with Census 2000 Data.” US Department of Commerce, US Census Bureau: Washington, DC, 226. Available at: https://www.census.gov/2010census/partners/pdf/TractLevelCensus-2000Apr_2_09.pdf (accessed October 14, 2017).

Campanelli, P., P. Sturgis, and S. Purdon. 1997. Can You Hear Me Knocking?: Investigation into the Impact of Interviewers on Survey Response Rates, National Centre for Social Research, London.

Chen, B.-C. 2012. “Simulating NHIS Field Operations.” Proceedings of the 2012 Research Conference, Federal Committee on Statistical Methodology. Office of Management and Budget. Washington, DC. Available at: https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/05/Chen_2012FCSM_II-A.pdf (accessed October 14, 2017).

Cochran, W.G. 1977. Sampling Techniques (Third ed.). New York: John Wiley and Sons. Durrant, G.B. and F. Steele. 2009. “Multilevel Modelling of Refusal and Non-Contact in Household Surveys: Evidence from Six UK Government Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 172(2): 361-381. Doi: http://dx.doi.org/10.1111/j.1467-985X.2008.00565.x.

Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70(5): 646-675. Doi: http://dx.doi.org/10.1093/poq/nfl0.

Groves, R.M. and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72(2): 167-189.

Hansen, M.H., W.N. Hurwitz, and W.G. Madow. 1953. Sample Survey Methods and Theory. New York: Wiley.

Health and Retirement Study. 2008. Sample Evolution: 1992-1998. Available at: http://hrsonline.isr.umich.edu/sitedocs/surveydesign.pdf (accessed February 6, 2014).

Heeringa, S.G. and J.H. Connor. 1995. Technical Description of the Health and Retirement Survey Sample Design. Available at: http://hrsonline.isr.umich.edu/sitedocs/userg/HRSSAMP.pdf (accessed February 6, 2014).

Judkins, D., J. Waksberg, and D. Northrup. 1990. Cost Functions for NHIS and Implications for Survey Design. Proceedings of the American Statistical Association, Survey Research Methods Section, 34-43. Anaheim, CA.

Kalsbeek, W.D., O.M. Mendoza, and D.V. Budescu. 1983. “Cost Models for Optimum Allocation in Multi-Stage Sampling.” Survey Methodology 9(2): 154-177.

Kalsbeek, W.D., S.L. Botman, J.T. Massey, and P.-W. Liu. 1994. “Cost-Efficiency and the Number of Allowable Call Attempts in the National Health Interview Survey.” Journal of Official Statistics 10(2): 133-152.

Kish, L. 1965. Survey Sampling. New York: John Wiley & Sons, Inc.

Lepkowski, J.M., W.D. Mosher, K.E. Davis, R.M. Groves, and J. Van Hoewyk. 2010. The 2006-2010 National Survey of Family Growth: Sample design and analysis of a continuous survey. Washington, DC: National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/series/sr_02/sr02_150.pdf (accessed October 14, 2017).

Mayer, C.S. 1968. “A Computer System for Controlling Interviewer Costs.” Journal of Marketing Research 5: 312-318.

Montaquila, J.M., J.M. Brick, M.C. Hagedorn, C. Kennedy, and S. Keeter. 2008. “Aspects of Nonrepsonse Bias in RDD Telephone Surveys.” In Advances in Telephone Survey Methodology, edited by J.M. Lepkowski, C. Tucker, J.M. Brick, E.D. de Leeuw, L. Japec, P.J. Lavrakas, M.W. Link, and R.L. Sangster, 561-586. Hoboken, NJ: John Wiley & Sons.

Morton-Williams, J. 1993. Interviewer Approaches. Hants, England: Aldershot.

Olson, K. and A. Peytchev. 2007. “Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes.” Public Opinion Quarterly 71(2): 273-286.

Olson, K. and J. Wagner. 2015. “A Feasibility Test of Using Smartphones to Collect GPS Information in Face-to-Face Surveys.” Survey Research Methods 9(1): 1-13. Doi: http://dx.doi.org/10.18148/srm/2015.v9i1.6036.

O’Muircheartaigh, C. and P. Campanelli. 1999. “A Multilevel Exploration of the Role of Interviewers in Survey Non-Response.” Journal of the Royal Statistical Society, Series A 162: 437-446. Doi: http://dx.doi.org/10.1111/1467-985X.00147.

Peachman, J. 1992. Design and Methodology of 1991/92 Household Interview Survey. Paper presented at the Australasian Transport Research Forum 17th Conference, 149-161. Canberra, Australia. Available at: http://atrf.info/papers/1992/1992_Peachman.pdf (accessed October 14, 2017).

Pickery, J. and G. Loosveldt. 2002. “A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse.” Quality and Quantity 36: 427-437. Doi: https://doi.org/10.1023/A:1020905911108.

Purdon, S., P. Campanelli, and P. Sturgis. 1999. “Interviewers’ Calling Strategies on Faceto-Face Interview Surveys.” Journal of Official Statistics 15(2): 199-219.

Raudenbush, S.W. and A.S. Bryk. 2002. Hierarchical linear models : applications and data analysis methods. Thousand Oaks, Sage Publications.

Schouten, B., F. Cobben, P. Lundquist, and J. Wagner. 2016. “Does More Balanced Survey Response Imply Less Non-Response Bias?” Journal of the Royal Statistical Society: Series A (Statistics in Society) 179(3): 727-748. Doi: http://dx.doi.org/10.1111/rssa.12152.

Stoop, I., J. Billiet, A. Koch, and R. Fitzgerald. 2010. Improving Survey Response: Lessons Learned from the European Social Survey. West Sussex, UK: Wiley.

Stokes, S.L. and P.A. Jones. 1989. Evaluation of the Interviewer Quality Control Procedure for the Post-Enumeration Survey. 1989 Proceedings of the Survey Research Methods Section. American Statistical Association. Alexandria, VA, 696-698.

Stroup, W. 2011. Living with Generalized Linear Models. Paper 349-2011, SAS Global Forum 2011. Available at: http://support.sas.com/resources/papers/proceedings11/349-2011.pdf (accessed October 14, 2017).

Sudman, S. 1965-66. “Time Allocation in Survey Interviewing and in Other Field Occupations.” Public Opinion Quarterly 29: 638-648.

Sudman, S. 1967. Reducing the Cost of Surveys. Chicago: Aldine.

Sudman, S. 1978. “Optimum Cluster Designs Within a Primary Unit Using Combined Telephone Screening and Face-to-Face Interviewing.” Journal of the American Statistical Association 73: 300-304. Doi: http://dx.doi.org/10.2307/2286656.

United States Census Bureau. 2013. “Metropolitan and Micropolitan Statistical Areas Main.” Available at: http://www.census.gov/population/metro/. Last Revised: May 6, 2013; (accessed January 15, 2015).

Wagner, J., K. Olson, and M. Edgar. 2013. Using GPS and Other Data to Assess Errors in Level-of-Effort Data in Field Surveys. Paper presented at the Joint Statistical Meetings, August 2013. Montreal, Canada.

Wang, K. and P. Biemer. 2010. The Accuracy of Interview Paradata: Results from a Field Investigation. Paper presented at the Annual Conference of the American Association for Public Opinion Research, Chicago, IL. Available at: https://www.rti.org/sites/default/files/resources/aapor10_biemer_pres.pdf (accessed October 14, 2017).

Weeks, M.F., R.A. Kulka, J.T. Lessler, and R.W. Whitmore. 1983. “Personal versus Telephone Surveys for Collecting Household Health Data at the Local Level.” American Journal of Public Health 73: 1389-1394.

West, B.T. and K. Olson. 2010. “How Much of Interviewer Variance is Really Nonresponse Error Variance?” Public Opinion Quarterly 74(5): 1004-1026. Doi: http://dx.doi.org/10.1093/poq/nfq061.

West, B.T., K.B. Welch, and A.T. Galecki. 2015. Linear Mixed Models: A Practical Guide Using Statistical Software (Second ed.). Boca Raton, FL: CRC Press.

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2018: 0,837
5-year IMPACT FACTOR: 0,934

CiteScore 2018: 1.04

SCImago Journal Rank (SJR) 2018: 0.963
Source Normalized Impact per Paper (SNIP) 2018: 1.020

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 704 511 101
PDF Downloads 269 194 17