This study attempted to integrate key assumptions in Respondent-Driven Sampling (RDS) into the Total Survey Error (TSE) perspectives and examine TSE as a new framework for a systematic assessment of RDS errors. Using two publicly available data sets on HIV-at-risk persons, nonresponse error in the RDS recruitment process and measurement error in network size reports were examined. On nonresponse, the ascertained partial nonresponse rate was high, and a substantial proportion of recruitment chains died early. Moreover, nonresponse occurred systematically: recruiters with lower income and higher health risks generated more recruits; and peers of closer relationships were more likely to accept recruitment coupons. This suggests a lack of randomness in the recruitment process, also shown through sizable intra-chain correlation. Self-reported network sizes suggested measurement error, given their wide dispersion and unreasonable reports. This measurement error has further implications for the current RDS estimators, which use network sizes as an adjustment factor on the assumption of a positive relationship between network sizes and selection probabilities in recruitment. The adjustment resulted in nontrivial unequal weighting effects and changed estimates in directions that were difficult to explain and, at times, illogical. Moreover, recruiters’ network size played no role in actual recruitment. TSE may serve as a tool for evaluating errors in RDS, which further informs study design decisions and inference approaches.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Baker, R., J.M. Brick, N.A. Bates, M. Battaglia, M.P. Couper, J.A. Dever, K.J. Gile, and R. Tourangeau. 2013. “Summary Report of the AAPOR Task Force on Non-Probability Sampling.” Journal of Survey Statistics and Methodology 1(2): 90–143. Doi: https://doi.org/10.1093/jssam/smt008.
Burt, R.D., H. Hagan, K. Sabin, and H. Thiede. 2010. “Evaluating Respondent-Driven Sampling in a Major Metropolitan Area: Comparing Injection Drug Users in the 2005 Seattle Area National HIV Behavioral Surveillance System Survey with Participants in the RACEN and Kiwi Studies.” Annals of Epidemiology 20(2): 159–167. Doi: https://doi.org/10.1016/j.annepidem.2009.10.002.
Centers for Disease Control and Prevention (CDC). 2009. HIV-Associated Behaviors Among Injecting-Drug Users––23 Cities, United States, May 2005–February 2006. Morbidity and Mortality Weekly Report, 58, 329–332. Available at: http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5813a1.htm (accessed September 2015).
Cole, S.R. and M.A. Hernan. 2008. “Constructing Inverse Probability Weights for Marginal Structural Models.” American Journal of Epidemiology 168: 656–664. Doi: https://doi.org/10.1093/aje/kwn164.
Compton, W., J. Normand, and E. Lambert. 2009. “Sexual Acquisition and Transmission of HIV Cooperative Agreement Program (SATHCAP).” Journal of Urban Health 86(1): 1–4. Doi: https://doi.org/10.1007/s11524-009-9373-4.
Constantine, M. 2010. “Disentangling Methodologies: The Ethics of Traditional Sampling Methodologies, Community-Based Participatory Research, and Respondent-Drive Sampling.” American Journal of Bioethics 10(3): 22–24. Doi: https://doi.org/10.1080/15265160903585628.
Dombrowski, R., B. Khan, J. Moses, E. Channell, and E. Misshula. 2013. “Assessing Respondent Driven Sampling for Network Studies in Ethnographic Contexts.” Advances in Anthropology 3(1): 1–9. Doi: https://doi.org/10.4236/aa.2013.31001.
Elliott, M.R. 2009. “Model Averaging Methods for Weight Trimming in Generalized Linear Regression Models.” Journal of Official Statistics 25(1): 1–21. Doi: https://doi.org/10.1.1.552.9050.
Fay, R.E. and R.A. Herriot. 1979. “Estimates of Income for Small Places: An Application of James-Stein Procedures to Census Data.” Journal of American Statistical Association 74: 269–277. Doi: https://doi.org/10.2307/2286322.
Frost, S.D.W., K.C. Brouwer, M.A.F. Cruz, R. Ramos, M.E. Ramos, R.M. Lozada, C. Magis-Rodriguez, and S.A. Strathdee. 2006. “Respondent-Driven Sampling of Injection Drug Users in Two U.S.-Mexico Border Cities: Recruitment Dynamics and Impact on Estimates of HIV and Syphilis Prevalence.” Journal of Urban Health 83(1): 83–97. Doi: https://doi.org/10.1007/s11524-006-9104-z.
Gile, K.J. 2011. “Improved Inference for Respondent-Driven Sampling Data with Application to HIV Prevalence Estimation.” Journal of American Statistical Association 106(493): 135–146. Doi: https://doi.org/10.1198/jasa.2011.ap09475.
Gile, K.J., L.G. Johnston, and M.J. Salganik. 2015. “Diagnostics for Respondent-Driven Sampling.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 178(1): 241–269. Doi: https://doi.org/10.1111/rssa.12059.
Groves, R.M. 1989. Survey Errors and Survey Costs. New York: Wiley.
Groves, R.M. and M.P. Couper. 1998. Nonresponse in Household Surveys. New York: Wiley.
Handcock, M.S. 2012. Estimating the Size of Hard-to-Reach Populations Using Respondent-Driven Sampling Data. Paper for the International Conference on Methods for Surveying and Enumerating Hard–to-Reach Populations, October 31–November 3, New Orleans, LA.
Heimer, R. 2005. “Critical Issues and Further Questions About Respondent-Driven Sampling: Comment on Ramierz-Valles et al. (2005).” AIDS and Behavior 9(4): 403–408. Doi: https://doi.org/10.1007/s10461-005-9030-1.
Iguchi, M.Y., S.H. Berry, A.J. Ober, T. Fain, D.D. Heckathorn, P.M. Gorbach, R. Heimer, A. Kozlov, L.J. Ouellet, S. Shoptaw, and W. Zule. 2010. Sexual Acquisition and Transmission of HIV Cooperative Agreement Program (SATHCAP) 2006–2008 [United States]. ICPSR29181-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. Doi: https://doi.org/10.3886/ICPSR29181.
Iguchi, M.Y., A.J. Ober, S.H. Berry, T. Fain, D.D. Heckathorn, P.M. Gorbach, R. Heimer, A. Kozlov, L.J. Ouellet, S. Shoptaw, and W.A. Zule. 2009. “Simultaneous Recruitment of Drug Users and Men Who Have Sex with Men in the United States and Russia Using Respondent-Driven Sampling: Sampling Methods and Implications.” Journal of Urban Health 88(1): 5–31. Doi: https://doi.org/10.1007/s11524-009-9365-4.
Johnston, L.G. and K. Sabin. 2010. “Sampling Hard-to-Reach Populations with Respondent Driven Sampling.” Methodological Innovations Online 5(2): 38–48. Doi: https://doi.org/10.4256/mio.2010.0017.
Kish, L. 1992. “Weighting for unequal Pi.” Journal of Official Statistics 8(2): 183–200.
Lansky, A., A. Abdul-Quader, M. Cribbin, T. Hall, T.J. Finlayson, R.S. Garfein, L.S. Lin, and P.S. Sullivan. 2007. “Developing an HIV Behavioral Surveillance System for Injecting Drug Users: The National HIV Behavioral Surveillance System.” Public Health Reports 122: 48–55. Doi: https://doi.org/10.1177/00333549071220S108.
Laumann, E.O., P.V. Marsden, and D. Prensky. 1983. “The Boundary Specification Problem in Network Analysis.” In Applied Network Analysis. A Methodological Introduction, edited by R.S. Burt and M.J. Minor. 18–34. Beverly Hills, CA: Sage.
Lee, R., J. Ranaldi, M. Cummings, J.N. Crucetti, H. Stratton, and L.-A. McNutt. 2011. “Given the Increasing Bias in Random Digit Dial Sampling, Could Respondent-Driven Sampling be a Practical Alternative?” Annals of Epidemiology 21(4): 272–279. Doi: https://doi.org/10.1016/j.annepidem.2010.11.018.
Lee, S., Z.T. Suzer-Gurtekin, J. Wagner, and R. Valliant. 2012. Exploring Error Properties of Respondent Driven Sampling. Paper presented at the Joint Statistical Meeting, July 28–August 2, San Diego, CA.
Lin, L., T. Finlayson, R. Iachan, M.C.B. Mendoza, and C. Wejnert. 2013. “Sampling Designs for Populations at High Risk for HIV.” Paper presented at the Joint Statistical Meeting, August 3–August 8, Montréal, Canada.
Little, R.J.A., S. Lewitzky, S. Heeringa, J. Lepkowski, and R.C. Kessler. 1997. “Assessment of Weighting Methodology of the National Comorbidity Survey.” American Journal of Epidemiology 146(5): 439–449. Doi: https://doi.org/10.1093/oxfordjournals.aje.a009297.
Lu, X., L. Bengtsson, T. Britton, M. Camitz, B.J. Kim, A. Thorson, and F. Liljeros. 2012. “The Sensitivity of Respondent-Driven Sampling.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 175(1): 1–26. Doi: https://doi.org/10.1111/j.1467-985X.2011.00711.x.
Martin, J.L., J. Wiley, and D. Osmond. 2003. “Social Networks and Unobserved Heterogeneity in Risk for AIDS.” Population Research and Policy Review 22(1): 65–90. Doi: https://doi.org/10.1023/A:1023509211339.
McCreesh, N., S.D. Frost, J. Seeley, J. Katongole, M.N. Tarsh, R. Ndunguse, F. Jichi, N.L. Lunel, D. Maher, L.G. Johnston, P. Sonnenberg, A.J. Copas, R.J. Hayes, and R.G. White. 2012. “Evaluation of Respondent-Driven Sampling.” Epidemiology 23(1): 138–147. Doi: https://doi.org/10.1097/EDE.0b013e31823ac17c.
Montealegre, J.R., J.M. Risser, B.J. Selwyn, S.A. McCurdy, and K. Sabin. 2013. “Effectiveness of Respondent Driven Sampling to Recruit Undocumented Central American Immigrant Women in Houston, Texas for an HIV Behavioral Survey.” AIDS and Behavior 17(2): 719–727. Doi: https://doi.org/10.1007/s10461-012-0306-y.
Ramirez-Valles, J. 2013. Latino MSM Community Involvement: HIV Protective Effects. ICPSR34385-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. Doi: https://doi.org/10.3886/ICPSR34385.v1.
Ramirez-Valles, J., D.D. Heckathorn, R. Vázquez, R.M. Diaz, and R.T. Campbell. 2005. “From Networks to Populations: the Development and Application of Respondent-Driven Sampling Among IDUs and Latino Gay Men.” AIDS and Behavior 9(4): 387–402. Doi: https://doi.org/10.1007/s10461-005-9012-3.
Salganik, M.J. 2006. “Variance Estimation, Design Effects and Sample Size Calculations for Respondent Driven Sampling.” Journal of Urban Health 83(7): 98–112. Doi: https://doi.org/10.1007/s11524-006-9106-x.
Valliant, R., J.A. Dever, and F. Kreuter. 2013. Practical Tools for Designing and Weighting Survey Samples. New York: Springer.
Verdery, A.M. and T.D. Mouw. 2012. Estimated Sampling Variance in Respondent Driven Sampling Data: Mathematical Derivations, Simulated Tests on Empirical Data, and Evidence from Other Forms of Chain-Referral Data Collection. Paper for the International Conference on Methods for Surveying and Enumerating Hard-to-Reach Populations, October 31–November 3, New Orleans, LA.
Volz, E. and D.D. Heckathorn. 2008. “Probability Based Estimation Theory for Respondent Driven Sampling.” Journal of Official Statistics 24(1): 79–97.
Wejnert, C. and D.D. Heckathorn. 2008. “Web-Based Network Sampling: Efficiency and Efficacy of Respondent-Driven Sampling for Online Research.” Sociological Methods and Research 37: 105–134. Doi: https://doi.org/10.1177/0049124108318333.
Zheng, T., M.J. Salganik, and A. Gelman. 2006. “How Many People Do You Know in Prison? Using Overdispersion in Count Data to Estimate Social Structure in Networks” Journal of American Statistical Association 101(474): 409–423. Doi: https://doi.org/10.1198/01621450500000116.