Comparing Survey and Sampling Methods for Reaching Sexual Minority Individuals in Flanders

Open access


As part of a large sexual health study, we used two different approaches to target Sexual Minority Individuals (SMIs). Firstly, we drew on a probability sample (1,832 respondents aged 14-80) of the Flemish population in Belgium. Secondly, we set up a targeted sampling design followed by an Internet survey. Our focus was to explore how two different sampling procedures and survey designs could lead to differences in sample characteristics. Results showed that for female SMIs (we excluded male SMIs from the analyses due to their low numbers) the population sample differed from the Internet sample in terms of sociodemographic characteristics (the latter included younger and more highly educated respondents) and scores on sexual orientation dimensions (the population sample included more respondents who didn’t identify as lesbian or bisexual but reported same-sex sexual experiences and desire). Respondents’ scores on sexual health indicators differed between the samples for two of the seven variables. We discuss implications for improving the quality and validity of nonrandom samples.

Aerts, S., Van Houtte, M., Dewaele, A., Cox, N., and Vincke, J. (2012). Sense of Belonging in Secondary Schools: A Survey of LGB and Heterosexual Students in Flanders. Journal of Homosexuality, 59, 90-113. DOI:

Bajos, N. and Bozon, M. (2008). Enqueˆte sur la sexualite´ en France. Paris: E ´ ditions La De´couverte.

Bakker, F., de Graaf, H., de Haas, S., Kedde, H., Kruijer, H., and Wijsen, C. (2009). Seksuele gezondheid in Nederland 2009. Utrecht: Ruthers Nisso Groep. Available at: (accessed March 30, 2014)

Bauermeister, J., Pingel, E., Zimmerman, M., Couper, M., Carballo-Die´guez, A., and Strecher, V.J. (2012). Data Quality in Web-Based HIV/AIDS Research: Handling Invalid and Suspicious data. Field Methods, 24, 272-291. DOI:

Best, S.J. and Krueger, B.S. (2004). Internet Data Collection. Thousand Oaks, CA: Sage.

Buysse, A., Caen, M., Dewaele, A., Enzlin, P., Lievens, J., T’Sjoen, G., Van Houtte, M., and Vermeersch, H (2013). Seksuele Gezondheid in Vlaanderen. Ghent: Academia Press.

Claeys, L. and Spee, S. (2005). Een Virtuele Illusie of Ree¨le Kansen? - Gender in de Netwerkmaatschappij. Antwerp: Steunpunt Gelijkekansenbeleid, University of Antwerp/University of Hasselt. Available at: (accessed March 30, 2014)

Couper, M.P. (2000). Web Surveys: A Review of Issues and Approaches. Public Opinion Quarterly, 64, 464-494. DOI:

Couper, M.P. (2008). Designing Effective Web Surveys. Cambridge: Cambridge University Press. de Leeuw, E.D. (2005). To Mix or not to Mix Data Collection Modes in Surveys. Journal of Official Statistics, 21, 233-255.

Denscombe, M. (2006). Web-Based Questionnaires and the Mode Effect - An Evaluation Based on Completion Rates and Data Contents of Near-Identical Questionnaires Delivered in Different Modes. Social Science Computer Review, 24, 246-254. DOI:

Dewaele, A., Cox, N., van den Berghe, W., and Vincke, J. (2011). Families of Choice? Exploring the Supportive Networks of Lesbians, Gay Men and Bisexuals. Journal of Applied Social Psychology, 41, 312-331. DOI: 1559-1816.2010.00715.x Dillman, D.A. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley.

Dillman, D.A. (2000). Mail and Internet Surveys: The Tailored Design Method. London: Wiley.

Edelman, M. (1993). Understanding the Gay and Lesbian Vote in 1992. Public Perspective, 4, 32-33.

Evans, J.R. and Mathur, A. (2005). The Value of Online Surveys. Internet Research, 15, 195-219.

Fernee, H. and Keuzenkamp, S. (2011). Selectiviteit van de Roze Vragenlijst.

Een Vergelijking met Paneldata (paper presented at Sociology Day on May 26, 2011, Den Haque). Available at:¼web&cd=1&ved=0CC4QFjAA&,d.ZG4. (accessed March 30, 2014)

Gronow, J., Haavio-Mannila, E., Kivinen, M., Lonkila, M., and Rotkirch, A. (1997). Cultural Inertia and Social Change in Russia. Helsinki: University of Helsinki, Department of Sociology.

Grov, C., Bimbi, D.S., Nanı´n, J.E., and Parsons, J.T. (2006). Ethnicity, Gender, and Generational Factors Associated with the Coming-Out Process Among Gay, Lesbian, and Bisexual Individuals. Journal of Sex Research, 43, 115-121.

Groves, R.M. (2006). Non-Response Rates and Non-Response Bias in Household Surveys. Public Opinion Quarterly, 70, 646-675. DOI:

Haavio-Mannila, E. and Kontula, O. (2001). Seksin Trendit Meilla¨ ja Naapureissa. Helsinki: WSOY.

Heerwegh, D. (2001). Surveyonderzoek Middels het Internet: Een exploratie van het Terrein. Leuven: Katholieke Universiteit Leuven, Departement Sociologie, Afdeling voor dataverzameling en analyse. Available at: (accessed March 30, 2014)

Kerker, B.D., Mostashari, F., and Thorpe, L. (2006). Health Care Access and Utilization Among Women Who Have Sex With Women: Sexual Behavior and Identity. Journal of Urban Health, 83, 970-979. DOI:

Koch, S.N. and Emrey, J.A. (2001). The Internet and Opinion Measurement: Surveying Marginalized Populations. Social Science Quarterly, 82, 131-138. DOI:

Kreuter, F., Presser, S., and Tourangeau, R. (2008). Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public Opinion Quarterly, 72, 847-865. DOI:

Laumann, E., Gagnon, J.H., Michael, R.T., and Michaels, S. (1994). The Social Organization of Sexuality: Sexual Practices in the United States. Chicago: University of Chicago Press.

Lievens, J. and Waege, H. (2009). Participatie in Vlaanderen. Basisgegevens van de participatiesurvey. Leuven: Acco.

Loosier, P.S. and Dittus, P.J. (2010). Group Differences in Risk Across Three Domains Using an Expanded Measure of Sexual Orientation. Journal of Primary Prevention, 31, 261-272. DOI:

Mathy, R.M., Schillace, M., Coleman, S.M., and Berquist, B.E. (2002). Methodological Rigor With Internet Samples: New Ways to Reach Underrepresented Populations. CyberPsychology and Behavior, 5, 253-266.

McCabe, S.E., Hughes, T.L., Bostwick, W., Morales, M., and Boyd, C.J. (2012).

Measurement of Sexual Identity in Surveys: Implications for Substance Abuse Research. Archives of Sexual Behavior, 41, 649-657. DOI: 1007/s10508-011-9768-7 Mercer, C.H., Bailey, J.V., Johnson, A.M., Erens, B., Wellings, K., Fenton, K.A., and Copas, A.J. (2007). Women Who Report Having Sex With Women: British National Probability Data on Prevalence, Sexual Behaviors, and Health Outcomes. American Journal of Public Health, 97, 1126-1133. DOI:

Meyer, I.H. and Wilson, P.A. (2009). Sampling Lesbian, Gay, and Bisexual Populations. Journal of Counseling Psychology, 56, 23-31. DOI:

Ramirez-Valles, J. (2002). The Protective Effects of Community Involvement for HIV Risk Behavior: A Conceptual Framework. Health Education Research, 17, 389-403.

Raymond, H.F., Rebchook, G., Curotto, A., Vaudrey, J., Amsden, M., Levine, D., and McFarland, W. (2010). Comparing Internet-Based and Venue-Based Methods to Sample MSM in the San Francisco Bay Area. Aids and Behavior, 14, 218-224. DOI:

Roberts, A.L., Austin, S.B., Corliss, H.L., Vandermorris, A.K., and Koenen, K.C. (2010). Pervasive Trauma Exposure Among US Sexual Orientation Minority Adults and Risk of Posttraumatic Stress Disorder. American Journal of Public Health, 100, 2433-2441. DOI:

Rothblum, E. (2007). From Science Fiction to Computer-Generated Technology: Sampling Lesbian, Gay, and Bisexual Individuals. In The Health of Sexual Minorities: Public Health Perspectives on Lesbian, Gay, Bisexual and Transgender Populations, I.H. Meyer and M.E. Northridge (eds). New York: Springer, 441-454.

Schillewaert, N. and Meulemeester, P. (2005). Comparing Response Distributions of Offline and Online Data Collection Methods. International Journal of Market Research, 47, 163-178.

Schonlau, M., van Soest, A., Kapteyn, A., and Couper, M. (2009). Selection Bias in Web-Surveys and the Use of Propensity Scores. Sociological Methods and Research, 37, 291-318.

Schwarcz, S., Spindler, H., Scheer, S., Valleroy, L., and Lansky, A. (2007). Assessing Representativeness of Sampling Methods for Reaching Men Who Have Sex With Men: A Direct Comparison of Results Obtained from Convenience and Probability Samples. AIDS and Behavior, 11, 596-602.

Silenzio, V.M.B., Duberstein, P.R., Tang, W., Lu, N., Tu, X., and Homan, C.M. (2009).

Connecting the Invisible Dots: Reaching Lesbian, Gay, and Bisexual Adolescents and Young Adults at Risk for Suicide Through Online Social Networks. Social Science and Medicine, 69, 469-474. DOI:

Tourangeau, R., Couper, M.P., and Steiger, D.M. (2003). Humanizing Self-Administered Surveys: Experiments on Social Presence in Web and IVR Surveys: Experiments on Social Presence in Web and IVR Surveys. Computers in Human Behavior, 19, 1-24. DOI:

Van Kesteren, N.M.C., Hospers, H., and Kok, G. (2007). Sexual Risk Behavior Among HIV-Positive Men Who Have Sex With Men: A Literature Review. Patient Education and Counseling, 65, 5-20.

Vincke, J. and Bleys, R. (2003). Vitale Vragen 2001. Eindrapport. Antwerp: Sensoa. Vincke, J. and Stevens, P. (1999). Een Beleidsgerichte Algemene Survey van Vlaamse Homoseksuele Mannen en Vrouwen--Basisrapport. Brussels: Ministerie van de Vlaamse Gemeenschap, Cel Gelijke Kansen, Universiteit Gent. Available at: (accessed March 30, 2014).

Vincke, J. and van Heeringen, K. (2004). Summer Holiday Camps for Gay and Lesbian Young Adults: An Evaluation of Their Impact on Social Support and Mental Well-Being. Journal of Homosexuality, 47, 33-46. DOI: http://www.dx.doi.org10.1300/J082v47n02_02

Wright, K.B. (2005). Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services. Journal of Computer-Mediated Communication, 10, article 11. Available at: (accessed March 30, 2014).

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information

IMPACT FACTOR 2017: 0.662
5-year IMPACT FACTOR: 1.113

CiteScore 2017: 0.74

SCImago Journal Rank (SJR) 2017: 1.158
Source Normalized Impact per Paper (SNIP) 2017: 0.860

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 158 158 17
PDF Downloads 53 53 2