Until quite recently, telephone surveys have typically relied on landline telephone numbers. However, with the increasing popularity and affordability of mobile phones, there has been a surge in households that do not have landline connections. Additionally, there has been a decline in the response rates and population coverage of landline telephone surveys, creating a challenge to collecting representative social data. Dual-frame telephone surveys that use both landline and mobile phone sampling frames can overcome the incompleteness of landline-only telephone sampling. However, surveying mobile phone users introduces new complexities in sampling, nonresponse measurement and statistical weighting. This article examines these issues and illustrates the consequences of failing to include mobile-phone-only users in telephone surveys using data from Australia. Results show that there are significant differences in estimates of populations’ characteristics when using information solely from the landline or mobile telephone sample. These biases in the population estimates are significantly reduced when data from the mobile and landline samples are combined and appropriate dual-frame survey estimators are used. The optimal choice of a dual-frame estimation strategy depends on the availability of good-quality information that can account for the differential patterns of nonresponse by frame.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Akter S. and J. Bennett. 2011. “Household Perceptions of Climate Change and Preferences for Mitigation Action: The Case of the Carbon Pollution Reduction Scheme in Australia.” Climatic Change 109: 417–436. Doi: http://dx.doi.org/10.1007/s10584-011-0034-8.
Arcos A. M.M. Rueda M. Trujillo and D. Molina. 2014. “Review of Estimation Methods for Landline and Cell Phone Surveys.” Sociological Methods and Research 44: 458–485. Doi: http://dx.doi.org/10.1177/0049124114546904.
American Association for Public Opinion Research (AAPOR). 2010. “Cell Phone Task Force Report. New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the US With Respondents Reached via Cell Phone Numbers.” AAPOR. Available at: http://www.aapor.org/AAPORKentico/Education-Resources/Reports/Cell-Phone-Task-Force-Report.aspx. (accessed 25 November 2015).
American Association for Public Opinion Research (AAPOR). 2011. “Standard Definitions: Final Disposition of Case Codes and Outcome Rates for Surveys 7th Edition.” AAPOR. Available at: http://www.esomar.org/knowledge-and-standards/research-resources/aapor-standard-definitions.php. (accessed 25 November 2015).
Australian Bureau of Statistics (ABS). 2008. “How Australians Use Their Time: Time spent on cultural activities. ABS cat.no.4153.0.” Canberra: Australian Bureau of Statistics. Available at: http://www.abs.gov.au/ausstats/abs@.nsf/mf/4153.0 (accessed 25 November 2015).
Australian Bureau of Statistics (ABS). 2012. “Basic Community Profiles Census 2011. Canberra: Australian Bureau of Statistics.” Available at: http://www.abs.gov.au/websitedbs/censushome.nsf/home/communityprofiles. (accessed 25 November 2015).
Australian Bureau of Statistics (ABS). 2014. “Australian Health Survey: Users’ Guide 2011-13.” ABS cat.no. 4363.0.55.001. Canberra: Australian Bureau of Statistics. Available at: http://www.abs.gov.au/ausstats/abs@.nsf/mf/4363.0.55.001 (accessed 25 November 2015).
Australian Communications and Media Authority (ACMA). 2006 2011 2014 2015. Communications Report 2005–2006 2010–2011 2013–2014 & 2014–2015 Series “Converging Communications Channels: Preferences and Behaviour of Australian Communications Users.” ACMA: Canberra. Available at: www.acma.gov.au/Comms-Report. (accessed 25 November 2015).
Bankier M.D. 1986. “Estimators Based on Several Stratified Samples With Applications to Multiple Finite Surveys.” Journal of the American Statistical Association 81: 1074–1079. Doi: http://dx.doi.org/10.1080/01621459.1986.10478376.
Barnes G.M. J.H. Hoffman M.O. Tidwell and J.W. Welte. 2015. “Gambling and Substance Use: Co-Occurrence Among Adults in a Recent General Population Study in the United States.” International Gambling Studies 15: 55–71. Doi: http://dx.doi.org/10.1080/14459795.2014.990396.
Barr M.L. J.J. van Ritten D.G. Steel and S.V. Thackway. 2012. “Inclusion of Mobile Phone Numbers Into an Ongoing Population Health Survey in New South Wales Australia: Design Methods Call Outcomes Costs and Sample Representativeness.” BMC Medical Research Methodology 12: 177–185. Doi: http://dx.doi.org/10.1186/1471-2288-12-177.
Barr M.L. R.A. Ferguson P.J. Hughes and D.G. Steel. 2014. “Developing a Weighting Strategy to Include Mobile Telephone Numbers Into an Ongoing Population Health Survey Using an Overlapping Dual Frame Design With Limited Benchmark Information.” BMC Medical Research Methodology 14: 102–112. Doi: http://dx.doi.org/10.1186/1471-2288-14-102.
Béland Y. 2002. “Canadian Community Health Survey – Methodological Overview.” Health Reports - Statistics Canada 13: 9–14.
Blumberg S.J. and J.V. Luke. 2007. “Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults.” Public Opinion Quarterly 71: 734–749. Doi: http://dx.doi.org/10.1093/poq/nfm047.
Blumberg S.J. and J.V. Luke. 2014. “Wireless Substitution: Early Release Estimates from the National Health Interview Survey January-June 2014.” National Center for Health Statistics. Available at: http://www.cdc.gov/nchs/nhis/releases.htm (accessed 25 November 2015).
Blumberg S.J. and J.V. Luke. 2015. “Wireless Substitution: Early Release Estimates from the National Health Interview Survey July-December 2014.” National Center for Health Statistics. Available at: http://www.cdc.gov/nchs/nhis/releases.htm (accessed 25 November 2015).
Brick J.M. 2011. “The future of survey sampling.” Public Opinion Quarterly 75: 872–888. Doi: http://dx.doi.org/10.1093/poq/nfr045.
Brick J.M. 2013. “Unit Nonresponse and Weighting Adjustments: a Critical Review.” Journal of Official Statistics 29: 329 – 353. Doi: http://dx.doi.org/10.2478/jos-2013-0026.
Brick J.M. I.F. Cervantes S. Lee and G. Norman. 2011. “Nonsampling Errors in Dual Frame Telephone Surveys.” Survey Methodology 37: 1–12.
Brick J.M. S. Dipko S. Presser and C. Tucker. 2006. “Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers.” Public Opinion Quarterly 70: 780–793. Doi: http://dx.doi.org/10.1093/poq/nfl031.
Busse B. and M. Fuchs. 2012. “The Components of Landline Telephone Survey Coverage Bias. The Relative Importance of No-Phone and Mobile-Only Populations.” Quality & Quantity 46: 1209–1225. Doi: http://dx.doi.org/10.1007/s11135-011-9431-3.
Callegaro M. and T. Possio. 2004. “Mobile Telephone Growth and Coverage Error in Telephone Surveys.” Polis: Research and Studies in Italian Society and Politics 18: 477–506.
Deville J. and C.-E. Särndal. 1992. “Calibration Estimators in Survey Sampling.” Journal of the American Statistical Association 87: 376–382.
Flores Cervantes I. and G. Kalton. 2008. “Methods for Sampling Rare Populations in Telephone Surveys.” In Advances in Telephone Survey Methodology edited by J.M. Lepkowski C. Tucker J.M. Brick E.D. de Leeuw L. Japec P. Lavrakas M.W. Link and R.L. Sangster 113–132. Hoboken NJ: Wiley.
Fuller W.A. and L.F. Burmeister. 1972. “Estimators for Samples Selected From Two Overlapping Frames.” In Proceedings of the Social Statistics Section of American Statistical Association August 14–17 245–249. Alexandria VA: American Statistical Association.
Gong G. and F.J. Samaniego. 1981. “Pseudo Maximum Likelihood Estimation: Theory and Applications.” The Annals of Statistics 9: 861–869. Doi: http://dx.doi.org/10.1214/aos/1176345526.
Groves R.M. and E. Peytcheva. 2008. “The Impact of Non-Response Rates on Non-Response Bias.” Public Opinion Quarterly 72: 167–189. Doi: http://dx.doi.org/10.1093/poq/nfn011.
Hartley H.O. 1962. “Multiple Frame Surveys.” In Proceedings of the Social Statistics Section of American Statistical Association date of conference 203–206. Alexandria VA: American Statistical Association.
Hartley H.O. 1974. “Multiple Frame Methodology and Selected Applications.” Sankhya C 36: 99–118.
Holt D. and T.M.F. Smith. 1979. “Post-Stratification.” Journal of the Royal Statistical Society Series A 142: 33–46.
Hu S.S. L. Balluz M.P. Battaglia and M.R. Frankel. 2011. “Improving Public Health Surveillance Using a Dual Frame Survey of Landline and Cell Phone Numbers.” American Journal of Epidemiology 173: 703–711. Doi: http://dx.doi.org/10.1093/aje/kwq442.
Hu F.B. T.Y. Li G.A. Colditz W.C. Willett and J.E. Manson. 2003. “Television Watching and Other Sedentary Behaviours in Relation to Risk of Obesity and Type 2 Diabetes Mellitus in Women.” Journal of the American Medical Association (JAMA) 289: 1785–1791. Doi: http://dx.doi.org/10.1001/jama.289.14.1785.
Iachan R. and M.L. Dennis. 1993. “A Multiple Frame Approach to Sampling the Homeless and Transient Population.” Journal of Official Statistics 9: 747–764.
Kalton G. and D.W. Anderson. 1986. “Sampling Rare Populations.” Journal of the Royal Statistical Society Series A 149: 65–82. Doi: http://dx.doi.org/10.2307/2981886.
Keeter S. C. Miller A. Kohut R.M. Groves and S. Presser. 2000. “Consequences of Reducing Non-Responses in a National Telephone Survey.” Public Opinion Quarterly 64: 125–148. Doi: http://dx.doi.org/10.1086/317759.
Keeter S. C. Kennedy A. Clark T. Tompson and M. Mokrzycki. 2007. “What’s Missing From National Landline RDD surveys? The Impact of the Growing Cell-Only Population.” Public Opinion Quarterly 71: 772–792. Doi: http://dx.doi.org/10.1093/poq/nfm053.
Keeter S. S.C. Kennedy M. Dimock J. Best and P. Craighill. 2006. “Gauging the Impact of Growing Non-Response on Estimates from a National RDD Telephone Survey.” Public Opinion Quarterly 70: 759–779. Doi: http://dx.doi.org/10.1093/poq/nfl035.
Kennedy C. 2007. “Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD surveys.” Public Opinion Quarterly 71: 750–771. Doi: http://dx.doi.org/10.1093/poq/nfm050.
Kuusela V. M. Callegaro and V. Vehovar. 2008. “The Influence of Mobile Telephones on Telephone Surveys.” In Advances in Telephone Survey Methodology edited by J.M. Lepkowski C. Tucker J.M. Brick E.D. de Leeuw L. Japec P.J. Lavrakas M.W. Link and R.L. Sangster 87–112. Doi: http://dx.doi.org/10.1002/9780470173404.ch4.
Lee S. J.M. Brick E.R. Brown and D. Grant. 2010. “Growing Cell-Phone Population and Noncoverage Bias in Traditional Random Digit Dial Telephone Health Surveys.” Health Services Research 45: 1121–1139. Doi: http://dx.doi.org/10.1111/j.1475-6773.2010.01120.x.
Little R.J.A. 1993. “Post-Stratification: A Modeler’s Perspective.” Journal of the American Statistical Association 88: 1001–1012.
Livingston M. P. Dietze J. Ferris D. Pennay L. Hayes and S. Lenton. 2013. “Surveying Alcohol and Other Drug Use Through Telephone Sampling: A Comparison of Landline and Mobile Phone Samples.” BMC Medical Research Methodology 13: 41. Doi: http://dx.doi.org/10.1186/1471-2288-13-41.
Lohr S. 2009. “Multiple Frame Surveys.” In Handbook of Statistics Sample Surveys: Design Methods and Applications edited by D. Pfeffermann and C.R. Rao 71–88. Amsterdam: North Holland.
Lohr S. and J. Rao. 2000. “Inference From Dual Frame Surveys.” Journal of the American Statistical Association 95: 271–280. Doi: http://dx.doi.org/10.1080/01621459.2000.10473920.
Lohr S. and J. Rao. 2006. “Estimation in Multiple-Frame Surveys.” Journal of the American Statistical Association 101: 1019–1030. Doi: http://dx.doi.org10.1198/0162144506000000195.
Lopez M.H. and A. Gonzalez-Barrera. 2013. “Inside the 2012 Latino Electorate.” Washington D.C.: Pew Research Center’s Hispanic Trends Project. Available at: http://www.pewhispanic.org/2013/06/03/inside-the-2012-latino-electorate. (accessed 25 November 2015).
Mecatti F. 2007. “A Single Frame Multiplicity Estimator for Multiple Frame Surveys.” Survey Methodology 33: 151–157.
Mohorko A. E. de Leeuw and J. Hox. 2013. “Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage Across Countries and Over Time.” Survey Methods: Insights from the Field (SMIF) 2013 1–13. Doi: http://dx.doi.org/10.13094/SMIF-2013-00002.
Pennay D.W. 2010. “Profiling the ‘Mobile Only’ Population. Results From a Dual-Frame Telephone Survey Using a Landline and Mobile Phone Sample Frame.” In Proceedings of the Australian Consortium for Social and Political Research Incorporated (ACSPRI) Social Science Methodology Conference 1–3 December Sydney Australia. Available at: http://www.srcentre.com.au/docs/publications/dual_frame-survey_acspri-conference-paper_finalv2.pdf?sfvrsn=0 (accessed 25 November 2015).
Pennay D.W. and N. Vickers. 2012. “Dual-Frame Omnibus Survey.” Technical and Methodological Summary Report. Social Research Centre Pty Ltd Melbourne Australia. Available at: http://www.srcentre.com.au/docs/event-workshop-july-2012/dual-frame-omnibus-technical-report-(pennay).pdf?sfvrsn=2 (accessed 25 November 2015).
Pennay D.W. and N. Vickers. 2013. “Second Dual-Frame Omnibus Survey.” Technical and Methodological Summary Report. Social Research Centre Pty Ltd Melbourne Australia. Available at: http://www.srcentre.com.au/docs/publications/full-report-here.pdf?sfvrsn=0 (accessed 27 November 2015).
Pew Research Centre. 2012. “Assessing the Representativeness of Public Opinion Surveys.” Available at: http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ (accessed 25 November 2015).
Skinner C.J. 1991. “On Efficiency of Raking Ratio Estimation for Multiple Frame Surveys.” Journal of the American Statistical Association 86: 779–784.
Skinner C.J. and J.N.K. Rao. 1996. “Estimation in Dual Frame Surveys with Complex Designs.” Journal of the American Statistical Association 91: 349–356.
Steeh C. 2008. “Telephone Surveys.” In International Handbook of Survey Methodology edited by E.D. de Leuuw J.J. Hox and D.A. Dillman 221–238. New York: Routledge.
Thomée S. A. Härenstam and M. Hagberg. 2011. “Mobile Phone Use and Stress Sleep Disturbances and Symptoms of Depression Among Young Adults - A Prospective Cohort Study.” BMC Public Health 11: 66. Doi: http://dx.doi.org/10.1186/1471-2458-11-66.
Tucker C. and J.M. Brick. 2007. “Household Telephone Service and Usage Patterns in the United States in 2004: Implications for Telephone Samples.” Public Opinion Quarterly 71: 3–22. Doi: http://dx.doi.org/10.1093/poq/nfl047.
Vicente P. and E. Reis E. 2009. The “Mobile Only Population in Portugal and its Impact in a Dual Frame Telephone Survey.” Survey Research Methods 3: 105–111.