<abstract xmlns="http://www.w3.org/1999/xhtml">More and more surveys are conducted online. While web surveys are generally cheaper and tend to have lower measurement error in comparison to other survey modes, especially for sensitive questions, potential advantages might be offset by larger nonresponse bias. This article compares the data quality in a web survey administration to another common mode of survey administration, the telephone.The unique feature of this study is the availability of administrative records for all sampled individuals in combination with a random assignment of survey mode. This specific design allows us to investigate and compare potential bias in survey statistics due to 1) nonresponse error, 2) measurement error, and 3) combined bias of these two error sources and hence, an overall assessment of data quality for two common modes of survey administration, telephone and web.Our results show that overall mean estimates on the web are more biased compared to the telephone mode. Nonresponse and measurement bias tend to reinforce each other in both modes, with nonresponse bias being somewhat more pronounced in the web mode. While measurement error bias tends to be smaller in the web survey implementation, interestingly, our results also show that the web does not consistently outperform the telephone mode for sensitive questions.</abstract>

More and more surveys are conducted online. While web surveys are generally cheaper and tend to have lower measurement error in comparison to other survey modes, especially for sensitive questions, potential advantages might be offset by larger nonresponse bias. This article compares the data quality in a web survey administration to another common mode of survey administration, the telephone.The unique feature of this study is the availability of administrative records for all sampled individuals in combination with a random assignment of survey mode. This specific design allows us to investigate and compare potential bias in survey statistics due to 1) nonresponse error, 2) measurement error, and 3) combined bias of these two error sources and hence, an overall assessment of data quality for two common modes of survey administration, telephone and web.Our results show that overall mean estimates on the web are more biased compared to the telephone mode. Nonresponse and measurement bias tend to reinforce each other in both modes, with nonresponse bias being somewhat more pronounced in the web mode. While measurement error bias tends to be smaller in the web survey implementation, interestingly, our results also show that the web does not consistently outperform the telephone mode for sensitive questions.

The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias

Journal of Official Statistics

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

More and more surveys are conducted online. While web surveys are generally cheaper and tend to have lower measurement error in comparison to other survey...

The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias

Published Online: Mar 26, 2019

Page range: 93 - 115

Received: Apr 01, 2016

Accepted: May 01, 2018

DOI: https://doi.org/10.2478/jos-2019-0005

Keywords
Mode effects, telephone survey, web survey, combined bias

© 2019 Barbara Felderer et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias

Published Online: Mar 26, 2019

Page range: 93 - 115

Received: Apr 01, 2016

Accepted: May 01, 2018

DOI: https://doi.org/10.2478/jos-2019-0005

KeywordsMode effects, telephone survey, web survey, combined bias

© 2019 Barbara Felderer et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Keywords
Mode effects, telephone survey, web survey, combined bias