The Impact of Pictures on Best-Worst Scaling in Web Surveys

Open access


Motivation and burden are two of the most important aspects that influence response rates and dropouts in online surveys. As a result, we focus our analyses on how pictures and Best Worst Scaling (BWS), two solutions for each problem, interact in the Web medium. We use an experimental design that compares a BWS with pictures, the experimental group, and BWS without pictures, the control group. Results show that pictures influence measurement of BWS in six out of 16 items. We also observe that Couper's (2001) conclusion that concordant text and images have an accentuation effect while a discordant relationship between the two has an interference impact is partly true in our data. Eight out of the 16 items are at least partially influenced by the concordant/discordant variable while four fully respect this model. We conclude by discussing the impact of our findings and its limitations.

Cook, C., Heath, F., & Thompson, R. L. (2000). A Meta-Analysis of Response Rates in Web-or Internet-Based Surveys. Educational and Psychological Measurement, 60(6), 821–836. doi:10.1177/00131640021970934

Couper, M. P. (2000). Review: Web surveys: A review of issues and approaches. The Public Opinion Quarterly, 64(4), 464-494.

Couper, M. P. (2001). Web surveys: The questionnaire design challenge. Proceedings of the 53rd session of the ISI. Retrieved from

Couper, M. P., Conrad, F. G., & Tourangeau, R. (2007). Visual Context Effects in Web Surveys. Public Opinion Quarterly, 71(4), 623–634. doi:10.1093/poq/nfm044

Couper, M. P., Tourangeau, R., & Kenyon, K. (2004). Picture This!: Exploring Visual Effects in Web Surveys. Public Opinion Quarterly, 68(2), 255-266. doi:10.1093/poq/nfh013

Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web Surveys: Perceptions of Burden. Social Science Computer Review, 19(2), 146–162. doi:10.1177/089443930101900202

Dillman, D. A., & Smyth, J. D. (2007). Design Effects in the Transition to Web-Based Surveys. American Journal of Preventive Medicine, 32(5), S90-S96. doi:10.1016/j.amepre.2007.03.008

Dillman, D. A., Tortora, R. D., Conradt, J., & Bowker, D. (1998). Influence of Plain Vs. Fancy Design on Response Rates for Web Surveys. Proceedings of the Survey Research Methods Section, American Statistical Association, 1998.

Fan, W., & Yan, Z. (2010). Factors Affecting Response Rates of the Web Survey: A Systematic Review. Computers in Human Behavior, 26(2), 132–139. doi:10.1016/j.chb.2009.10.015

Finn, A., & Louviere, J. J. (1992). Determining the Appropriate Response to Evidence of Public Concern: The Case of Food Safety. Journal of Public Policy & Marketing, 11(2), 12-25.

Galesic, M. (2006). Dropouts on the Web: Effects of Interest and Burden Experienced During an Online Survey. Journal of Official Statistics, 22(2), 313.

Göritz, A. S. (2006). Incentives in Web studies: Methodological issues and a review. International Journal of Internet Science, 1(1), 58-70.

Kahle, L. R., Beatty, S. E., & Homer, P. (1986). Alternative measurement approaches to consumer values: the list of values (LOV) and values and life style (VALS). Journal of consumer research, 13(3), 405-409.

Kivu, M. (2010). Long Questionnaires: Impact on Abandon Rate (1-5). Bucharest: IPSOS - Romania.

Lee, J. A., Soutar, G., & Louviere, J. (2008). The Best–Worst Scaling Approach: An Alternative to Schwartz's Values Survey. Journal of Personality Assessment, 90(4), 335-347. doi:10.1080/00223890802107925

Louviere, J. J. (1988). Conjoint Analysis Modelling of Stated Preferences: A Review of Theory, Methods, Recent Developments and External Validity. Journal of Transport Economics and Policy, 93-119.

Mahon-Haft, T. A., & Dillman, D. A. (2010). Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality. Survey Research Methods, 4, 43-59.

Manfreda, K., & Vehovar, V. (2002). Survey design features influencing response rates in web surveys. The International Conference on Improving Surveys Proceedings. Retrieved from

Marley, A. A. J., & Louviere, J. J. (2005). Some Probabilistic Models of Best, Worst, and Best–Worst Choices. Journal of Mathematical Psychology, 49(6), 464-480. doi:10.1016/

Moore, W. L., Jason Gray-Lee, & Louviere, J. J. (1998). A Cross-Validity Comparison of Conjoint Analysis and Choice Models at Different Levels of Aggregation. Marketing Letters, 9(2), 195-207.

Orme, B. (2000). Hierarchical Bayes: Why all the Attention? Sawtooth Software Research Paper Series, 1-7.

Sproull, L., Subramani, M., Kiesler, S., Walker, J. H., & Waters, K. (1996). When the Interface is a Face. Human-Computer Interaction, 11(2), 97-124.

Tavares, S., Cardoso, M., & Dias, J. G. (2010). The Heterogeneous Best-Worst Choice Method in Market Research. International Journal of Market Research, 52(4), 533. doi:10.2501/S1470785309201430

The American Association for Public Opinion Research. (2011). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 7th edition.

Toepoel, V., & Couper, M. P. (2010). Can Verbal Instructions Counteract Visual Context Effects in Web Surveys? Public Opinion Quarterly, 75(1), 1-18. doi:10.1093/poq/nfq044

Winters, L. (1989). SRI Announces VALS 2. Marketing Research, 1(2), 67.

Journal Information


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 103 103 4
PDF Downloads 39 39 2