Open access

Abstract

The U.S. Census Bureau conducted a series of experiments to evaluate alternative contact strategies. We hoped to identify effective mail strategies that increase timeliness of response and reduce the number of cases receiving more-costly follow-ups. These experiments were incorporated into the collection of several annual business surveys and one quarterly survey. This article summarizes results from five experiments whose designs are based on business survey decision-making and response processes. We obtained clear, positive results from tests involving manipulation of mail sequences, but only modest results from tests in which we varied messages and envelope appearance, whose effects were seen only in specific industry subgroups.

Barron, S., C. Reiser, M. Bentley, J. Hill, and A. Meier. 2012. “2010 Census Deadline Messaging and Compressed Mailing Schedule Experiment.” Paper presented at the 2012 Joint Statistical Meetings, San Diego, CA.

Barry, T. E. 1987. “The Development of the Hierarchy of Effects: An Historical Perspective.” Current Issues and Research in Advertising 10(1–2): 251–295.

Barth, D., M. Zelenak, M. Asiala, E. Castro, and A. Roberts. 2016. 2015 Envelope Mandatory Messaging Test: Final Report. Suitland, MD: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2016/acs/2016_Barth_01.pdf (accessed April 2018).

Bavdaž, M. 2010. “The Multidimensional Integral Business Survey Response Model.” Survey Methodology 36(1): 81–93.

Britt, R. and F. Featherston. 2007. “Increasing Survey Cooperation: Motivating Chronic Late Responders to an Annual Survey.” Paper presented at the Third International Conference on Establishment Surveys, June 18–21, 2007, Montreal, Quebec, Canada.

Chun, Y.I. and K. Robertson. 1995. “The Effects of Advance Letters and Reminder/Thank You Letters on Reducing Nonresponse in an Establishment Survey: An Experimental Study.” Washington, D.C., USA: Bureau of Labor Statistics. Available at: https://stats.bls.gov/osmr/pdf/st950020.pdf (accessed April 2018).

Cialdini, R. 1984. Influence: The Psychology of Persuasion. NY: Quill.

Claveau, J. and C. Turmelle. 2012. “An Embedded Experiment to Test Non-Response Follow-Up Strategies when Using Electronic Questionnaires.” Paper presented at the Conference of European Statisticians, June 6–8, 2012. Paris, France.

Crawford, S., M. Couper, and M. Lamias. 2001. “Web Surveys: Perceptions of Burden.” Social Science Computer Review 19(2001): 146–162. Doi: http://dx.doi.org/10.1177/089443930101900202.

Davis, W. and N. Pihama. 2009. “Survey Response as Organizational Behavior: An Analysis of the Annual Enterprise Survey, 2003–2007.” New Zealand Association of Economists Conference, July 1–3, 2009. Wellington, New Zealand.

Dillman, D., J. Smith, and L. Christian. 2009. Internet, Mail, and Mixed Mode Surveys: the Tailored Design Method. Hoboken, NJ: John Wiley & Sons.

Dillman, D., E. Singer, J. Clark, and J. Treat. 1996. “Effects of Benefits Appeals, Mandatory Appeals, and Variations in Statements of Confidentiality on Completion Rates for Census Questionnaires.” Public Opinion Quarterly 60(3): 376–389. Doi: https://dx.doi.org/10.1086/297759.

Eggleston, C., J. Childs, and E. Nichols. 2016. “Conjoint Analysis of Survey Invitation Messages: A New Method to Understand What Moves People to Respond.” Presented at the Federal Computer-Assisted Survey Instrument Collection Conference, May 3–4, 2016. Suitland, MD.

Erikson, J. and P. Hartwig. 2012. “Trying to Improve Response Rates by Lowering the Perceived Response Burden of a Survey.” Presented at the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montreal, Quebec, Canada.

Franklin, J., J. Simpson, J. Wine, and N. Paoli. 2007. “Developing a Contact Strategy to Maximize Self-Administered Web Participation.” Presented at the American Association for Public Opinion Research Conference, May 17–20, 2007. Anaheim, CA.

Fulton, J., G. Morales, and J. Childs. 2016. “Effects of Messaging to Encourage Response to the American Community Survey.” Presented at the American Association of Public Opinion Research Conference, May 12–15, 2016. Austin, TX.

Giesen, D. 2012. “Exploring Causes and Effects of Perceived Response Burden.” Paper presented at the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montreal, Quebec, Canada.

Groves, R., D. Cantor, M. Couper, K. Levin, K. McGonagle, E. Singer, and J. Van Hoewyk. 1997. “Research Investigations in Gaining Participation from Sample Firms in the Current Employment Statistics Program.” In proceedings of the Survey Research Methods Section, American Statistical Association: 289–294. Anaheim, California, USA, August 10–14, 1997.

Groves, R., R. Cialdini, and M. Couper. 1992. “Understanding the Decision to Participate in a Survey.” Public Opinion Quarterly 56(4): 475–495. Doi: http://dx.doi.org/10.1086/269338.

Groves, R. and M. Couper. 1998. Nonresponse in Household Interview Surveys. Hoboken, NJ: John Wiley & Sons.

Haraldsen, G. 2013. “Quality Issues in Business Surveys.” In Designing and Conducting Business Surveys, edited by Snijkers, G., Haraldsen, G., Jones, J. and Willimack, D. John Wiley and Sons: Hoboken, NJ.

Hedlin, D., H. Lindkvist, H. Bäckström, and J. Erikson. 2008. “An Experiment on Perceived Survey Response Burden among Businesses.” Journal of Official Statistics 24(2): 301–318.

Her Majesty’s Revenue Commission. 2010. Large Business Methodology Review: Stage Two Report. Her Majesty’s Revenue and Customs Research Report 98. London, UK: Her Majesty’s Revenue Commission. Available at: https://www.gov.uk/government/publications/government-sponsored-research-of-large-businesses (accessed April 2018).

Hernandez, A., C. Fan, and A. Tuttle. 2016. “The Effect of a Pre-Due Date Reminder Letter on Nonresponse in a Business Survey.” Presented at the American Association of Public Opinion Research Conference, May 12–15, 2016. Austin, TX.

Herndandez, A. 2016. 2015 ARTS Contact Strategies Experiment Results. Suitland, MD: U.S. Census Bureau. Internal report.

Hughes, T. and J. Tancreto. 2015. “Refining the Web Response Option in the Multiple Mode Collection of the American Community Survey.” Presented at the European Survey Research Association Conference, July 13–17, 2015. Reykjavik, Iceland.

Kaplowitz, M.D., T.D. Hadlock, and R. Levine. 2004. “A Comparison of Web and Mail Survey Response Rates.” Public Opinion Quarterly 68(1): 94–101. Doi: http://dx.doi.org/10.1093/poq/nfh006.

Keller, J. 2015. 2017 Economic Census Publicity and Outreach Phase 1: Focus Group Research. Suitland, MD: U.S. Census Bureau. Internal presentation.

Keller, S., R. Hough, T. Falconer, and K. Curcio. 2011. “Characteristics of Large Company Respondents to an Economic Survey.” Presented at the 2011 Joint Statistical Meetings, July 30 – August 4, 2011. Miami Beach, FL.

Landreth, A. 2002. Results and Recommendations from Cognitive Interviews with Selected Materials from the American Community Survey. Suitland, MD: U.S. Census Bureau. Available at: http://www.census.gov/srd/papers/pdf/ssm2003-10.pdf (accessed April 2018).

Landreth, A. 2003. SIPP Advance Letter Focus Group Results: Issues and Recommendations. Suitland, MD: U.S. Census Bureau. Available at: http://www.census.gov/srd/papers/pdf/ssm2003-07.pdf (accessed April 2018).

Landreth, A. 2004. “Survey Letters: A Respondent’s Perspective.” Paper presented at the American Association for Public Opinion Research, May 13–16, 2004. Phoenix, Arizona.

Langeland, J. and A. Tuttle. 2016. “Improving Data Collection: Results from Two Contact Strategy Experiments in Economic Surveys.” Presented at the Fifth International Conference on Establishment Surveys, June 20–23, 2016. Geneva, Switzerland.

Leslie, T., D. Griffin, P. McGovern, and D. Raglin. 2004. “Measuring the Impact of Questionnaire and Envelope Messages on Respondent Behavior.” Presented at the Joint Statistical Meetings, August 8–12, 2004. Toronto, Ontario, Canada.

Lor, M., B. Bowers, and A. Krupp. 2017. “Tailored Explanation: A Strategy to Minimize Nonresponse in Demographic Items Among Low-Income Racial and Ethnic Minorities.” Survey Practice 10(3). Doi: http://dx.doi.org/10.29115/SP-2017-0015.

Lorenc, B., M. Bavdazˇ, D. Diesen, R. Seljak, and V. Torres van Grinsven. 2012. “Businesses as Users of Official Statistics.” Paper presented at the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montreal, Quebec, Canada.

Lykes, V. and J. Meyers. 2017. “Got Mail? Drivers of Mail-to-Online Response Rates.” Presented at the American Association of Public Opinion Research Conference, May 18–21, 2017. New Orleans, Louisiana.

Marquette, E., M. Kornbau, and J. Toribio. 2015. “Testing Contact Strategies to Improve Response in the 2012 Economic Census.” Paper presented at the Joint Statistical Meetings, August 8–13, 2015. Seattle, Washington.

McCarthy, J., T. Wilson, and A. Dau. 2015. “Adept Adaptations: Developing Alternative Strategies in Adaptive Survey Designs.” Presented at the Federal Committee on Statistical Methods Conference, December 1–3, 2015. Washington, D.C.

McCarthy, J., D. Beckler, and M. Qualey. 2006. “An Analysis of the Relationship Between Survey Burden and Nonresponse: If We Bother Them More, Are They Less Cooperative?” Journal of Official Statistics 22(1): 97–112.

Matthews, B. 2012. 2011 American Community Survey Internet Tests: Results from Second Test in November 2011. Suitland, MD: U.S. Census Bureau. Internal report.

Nichols, E. 2012. The April 2011 American Community Survey Internet Test: Attitudes and Behavior Study Follow up. Suitland, MD: U.S. Census Bureau. Available at: https://www.census.gov/content/dam/Census/library/working-papers/2012/acs/2012_Nichols_01.pdf (accessed April 2018).

Perrins, G., P. Phipps, J. Pinkos, T. Shaffer, G. Stamas, S. Suchman, C. Tucker, and S. Watkins. 2009. Results from the Occupational Employment Survey Letter and Brochure Experiment. Washington, D.C: Bureau of Labor Statistics. Internal report.

Phipps, P. 2008. “The Occupational Employment Statistics (OES) Program: Cover Letter and Brochure Experiments.” Presented at the Federal Computer-Assisted Survey Instrument Collection Conference, March 11–13, 2008. Washington, D.C.

Reiser, C. 2013. “Internet Response for the Decennial Census – 2012 National Census Test.” Presented at the American Association of Public Opinion Research Conference, May 16–19, 2013. Boston, MA.

Shih, T. and X. Fan. 2008. “Comparing Response Rates from Web and Mail Surveys: A Meta-Analysis.” Field Methods 20(3): 249–271. Doi: http://dx.doi.org/10.1177/1525822X08317085.

Snijkers, G., B. Berkenbosch, and M. Luppes. 2007. “Understanding the Decision to Participate in a Business Survey.” Paper presented at the Third International Conference on Establishment Surveys, June 18–21, 2007. Montreal, Quebec, Canada.

Snijkers, G. and J. Jones. 2013. “Business Survey Communication.” In Designing and Conducting Business Surveys, edited by Snijkers, G., Haraldsen, G., Jones, J. and Willimack, D., 359–430. John Wiley and Sons: Hoboken, NJ.

Snijkers, G. and G. Haraldsen. 2013. “Managing the Data Collection.” In Designing and Conducting Business Surveys, edited by Snijkers, G., Haraldsen, G., Jones, J. and Willimack, D., 431–457. John Wiley and Sons: Hoboken, NJ.

Tancreto, J. 2012. 2011 American Community Survey Internet Tests: Results from First Test in April 2011. Suitland, MD: U.S. Census Bureau. Internal report.

Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1994. “Organizational Survey Nonresponse.” Administrative Science Quarterly 39: 439–457. Doi: http://dx.doi.org/10.2307/2393298.

Torres van Grinsven, V., I. Bolko, and M. Bavdazˇ. 2012. “Sources of Motivation in Business Surveys.” Paper presented at the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montreal, Quebec, Canada.

Tulp, D., C. Hoy, G. Kusch, and S. Cole. 1991. “Nonresponse under Mandatory vs. Voluntary Reporting in the 1989 Survey of Pollution Abatement Costs and Expenditures (PACE).” In JSM Proceedings, Survey Research Methods Section. August 18–22, 1991. Alexandria, VA: American Statistical Association: 272–277.

Tuttle, A. 2009. “Establishment Respondents as Survey Managers: Using Survey Design Features to Empower Respondents to Find and Recruit Knowledgeable Company Personnel for Assistance.” Presented at the Federal Committee on Statistical Methodology Research Conference, November 2–4, 2009. Washington D.C.

Tuttle, A. 2011. Large Company Reporting Practices Program – Company Visits Highlights, Round 1. Suitland, MD: U.S. Census Bureau. Internal report.

Tuttle, A. 2016. “Developing Electronic Tools to Support the Logistical and Collaborative Needs of Respondents to the Economic Census.” Presented at the Federal Computer-Assisted Survey Instrument Collection Conference, May 3–4, 2016. Suitland, MD.

Tuttle, A., K. Pick, R. Hough, and J. Mulrow. 2010. “Experimenting with Pre-contact Strategies for Reducing Nonresponse in an Economic Survey.” In JSM Proceedings, Survey Research Methods Section. July 31 – August 5, 2010. Alexandria, VA: American Statistical Association: 5253–5262.

Tuttle, A. and D. Willimack. 2005. “Privacy Principles and Data Sharing: Implications of CIPSEA for Economic Survey Respondents.” Presented at the Federal Committee on Statistical Methodology Research Conference, November 14–16, 2005. Washington D.C.

Tuttle, A. and D. Willimack. 2012. “Social Network Analysis as a Tool for Assessing Data Quality and Respondent Burden in Establishment Surveys.” Presented at the Fourth International Conference on Establishment Surveys, June 11–14, 2012. Montreal, Quebec, Canada.

Willimack, D., E. Nichols, and S. Sudman. 2002. “Understanding Unit and Item Nonresponse in Business Surveys.” In Survey Nonresponse, edited by Groves, R. et al. New York: Wiley, 213–227.

Willimack, D. and E. Nichols. 2010. “A Hybrid Response Process Model for Business Surveys.” Journal of Official Statistics 26(1): 3–24.

Willimack, D., E. Nichols, and S. Sudman. 1999. “Understanding the Questionnaire in Business Surveys.” In JSM Proceedings, Survey Research Methods Section. August 8–12, 1999. Alexandria, VA: American Statistical Association: 889–94.

Willimack, D. and G. Snijkers. 2013. “The Business Context and its Implications for the Survey Response Process.” In Designing and Conducting Business Surveys, edited by Snijkers, G., Haraldsen, G., Jones, J. and Willimack, D. page numbers. Wiley: Hoboken, NJ.

Zotti, A., L. Avenilla, S. Coffey, and G. Walejko. 2016. “The Effects of Sending a Pre-notice Postcard in a CAPI-conducted Survey.” Presented at the American Association of Public Opinion Research Conference, May 12–15, 2016. Austin, TX.

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2017: 0.662
5-year IMPACT FACTOR: 1.113

CiteScore 2017: 0.74

SCImago Journal Rank (SJR) 2017: 1.158
Source Normalized Impact per Paper (SNIP) 2017: 0.860

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 224 224 69
PDF Downloads 93 93 46