Assessing Nonresponse in a Longitudinal Establishment Survey Using Regression Trees

Open access


This article introduces and discusses a method for conducting an analysis of nonresponse for a longitudinal establishment survey using regression trees. The methodology consists of three parts: analysis during the frame refinement and enrollment phases, common in longitudinal surveys; analysis of the effect of time on response rates during data collection; and analysis of the potential for nonresponse bias. For all three analyses, regression tree models are used to identify establishment characteristics and subgroups of establishments that represent vulnerabilities during the data collection process. This information could be used to direct additional resources to collecting data from identified establishments in order to improve the response rate.

6. References

  • Andridge, R.R. and R.J.A. Little. 2011. “Proxy Pattern-Mixture Analysis for Survey Nonresponse.” Journal of Official Statistics 27: 153–180.

  • Andridge, R.R. and K.J. Thompson. 2015. “Assessing Nonresponse Bias in a Business Survey: Proxy Pattern-Mixture Analysis for Skewed Data.” Annals of Applied Statistics 9(4): 2237–2265. Doi:

  • Atrostic, B.K., N. Bates, and A. Silberstein. 2001. “Nonresponse in US Government Household Surveys: Consistent Measures, Recent Trends, and New Insights.” Journal of Official Statistics 17: 209–226.

  • Bureau of Labor Statistics. 2015. Handbook of Methods. “Job Openings and Labor Turnover Survey” Chapter 18. Available at: (accessed October 2017).

  • Davis, W.R. and N. Pihama. 2009. “Survey Response as Organisational Behaviour: an Analysis of the Annual Enterprise Survey, 2003–2007.” New Zealand Association of Economists Conference. New Zealand: New Zealand Association of Economists. Available at: (accessed October 2017).

  • De Heer, W. and E. De Leeuw. 2002. “Trends in Household Survey Nonresponse: a Longitudinal and International Comparison.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 41–54. New York: John Wiley and Sons.

  • Dillman, D.A. 2000. Mail and Internet Surveys: The Tailored Design Method (second edition). New York: John Wiley & Sons.

  • Dillman, D.A., J.D. Smyth, and L.M. Christian. 2009. Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method (third edition). Hoboken, NJ: John Wiley & Sons.

  • Freedman, D.S., A. Thornton, and D. Camburn. 1980. “Maintaining Response Rates in Longitudinal Studies.” Sociological Methods & Research 9: 87–98.

  • Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646–675.

  • Groves, R.M., D.A. Dillman, J.L. Eltinge, and R.J.A. Little. 2002. Survey Nonresponse. New York: John Wiley and Sons.

  • Heberlein, T.A. and R. Baumgartner. 1978. “Factors Affecting Response Rates to Mailed Questionnaires: A Quantitative Analysis of the Published Literature.” American Sociological Review 43: 447–462.

  • Holbrook, A., J.A. Krosnick, and A. Pfent. 2007. “The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms.” In Advances in Telephone Survey Methodology, edited by J.M. Lepkowski, C. Tucker, J.M. Brick, E. De Leeuw, L. Japec, P.J. Lavrakas, M.W. Link and R.L. Sangster, 499–528. Hoboken, NJ: Wiley.

  • Hothorn, T., K. Hornik, and A. Zeileis. 2006. “Unbiased Recursive Partitioning: a Conditional Inference Framework.” Journal of Computational and Graphical Statistics 15: 651–674.

  • Janik, F. and S. Kohaut. 2012. “Why Don’t They Answer? Unit Nonresponse in the IAB Establishment Panel.” Quality & Quantity 46: 917–934.

  • Kreuter, F., K. Olson, J. Wagner, T. Yan, T.M. Ezzati-Rice, C. Casas-Cordero, M. Lemay, A. Peytchev, R.M. Groves, and T.E. Raghunathan. 2010. “Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-Response: Examples from Multiple Surveys.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 173: 389–407.

  • Lepkowski, J.M. and M.P. Couper. 2002. “Nonresponse in the Second Wave of Longitudinal Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 259–272. New York: John Wiley and Sons.

  • Little, R.J.A. and S. Vartivarian. 2005. “Does Weighting for Nonresponse Increase the Variance of Survey Means?” Survey Methodology 31: 161–168.

  • Lohr, S., V. Hsu, and J. Montaquila. 2015. “Using Classification and Regression Trees to Model Survey Nonresponse.” In Joint Statistical Meetings, Proceedings of the Survey Research Methods Section: American Statistical Association. 2071–2085. Alexandria, VA: American Statistical Association. Available at: (accessed October 2017).

  • Morgan, J.N. and J.A. Sonquist. 1963. “Problems in the Analysis of Survey Data, and a Proposal.” Journal of the American Statistical Association 58: 415–434.

  • Office of Management and Budget. 2006. Statistical Directive No. 2, Standards and Guidelines for Statistical Surveys 2006. Washington DC: OMB. Available at: (accessed October 2017).

  • Paxson, M.C., D.A. Dillman, and J. Tarnai. 1995. “Improving Response to Business Mail Surveys.” In Business Survey Methods, edited by B.G. Cox, D.A. Binder, B.N. Chinnappa, A. Christianson, M.J. Colledge, and P.S. Kott, 303–316. New York: John Wiley and Sons.

  • Phipps, P. and D. Toth. 2012. “Analyzing Establishment Nonresponse Using an Interpretable Regression Tree Model with Linked Administrative Data.” The Annals of Applied Statistics 6: 772–794.

  • Särndal, C. 2011. “The 2010 Morris Hansen Lecture. Dealing with Survey Nonresponse in Data Collection, in Estimation.” Journal of Official Statistics 27(1): 1–21.

  • Särndal, C. and P. Lundquist. 2014. “Accuracy in Estimation with Nonresponse: A Function of Degree of Imbalance and Degree of Explanation.” Journal of Survey Statistics and Methodology 2(4): 361–387. Doi:

  • Schouten, B., F. Cobben, and J. Bethlehem. 2009. “Indicators for the Representativeness of Survey Response.” Survey Methodology 35(1): 101–113.

  • Schouten, B., N. Shlomo, and C. Skinner. 2011. “Indicators for Monitoring and Improving Representativeness of Response.” Journal of Official Statistics 27(2): 231–253.

  • Seiler, C. 2010. Dynamic Modelling of Nonresponse in Business Surveys. No. 93. Ifo Working Paper. Available at: (accessed October 2017).

  • Singer, E. 2002. “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 163–177. New York: John Wiley and Sons.

  • Smaill, K. 2012. “Trajectory Modelling of Longitudinal Non-Response in Business Surveys.” Statistical Journal of the International Association of Official Statistics 28: 137–144.

  • Su, X., M. Wang, and J. Fan. 2004. “Maximum Likelihood Regression Trees.” Journal of Computational and Graphical Statistics 13: 586–598.

  • Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1994. “Organizational Survey Nonresponse.” Administrative Science Quarterly 39: 439–457.

  • Toth, D. 2017. rpms: Recursive Partitioning for Modeling Survey Data. R package version 0.2.0. Available at: (accessed October 2017).

  • Wagner, J. 2010. “The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data.” Public Opinion Quarterly 74: 223–243. Doi:

  • Wagner, J. 2012. “A Comparison of Alternative Indicators for the Risk of Nonresponse Bias.” Public Opinion Quarterly 76: 555–575. Doi:

  • Watson, N. and M. Wooden. 2009. “Identifying Factors Affecting Longitudinal Survey Response.” In Methodology of Longitudinal Surveys, edited by Peter Lynn, 151–181. Hoboken, NJ: John Wiley and Sons.

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information

IMPACT FACTOR 2016: 0.411
5-year IMPACT FACTOR: 0.776

CiteScore 2016: 0.63

SCImago Journal Rank (SJR) 2016: 0.710
Source Normalized Impact per Paper (SNIP) 2016: 0.975


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 71 71 71
PDF Downloads 35 35 35