<abstract xmlns="http://www.w3.org/1999/xhtml"><p>To mitigate the potentially harmful effects of nonresponse, most surveys repeatedly follow up with nonrespondents, often targeting a response rate or predetermined number of completes. Each additional recruitment attempt generally brings in a new wave of data, but returns gradually diminish over the course of a fixed data collection protocol, as each subsequent wave tends to consist of fewer responses than the last. Consequently, point estimates begin to stabilize. This is the notion of phase capacity, suggesting some form of design change is in order, such as switching modes, increasing the incentive, or, as is considered exclusively in this research, discontinuing the nonrespondent follow-up campaign altogether. A previously proposed test for phase capacity calls for multiply imputing nonrespondents’ missing data to assess, retrospectively, whether the most recent wave of data significantly altered a key, nonresponse-adjusted point estimate. This study introduces a more flexible adaptation amenable to surveys that instead reweight the observed data to compensate for nonresponse. Results from a simulation study and application indicate that, all else equal, the weighting version of the test is more sensitive to point estimate changes, thereby dictating more follow-up attempts are warranted.</p></abstract>

To mitigate the potentially harmful effects of nonresponse, most surveys repeatedly follow up with nonrespondents, often targeting a response rate or predetermined number of completes. Each additional recruitment attempt generally brings in a new wave of data, but returns gradually diminish over the course of a fixed data collection protocol, as each subsequent wave tends to consist of fewer responses than the last. Consequently, point estimates begin to stabilize. This is the notion of phase capacity, suggesting some form of design change is in order, such as switching modes, increasing the incentive, or, as is considered exclusively in this research, discontinuing the nonrespondent follow-up campaign altogether. A previously proposed test for phase capacity calls for multiply imputing nonrespondents’ missing data to assess, retrospectively, whether the most recent wave of data significantly altered a key, nonresponse-adjusted point estimate. This study introduces a more flexible adaptation amenable to surveys that instead reweight the observed data to compensate for nonresponse. Results from a simulation study and application indicate that, all else equal, the weighting version of the test is more sensitive to point estimate changes, thereby dictating more follow-up attempts are warranted.

Univariate Tests for Phase Capacity: Tools for Identifying When to Modify a Survey’s Data Collection Protocol

U.S. Office of Personnel Management (OPM)

Journal of Official Statistics

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

To mitigate the potentially harmful effects of nonresponse, most surveys repeatedly follow up with nonrespondents, often targeting a response rate or...

Univariate Tests for Phase Capacity: Tools for Identifying When to Modify a Survey’s Data Collection Protocol

Published Online: Sep 09, 2017

Page range: 601 - 624

Received: Mar 01, 2016

Accepted: May 01, 2017

DOI: https://doi.org/10.1515/jos-2017-0029

Keywords
Responsive survey design, multiple imputation, weighting, nonresponse

© 2017 Taylor Lewis, published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Univariate Tests for Phase Capacity: Tools for Identifying When to Modify a Survey’s Data Collection Protocol

Published Online: Sep 09, 2017

Page range: 601 - 624

Received: Mar 01, 2016

Accepted: May 01, 2017

DOI: https://doi.org/10.1515/jos-2017-0029

KeywordsResponsive survey design, multiple imputation, weighting, nonresponse

© 2017 Taylor Lewis, published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Keywords
Responsive survey design, multiple imputation, weighting, nonresponse