Detecting Reporting Errors in Data from Decentralised Autonomous Administrations with an Application to Hospital Data

Open access

Abstract

Administrative data sources are increasingly used by National Statistical Institutes to compile statistics. These sources may be based on decentralised autonomous administrations, for instance municipalities that deliver data on their inhabitants. One issue that may arise when using these decentralised administrative data is that categorical variables are underreported by some of the data suppliers, for instance to avoid administrative burden. Under certain conditions overreporting may also occur.

When statistical output on changes is estimated from decentralised administrative data, the question may arise whether those changes are affected by shifts in reporting frequencies. For instance, in a case study on hospital data, the values from certain data suppliers may have been affected by changes in reporting frequencies. We present an automatic procedure to detect suspicious data suppliers in decentralised administrative data in which shifts in reporting behaviour are likely to have affected the estimated output. The procedure is based on a predictive mean matching approach, where part of the original data values are replaced by imputed values obtained from a selected reference group. The method is successfully applied to a case study with administrative hospital data.

Backor, K., S. Golde, and N. Nie. 2007. “Estimating Survey Fatigue in Time Use Study.” Paper presented at the 29th Annual Conference of the International Association of Time Use Research, 17–19 October 2007, Washington, DC, U.S.A. Available at http://www.atususers.umd.edu/wip2/papers_i2007/Backor.pdf (accessed October 2018).

Bakker, B.F.M. and P.J.H. Daas. 2012. Methodological Challenges of Register-based Research. Statistica Neerlandica, 66: 2–7. Doi: http://dx.doi.org/10.1111/j.1467-9574.2011.00505.x.

Berenschot. 2012. Inventarisatie informatiebehoefte brandweerstatistiek. Eindrapport (in Dutch). Available at https://www.wodc.nl/binaries/2131-volledige-tekst_tcm28-72208.pdf (accessed February 2018).

Bottle, A., B. Jarman, and P. Aylin. 2011. “Hospital Standardized Mortality Ratios: Sensitivity Analyses on the Impact of Coding.” Health Services Research 46: 1741–1761. Doi: http://dx.doi.org/10.1111/j.1475-6773.2011.01295.x.

Brackstone, G.J. 1987. “Issues in the Use of Administrative Records for Statistical Purposes.” Survey Methodology 13: 29–43. Available at https://www150.statcan.gc.ca/n1/en/pub/12-001-x/1987001/article/14467-eng.pdf?st=DEZo9O3B (accessed October 2018).

Charlson, M.E., P. Pompei, K.L. Ales, and R. MacKenzie. 1987. “A New Method of Classifying Prognostic Comorbidity in Longitudinal Studies: Development and Validation.” Journal of Chronic Diseased 40: 373–383. Doi: http://dx.doi.org/10.1016/0021-9681(87)90171-8.

De Waal, T., J. Pannekoek, and S. Scholtus. 2011. “Handbook of Statistical Data Editing and Imputation.” New York: John Wiley and Sons.

Efron, B. and C.N. Morris. 1975. “Data Analysis using Stein’s Estimator and Its Generalizations.” Journal of the American Statistical Association 74: 311–319. Doi: http://dx.doi.org/10.1080/01621459.1975.10479864.

Elixhauser, A., C. Steiner, D.R. Harris, and R.M. Coffey. 1998. “Comorbidity Measures for Use with Administrative Data.” Medical Care 36: 8–27. Doi: http://dx.doi.org/10.1097/00005650-199801000-00004.

Groen, J.A. 2012. “Sources of Error in Survey and Administrative Data: The Importance of Reporting Procedures.” Journal of Official Statistics 28: 173–198. Available at http://www.sverigeisiffror.scb.se/contentassets/ff271eeeca694f47ae99b942de61df83/sources-of-error-in-survey-and-administrative-data-the-importance-of-reporting-procedures.pdf (accessed October 2018).

Harteloh, P., K. de Bruin, and J. Kardaun. 2010. “The Reliability of Cause-of-death Coding in The Netherlands.” The European Journal of Epidemiology 25: 531–538. Doi: http://dx.doi.org/10.1007/s10654-010-9445-5.

Hosmer, D.W. and S. Lemeshow. 2004. Applied Logistic Regression. New York: John Wiley and Sons.

Israëls, A., J. van der Laan, J. van der Akker-Ploemacher, and A. de Bruin. 2012. HSMR 2011: Methodological report. Technical report, Statistics Netherlands. Available at https://www.cbs.nl/NR/rdonlyres/E7EC3032-B244-4566-947D-543B8AAE6E4A/0/2012hsmr2011methoderapport.pdf (accessed February 2018).

Jarman, B. 2008. “In Defence of the Hospital Standardised Mortality Ratio.” Healthcare Papers 8: 37–41. Doi: http://dx.doi.org/10.12927/hcpap.2008.19974.

Jarman, B., S. Gault, B. Alves, A. Hider, S. Dolan, A. Cook, B. Hurwitz, and L.I. Iezzoni. 1999. “Explaining Differences in English Hospital Death Rates Using Routinely Collected Data.” Biomedicial Journal (BMJ) 318: 1515–1520. Doi: http://dx.doi.org/10.1136/bmj.318.7197.1515.

Kim, Y., Y-K. Choi, and S. Emery. 2013. “Logistic Regression with Multiple Random Effects: A Simulation Study of Estimation Methods and Statistical Packages.” The American Statistician 67: 171–182. Doi: http://dx.doi.org/10.1080/00031305.2013.817357.

Oberski, D.L., A. Kirchner, S. Eckman, and F. Kreuter. 2017. “Evaluating the Quality of Survey and Administrative Data with Generalized Multitrait-Multimethod Models.” Journal of the American Statistical Association. Available at http://dx.doi.org/10.1080/01621459.2017.1302338 (accessed February 2018).

Pieter, D., R.B. Kool, and G.P. Westert. 2010. Nederlands Tijdschrift voor Geneeskunde, 154, A2186 [in Dutch]. Available at https://www.ntvg.nl/artikelen/beperkte-invloedgegevensregistratie-op-gestandaardiseerd-ziekenhuissterftecijfer-hsmr/volledig (accessed February 2018).

Pitches, D.W., M.A. Mohammed, and R.J. Lilford. 2007. “What Is the Empirical Evidence That Hospitals with Higher-Risk Adjusted Mortality Rates Provide Poorer Quality Care? A Systematic Review of the Literature.” BMC Health Services Research 7: 91–98. Doi: http://dx.doi.org/10.1186/1472-6963-7-91.

Prins, M.J. 2016. The Effect of Coding Practice on the Hospital Standardised Mortality Ratio, Master Thesis. Utrecht University. (available upon request).

Quan, H., B. Li, L.D. Saunders, G.A. Parsons, C.I. Nilsson, A. Alibhai, and W.A. Ghali. 2008. “Assessing Validity of ICD-9-CM and ICD-10 Administrative Data in Recording Clinical Conditions in a Unique Dually Coded Database.” Health Services Research 43: 1424–1441. Available at http://onlinelibrary.wiley.com/doi/10.1111/j.1475-6773.2007.00822.x/full (accessed February 2018).

Rousseeuw, P.J. and A.M. Leroy. 1987. Robust Regression and Outlier Detection. New York: John Wiley and Sons.

Rubin, D.B. 1978. Multiple Imputations in Sample Surveys – a Phenomenological Bayesian Approach to Nonresponse. Proceedings of the Section on Survey Research Methods. American Statistical Association. Available at http://ww2.amstat.org/sections/srms/proceedings/papers/1978_004.pdf (accessed February 2018).

Rubin, D.B. 1987. Multiple Imputation for Non-response in Surveys. New York: John Wiley and Sons.

Shields, J. and N. To. 2005. “Learning to Say No: Conditioned Underreporting in an Expenditure Survey.” Paper presented at the American Association for Public Opinion Research Annual Conference, 12–15 May 2005, Miami Beach, U.S.A. Available at http://ww2.amstat.org/sections/srms/proceedings/y2005/Files/JSM2005-000432.pdf (accessed October 2018).

Silberstein, A.R. and C.A. Jacobs. 1989. Symptoms of Repeated Interview Effects in the Consumer Expenditure Survey. In Panel Surveys, edited by D. Kasprzyk, G. Duncan, G. Kalton, and M.P. Singh, 289–303. New York: John Wiley and Sons.

Tourangeau, R., R.M. Groves, and C. Redline. 2010. “Sensitive Topics and Reluctant Respondents: Demonstrating a Link between Nonresponse Bias and Measurement Error.” Public Opinion Quarterly 74(3): 413–432. Doi: http://dx.doi.org/10.1093/poq/nfq004.

Tourangeau, R. and T. Yan. 2007. “Sensitive Questions in Surveys.” Psychological Bulletin 133(5): 859–883. Doi: http://dx.doi.org/10.1037/0033-2909.133.5.859.

United Nations Economic Commission for Europe. 2011. Using Administrative and Secondary Sources for Official Statistics: a Handbook of Principles and Practices. New York and Geneva: United Nations. Available at http://www.unece.org/fileadmin/DAM/stats/publications/Using_Administrative_Sources_Final_for_web.pdf (accessed February 2018).

Van Delden, A. and S. Scholtus. 2017. “Correspondence between survey and admin data on quarterly turnover.” CBS Discussion Paper 2017-03. Available at https://www.cbs.nl/en-gb/background/2017/07/correspondence-between-survey-and-admin-data-on-quarterly-turnover (accessed February 2018).

Van den Bosch, W.F., J. Silberbusch, K.J. Roozendaal, and C. Wagner. 2010. Variatie in codering Patiëntengegevens beïnvloedt gestandaardiseerd ziekenhuissterftecijfer (HSMR). Nederlands Tijdschrift voor Geneeskunde, 154 A1189 [in Dutch]. Available at https://www.ntvg.nl/artikelen/variatie-codering-patiëntengegevens-beïnvloedtgestandaardiseerd-ziekenhuissterftecijfer/volledig (accessed February 2018).

Van der Laan, J. 2013. Quality of the Dutch Medical Registration (LMR) for the calculation of the Hospital Standardised Mortality Ratio. Discussion Paper. Statistics Netherlands. Available at https://www.cbs.nl/NR/rdonlyres/6290A0A8-4CC9-4DBFAF0B-A3C6742EEA89/0/201308x10pub.pdf (accessed February 2018).

Van der Laan, J., A. de Bruin, J. van den Akker-Ploemacher, C. Penning, and F. Pijpers. 2015. HSMR 2014: Methodological Report. Technical Report. Statistics Netherlands. Available at https://www.cbs.nl/NR/rdonlyres/2AFF4E96-C02F-4BB0-97A9-035D17DF1104/0/2015hsmrmethodologicalreport2014.pdf (accessed February 2018).

Wallgren, A. and B. Wallgren. 2014. Register-based Statistics. Statistical Methods for Administrative Data (2nd edition). New York: John Wiley and Sons.

West, B.T. and A.G. Blom. 2017. “Explaining Interviewing Effects: A Research Synthesis.” Journal of Survey Statistics and Methodology 5: 175–211. Doi: http://dx.doi.org/10.1093/jssam/smw024.

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2017: 0.662
5-year IMPACT FACTOR: 1.113

CiteScore 2017: 0.74

SCImago Journal Rank (SJR) 2017: 1.158
Source Normalized Impact per Paper (SNIP) 2017: 0.860

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 130 130 62
PDF Downloads 96 96 43