Measuring and Detecting Errors in Occupational Coding: an Analysis of SHARE Data

Open access

Abstract

This article studies coding errors in occupational data, as the quality of this data is important but often neglected. In particular, we recoded open-ended questions on occupation for last and current job in the Dutch sample of the “Survey of Health, Ageing and Retirement in Europe” (SHARE) using a high-quality software program for ex-post coding (CASCOT software). Taking CASCOT coding as our benchmark, our results suggest that the incidence of coding errors in SHARE is high, even when the comparison is made at the level of one-digit occupational codes (28% for last job and 30% for current job). This finding highlights the complexity of occupational coding and suggests that processing errors due to miscoding should be taken into account when undertaking statistical analyses or writing econometric models. Our analysis suggests strategies to alleviate such coding errors, and we propose a set of equations that can predict error. These equations may complement coding software and improve the quality of occupational coding.

Autor, D. 2013. “The ‘Task Approach’ to Labour Markets: an Overview.” Journal of Labour Market Research 46: 185-199. Doi: http://dx.doi.org/10.1007/s12651-013-0128-z.

Autor, D., L.F. Katz, and M.S. Kearney. 2006. “The Polarization of the US Labor Market.” American Economic Review 96: 189-194. Doi: http://dx.doi.org/10.1257/000282806777212620.

Autor, D., F. Levy, and R.J. Murnane. 2003. “The Skill Content Of Recent Technological Change: An Empirical Exploration.” Quarterly Journal of Economics 118: 1279-1333. Doi: http://dx.doi.org/10.1162/003355303322552801.

Bethmann, A., M. Schierholz, K. Wenzig, and M. Zielonka. 2014. Automatic Coding of Occupations Using Machine Learning Algorithms for Occupation Coding in Several German Panel Surveys. In: Statistics Canada (Ed.), Beyond traditional survey taking. Adapting to a changing world. Proceedings of Statistics Canada Symposium 2014, Quebec. Available at: http://fdz.iab.de/342/section.aspx/Publikation/k151124301 (accessed October 2016).

Biemer, P.B. and L.E. Lyberg. 2003. Introduction to Survey Quality. New York: John Wiley & Sons, Inc.

CBS (Statistics Netherlands). 2012. Coding Tool Implemented in 2012 for Coding Occupations in Social Surveys. Internal Document. The Hague: Statistics Netherlands.

Cheeseman Day, J. 2014. Using an Autocoder to Code Industry and Occupation in the American Community Survey. Presentation held at the Federal Economic Statistics Advisory Committee Meeting, 13 June 2014. Available at: http://www2.census.gov/adrm/fesac/2014-06-13_day.pdf (accessed October 2016).

Christelis, D., T. Jappelli, and M. Padula. 2010. “Cognitive Abilities and Portfolio Choice.” European Economic Review 54: 18-38. Doi: http://dx.doi.org/10.1016/j.euroecorev.2009.04.001.

Commission of the European Communities. 2009. “Commission Regulation (EC) No 1022/2009 of 29 October 2009 amending Regulations (EC) No 1738/2005, (EC) No 698/2006 and (EC) No 377/2008 as regards the International Standard Classification of Occupations (ISCO).” Official Journal of the European Union, L 283/3, 30 October 2009. Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32009R1022&from=EN (accessed October 2016).

DESA. 2010. “Handbook on Population and Housing Census Editing.” Revision 1, Series F, No 82 (Studies in Methods (Ser. F)), United Nations Statistics Division. New York: United Nations Publication. Available at: http://unstats.un.org/unsd/publication/SeriesF/seriesf_82rev1e.pdf (accessed October 2016).

Elias, P., K. Halstead, and K. Prandy. 1993. Computer-Assisted Standard Occupational Coding. London: HMSO.

Elias, P. 1997. “Occupational Classification (ISCO-88). Concepts, Methods, Reliability, Validity and Cross-National Comparability.” OECD Labour Market and Social Policy Occasional Papers No. 20, OECD Publishing. Doi: http://dx.doi.org/10.1787/304441717388.

Ellison, R. 2014. Demonstration of Performance of CASCOT 5.0. Presentation held at the CASCOT: Occupational Coding in Multi-national Surveys Workshop, 10-11 April 2014, Venice. Available at: http://dasish.eu/dasishevents/cascotworkshop/programmepres/ (accessed 15 April, 2014).

Feenstra, R.C. and G.H. Hanson. 1996. “Globalization, Outsourcing, and Wage Inequality.” American Economic Review 86: 240-245. Available at: http://www.jstor.org/stable/2118130.

Fletcher, J.M., J.L. Sindelar, and S. Yamaguchi. 2011. “Cumulative Effects of Job Characteristics on Health.” Health Economics 20: 553-570. Doi: http://dx.doi.org/10.1002/hec.1616.

Ganzeboom, H. 2008. Occupation Coding: Do’s And Dont’s. Version 2, 2 August 2008. Available at: http://www.gesis.org/fileadmin/upload/dienstleistung/daten/umfragedaten/issp/members/codinginfo/ISCO-coding_dos_donts_HG2008.pdf (accessed October 2016).

Goos, M. and A. Manning. 2007. “Lousy and Lovely Jobs: The Rising Polarization of Work in Britain.” Review of Economics and Statistics 89: 118-133. Doi: http://dx.doi.org/10.1162/rest.89.1.118.

Hartog, J. 2000. “Over-Education and Earnings: Where Are We, Where Should We Go?” Economics of Education Review 19: 131-147. Doi: http://dx.doi.org/10.1016/S0272-7757(99)00050-3.

Hoffmann, E., P. Elias, B. Embury, and R. Thomas. 1995. What Kind Of Work Do You Do? Data Collection and Processing Strategies When Measuring “Occupation” for Statistical Surveys and Administrative Records. ILO working paper, N.95-1. Geneva: ILO. Available at: http://www.ilo.org/global/statistics-and-databases/WCMS_087880/lang-en/index.htm (accessed October 2016).

Jackle, A. 2008. “Dependent Interviewing: Effects on Respondent Burden and Efficiency of Data Collection.” Journal of Official Statistics 24: 411-430.

Jones, R. and P. Elias. 2004. “CASCOT: Computer-Assisted Structured Coding Tool.” Coventry: Warwick Institute for Employment Research, University of Warwick.

ILO. 2014. ISCO: International Standard Classification of Occupations. Available at: http://www.ilo.org/public/english/bureau/stat/isco/ (accessed 3 October, 2016).

ILO. 2012. International Standard Classification of Occupations: Structure, Group Definitions and Correspondence tables. Vol. 1. Geneva: ILO. Available at: http://www.ilo.org/wcmsp5/groups/public/--dgreports/--dcomm/--publ/documents/publication/wcms_172572.pdf (accessed October 2016).

ILO. 2010. Measuring the Economically Active in Population Censuses: A Handbook. Studies in Methods Series F, No. 102. New York: ILO and UN.

Leist, A.K., M.M. Glymour, J.P. Mackenbach, F.J. van Lenthe, and M. Avendano. 2013. “Time Away from Work Predicts Later Cognitive Function: Differences by Activity During Leave.” Annals of Epidemiology 23: 455-462. Doi: http://dx.doi.org/10.1016/j.annepidem.2013.05.014.

Leuven, E. and H. Oosterbeek. 2011. “Overeducation and Mismatch in the Labor Market.” In Handbook of the Economics of Education, edited by E. Hanushek, S. Machin and L. Woessmann, 283-326. Amsterdam: Elsevier.

MEA. 2013. SHARE Release Guide 2.6.0. Waves 1 & 2. Munich: publisher. Available at: Munich Center for the Economics of Ageing (MEA) at the Max Planck Institute for Social Law and Social Policy (MPISOC) publishing. (accessed October 2016).

Moscarini, G. and K. Thomsson. 2007. “Occupational and Job Mobility in the US.” The Scandinavian Journal of Economics 109: 807-836. Doi: http://dx.doi.org/10.1111/j.1467-9442.2007.00510.x.

Perales, F. 2014. “How Wrong Were We? Dependent Interviewing, Self-Reports and Measurement Error in Occupational Mobility in Panel Surveys.” Longitudinal and Life Course Studies 4: 299-316. Doi: http://dx.doi.org/10.14301/llcs.v5i3.295.

Ravesteijn, B., H. van Kippersluis, and E. van Doorslaer. 2013. The Wear and Tear on Health: What is the Role of Occupation? Tinbergen Institute Discussion Paper 13-143. Amsterdam: Tinbergen Institute. Available at: http://papers.tinbergen.nl/13143.pdf (accessed October 2016).

Rose, D. and E. Harrison. 2007. “The European Socio-Economic Classification: A New Social Class Schema For Comparative European Research.” European Societies 9: 459-490. Doi: http://dx.doi.org/10.1080/14616690701336518.

Tijdens, K.G. 2014a. “Drop-Out Rates During Completion of an Occupation Search Tree in Web-Surveys.” Journal of Official Statistics 30: 23-43. Doi: http://dx.doi.org/10.2478/jos-2014-0002.

Tijdens, K.G. 2014b. Reviewing the Measurement and Comparison of Occupations Across Europe. AIAS Working Paper 149. Amsterdam: University of Amsterdam. Available at: http://dare.uva.nl/record/1/432281 (accessed October 2016).

United Nations. 2007. “Updating the International Standard Classification of Occupations (ISCO): Summary of major changes between ISCO-88 and ISCO-08 (Feb 2007 draft).” Paper for discussion by the Expert Group on International Economic and Social Classifications, New York, 16-18 April 2007. Available at: http://unstats.un.org/unsd/class/intercop/expertgroup/2007/AC124-11.PDF (accessed October 2016).

United Nations (UN). 2014. National Classifications. Available at: http://unstats.un.org/unsd/cr/ctryreg/ctrylist2.asp (accessed March 2015).

Westerman, S. 2014. “CBS and CASCOT: tuning CASCOT for improved performance.” Presentation held at the CASCOT: Occupational Coding in Multi-national Surveys Workshop, 10-11 April 2014, Venice. Available at: http://dasish.eu/dasishevents/cascotworkshop/programmepres/ (accessed 15 April, 2014).

Wilcoxon, F. 1945. “Individual Comparisons by Ranking Methods.” Biometrics 1: 80-83. Available at: http://www.jstor.org/stable/3001968.

Wooldridge, J.M. 2010. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press.

Journal of Official Statistics

The Journal of Statistics Sweden

Journal Information


IMPACT FACTOR 2017: 0.662
5-year IMPACT FACTOR: 1.113

CiteScore 2017: 0.74

SCImago Journal Rank (SJR) 2017: 1.158
Source Normalized Impact per Paper (SNIP) 2017: 0.860

Cited By

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 294 294 22
PDF Downloads 123 123 7