Big Data in Market Research: Why More Data Does Not Automatically Mean Better Information

Volker Bosch 1
  • 1 Head of Marketing & Data Sciences, GfK SE, Nuremberg, Germany

Abstract

Big data will change market research at its core in the long term because consumption of products and media can be logged electronically more and more, making it measurable on a large scale. Unfortunately, big data datasets are rarely representative, even if they are huge. Smart algorithms are needed to achieve high precision and prediction quality for digital and non-representative approaches. Also, big data can only be processed with complex and therefore error-prone software, which leads to measurement errors that need to be corrected. Another challenge is posed by missing but critical variables. The amount of data can indeed be overwhelming, but it often lacks important information. The missing observations can only be filled in by using statistical data imputation. This requires an additional data source with the additional variables, for example a panel. Linear imputation is a statistical procedure that is anything but trivial. It is an instrument to “transport information,” and the higher the observed data correlates with the data to be imputed, the better it works. It makes structures visible even if the depth of the data is limited.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Fenn, Jackie (1995): The Microsoft System Software Hype Cycle Strikes Again

  • Gaffert P., Bosch V., Meinfelder, F. (2016): “Interactions and squares. Don’t transform, just impute!,” Conference Paper, Joint Statistical Meetings, Chicago

  • http://www.ibmbigdatahub.com/infographic/four-vs-big-data

  • http://fivethirtyeight.blogs.nytimes.com/2012/11/10/which-polls-fared-best-and-worst-inthe-2012-presidential-race/?_r=0

OPEN ACCESS

Journal + Issues

Search