Big Data in Market Research: Why More Data Does Not Automatically Mean Better Information

Open access

Abstract

Big data will change market research at its core in the long term because consumption of products and media can be logged electronically more and more, making it measurable on a large scale. Unfortunately, big data datasets are rarely representative, even if they are huge. Smart algorithms are needed to achieve high precision and prediction quality for digital and non-representative approaches. Also, big data can only be processed with complex and therefore error-prone software, which leads to measurement errors that need to be corrected. Another challenge is posed by missing but critical variables. The amount of data can indeed be overwhelming, but it often lacks important information. The missing observations can only be filled in by using statistical data imputation. This requires an additional data source with the additional variables, for example a panel. Linear imputation is a statistical procedure that is anything but trivial. It is an instrument to “transport information,” and the higher the observed data correlates with the data to be imputed, the better it works. It makes structures visible even if the depth of the data is limited.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Fenn Jackie (1995): The Microsoft System Software Hype Cycle Strikes Again

  • Gaffert P. Bosch V. Meinfelder F. (2016): “Interactions and squares. Don’t transform just impute!” Conference Paper Joint Statistical Meetings Chicago

  • http://www.ibmbigdatahub.com/infographic/four-vs-big-data

  • http://fivethirtyeight.blogs.nytimes.com/2012/11/10/which-polls-fared-best-and-worst-inthe-2012-presidential-race/?_r=0

Search
Journal information
Cited By
Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 916 436 18
PDF Downloads 533 263 27