<abstract xmlns="http://www.w3.org/1999/xhtml"><p>Classification models enable optimal actions to be taken at every stage of the customer’s lifecycle. A circumstance affecting both the model building process and the assessment of their discriminatory power is the unbalanced distribution of the dichotomous dependent variable. The article focuses on the question of reliable assessment of the goodness of fit. The first part of the article reviews the measures of predictive power and then assesses the impact of the distribution of the dependent variable on the selected measures of goodness of fit. As a result, the high sensitivity of a number of measures such as lift, accuracy (ACC), or F-Score was observed. The sensitivity of MCC and Kappa Cohen’s measurements was also observed. Sensitivity (SENS) and specificity (SPEC), Youden’s index and measures based on ROC curves showed no such sensitivity. The conclusions obtained may allow the avoidance of misjudging the predictive power of models built for both learning and business practice.</p></abstract>

Classification models enable optimal actions to be taken at every stage of the customer’s lifecycle. A circumstance affecting both the model building process and the assessment of their discriminatory power is the unbalanced distribution of the dichotomous dependent variable. The article focuses on the question of reliable assessment of the goodness of fit. The first part of the article reviews the measures of predictive power and then assesses the impact of the distribution of the dependent variable on the selected measures of goodness of fit. As a result, the high sensitivity of a number of measures such as lift, accuracy (ACC), or F-Score was observed. The sensitivity of MCC and Kappa Cohen’s measurements was also observed. Sensitivity (SENS) and specificity (SPEC), Youden’s index and measures based on ROC curves showed no such sensitivity. The conclusions obtained may allow the avoidance of misjudging the predictive power of models built for both learning and business practice.

Assessment of the Influence of Dependent Variable Distribution on Selected Goodness of Fit Measures Using the Example of Customer Churn Model

Econometrics

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Classification models enable optimal actions to be taken at every stage of the customer’s lifecycle. A circumstance affecting both the model building process...

Assessment of the Influence of Dependent Variable Distribution on Selected Goodness of Fit Measures Using the Example of Customer Churn Model

Published Online: May 29, 2020

Page range: 51 - 70

DOI: https://doi.org/10.15611/eada.2020.1.05

Keywords
classification models, goodness of fit, unbalanced datasets, customer churn analysis

© 2020 Grzegorz Migut, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Assessment of the Influence of Dependent Variable Distribution on Selected Goodness of Fit Measures Using the Example of Customer Churn Model

Published Online: May 29, 2020

Page range: 51 - 70

DOI: https://doi.org/10.15611/eada.2020.1.05

Keywordsclassification models, goodness of fit, unbalanced datasets, customer churn analysis

© 2020 Grzegorz Migut, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Keywords
classification models, goodness of fit, unbalanced datasets, customer churn analysis