Can Confirmation Measures Reflect Statistically Sound Dependencies in Data? The Concordance-based Assessment

Open access


The paper considers particular interestingness measures, called confirmation measures (also known as Bayesian confirmation measures), used for the evaluation of “if evidence, then hypothesis” rules. The agreement of such measures with a statistically sound (significant) dependency between the evidence and the hypothesis in data is thoroughly investigated. The popular confirmation measures were not defined to possess such form of agreement. However, in error-prone environments, potential lack of agreement may lead to undesired effects, e.g. when a measure indicates either strong confirmation or strong disconfirmation, while in fact there is only weak dependency between the evidence and the hypothesis. In order to detect and prevent such situations, the paper employs a coefficient allowing to assess the level of dependency between the evidence and the hypothesis in data, and introduces a method of quantifying the level of agreement (referred to as a concordance) between this coefficient and the measure being analysed. The concordance is characterized and visualised using specialized histograms, scatter-plots, etc. Moreover, risk-related interpretations of the concordance are introduced. Using a set of 12 confirmation measures, the paper presents experiments designed to establish the actual concordance as well as other useful characteristics of the measures.

[1] Bell, C., Mutual information and maximal correlation as measure dependence, The Annals of Mathematical Statistics, 33, 1962, 587–595.

[2] Bjerve, S., Doksum, K., Correlation curves: Measures of association as functions of covariate value, The Annals of Statistic, 21, 1993, 890–902.

[3] Brzezińska, I., Greco, S., Słowiński, R., Mining pareto-optimal rules with respect to support and anti-support, Engineering Applications of Artificial Intelligence, 20, 5, 2007, 587–600.

[4] Carnap, R., Logical Foundations of Probability, 2nd ed., University of Chicago Press, 1962.

[5] Christensen, D., Measuring confirmation, Journal of Philosophy, 96, 1999, 437–461.

[6] Crupi, V., Tentori, K., Gonzalez, M., On bayesian measures of evidential support: Theoretical and empirical issues, Philosophy of Science, 74, 2007, 229–252.

[7] Earman, J., Bayes or Bust: A Critical Examination of Bayesian Confirmation Theory, MIT Press, Cambridge, MA, 1992.

[8] Eells, E., Fitelson, B., Symmetries and asymmetries in evidential support, Philosophical Studies, 107, 2, 2002, 129–142.

[9] Everitt, B., The Analysis of Contingency Tables, Chapman & Hall, 1992.

[10] Fitelson, B., The plurality of bayesian measures of confirmation and the problem of measure sensitivity, Philosophy of Science, 66, 1999, 362–378.

[11] Fitelson, B., Studies in Bayesian Confirmation Theory, Ph.D. thesis, University of Wisconsin, Madison, 2001.

[12] Geng, L., Hamilton, H., Interestingness measures for data mining: A survey, ACM Computing Surveys, 38, 3, 2006.

[13] Glass, D.H., Confirmation measures of association rule interestingness, Knowlegde Based Systems, 44, 2013, 65–77.

[14] Greco, S., Pawlak, Z., Słowiński, R., Can bayesian confirmation measures be useful for rough set decision rules?, Engineering Applications of Artifficial Intelligence, 17, 2004, 345–361.

[15] Greco, S., Słowiński, R., Szczęch, I., Properties of rule interestingness measures and alternative approaches to normalization of measures, Information Sciences, 216, 2012, 1–16.

[16] Greco, S., Słowiński, R., Szczęch, I., Measures of rule interestingness in various perspectives of confirmation, Inf. Sci., 346-347, 2016, 216–235, doi:10.1016/j.ins.2016.01.056.

[17] Hämäläinen, W., Statapriori: an efficient algorithm for searching statistically significant association rules, Knowl. Inf. Syst., 23, 3, 2010, 373–399.

[18] Hämäläinen, W., Kingfisher: an efficient algorithm for searching for both positive and negative dependency rules with statistical significance measures, Knowl. Inf. Syst., 32, 2, 2012, 383–414.

[19] Hastie, T., Tibshirani, R., Friedman, J., Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, 2003.

[20] Hempel, C., Studies in the logic of confirmation (i), Mind, 54, 1945, 1–26.

[21] Joyce, J., The Foundations of Causal Decision Theory, Cambridge University Press, 1999.

[22] Keeney, R., Raiffa, H., Decisions With Multiple Objectives: Preferences and Value Tradeoffs, John Wiley and Sons, Inc, 1976.

[23] Kemeny, J., Oppenheim, P., Degrees of factual support, Philosophy of Science, 19, 1952, 307–324.

[24] Lenca, P., Meyer, P., Vaillant, B., Lallich, S., On selecting interestingness measures for association rules: User oriented description and multiple criteria decision aid, European Journal of Operational Research, 184, 2, 2008, 610–626.

[25] Luce, R., Raiffa, H., Games and Decisions: Introduction and Critical Survey, John Wiley and Sons, Inc, 1957.

[26] Mortimer, H., The Logic of Induction, Paramus, Prentice Hall, 1988.

[27] Nozick, R., Philosophical Explanations, Clarendon Press, Oxford, UK, 1981.

[28] Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufman, San Francisco, 1988.

[29] Rayner, J., Best, D., A Contingency Table Approach to Nonparametric Testing, Taylor & Francis Group, 2001.

[30] Słowiński, R., Szczęch, I., Urbanowicz, M., Greco, S., Mining association rules with respect to support and anti-support-experimental results, Rough Sets and Intelligent Systems Paradigms, International Conference, RSEISP 2007, Warsaw, Poland, June 28-30, 2007, Proceedings, 2007, 534–542.

[31] Susmaga, R., Szczęch, I., Statistical significance of bayesian confirmation measures, Technical report, RA-010/12, Poznań University of Technology, 2012.

[32] Susmaga, R., Szczęch, I., The property of χ012$\chi _{01}^2$-concordance for bayesian confirmation measures, Lecture Notes in Artificial Intelligence, 8234, 2013, 226–236.

[33] Susmaga, R., Szczęch, I., Visualization support for the analysis of properties of interestingness measures, Bulletin of the Polish Academy of Sciences. Technical Sciences, 63, 1, 2015, 315–327.

[34] Susmaga, R., Szcze¸ch, I., Selected group-theoretic aspects of confirmation measure symmetries, Inf. Sci., 346-347, 2016, 424–441, doi:10.1016/j.ins.2016.01.041.

[35] Szczęch, I., Multicriteria attractiveness evaluation of decision and association rules, Transactions on Rough Sets X, LNCS series, 5656, 2009, 197–274.

[36] Tentori, K., Crupi, V., Bonini, N., Osherson, D., Comparison of confirmation measures, Cognition, 103, 2007, 107–119.

[37] Tick, J., Fodor, J., Fuzzy implications and inference processes, Computing and Informatics, 24, 2005, 591–602.

[38] Venables, W., Ripley, B., Modern Applied Statistics with S, Springer-Verlag, 2002.

Foundations of Computing and Decision Sciences

The Journal of Poznan University of Technology

Journal Information

CiteScore 2017: 0.82

SCImago Journal Rank (SJR) 2017: 0.212
Source Normalized Impact per Paper (SNIP) 2017: 0.523

Mathematical Citation Quotient (MCQ) 2017: 0.02


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 273 254 25
PDF Downloads 137 135 13