<abstract xmlns="http://www.w3.org/1999/xhtml"><p>The paper considers particular interestingness measures, called confirmation measures (also known as Bayesian confirmation measures), used for the evaluation of “<italic>if</italic> evidence, <italic>then</italic> hypothesis” rules. The agreement of such measures with a statistically sound (significant) dependency between the evidence and the hypothesis in data is thoroughly investigated. The popular confirmation measures were not defined to possess such form of agreement. However, in error-prone environments, potential lack of agreement may lead to undesired effects, e.g. when a measure indicates either strong confirmation or strong disconfirmation, while in fact there is only weak dependency between the evidence and the hypothesis. In order to detect and prevent such situations, the paper employs a coefficient allowing to assess the level of dependency between the evidence and the hypothesis in data, and introduces a method of quantifying the level of agreement (referred to as a concordance) between this coefficient and the measure being analysed. The concordance is characterized and visualised using specialized histograms, scatter-plots, etc. Moreover, risk-related interpretations of the concordance are introduced. Using a set of 12 confirmation measures, the paper presents experiments designed to establish the actual concordance as well as other useful characteristics of the measures.</p></abstract>

The paper considers particular interestingness measures, called confirmation measures (also known as Bayesian confirmation measures), used for the evaluation of “if evidence, then hypothesis” rules. The agreement of such measures with a statistically sound (significant) dependency between the evidence and the hypothesis in data is thoroughly investigated. The popular confirmation measures were not defined to possess such form of agreement. However, in error-prone environments, potential lack of agreement may lead to undesired effects, e.g. when a measure indicates either strong confirmation or strong disconfirmation, while in fact there is only weak dependency between the evidence and the hypothesis. In order to detect and prevent such situations, the paper employs a coefficient allowing to assess the level of dependency between the evidence and the hypothesis in data, and introduces a method of quantifying the level of agreement (referred to as a concordance) between this coefficient and the measure being analysed. The concordance is characterized and visualised using specialized histograms, scatter-plots, etc. Moreover, risk-related interpretations of the concordance are introduced. Using a set of 12 confirmation measures, the paper presents experiments designed to establish the actual concordance as well as other useful characteristics of the measures.

Can Confirmation Measures Reflect Statistically Sound Dependencies in Data? The Concordance-based Assessment

Foundations of Computing and Decision Sciences

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

The paper considers particular interestingness measures, called confirmation measures (also known as Bayesian confirmation measures), used for the evaluation...

Can Confirmation Measures Reflect Statistically Sound Dependencies in Data? The Concordance-based Assessment

Published Online: Feb 16, 2018

Page range: 41 - 66

Received: Sep 29, 2017

Accepted: Jan 30, 2018

DOI: https://doi.org/10.1515/fcds-2018-0004

Keywords
Interestingness measures, confirmation measures, statistical dependency, concordance

© 2018 Robert Susmaga et al., published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Can Confirmation Measures Reflect Statistically Sound Dependencies in Data? The Concordance-based Assessment

Published Online: Feb 16, 2018

Page range: 41 - 66

Received: Sep 29, 2017

Accepted: Jan 30, 2018

DOI: https://doi.org/10.1515/fcds-2018-0004

KeywordsInterestingness measures, confirmation measures, statistical dependency, concordance

© 2018 Robert Susmaga et al., published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
Interestingness measures, confirmation measures, statistical dependency, concordance