Comparison of the Effects of Cross-validation Methods on Determining Performances of Classifiers Used in Diagnosing Congestive Heart Failure

Open access


Congestive heart failure (CHF) occurs when the heart is unable to provide sufficient pump action to maintain blood flow to meet the needs of the body. Early diagnosis is important since the mortality rate of the patients with CHF is very high. There are different validation methods to measure performances of classifier algorithms designed for this purpose. In this study, k-fold and leave-one-out cross-validation methods were tested for performance measures of five distinct classifiers in the diagnosis of the patients with CHF. Each algorithm was run 100 times and the average and the standard deviation of classifier performances were recorded. As a result, it was observed that average performance was enhanced and the variability of performances was decreased when the number of data sections used in the cross-validation method was increased.

[1] U.S. National Library of Medicine. Heart failure (Medical Encyclopedia).

[2] Flavell, C., Stevenson, L.W. (2001). Take Heart with Heart Failure. Circulation, 104, 89.

[3] Wilbur, J., James, P. (2005). Diagnosis and management of heart failure in the outpatient setting. Primary Care, 32, 1115-1129.

[4] American Heart Association (2006). Heart Disease and Stroke Statistics-2006 Update: A Report From the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation, 113, 85-151.

[5] Isler, Y., Kuntalp, M. (2007). Combining Classical HRV Indices with Wavelet Entropy Measures Improves to Performance in Diagnosing Congestive Heart Failure. Computers in Biology and Medicine, 37(10), 1502-1510.

[6] Isler, Y., Kuntalp, M. (2010). Heart Rate Normalization in the Analysis of Heart Rate Variability in Congestive Heart Failure. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 224(3), 453-463.

[7] Yu, S.N., Lee, M.Y. (2012). Bispectral analysis and genetic algorithm for congestive heart failure recognition based on heart rate variability. Computers in Biology and Medicine, 42, 816-825.

[8] Yu, S.N., Lee, M.Y. (2012). Conditional mutual information-based feature selection for congestive heart failure recognition using heart rate variability. Computer Methods and Programs in Biomedicine, 108, 299-309.

[9] Jovic, A., Bogunovic, N. (2011). Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features. Artificial Intelligence in Medicine, 51, 175-186.

[10] Pecchia, L., Melillo, P., Sansone, M., Bracale, M. (2011). Discrimination power of short-term heart rate variability measures for CHF assessment. IEEE Transactions on Information Technology in Biomedicine, 15(1), 40-46.

[11] Narin, A., Isler, Y., Ozer, M. (2014). Investigating the performance improvement of HRV Indices in CHF using feature selection methods based on backward elimination and statistical significance. Computers in Biology and Medicine, 45, 72-79.

[12] Narin, A., Isler, Y. (2012). Effect of Principal Component Analysis on Diagnosing Congestive Heart Failure Patients using Heart Rate Records. In IEEE 20th Signal Processing and Communications Applications Conference (SIU2012), 18-20 April 2012, Fethiye / Mugla.

[13] Isler, Y., Selver, M.A., Kuntalp, M. (2005). Effects of Detrending in Heart Rate Variability Analysis. In II. Muhendislik Bilimleri Genc Arastirmacilar Kongresi MBGAK’2005, 17-19 October 2005, Istanbul, 213-219.

[14] Duda, R.O., Hart, P.E., Stork, D.G. (2000). Pattern Classification, 2nd edition, New York. Wiley.

[15] Goldberger, A.L., (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation, 101(23), e215-e220.

[16] Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology. (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043-1065.

[17] Lomb, N.R. (1976). Least-squares frequency analysis of unequally spaced data. Astrophysics and Space Science, 39, 447-462.

[18] Quiraga, R.Q., Rosso, O.A., Basar, E., Schurmann, M. (2001). Wavelet entropy in event-related potentials: A new method shows ordering of EEG oscillations. Biological Cybernetics, 84(4), 291-299.

[19] Woo, M.A., Stevenson, W.G., Moser, D.K., Trelease, R.B., Harper, R.H. (1992). Patterns of beat-to-beat heart rate variability in advanced heart failure. American Heart Journal, 123, 704-710.

[20] Kamen, P.W., Krum, H., Tonkin, A.M. (1996). Poincare plot of heart rate variability allows quantitative display of parasympathetic nervous activity. Clinical Science, 92, 201-208.

[21] Kamen, P.W. (1996). Heart rate variability. Australian Family Physician, 25, 1087-1094.

[22] Brennan, M., Palaniswami, M., Kamen, P. (2001). Do existing measures of poincare plot geometry reflect nonlinear features of heart rate variability? IEEE Transactions on Biomedical Engineering, 48(11), 1342-1347.

[23] Huikuri, H.V., Makikallio, T.H., Peng, C.K., Goldberger, A.L., Hintze, U., Moller, M. (2000). Fractal correlation properties of R-R interval dynamics and mortality in patients with depressed left ventricular function after an acute myocardial infarction. Circulation, 101, 47-53.

[24] Acharya, U.R., Kannathal, N., Seng, O.W., Ping, L.Y., Chua, T. (2004). Heart rate analysis in normal subjects of various age groups. BioMedical Engineering OnLine, 3(24).

[25] Caminal, P., Vallverdu, M., Giraldo, B., Benito, S., Vazquez, G., Voss, A. (2005). Optimized symbolic dynamics approach for the analysis of the respiratory pattern. IEEE Transactions on Biomedical Engineering, 52(11), 1832-1839.

[26] Xu, J.H., Liu, Z.R., Liu, R. (1994). The measures of sequence complexity for EEG studies. Chaos, 4(11), 2111-2119.

[27] Richman, J.S., Randall, M.J. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology - Heart and Circulatory Physiology, 278, H2039-H2049.

[28] Akgul, A. (2003). Tibbi Arastirmalarda Istatistiksel Analiz Teknikleri: SPSS Uygulamalari (Statistical Analysis Techniques in Medical Researches: SPSS Experiments). 2nd edition, Ankara, Turkey, Seckin Yayincilik.

[29] Kohavi, R. (1995). A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In 14th International Joint Conference on Artificial Intelligence (IJCAI), 20-25 August 1995, Montreal, Quebec, Canada.

Measurement Science Review

The Journal of Institute of Measurement Science of Slovak Academy of Sciences

Journal Information

IMPACT FACTOR 2017: 1.345
5-year IMPACT FACTOR: 1.253

CiteScore 2017: 1.61

SCImago Journal Rank (SJR) 2017: 0.441
Source Normalized Impact per Paper (SNIP) 2017: 0.936

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 141 141 29
PDF Downloads 49 49 14