Recursive-Rule Extraction Algorithm With J48graft And Applications To Generating Credit Scores

Open access


The purpose of this study was to generate more concise rule extraction from the Recursive-Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

[1] Marqus, A.I., Garca, V., & Snchez, J.S. On the suitability of resampling techniques for the class imbalance problem in credit scoring. Journal of the Operational Research Society 64, pp. 1060–1070, 2013.

[2] Zhao Z., Xu, S., Kang, B. H., Kabir, M. M. J., & Liu, Y. Investigation and improvement of multilayer perceptron neural networks for credit scoring Expert Systems with Applications 42, pp. 3508–3516, 2015

[3] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.

[4] Quinlan, J.R. Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning, San Mateo, CA, 1993, Morgan Kaufman.

[5] Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. Comprehensible credit scoring models using support vector machines. European Journal of Operational Research 183, pp. 1497–1488, 2007.

[6] Baesens, B., et al. Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49, No. 3, pp. 312–329, 2004.

[7] Abellan, J., and Mantas, C. Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications, pp. 3825–3830, 2014.

[8] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.

[9] Braket, N., and Bradely, A.P. Rule extraction from support vector machine: a review. Neurocomputing 74, pp. 178–190, 2010.

[10] Setiono, R., and Liu, H. Neurolinear: From neural networks to oblique decision rules. Neurocomputing 17, No. 1, pp. 1–24, 1997.

[11] Setiono, R. and Liu, H. A connectionist approach to generating oblique decision trees. IEEE Trans. Syst., Man, Cybern. B, Cybern. Vol. 29, No. 3, pp. 440–444, Jun. 1999.

[12] Setiono, R., Baesens, B. & Mues, C. A note on knowledge discovery using neural Setiono networks and its application to credit card screening. European Journal of Operational Research 192, pp.326-332, 2009.

[13] Setiono, R., et al. Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19, No. 2, pp. 299–307, 2008.

[14] Setiono, R., and Liu, H. Symbolic representation of neural networks. IEEE Computer 29, No. 3, pp. 71–77, 1996.

[15] Gupta, A, Park, S. and Lam, S.M. Generalized analytic rule extraction for feedforward neural networks, IEEE Trans. Knowledge and Data Engineering, 11, pp.985-991, 1999.

[16] Etchell, T.A. and Lisboa, J.P.G., Orthogonal search-based rule extraction (OSRE) for trained neural-networks: A practical and efficient approach, IEEE Trans. Neural Networks 17, pp.374-384, 2006.

[17] Hansen, L.K., and Salamon, P., Neural network ensembles. IEEE Trans. Patter Analysis and Machine Intelligence 12, pp. 993–1001, 1990.

[18] Igelnik, S., Pao, Y.-H., LeClair, S. R., and Shen, C. Y. The ensemble approach to neural-network learning and generalization. IEEE Trans. Neural Networks 10, pp. 19–30, 1999.

[19] Liao, J.-J., Shih, C.-H., Chen, T.-F., and Hsu, M.-F. An example-based model for two-class imbalanced financial problem. Economic Modelling 37, pp. 175–183, 2014.

[20] Setiono R. et al., Rule extraction from minimal neural networks for credit card screening, Inter. J. of Neural Systems., Vol. 21, No. 4, pp. 265–276, 2011.

[21] Setiono R. et al., A note on knowledge discovery using neural networks and its application to credit card screening, European J. Operational Research, Vol. 192, pp. 326–332, 2009.

[22] Hayashi Y. et al., Understanding consumer heterogeneity: A business intelligence application of neural networks, Knowledge-Based Systems, Vol. 23, No. 8, pp. 856–863, 2010.

[23] Bologna G., Is it worth generating rules from neural network ensemble?, J. of Applied Logic, Vol. 2, pp. 325–348, 2004.

[24] Zhou, Z.-H. Extracting symbolic rules from trained neural network ensembles. AI Communications 16, pp. 3–15, 2003.

[26] Setiono R. et al., A penalty-function approach for pruning feedforward neural networks, Neural Comp., Vol. 9, No. 1, pp. 185–204, 1997.

[27] Witten, I.H. and Frank, E., Data Ming: Practical Machine Learning Tools With Java Implementations. San Francisco, CA: Morgan Kaufmann, 1999.

[28] Quinlan J.R., Induction of decision trees, Machine Learning, Vol.1, pp.81-106, 1986.

[29] Webb. G.I., Decision Tree Grafting from the All-Tests-But-One Partition, in Proc. 16th International Joint Conference on Artificial Intelligence (IJCAI), Vol. 2, pp. 702–707, 1999.

[30] Webb, G. I., Decision Tree Grafting, Learining, IJCAI’97 Proceedings of the 15th International Conference on Artificial Intelligence, Vol.2, pp. 846-885, 1997.

[31] Frank, A. & Asuncion, A. University of California. Irvine Machine Learning Repository., 2010

[32] Prechelt, L. Proben1 — A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultt fr Informatik, Universitt Karlsruhe, Germany. Anonymous ftp available from ftp://pub/papers/techreport/1994/ on, 1994.

[33] Smith, M. Neural Networks for Statistical Modeling, New York: Van Nostrand Reinhold, 1993.

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

Journal Information

CiteScore 2017: 5.00

SCImago Journal Rank (SJR) 2017: 0.492
Source Normalized Impact per Paper (SNIP) 2017: 2.813

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 143 143 18
PDF Downloads 56 56 12