Fuzzy interpretation for temporal-difference learning in anomaly detection problems

Open access


Nowadays, information control systems based on databases develop dynamically worldwide. These systems are extensively implemented into dispatching control systems for railways, intrusion detection systems for computer security and other domains centered on big data analysis. Here, one of the main tasks is the detection and prediction of temporal anomalies, which could be a signal leading to significant (and often critical) actionable information. This paper proposes the new anomaly prevent detection technique, which allows for determining the predictive temporal structures. Presented approach is based on a hybridization of stochastic Markov reward model by using fuzzy production rules, which allow to correct Markov information based on expert knowledge about the process dynamics as well as Markov’s intuition about the probable anomaly occurring. The paper provides experiments showing the efficacy of detection and prediction. In addition, the analogy between new framework and temporal-difference learning for sequence anomaly detection is graphically illustrated.


  • [1] V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey”, ACM Computing Surveys (CSUR) 41(3), 1–72 (2009).

  • [2] V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection for discrete sequences: A survey”, Transactions on Knowledge and Data Engineering 24(5), 823–839 (2012).

  • [3] S.M. Kovalev, A.N. Guda, and M.A. Butakova, “Hybrid stochastic detection model of specific patterns in temporal data”, Vestnik RGUPS 3 (51), 36–42 (2013) (in Russian).

  • [4] R.J. Povinelli and X. Feng, “A new temporal pattern identification method for characterization and prediction of complex time series events”, Transactions on Knowledge and Data Engineering 15 (2), 339–352 (2003).

  • [5] S.M. Kovalev and A.V. Sukhanov, “Hybrid stochastic model based detection of specific patterns in time series”, Izvestiya SFesU. Engineering sciences 4(153), 142–149 (2014) (in Russian).

  • [6] P. Cunningham, M. Cord, and S. J. Delany, “Supervised learning”, Machine Learning Techniques for Multimedia, 21–49 (2008).

  • [7] S.S. Khan and M.G. Madden, “A survey of recent trends in one class classification”, Artificial Intelligence and Cognitive Science, 188–197 (2010).

  • [8] Z. Ghahramani, “Unsupervised learning”, Advanced Lectures on Machine Learning, 72–112 (2004).

  • [9] M. V. Mahoney and P. K. Chan, “Learning nonstationary models of normal network traffic for detecting novel attacks”, Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining, 376–385 (2002).

  • [10] I. Aydin, M. Karakose, and E. Akin, “The prediction algorithm based on fuzzy logic using time series data mining method”, World Academy of Science, Engineering and Technology 51 (27), 91–98 (2009).

  • [11] E. Zitzler and L. Thiele, “Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach”, IEEE transactions on evolutionary computation 3 (4), 257–271 (1999).

  • [12] N. Ye et al., “A Markov chain model of temporal behavior for anomaly detection”, Proceedings of the 2000 IEEE Systems, Man, and Cybernetics Information Assurance and Security Workshop, 171–174 (2000).

  • [13] X. Xu, “Sequential anomaly detection based on temporal-difference learning: Principles, models and case studies”, Applied Soft Computing 10 (3), 859–867 (2010).

  • [14] A.V. Leonenkov, Fuzzy Simulation in the Environment MATLAB and FuzzyTECH, BKV, Petersburg, 2005 (in Russian).

  • [15] A.V. Sukhanov, S.M. Kovalev, and V. Styskala, “Advanced temporal-difference learning for intrusion detection”, 13th IFAC and IEEE Conference on Programmable Devices and Embedded Systems PDES 48 (4), 3–48 (2015).

  • [16] A. Sukhanov, “Behavior prediction for time series”, Results and Solutions of Young R&S for Innovations and Progress Final Report, 88–91 (2014).

  • [17] S.M. Kovalev, A.N. Guda, and A.V. Sukhanov, “Hybrid method based on fuzzy productions for prediction stochastic model learning”, Vestnik RGUPS 59 (3), 40–46 (2015) (in Russian).

  • [18] S. Forrest, Computer immune systems – data sets, http://www.cs.unm.edu/~immsec/systemcalls.htm. [accessed 11.12.2014], 2006.

  • [19] A.N. Shabenikov, A.V. Sukhanov, and S.M. Kovalev, “Intelligent technique for faults prediction on hump yards”, Inženernyj vestnik Dona 4, http://ivdon.ru/ru/magazine/archive/n4y2015/3334 [accessed 11.11.2015] (2015) (in Russian).

Bulletin of the Polish Academy of Sciences Technical Sciences

The Journal of Polish Academy of Sciences

Journal Information

IMPACT FACTOR 2016: 1.156
5-year IMPACT FACTOR: 1.238

CiteScore 2016: 1.50

SCImago Journal Rank (SJR) 2016: 0.457
Source Normalized Impact per Paper (SNIP) 2016: 1.239


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 27 27 21
PDF Downloads 2 2 0