Affective State Based Anomaly Detection in Crowd

  • 1 Riga Technical University, , Riga, Latvia


To distinguish individuals with dangerous abnormal behaviours from the crowd, human characteristics (e.g., speed and direction of motion, interaction with other people), crowd characteristics (such as flow and density), space available to individuals, etc. must be considered. The paper proposes an approach that considers individual and crowd metrics to determine anomaly. An individual’s abnormal behaviour alone cannot indicate behaviour, which can be threatening toward other individuals, as this behaviour can also be triggered by positive emotions or events. To avoid individuals whose abnormal behaviour is potentially unrelated to aggression and is not environmentally dangerous, it is suggested to use emotional state of individuals. The aim of the proposed approach is to automate video surveillance systems by enabling them to automatically detect potentially dangerous situations.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] IHS Markit video surveillance, “Top Video Surveillance Trends for 2017,” Mark. Week, pp. 14–18, 2016.

  • [2] M. Andersson, J. Rydell, and J. Ahlberg, “Estimation of Crowd Behavior Using Sensor Networks and Sensor Fusion,” 12th Int. Conf. Inf. Fusion, Aug. 2009.

  • [3] A. Kondrova, Kongnitīvo procesu sistēma. Rīga, 2010.

  • [4] H. U. Keval, “Effective, Design, Configuration, and Use of Digital CCTV,” Doctoral thesis, University College London, 2009.

  • [5] M. W. Baig, E. I. Barakova, L. Marcenaro, M. Rauterberg, and C. S. Regazzoni, “Crowd Emotion Detection Using Dynamic Probabilistic Models,” in From Animals to Animals 13, A. P. del Pobil, E. Martinez-Martin, J. Hallam, E. Cervera, A. Morales, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2014, pp. 328–337.

  • [6] W. Little, Introduction to Sociology – 1st Canadian Edition Edition. pp. 141–168, 2014.

  • [7] Y. Koizumi, S. Saito, H. Uematsu, Y. Kawachi, and N. Harada, “Unsupervised Detection of Anomalous Sound based on Deep Learning and the Neyman-Pearson Lemma,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 1, pp. 212–224, Jan. 2019.

  • [8] D. Chakrabarty and M. Elhilali, “Abnormal Sound Event Detection Using Temporal Trajectories Mixtures,” 2016 IEEE Int. Conf. Acoust. Speech Signal Process., pp. 216–220, 2016.

  • [9] A. G. Abuarafah, M. O. Khozium, and E. Abdrabou, “Real-Time Crowd Monitoring Using Infrared Thermal Video Sequences,” J. Am. Sci., vol. 8, no. 3, 2012.

  • [10] V. Chandola, A. Banerjee, and V. Kumar, “Anomaly Detection: A Survey,” ACM Computing Surveys, vol. 41, no. 3, pp. 1–58, Jul. 2009.

  • [11] H. Parvin, H. Alizadeh, and B. Minati, “A Modification on k-Nearest Neighbor Classifier,” Global Journal of Computer Science and Technology, vol. 10, no. 14, pp. 37–41, 2010.

  • [12] R. Chalapathy and S. Chawla, “Deep Learning for Anomaly Detection: A Survey,” pp. 1–50, 2019.

  • [13] Y. Chen, J. Qian, and V. Saligrama, “A New One-Class SVM for Anomaly Detection,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3567–3571, 2013.

  • [14] F. T. Liu and K. M. Ting, “Isolation Forest,” 2008 Eighth IEEE Int. Conf. Data Min., pp. 413–422, Dec. 2008.

  • [15] C. Chen, Y. Shao, and X. Bi, “Detection of Anomalous Crowd Behavior Based on the Acceleration Feature,” IEEE Sens. J., vol. 15, no. 12, pp. 7252–7261, 2015.

  • [16] J. Z. C. Lai and T. J. Huang, “Fast Global k-Means Clustering Using Cluster Membership and Inequality,” Pattern Recognit., vol. 43, no. 5, pp. 1954–1963, 2010.

  • [17] M. M. Breunig, H. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying Density-Based Local Outliers,” in Proceedings of the 2000 ACM SIGMOD international conference on Management of data (SIGMOD ’00), pp. 1–12, 2000.

  • [18] M. Markou and S. Singh, “Novelty Detection: A Review – Part 1: Statistical Approaches,” Signal Processing, vol. 83, no. 12, pp. 2481–2497, Dec. 2003.

  • [19] W.-L. Hsu, Y.-C. Wang, and C.-L. Lin, “Abnormal Crowd Event Detection Based on Outlier in Time,” in 2014 Int. Conf. Mach. Learn. Cybern., vol. 1, pp. 359–363, 2014.

  • [20] T. Hassner, Y. Itcher, and O. Kliper-Gross, “Violent Flows: Real-Time Detection of Violent Crowd Behavior *,” 2012 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., pp. 1–6, 2012.

  • [21] C.-L. L. Wei-Lieh Hsu, Yu-Cheng Wang, “Spatio-Temporal Anomaly Detection in Crowd Movement Using SIFT,” in Proc. 2014 Int. Conf. Mach. Learn. Cybern., 2014.

  • [22] Y.-T. Matsuda, T. Fujimura, K. Katahira, M. Okada, K. Ueno, K. Cheng, and K. Okanoya, “The Implicit Processing of Categorical and Dimensional Strategies: An fMRI Study of Facial Emotion Perception,” Front. Hum. Neurosci., vol. 7, no. September, 2013.

  • [23] R. Fan, K. Xu, and J. Zhao, “Higher Contagion and Weaker Ties Mean Anger Spreads Faster Than Joy in Social Media,” pp. 1–23, 2016.

  • [24] L. Coviello, J. H. Fowler, and M. Franceschetti, “Words on the Web: Noninvasive Detection of Emotional Contagion in Online Social Networks,” in Proc. IEEE, vol. 102, no. 12, pp. 1911–1921, 2014.

  • [25] D. Mehta, M. F. H. Siddiqui, and A. Y. Javaid, “Facial Emotion Recognition: A Survey and Real-World User Experiences in Mixed Reality,” Sensors, vol. 18, no. 2, pp. 416, 2018.

  • [26] S. Petrovica and H. K. Ekene, “Emotion Recognition for Intelligent Tutoring,” CEUR Workshop Proc., vol. 1684, 2016.

  • [27] Z. S. Hippe, J. L. Kulikowski, T. Mroczek, and J. Wtorek, “Emotion Recognition and Its Applications,” Adv. Intell. Syst. Comput., vol. 300, no. July, 2014.

  • [28] K. Glanz, “Social and Behavioral Theories 1. Learning Objectives,” Off. Behav. Soc. Sci. Res., 2005.

  • [29] G. Bradski, “The OpenCV Library,” Dr. Dobb’s J. Softw. Tools, vol. 120, pp. 122–125, 2000.

  • [30] J. Redmon and A. Farhadi, YOLO v.3, 2018.

  • [31] J. Redmon, “Darknet: Open Source Neural Networks in C,” Pjreddie. [Online]. Available:

  • [32] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common Objects in Context,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8693 LNCS, no. PART 5, pp. 740–755, 2014.

  • [33] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple Online and Realtime Tracking,” in 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 3464–3468.

  • [34] T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand Keypoint Detection in Single Images Using Multiview Bootstrapping,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4645–4653, 2017.

  • [35] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310, 2017.

  • [36] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional Pose Machines,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 4724–4732, 2016.


Journal + Issues