Score Level and Rank Level Fusion for Kinect-Based Multi-Modal Biometric System

Open access

Abstract

Computational intelligence firmly made its way into the areas of consumer applications, banking, education, social networks, and security. Among all the applications, biometric systems play a significant role in ensuring an uncompromised and secure access to resources and facilities. This article presents a first multimodal biometric system that combines KINECT gait modality with KINECT face modality utilizing the rank level and the score level fusion. For the KINECT gait modality, a new approach is proposed based on the skeletal information processing. The gait cycle is calculated using three consecutive local minima computed for the distance between left and right ankles. The feature distance vectors are calculated for each person’s gait cycle, which allows extracting the biometric features such as the mean and the variance of the feature distance vector. For Kinect face recognition, a novel method based on HOG features has been developed. Then, K-nearest neighbors feature matching algorithm is applied as feature classification for both gait and face biometrics. Two fusion algorithms are implemented. The combination of Borda count and logistic regression approaches are used in the rank level fusion. The weighted sum method is used for score level fusion. The recognition accuracy obtained for multi-modal biometric recognition system tested on KINECT Gait and KINECT Eurocom Face datasets is 93.33% for Borda count rank level fusion, 96.67% for logistic regression rank-level fusion and 96.6% for score level fusion.

[1] Gavrilova M., Monwar M., Multimodal Biometrics and Intelligent Image Processing for Security Systems, Hardcover, IGI Global, 350 pages, 2012

[2] Ross A., Nandakumar K., Jain A., Handbook of Multibiometrics, New York: Springer-Verlag, 2006

[3] Kastaniotis D. et.al. A Framework for Gait-based Recognition Using Kinect, Pattern Recognition, Elsevier, Vol. 68(2), pp. 327-335, 2015

[4] Leszek Rutkowski, Computational Intelligence: Methods and Techniques, 1st Ed, book, Springer, 2008

[5] Y Wang et.al., Cognitive Intelligence: Deep Learning, Thinking, and Reasoning by Brain-Inspired Systems, IJCINI, vol. 10, no. 4, pp1-20, 2016

[6] Paul P.P., Gavrilova M., Alhajj R., Decision Fusion for Multimodal Biometrics Using Social Network Analysis, IEEE Trans. On Systems, Man, and Cybernetics, vol. 44, no. 11, pp. 1522-1533, 2014

[7] Monwar M., Gavrilova M., Multimodal Biometric System using Rank-Level Fusion Approach, IEEE Trans. on Systems, Man, and Cybernetics, vol. 39, no. 4, pp. 867-878, 2009

[8] Rahman M.W., Gavrilova M., Kinect Gait Skeletal Joint Feature Based Person Identification, 16th Int. Conference on Cognitive Informatics & Cognitive Computing (ICCICC), IEEE, pp. 423-430, 2017

[9] Shakhnarovich G., Darrel T., On Probabilistic Combination of Face and Gait Cues for Identification, 5th International Conference on Automatic Face and Gesture Recognition, IEEE, pp. 1-6, 2002

[10] Rahman M.W., Zohra F.T., Gavrilova M., Rank Level Fusion for Kinect Gait and Face Biometric Identification, Symposium Series on Computational Intelligence (SSCI), IEEE, pp. 2218-2224, 2017

[11] Kale A., Roychowdhury A., Chellappa R., Fusion of Gait and Face for Human Identification, International Conference on Acoustics, Speech and Signal Processing, IEEE, pp. 1-5, 2004

[12] Ranganath T., Lee T., Sanei S., Fusion of Chaotic Measure into a New Hybrid Face-Gait System for Human Recognition, 18th International Conference on Pattern Recognition, IEEE, vol. 4, pp. 541-544, 2006

[13] Zhou X., Bahanu B., Feature Fusion of Face and Gait for Human Recognition at a Distance in Video, 18th International Conference on Pattern Recognition (ICPR), IEEE, vol. 4, pp. 529-532, 2006

[14] Zhou X., Bahanu B., Integrating Face and Gait for Human Recognition at a Distance in Video, IEEE Trans. on Systems, Man, and Cybernetics, vol. 37, no. 5, pp. 119-1137, 2007

[15] Geng X. et.al. Adaptive Fusion of Gait and Face for Human Identification in Video, Workshop on Application of Computer Vision, IEEE, pp. 1-6, 2008

[16] Hossain E., Chetty G., Multimodal Face-Gait Fusion for Biometric Person Authentication, International Conference on Embedded and Ubiquitous Computing (EUC), IEEE, pp. 332-337, 2011

[17] Hoffman M. et.al, Combined face and gait recognition using alpha matte preprocessing, 5th International Conference on Biometric (ICB), IEEE, pp. 390-395, 2012

[18] Almohammod M., Salma G., Mahmoud T., Human identification system based on feature level fusion using face and gait biometrics, International Conference on Engineering and Technology (ICET), IEEE, pp. 1-5, 2012

[19] Xing X., Wang K., Lv Z., Fusion of Gait and Facial Features using Coupled Projections for People Identification at a Distance, IEEE Signal Processing Letters, vol. 22, no. 12, pp. 2349-2353, 2015

[20] Zhang D. et.al., Ethnicity classification based on fusion of face and gait, International Conference on Biometric (ICB), IEEE, pp. 384-389, 2012

[21] Zhang D., Wang Y., Gender recognition based on fusion on face and gait information, International Conference on Machine Learning and Cybernetics, IEEE, vol. 1, pp. 62-67, 2008

[22] Ahmed F., Paul P.P., Gavrilova M., DTW based Kernel and Rank Level Fusion for 3D Gait Recognition using Kinect, The Visual Computer, vol. 31, no 6, pp. 915-924, 2015

[23] Vondrick C. et. al. HOGgles: Visualizing Object Detection Features, International Journal on Computer Vision, pp. 1-9, 2013

[24] http://www.upcv.upatras.gr/personal/kastaniotis/datasets.html Last accessed: September 20, 2017

[25] Min R., Kose N., Dugelay, J.L., KinectFaceDB: A Kinect database for face recognition, IEEE Transactions on Systems, Man and Cybernetics: Systems, vol. 44, no.11, pp. 1534-1548, 2014

[26] Paul P.P., Gavrilova M., Alhajj R., Decision Fusion for Multimodal Biometrics Using Social Network Analysis, Transactions on Systems, Man, and Cybernetics Systems, IEEE, vol. 44, no. 11, pp. 1522-1533, 2014

[27] Xing X., Wang K., Lv Z., Fusion of Gait and Facial Features using Coupled Projections for People Identification at a Distance, Signal Processing Letters, IEEE, vol. 22, no. 12, pp. 2349-2353, 2015

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

Journal Information


CiteScore 2018: 4.70

SCImago Journal Rank (SJR) 2018: 0.351
Source Normalized Impact per Paper (SNIP) 2018: 4.066

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 205 205 135
PDF Downloads 112 112 50