The Recognition Of Partially Occluded Objects with Support Vector Machines, Convolutional Neural Networks and Deep Belief Networks

Open access


Biologically inspired artificial neural networks have been widely used for machine learning tasks such as object recognition. Deep architectures, such as the Convolutional Neural Network, and the Deep Belief Network have recently been implemented successfully for object recognition tasks. We conduct experiments to test the hypothesis that certain primarily generative models such as the Deep Belief Network should perform better on the occluded object recognition task than purely discriminative models such as Convolutional Neural Networks and Support Vector Machines. When the generative models are run in a partially discriminative manner, the data does not support the hypothesis. It is also found that the implementation of Gaussian visible units in a Deep Belief Network trained on occluded image data allows it to also learn to effectively classify non-occluded images

[1] D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science, 9:147-169, 1985.

[2] Y. Bengio. Learning deep architectures for ai. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.

[3] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at

[4] Joseph Lin Chu and Adam Krzy˙zak. Application of support vector machines, convolutional neural networks and deep belief networks to recognition of partially occluded objects. In L. Rutkowski, editor, The 13th International Conference on Artificial Intelligence and Soft Computing ICAISC 2014, Lecture Notes on Artifical Intelligece (LNAI), volume 8467, pages 34-46. Springer International Publishing Switzerland, 2014.

[5] R. Collobert and S. Bengio. Links between perceptrons, mlps and svms. Proceedings of the 21st International Conference on Machine Learning, page 23, 2004.

[6] C. Cortes and V. N. Vapnik. Support-vector networks. Machine Learning, 20:273-297, 1995.

[7] K. Fukushima. Neocognitron for handwritten digit recognition. Neurocomputing, 51:161-180, 2003.

[8] Kunihiko Fukushima and Sei Miyake. Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition, 15(6):455-469, 1982.

[9] G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771-1800, 2002.

[10] G. E. Hinton. A practical guide to training restricted boltzmann machines. Momentum, 9(1):599-619, 2010.

[11] G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554, 2006.

[12] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313:504-507, 2006.

[13] J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the USA, 79(8):2554-2558, 1982.

[14] F. J. Huang and Y. LeCun. Large-scale learning with svm and convolutional nets for generic object categorization. Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1:284-291, 2006.

[15] D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in a cat’s visual cortex. Journal of Physiology (London), 160:106-154, 1962.

[16] Y. LeCun, L. Bottou, Y. Bengio, and Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.

[17] Y. LeCun, F.J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2:97-104, 2004.

[18] Aleix M. Mart´ınez. Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6):748-763, 2002.

[19] V. Nair and G. E. Hinton. 3d object recognition with deep belief nets. Advances in Neural Information Processing Systems (NIPS), pages 1339-1347, 2009.

[20] M. Ranzato, J. Susskind, V. Mnih, and G. Hinton. On deep generative models with applications to recognition. 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2857-2864, 2011.

[21] M. A. Ranzato, F. J. Huang, Y. L. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1-8, 2007.

[22] P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In David E. Rumelhart and James L. McLelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1, chapter 6, pages 194-281. MIT Press, 1986.

[23] P.W. M. Tsang and P. C. Yuen. Recognition of partially occluded objects. IEEE Transactions on Systems, Man and Cybernetics, 23(1):228-236, 1993.

[24] John Winn and Jamie Shotton. The layout consistent random field for recognizing and segmenting partially occluded objects. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 1:37-44, 2006.

[25] LaurenzWiskott and Christoph Von Der Malsburg. A neural system for the recognition of partially occluded objects in cluttered scenes: A pilot study. International Journal of Pattern Recognition and Artificial Intelligence, 7(4):935-948, 1993

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

Journal Information

CiteScore 2017: 5.00

SCImago Journal Rank (SJR) 2017: 0.492
Source Normalized Impact per Paper (SNIP) 2017: 2.813


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 168 168 23
PDF Downloads 30 30 4