A Novel Deep Neural Network that Uses Space-Time Features for Tracking and Recognizing a Moving Object

Open access


This work proposes a deep neural net (DNN) that accomplishes the reliable visual recognition of a chosen object captured with a webcam and moving in a 3D space. Autoencoding and substitutional reality are used to train a shallow net until it achieves zero tracking error in a discrete ambient. This trained individual is set to work in a real world closed loop system where images coming from a webcam produce displacement information for a moving region of interest (ROI) inside the own image. This loop gives rise to an emergent tracking behavior which creates a self-maintain flow of compressed space-time data. Next, short term memory elements are set to play a key role by creating new representations in terms of a space-time matrix. The obtained representations are delivery as input to a second shallow network which acts as “recognizer”. A noise balanced learning method is used to fast train the recognizer with real-world images, giving rise to a simple and yet powerful robotic eye, with a slender neural processor that vigorously tracks and recognizes the chosen object. The system has been tested with real images in real time.

[1] Y. Zhang, K. Sohn, R. Villegas, P. Gang and L. Honglak, Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction, Journal CoRR, vol abs/1504.03293, 2015

[2] C. Szegedy, A. Toshev, and D. Erhan, Deep Neural Networks for Object Detection, Advances in Neural Information Processing Systems 26, 2013, pp. 2553–2561

[3] K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. Le-Cun, What is the Best Multi-Stage Architecture for Object Recognition?, in ’ICCV’, IEEE, 2009, pp. 2146–2153

[4] S. Luck, Visual short term memory, in Scholarpedia, 2007, VOLUME 2, number 6, pp. 3328

[5] E. Averbach, and A. S. Coriell, Short-Term Memory in Vision, The Bell System Technical Journal, 1961, vol. 40, Issue 1, pp. 309–328

[6] Y. Jiang, R. M. Zur, L. L. Pesce and a. K. Drukker, A study of the effect of noise injection on the training of artificial neural networks, in Neural Network IJCNN International Joint Conference on, IEEE, 2009, pp. 1428–1432

[7] Y. Jiang R., M. Zur, L. L. Pesce and K. Drukker, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, in Medical Physics, vol. 36, 2009, pp. 4810–4818

[8] O. Chang, Reliable object recognition by using co-operative neural agents, in 2014 International Joint Conference on Neural Networks, (IJCNN), Beijing, China, July 6–11, 2014, pp. 2571–2578

[9] Chang Oscar, A Bio-Inspired Robot with Visual Perception of Affordances, In: Computer Vision - ECCV 2014 Workshops: Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part II, editors Agapito, Lourdes and Bronstein, M. Michael and Rother, Carsten, Springer International Publishing. Switzerland, 2015, pp. 420–426

[10] P. Constante, A. Gordon, O. Chang, E. Pruna, I. Escobar, F. Acua, Artificial Vision Techniques for Strawberrys Industrial Classification, IEEE Latin America Transactions, 2016, In-press

[11] M. H. Mahoor, R. Godzdanker, K. Dalamagkidis and K. P. Valavanis, Vision-Based Landing of Light Weight Unmanned Helicopters on a Smart Landing Platform, in Journal of Intelligent Robotic Systems, vol. 61, 2011, pp. 251–265

[12] I. F. Mondragon, P. Campoy, C. Martinez and M. A. Olivares-Mendez, 3D pose estimation based on planar object tracking for UAVs control, in Robotics and Automation (ICRA), 2010 IEEE International Conference on, 2010, pp. 35–41

[13] M. Woolridge and N. R. Jennings, Intelligent agents: Theory and practice, in: The Knowledge Engineering Review, vol. 10, no. 2, 1995, pp. 115–152

[14] Y. Bengio, Learning Deep Architectures for AI, In: Found. Trends Mach. Learn, vol. 2, number 1, 2009 pp. 1–127

[15] K. Suzuki, S. Wakisaka, and N. Fujii, Substitutional reality system: a novel experimental platform for experiencing alternative reality, In: Scientific reports, vol. 2, 2012

[16] A. Dorian, Can we build a conscious machine?, In: CoRR, 2014, vol. abs/1411.5224

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

Journal Information

CiteScore 2017: 5.00

SCImago Journal Rank (SJR) 2017: 0.492
Source Normalized Impact per Paper (SNIP) 2017: 2.813

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 324 324 52
PDF Downloads 162 162 27