A Novel Deep Neural Network that Uses Space-Time Features for Tracking and Recognizing a Moving Object

Open access


This work proposes a deep neural net (DNN) that accomplishes the reliable visual recognition of a chosen object captured with a webcam and moving in a 3D space. Autoencoding and substitutional reality are used to train a shallow net until it achieves zero tracking error in a discrete ambient. This trained individual is set to work in a real world closed loop system where images coming from a webcam produce displacement information for a moving region of interest (ROI) inside the own image. This loop gives rise to an emergent tracking behavior which creates a self-maintain flow of compressed space-time data. Next, short term memory elements are set to play a key role by creating new representations in terms of a space-time matrix. The obtained representations are delivery as input to a second shallow network which acts as “recognizer”. A noise balanced learning method is used to fast train the recognizer with real-world images, giving rise to a simple and yet powerful robotic eye, with a slender neural processor that vigorously tracks and recognizes the chosen object. The system has been tested with real images in real time.


  • [1] Y. Zhang, K. Sohn, R. Villegas, P. Gang and L. Honglak, Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction, Journal CoRR, vol abs/1504.03293, 2015

  • [2] C. Szegedy, A. Toshev, and D. Erhan, Deep Neural Networks for Object Detection, Advances in Neural Information Processing Systems 26, 2013, pp. 2553–2561

  • [3] K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. Le-Cun, What is the Best Multi-Stage Architecture for Object Recognition?, in ’ICCV’, IEEE, 2009, pp. 2146–2153

  • [4] S. Luck, Visual short term memory, in Scholarpedia, 2007, VOLUME 2, number 6, pp. 3328

  • [5] E. Averbach, and A. S. Coriell, Short-Term Memory in Vision, The Bell System Technical Journal, 1961, vol. 40, Issue 1, pp. 309–328

  • [6] Y. Jiang, R. M. Zur, L. L. Pesce and a. K. Drukker, A study of the effect of noise injection on the training of artificial neural networks, in Neural Network IJCNN International Joint Conference on, IEEE, 2009, pp. 1428–1432

  • [7] Y. Jiang R., M. Zur, L. L. Pesce and K. Drukker, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, in Medical Physics, vol. 36, 2009, pp. 4810–4818

  • [8] O. Chang, Reliable object recognition by using co-operative neural agents, in 2014 International Joint Conference on Neural Networks, (IJCNN), Beijing, China, July 6–11, 2014, pp. 2571–2578

  • [9] Chang Oscar, A Bio-Inspired Robot with Visual Perception of Affordances, In: Computer Vision - ECCV 2014 Workshops: Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part II, editors Agapito, Lourdes and Bronstein, M. Michael and Rother, Carsten, Springer International Publishing. Switzerland, 2015, pp. 420–426

  • [10] P. Constante, A. Gordon, O. Chang, E. Pruna, I. Escobar, F. Acua, Artificial Vision Techniques for Strawberrys Industrial Classification, IEEE Latin America Transactions, 2016, In-press

  • [11] M. H. Mahoor, R. Godzdanker, K. Dalamagkidis and K. P. Valavanis, Vision-Based Landing of Light Weight Unmanned Helicopters on a Smart Landing Platform, in Journal of Intelligent Robotic Systems, vol. 61, 2011, pp. 251–265

  • [12] I. F. Mondragon, P. Campoy, C. Martinez and M. A. Olivares-Mendez, 3D pose estimation based on planar object tracking for UAVs control, in Robotics and Automation (ICRA), 2010 IEEE International Conference on, 2010, pp. 35–41

  • [13] M. Woolridge and N. R. Jennings, Intelligent agents: Theory and practice, in: The Knowledge Engineering Review, vol. 10, no. 2, 1995, pp. 115–152

  • [14] Y. Bengio, Learning Deep Architectures for AI, In: Found. Trends Mach. Learn, vol. 2, number 1, 2009 pp. 1–127

  • [15] K. Suzuki, S. Wakisaka, and N. Fujii, Substitutional reality system: a novel experimental platform for experiencing alternative reality, In: Scientific reports, vol. 2, 2012

  • [16] A. Dorian, Can we build a conscious machine?, In: CoRR, 2014, vol. abs/1411.5224

Journal of Artificial Intelligence and Soft Computing Research

The Journal of Polish Neural Network Society, the University of Social Sciences in Lodz & Czestochowa University of Technology

Journal Information


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 25 25 18
PDF Downloads 12 12 6