A Novel Deep Neural Network that Uses Space-Time Features for Tracking and Recognizing a Moving Object

Oscar Chang 1 , Patricia Constante 2 , Andrés Gordon 2  and Marco Singaña 2
  • 1 Department of Software Innovation, Farmaenlace, Quito Ecuador
  • 2 Department of Energy and Mechanics, Universidad de las Fuerzas Armadas ESPE Latacunga Ecuador


This work proposes a deep neural net (DNN) that accomplishes the reliable visual recognition of a chosen object captured with a webcam and moving in a 3D space. Autoencoding and substitutional reality are used to train a shallow net until it achieves zero tracking error in a discrete ambient. This trained individual is set to work in a real world closed loop system where images coming from a webcam produce displacement information for a moving region of interest (ROI) inside the own image. This loop gives rise to an emergent tracking behavior which creates a self-maintain flow of compressed space-time data. Next, short term memory elements are set to play a key role by creating new representations in terms of a space-time matrix. The obtained representations are delivery as input to a second shallow network which acts as “recognizer”. A noise balanced learning method is used to fast train the recognizer with real-world images, giving rise to a simple and yet powerful robotic eye, with a slender neural processor that vigorously tracks and recognizes the chosen object. The system has been tested with real images in real time.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] Y. Zhang, K. Sohn, R. Villegas, P. Gang and L. Honglak, Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction, Journal CoRR, vol abs/1504.03293, 2015

  • [2] C. Szegedy, A. Toshev, and D. Erhan, Deep Neural Networks for Object Detection, Advances in Neural Information Processing Systems 26, 2013, pp. 2553–2561

  • [3] K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. Le-Cun, What is the Best Multi-Stage Architecture for Object Recognition?, in ’ICCV’, IEEE, 2009, pp. 2146–2153

  • [4] S. Luck, Visual short term memory, in Scholarpedia, 2007, VOLUME 2, number 6, pp. 3328

  • [5] E. Averbach, and A. S. Coriell, Short-Term Memory in Vision, The Bell System Technical Journal, 1961, vol. 40, Issue 1, pp. 309–328

  • [6] Y. Jiang, R. M. Zur, L. L. Pesce and a. K. Drukker, A study of the effect of noise injection on the training of artificial neural networks, in Neural Network IJCNN International Joint Conference on, IEEE, 2009, pp. 1428–1432

  • [7] Y. Jiang R., M. Zur, L. L. Pesce and K. Drukker, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, in Medical Physics, vol. 36, 2009, pp. 4810–4818

  • [8] O. Chang, Reliable object recognition by using co-operative neural agents, in 2014 International Joint Conference on Neural Networks, (IJCNN), Beijing, China, July 6–11, 2014, pp. 2571–2578

  • [9] Chang Oscar, A Bio-Inspired Robot with Visual Perception of Affordances, In: Computer Vision - ECCV 2014 Workshops: Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part II, editors Agapito, Lourdes and Bronstein, M. Michael and Rother, Carsten, Springer International Publishing. Switzerland, 2015, pp. 420–426

  • [10] P. Constante, A. Gordon, O. Chang, E. Pruna, I. Escobar, F. Acua, Artificial Vision Techniques for Strawberrys Industrial Classification, IEEE Latin America Transactions, 2016, In-press

  • [11] M. H. Mahoor, R. Godzdanker, K. Dalamagkidis and K. P. Valavanis, Vision-Based Landing of Light Weight Unmanned Helicopters on a Smart Landing Platform, in Journal of Intelligent Robotic Systems, vol. 61, 2011, pp. 251–265

  • [12] I. F. Mondragon, P. Campoy, C. Martinez and M. A. Olivares-Mendez, 3D pose estimation based on planar object tracking for UAVs control, in Robotics and Automation (ICRA), 2010 IEEE International Conference on, 2010, pp. 35–41

  • [13] M. Woolridge and N. R. Jennings, Intelligent agents: Theory and practice, in: The Knowledge Engineering Review, vol. 10, no. 2, 1995, pp. 115–152

  • [14] Y. Bengio, Learning Deep Architectures for AI, In: Found. Trends Mach. Learn, vol. 2, number 1, 2009 pp. 1–127

  • [15] K. Suzuki, S. Wakisaka, and N. Fujii, Substitutional reality system: a novel experimental platform for experiencing alternative reality, In: Scientific reports, vol. 2, 2012

  • [16] A. Dorian, Can we build a conscious machine?, In: CoRR, 2014, vol. abs/1411.5224


Journal + Issues