Smooth Non-increasing Square Spatial Extents of Filters in Convolutional Layers of CNNs for Image Classification Problems

[1] V. Chandrasekhar, J. Lin, O. Morère, H. Goh, and A. Veillard, “A practical guide to CNNs and Fisher Vectors for image instance retrieval,” Signal Processing, vol. 128, 2016, pp. 426–439. https://doi.org/10.1016/j.sigpro.2016.05.02110.1016/j.sigpro.2016.05.021Search in Google Scholar

[2] M. Elleuch, R. Maalej, and M. Kherallah, “A new design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition,” Procedia Computer Science, vol. 80, 2016, pp. 1712–1723. https://doi.org/10.1016/j.procs.2016.05.51210.1016/j.procs.2016.05.512Search in Google Scholar

[3] Q. Guo, F. Wang, J. Lei, D. Tu, and G. Li, “Convolutional feature learning and Hybrid CNN-HMM for scene number recognition,” Neurocomputing, vol. 184, 2016, pp. 78–90. https://doi.org/10.1016/j.neucom.2015.07.13510.1016/j.neucom.2015.07.135Search in Google Scholar

[4] M. Joo Er, Y. Zhang, N. Wang, and M. Pratama, “Attention pooling-based convolutional neural network for sentence modelling,” Information Sciences, vol. 373, 2016, pp. 388–403. https://doi.org/10.1016/j.ins.2016.08.08410.1016/j.ins.2016.08.084Search in Google Scholar

[5] Z. Chen, F. Cao, and J. Hu, “Approximation by network operators with logistic activation functions,” Applied Mathematics and Computation, vol. 256, 2015, pp. 565–571. https://doi.org/10.1016/j.amc.2015.01.04910.1016/j.amc.2015.01.049Search in Google Scholar

[6] D. Costarelli and R. Spigler, “Approximation results for neural network operators activated by sigmoidal functions,” Neural Networks, vol. 44, 2013, pp. 101–106. https://doi.org/10.1016/j.neunet.2013.03.01510.1016/j.neunet.2013.03.01523587719Search in Google Scholar

[7] G. A. Anastassiou, “Multivariate sigmoidal neural network approximation,” Neural Networks, vol. 24, iss. 4, 2011, pp. 378–386. https://doi.org/10.1016/j.neunet.2011.01.00310.1016/j.neunet.2011.01.00321310590Search in Google Scholar

[8] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, iss. 11, 1998, pp. 2278–2324. https://doi.org/10.1109/5.72679110.1109/5.726791Search in Google Scholar

[9] P. Simard, D. Steinkraus, and J. C. Platt, “Best practices for convolutional neural networks applied to visual document analysis,” International Conference on Document Analysis and Recognition (ICDAR), vol. 3, 2003, pp. 958–962. https://doi.org/10.1109/ICDAR.2003.122780110.1109/ICDAR.2003.1227801Search in Google Scholar

[10] D. Ciresan, U. Meier, and J. Schmidhuber, “Multi-column deep neural networks for image classification,” 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 3642–3649. https://doi.org/10.1109/CVPR.2012.624811010.1109/CVPR.2012.6248110Search in Google Scholar

[11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, iss. 6, 2017, pp. 84–90. https://doi.org/10.1145/306538610.1145/3065386Search in Google Scholar

[12] J. Mutch and D. G. Lowe, “Object class recognition and localization using sparse features with limited receptive fields,” International Journal of Computer Vision, vol. 80, iss. 1, 2008, pp. 45–57. https://doi.org/10.1007/s11263-007-0118-010.1007/s11263-007-0118-0Search in Google Scholar

[13] K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological Cybernetics, vol. 36, iss. 4, 1980, pp. 193–202. https://doi.org/10.1007/BF0034425110.1007/BF00344251Search in Google Scholar

[14] K. Fukushima, “Neocognitron: A hierarchical neural network capable of visual pattern recognition,” Neural Networks, vol. 1, iss. 2, 1988, pp. 119–130. https://doi.org/10.1016/0893-6080(88)90014-710.1016/0893-6080(88)90014-7Search in Google Scholar

[15] K. Fukushima, “Artificial vision by multi-layered neural networks: Neocognitron and its advances,” Neural Networks, vol. 37, 2013, pp. 103–119. https://doi.org/10.1016/j.neunet.2012.09.01610.1016/j.neunet.2012.09.01623098752Search in Google Scholar

[16] D. Ciresan, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber, “Flexible, high performance convolutional neural networks for image classification,” Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, vol. 2, 2011, pp. 1237–1242.Search in Google Scholar

[17] P. Connor, P. Hollensen, O. Krigolson, and T. Trappenberg, “A biological mechanism for Bayesian feature selection: Weight decay and raising the LASSO”, Neural Networks, vol. 67, 2015, pp. 121–130. https://doi.org/10.1016/j.neunet.2015.03.00510.1016/j.neunet.2015.03.00525897512Search in Google Scholar

[18] A. Mahendran and A. Vedaldi, “Visualizing deep convolutional neural networks using natural pre-images,” International Journal of Computer Vision, vol. 120, iss. 3, 2016, pp. 233–255. https://doi.org/10.1007/s11263-016-0911-810.1007/s11263-016-0911-8Search in Google Scholar

[19] L. Guo, S. Li, X. Niu, and Y. Dou, “A study on layer connection strategies in stacked convolutional deep belief networks,” Pattern Recognition, 6th Chinese Conference, CCPR 2014, Changsha, China, November 17–19, 2014 (Proceedings, Part I), 2014, pp. 81–90. https://doi.org/10.1007/978-3-662-45646-0_910.1007/978-3-662-45646-0_9Search in Google Scholar

[20] Z. Wang, Z. Deng, and S. Wang, “Accelerating convolutional neural networks with dominant convolutional kernel and knowledge preregression,” Computer Vision–ECCV 2016, 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VIII), 2016, pp. 533–548. https://doi.org/10.1007/978-3-319-46484-8_3210.1007/978-3-319-46484-8_32Search in Google Scholar

[21] Z.-Z. Li, Z.-Y. Zhong, and L.-W. Jin, “Identifying best hyperparameters for deep architectures using random forests,” Learning and Intelligent Optimization, 9th International Conference, LION 9, Lille, France, January 12–15, 2015 (Revised Selected Papers), 2015, pp. 29–42. https://doi.org/10.1007/978-3-319-19084-6_410.1007/978-3-319-19084-6_4Search in Google Scholar

[22] C. Ann Ronao and S.-B. Cho, “Deep convolutional neural networks for human activity recognition with smartphone sensors,” Neural Information Processing, 22nd International Conference, ICONIP 2015, November 9–12, 2015 (Proceedings, Part IV), 2015, pp. 46–53. https://doi.org/10.1007/978-3-319-26561-2_610.1007/978-3-319-26561-2_6Search in Google Scholar

[23] A. Azadeh, M. Saberi, A. Kazem, V. Ebrahimipour, A. Nourmohammadzadeh, and Z. Saberi, “A flexible algorithm for fault diagnosis in a centrifugal pump with corrupted data and noise based on ANN and support vector machine with hyper-parameters optimization,” Applied Soft Computing, vol. 13, iss. 3, 2013, pp. 1478–1485. https://doi.org/10.1016/j.asoc.2012.06.02010.1016/j.asoc.2012.06.020Search in Google Scholar

[24] Z. Bai, L. L. C. Kasun, and G.-B. Huang, “Generic object recognition with local receptive fields based extreme learning machine,” Procedia Computer Science, vol. 53, 2015, pp. 391–399. https://doi.org/10.1016/j.procs.2015.07.31610.1016/j.procs.2015.07.316Search in Google Scholar

[25] P. Date, J. A. Hendler, and C. D. Carothers, “Design index for deep neural networks,” Procedia Computer Science, vol. 88, 2016, pp. 131–138. https://doi.org/10.1016/j.procs.2016.07.41610.1016/j.procs.2016.07.416Search in Google Scholar

[26] N. van Noord and E. Postma, “Learning scale-variant and scale-invariant features for deep image classification,” Pattern Recognition, vol. 61, 2017, pp. 583–592. https://doi.org/10.1016/j.patcog.2016.06.00510.1016/j.patcog.2016.06.005Search in Google Scholar

[27] K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” Computer Vision and Pattern Recognition, arXiv:1312.6034v2 [cs.CV], 2014.Search in Google Scholar

[28] Y. Zhu, C. Zhang, D. Zhou, X. Wang, X. Bai, and W. Liu, “Traffic sign detection and recognition using fully convolutional network guided proposals,” Neurocomputing, vol. 214, 2016, pp. 758–766. https://doi.org/10.1016/j.neucom.2016.07.00910.1016/j.neucom.2016.07.009Search in Google Scholar

[29] J. Ma, F. Wu, J. Zhu, D. Xu, and D. Kong, “A pre-trained convolutional neural network based method for thyroid nodule diagnosis,” Ultrasonics, vol. 73, 2017, pp. 221–230. https://doi.org/10.1016/j.ultras.2016.09.01110.1016/j.ultras.2016.09.01127668999Search in Google Scholar

[30] J.-L. Buessler, P. Smagghe, and J.-P. Urban, “Image receptive fields for artificial neural networks,” Neurocomputing, vol. 144, 2014, pp. 258–270. https://doi.org/10.1016/j.neucom.2014.04.04510.1016/j.neucom.2014.04.045Search in Google Scholar

[31] J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, “Understanding neural networks through deep visualization,” Computer Vision and Pattern Recognition, arXiv:1506.06579v1 [cs.CV], 2015.Search in Google Scholar

[32] L. A. Gatys, A. S. Ecker, and M. Bethge, “Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks,” Computer Vision and Pattern Recognition, arXiv:1505.07376v1 [cs.CV], 2015.Search in Google Scholar

[33] H. Jégou, M. Douze, C. Schmid, and P. Pérez, “Aggregating local descriptors into a compact image representation,” 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 3304–3311.10.1109/CVPR.2010.5540039Search in Google Scholar

[34] A. Mahendran and A. Vedaldi, “Understanding deep image representations by inverting them,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 5188–5196. https://doi.org/10.1109/CVPR.2015.729915510.1109/CVPR.2015.7299155Search in Google Scholar

[35] C. Schmid and R. Mohr, “Local grayvalue invariants for image retrieval,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, iss. 5, 1997, pp. 530–535. https://doi.org/10.1109/34.58921510.1109/34.589215Search in Google Scholar

[36] V. Mayya, R. M. Pai, and M. M. M. Pai, “Automatic facial expression recognition using DCNN,” Procedia Computer Science, vol. 93, 2016, pp. 453–461. https://doi.org/10.1016/j.procs.2016.07.23310.1016/j.procs.2016.07.233Search in Google Scholar

[37] Y. LeCun, F. J. Huang, and L. Bottou, “Learning methods for generic object recognition with invariance to pose and lighting,” International Conference on Computer Vision and Pattern Recognition, vol. 2, 2004, pp. 97–104. https://doi.org/10.1109/CVPR.2004.131515010.1109/CVPR.2004.1315150Search in Google Scholar

[38] V. V. Romanuke, “Boosting ensembles of heavy two-layer perceptrons for increasing classification accuracy in recognizing shifted-turned-scaled flat images with binary features,” Journal of Information and Organizational Sciences, vol. 39, no. 1, 2015, pp. 75–84.Search in Google Scholar

[39] V. V. Romanuke, “Optimal training parameters and hidden layer neurons number of two-layer perceptron for generalized scaled objects classification problem,” Information Technology and Management Science, vol. 18, 2015, pp. 42–48. https://doi.org/10.1515/itms-2015-000710.1515/itms-2015-0007Search in Google Scholar

[40] V. V. Romanuke, “Two-layer perceptron for classifying flat scaledturned-shifted objects by additional feature distortions in training,” Journal of Uncertain Systems, vol. 9, no. 4, 2015, pp. 286–305.Search in Google Scholar

[41] V. V. Romanuke, “An attempt for 2-layer perceptron high performance in classifying shifted monochrome 60-by-80-images via training with pixel-distorted shifted images on the pattern of 26 alphabet letters,” Radio Electronics, Computer Science, Control, no. 2, 2013, pp. 112–118. https://doi.org/10.15588/1607-3274-2013-2-1810.15588/1607-3274-2013-2-18Search in Google Scholar

[42] E. Kussul and T. Baidyk, “Improved method of handwritten digit recognition tested on MNIST database,” Image and Vision Computing, vol. 22, iss. 12, 2004, pp. 971–981. https://doi.org/10.1016/j.imavis.2004.03.00810.1016/j.imavis.2004.03.008Search in Google Scholar

[43] V. V. Romanuke, “Training data expansion and boosting of convolutional neural networks for reducing the MNIST dataset error rate,” Research Bulletin of the National Technical University of Ukraine “Kyiv Polytechnic Institute”, no. 6, pp. 29–34, 2016. https://doi.org/10.20535/1810-0546.2016.6.8411510.20535/1810-0546.2016.6.84115Search in Google Scholar

[44] V. V. Romanuke, “Uniform sampling of fundamental simplexes as sets of players’ mixed strategies in the finite noncooperative game for finding equilibrium situations with possible concessions,” Journal of Automation and Information Sciences, vol. 47, iss. 9, 2015, pp. 76–85. https://doi.org/10.1615/JAutomatInfScien.v47.i9.7010.1615/JAutomatInfScien.v47.i9.70Search in Google Scholar

[45] V. V. Romanuke, “Sampling individually fundamental simplexes as sets of players’ mixed strategies in finite noncooperative game for applicable approximate Nash equilibrium situations with possible concessions,” Journal of Information and Organizational Sciences, vol. 40, no. 1, 2016, pp. 105–143.10.31341/jios.40.1.6Search in Google Scholar

[46] V. V. Romanuke, “Appropriate number and allocation of ReLUs in convolutional neural networks,” Research Bulletin of the National Technical University of Ukraine “Kyiv Polytechnic Institute”, no. 1, pp. 69–78, 2017. https://doi.org/10.20535/1810-0546.2017.1.8815610.20535/1810-0546.2017.1.88156Search in Google Scholar