Cross-task code reuse in genetic programming applied to visual learning

Open access

Abstract

We propose a method that enables effective code reuse between evolutionary runs that solve a set of related visual learning tasks. We start with introducing a visual learning approach that uses genetic programming individuals to recognize objects. The process of recognition is generative, i.e., requires the learner to restore the shape of the processed object. This method is extended with a code reuse mechanism by introducing a crossbreeding operator that allows importing the genetic material from other evolutionary runs. In the experimental part, we compare the performance of the extended approach to the basic method on a real-world task of handwritten character recognition, and conclude that code reuse leads to better results in terms of fitness and recognition accuracy. Detailed analysis of the crossbred genetic material shows also that code reuse is most profitable when the recognized objects exhibit visual similarity.

Bhanu, B., Lin, Y. and Krawiec, K. (2005). Evolutionary Synthesisof Pattern Recognition Systems, Springer-Verlag, New York, NY.

Caruana, R. (1997). Multitask learning, Machine Learning28(1): 41-75.

Chang, Y.F., Lee, J.C., Mohd Rijal, O. and Syed Abu Bakar, S.A.R. (2010). Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship model, International Journal of AppliedMathematics and Computer Science 20(4): 727-738, DOI: 10.2478/v10006-010-0055-x.

Ciresan, D.C., Meier, U. and Schmidhuber, J. (2012). Multi-column deep neural networks for image classification, Proceedings of the 2012 IEEE Conferenceon Computer Vision and Pattern Recognition(CVPR), Providence, RI, USA, pp. 3642-3649.

Fabijańska, A. (2012). A survey of subpixel edge detection methods for images of heat-emitting metal specimens, International Journal of Applied Mathematicsand Computer Science 22(3): 695-710, DOI: 10.2478/v10006-012-0052-3.

Galvan Lopez, E., Poli, R. and Coello Coello, C.A. (2004). Reusing code in genetic programming, in M. Keijzer, U.-M. O’Reilly, S.M. Lucas, E. Costa and T. Soule (Eds.), Genetic Programming-7th European Conference,EuroGP 2004, Proceedings, Lecture Notes in Computer Science, Vol. 3003, Springer-Verlag, Berlin/Heidelberg, pp. 359-368.

Ghosn, J. and Bengio, Y. (2003). Bias learning, knowledge sharing, IEEE Transactions on Neural Networks14(4): 748-765.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I.H. (2009). The Weka data mining software: An update, SIGKDD Explorations 11(1): 10-18.

Haynes, T. (1997). On-line adaptation of search via knowledge reuse, in J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba and R.L. Riolo (Eds.), GeneticProgramming 1997: Proceedings of the Second AnnualConference, Morgan Kaufmann, San Francisco, CA, pp. 156-161.

Holland, J. (1975). Adaptation in Natural and Artificial Systems, Vol. 1, University of Michigan Press, Ann Arbor, MI.

Hornby, G.S. and Pollack, J.B. (2002). Creating high-level components with a generative representation for body-brain evolution, Artificial Life 8(3): 223-246.

Howard, D. (2003). Modularization by multi-run frequency driven subtree encapsulation, in R.L. Riolo and B. Worzel (Eds.), Genetic Programming Theory and Practice, Kluwer, New York, NY, Chapter 10, pp. 155-172.

Howard, D., Roberts, S.C. and Ryan, C. (2006). Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance, Pattern RecognitionLetters 27(11): 1275-1288.

Hsu, W.H., Harmon, S.J., Rodriguez, E. and Zhong, C. (2004). Empirical comparison of incremental reuse strategies in genetic programming for keep-away soccer, in M. Keijzer (Ed.), Late Breaking Papers at the 2004 Genetic andEvolutionary Computation Conference, Association for Computing Machinery, Seattle, WA.

Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007a). Genetic programming for cross-task knowledge sharing, in D. Thierens (Ed.), Genetic and Evolutionary ComputationConference GECCO, Association for Computing Machinery, London, pp. 1620-1627.

Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007b). Knowledge reuse in genetic programming applied to visual learning, in D. Thierens (Ed.), Genetic and EvolutionaryComputation Conference GECCO, Association for Computing Machinery, London, pp. 1790-1797.

Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007c). Learning and recognition of hand-drawn shapes using generative genetic programming, in M. Giacobini (Ed.), EvoWorkshops2007, Lecture Notes in Computer Science, Vol. 4448, Springer-Verlag, Berlin/Heidelberg, pp. 281-290.

Koza, J. (1992). Genetic Programming, MIT Press, Cambridge, MA.

Koza, J.R. (1994). Genetic Programming II: Automatic Discoveryof Reusable Programs, MIT Press, Cambridge, MA.

Koza, J.R., Bennett III, F.H., Andre, D. and Keane, M.A. (1996). Reuse, parameterized reuse, and hierarchical reuse of substructures in evolving electrical circuits using genetic programming, in T. Higuchi (Ed.), Proceedings of InternationalConference on Evolvable Systems: From Biology toHardware (ICES-96), Lecture Notes in Computer Science, Vol. 1259, Springer-Verlag, Berlin.

Krawiec, K. (2006). Learning high-level visual concepts using attributed primitives and genetic programming, in F. Rothlauf (Ed.), EvoWorkshops 2006, Lecture Notes in Computer Science, Vol. 3907, Springer-Verlag, Berlin/Heidelberg, pp. 515-519.

Krawiec, K. (2007). Generative learning of visual concepts using multiobjective genetic programming, Pattern RecognitionLetters 28(16): 2385-2400.

Krawiec, K. and Bhanu, B. (2005). Visual learning by coevolutionary feature synthesis, IEEE Transactions onSystem, Man, and Cybernetics, Part B 35(3): 409-425.

Kurashige, K., Fukuda, T. and Hoshino, H. (2003). Reusing primitive and acquired motion knowledge for gait generation of a six-legged robot using genetic programming, Journal of Intelligent and Robotic Systems38(1): 121-134.

Langdon, W.B. and Poli, R. (2002). Foundations of Genetic Programming, Springer-Verlag, New York, NY.

Li, B., Li, X., Mabu, S. and Hirasawa, K. (2012). Towards automatic discovery and reuse of subroutines in variable size genetic network programming, in X. Li (Ed.), Proceedingsof the 2012 IEEE Congress on Evolutionary Computation,Brisbane, Australia, pp. 485-492.

Louis, S. and McDonnell, J. (2004). Learning with case-injected genetic algorithms, IEEE Transactions on EvolutionaryComputation 8(4): 316-328.

Luke, S. (2002). ECJ evolutionary computation system, http://cs.gmu.edu/eclab/projects/ecj/.

Mitchell, T.M. (2006). The discipline of machine learning, Technical Report CMU-ML-06-108, Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA.

Montana, D.J. (1993). Strongly typed genetic programming, BBN Technical Report #7866, Bolt Beranek and Newman, Inc., Cambridge, MA.

Moya, M.R., Koch, M.W. and Hostetler, L.D. (1993). One-class classifier networks for target recognition applications, World Congress on Neural Networks, Portland, OR, USA, pp. 797-801.

O’Sullivan, J. and Thrun, S. (1995). A Robot That ImprovesIts Ability to Learn, Carnegie Mellon University, School of Computer Science, Pittsburgh, PA.

Poli, R., Langdon, W.B. and McPhee I.N.F. (2008). A field guide to genetic programming, http://www.gp-field-guide.org.uk/.

Perez, C.B. and Olague, G. (2013). Genetic programming as strategy for learning image descriptor operators, IntelligentData Analysis 17(4): 561-583.

Pratt, L.Y., Mostow, J. and Kamm, C.A. (1991). Direct transfer of learned information among neural networks, Proceedingsof the 9th National Conference on Artificial Intelligence(AAAI-91), Anaheim, CA, USA, pp. 584-589.

Rizki, M.M., Zmuda, M.A. and Tamburino, L.A. (2002). Evolving pattern recognition systems, IEEE Transactionson Evolutionary Computation 6(6): 594-609.

Roberts, S.C., Howard, D. and Koza, J.R. (2001). Evolving modules in genetic programming by subtree encapsulation, in J.F. Miller (Ed.), Genetic Programming, Proceedings ofEuroGP’2001, Lecture Notes in Computer Science, Vol. 2038, Springer-Verlag, Berlin, pp. 160-175.

Rosca, J.P. and Ballard, D.H. (1996). Discovery of subroutines in genetic programming, in P.J.Angeline and K.E.Kinnear, Jr. (Eds.), Advances in Genetic Programming 2,MIT Press, Cambridge, MA, Chapter 9, pp. 177-202.

Seront, G. (1995). External concepts reuse in genetic programming, in E.V. Siegel and J.R. Koza (Eds.), WorkingNotes for the AAAI Symposium on Genetic Programming, AAAI/MIT, Cambridge, MA, pp. 94-98.

Tackett,W.A. (1993). Genetic generation of “dendritic” trees for image classification, Proceedings of theWorld Congress onNeural Networks, Portland, OR, USA, pp. IV 646-649.

Teller, A. and Veloso, M. (1997). PADO: A new learning architecture for object recognition, in K. Ikeuchi and M. Veloso (Eds.), Symbolic Visual Learning, Oxford Press, New York, NY, pp. 77-112.

Trujillo, L. and Olague, G. (2006). Synthesis of interest point detectors through genetic programming, in M. Cattolico (Ed.), Genetic and Evolutionary Computation ConferenceGECCO, Association for Computing Machinery, Seattle, WA, pp. 887-894.

Vilalta, R. and Drissi, Y. (2002). A perspective view and survey of meta-learning, Artificial Intelligence Review18(2): 77-95.

Whitley, D., Rana, S. and Heckendorn, R. (1999). The island model genetic algorithm: On separability, population size and convergence, Journal of Computing and InformationTechnology 7(1): 33-47.

International Journal of Applied Mathematics and Computer Science

Journal of the University of Zielona Góra

Journal Information


IMPACT FACTOR 2017: 1.694
5-year IMPACT FACTOR: 1.712

CiteScore 2017: 2.20

SCImago Journal Rank (SJR) 2017: 0.729
Source Normalized Impact per Paper (SNIP) 2017: 1.604

Mathematical Citation Quotient (MCQ) 2017: 0.13

Cited By

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 144 143 10
PDF Downloads 59 58 5