Swarm Algorithms for NLP - The Case of Limited Training Data

Open access


The present article describes a novel phrasing model which can be used for segmenting sentences of unconstrained text into syntactically-defined phrases. This model is based on the notion of attraction and repulsion forces between adjacent words. Each of these forces is weighed appropriately by system parameters, the values of which are optimised via particle swarm optimisation. This approach is designed to be language-independent and is tested here for different languages.

The phrasing model’s performance is assessed per se, by calculating the segmentation accuracy against a golden segmentation. Operational testing also involves integrating the model to a phrase-based Machine Translation (MT) system and measuring the translation quality when the phrasing model is used to segment input text into phrases. Experiments show that the performance of this approach is comparable to other leading segmentation methods and that it exceeds that of baseline systems.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] D. Klein and C. D. Manning A generative constituent-context model for improved grammar induction Proceedings of 40th ACL Meeting Philadelphia USA pages 128–135 July 2002.

  • [2] D. Klein and C. D. Manning Corpus-based induction of syntactic structure: Models of dependency and constituency Proceedings of 42nd ACL Meeting Barcelona Spain pages 478–485 July 21-26 2004.

  • [3] Y. Seginer Fast unsupervised incremental parsing Proceedings of 45th ACL Meeting Prague Czech Republic pages 384–391 June 2007.

  • [4] E. Ponvert J. Baldridge and K. Erk Simple unsupervised grammar induction from raw text with cascaded finite state models Proceedings of 49th ACL Meeting Portland Oregon USA pages 1077–1086 2011.

  • [5] D. Yarowsky and G. Ngai Inducing multilingual PoS taggers and np bracketers via robust projection across aligned corpora Proceedings of NAACL-2001 Conference Pittsburgh PA USA pages 200-207 2-7 June 2001.

  • [6] L. Zhu D.F. Wong and L.S. Chao Unsupervised chunking based on graph propagation from bilingual corpus The Scientific World Journal Article ID 401943 2014.

  • [7] S. Goldwater T.L. Griffiths and M. Johnson Contextual dependencies in unsupervised word segmentation Proceedings of 21st International Conference on Computational Linguistics and 44th ACL Meeting Sydney Australia pages 673–680 2006.

  • [8] D. Mochihashi T. Yamada and N. Ueda Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling Proceedings of 47th ACL Meeting Suntec Singapore pages 100–108 August 2009.

  • [9] T. Nguyen S. Vogel and N.A. Smith Nonparametric word segmentation for machine translation Proceedings of COLING-2010 Beijing China pages 815–823 August 2010.

  • [10] G. Tambouratzis M. Vassiliou & S. Sofianopoulos Machine Translation with Minimal Reliance on Parallel Resources Springer Briefs in Statistics series Springer-Verlag Berlin 2017.

  • [11] E.F. Tjong K. Sang and H. Dejean Introduction to the CONLL-2001 shared task: Clause identification Proceedings of CoNLL-2001 Toulouse France pages 53–57 6-7 July 2001.

  • [12] J. Lafferty A. McCallum and F.C.N. Pereira Conditional random fields: Probabilistic models for segmenting and labelling sequence data. ICML Conference Proceedings Williamstown USA pages 282–289 28 June-1 July 2001.

  • [13] F. Sha and F.C.N. Pereira Shallow parsing with conditional random fields Proceedings of HLTNAACL Conference Alberta Canada pages 213–220 27 May – 1 Jun 2003.

  • [14] Y. Tsuruoka J. Tsujii and S. Ananiadou Fast full parsing by linear-chain conditional random fields 12th EACL Conference Proceedings Athens Greece pages 790–798 30 March-3 April 2009.

  • [15] R.O. Duda P. E. Hart and D.G. Stork Pattern Classification (2nd edition) Wiley Interscience New York U.S.A. 2001.

  • [16] G. Tambouratzis Conditional Random Fields versus template-matching in MT phrasing tasks involving sparse training data Pattern Recognition Letters 53:44-52 2015.

  • [17] A.J. Booker J.E. Dennis Jr. P.D. Frank D.B. Serafini V. Torczon and M.W. Trosset A Rigorous Framework for Optimization of Expensive Functions by Surrogates Structural Optimisation 17:1-13 1999.

  • [18] Y. Jin Surrogate-assisted evolutionary computation: Recent advances and future challenges Swarm and Evolutionary Computation 1(2):61-70 2011.

  • [19] K. Stoeber P. Wagner J. Helbig S. Koester D. Stall M. Thomae J. Blauert W. Hess and H. Mangold Speech synthesis by multilevel selection and concatenation of units from large speech corpora. Verbmobil: Foundations of Speech-to-Speech Translation Symbolic Computation Springer Berlin pages 519–537 2000.

  • [20] J. Kennedy and R. Eberhart Particle swarm optimization IEEE International Conference on Neural Networks Perth WA USA pages 1942–1948 27 Nov-1 Dec 1995.

  • [21] G. Tambouratzis Applying PSO to Natural Language Processing Tasks: Optimizing the Identifi-cation of Syntactic Phrases Proceedings of CEC-2016 Conference Vancouver Canada pp. 1831-1838 July 2016.

  • [22] R. Mendes J. Kennedy and J. Neves The fully informed particle swarm: Simpler maybe better IEEE Transactions on Evolutionary Computation 8(3):204–210 2004.

  • [23] Z.-H. Zhan J. Zhang Y. Li and H.S.-H. Chung Adaptive particle swarm optimization IEEE Transactions on Systems Man Cybernetics Part B: Cybernetics 39(6):1362–1381 2009.

  • [24] G. Tambouratzis (2017) The effectiveness of surrogate functions in improving the accuracy of PSO-type algorithms in an NLP task Proceedings of 10th SSCI Conference Honolulu USA pp. 3214-3221 IEEE Press 27 November - 1 December 2017.

  • [25] K. Papineni S. Roukos T. Ward and W.J. Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. 40th ACL Meeting Proceedings Philadelphia USA pages 311–318.

  • [26] NIST. Automatic evaluation of machine translation quality using n-gram cooccurrences statistics. Report available at http://www.itl.nist.gov/iad/mig/tests/mt/doc/ngramstudy.pdf 2002.

  • [27] K.R. Harrison. A.P. Engelbrecht B.M. Ombuki-Berman An adaptive particle swarm optimization algorithm based on optimal parameter regions Proceedings of 10th SSCI Conference Honolulu USA pp. 1606-1613 IEEE Press 27 November - 1 December 2017.

Journal information
Impact Factor

CiteScore 2018: 4.70

SCImago Journal Rank (SJR) 2018: 0.351
Source Normalized Impact per Paper (SNIP) 2018: 4.066

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 277 277 24
PDF Downloads 157 157 13