Exploring lexical and syntactic features for language variety identification

Chris van der Lee, Antal van den Bosch

Onderzoeksoutput: Hoofdstuk in boek/boekdeelBijdrage aan conferentie proceedingsWetenschappelijkpeer review

76 Downloads (Pure)

Samenvatting

We present a method to discriminate between texts written in either the Netherlandic
or the Flemish variant of the Dutch language. The method draws on a feature
bundle representing text statistics, syntactic features, and word n-grams. Text
statistics include average word length and sentence length, while syntactic features
include ratios of function words and part-of-speech n-grams. The effectiveness of
the classifier was measured by classifying Dutch subtitles developed for either
Dutch or Flemish television. Several machine learning algorithms were compared
as well as feature combination methods in order to find the optimal generalization
performance. A machine-learning meta classifier based on AdaBoost attained the
best F-score of 0.92.
Originele taal-2Engels
TitelProceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)
Pagina's190-199
Aantal pagina's10
StatusGepubliceerd - 03 apr 2017

Vingerafdruk Duik in de onderzoeksthema's van 'Exploring lexical and syntactic features for language variety identification'. Samen vormen ze een unieke vingerafdruk.

  • Citeer dit

    van der Lee, C., & van den Bosch, A. (2017). Exploring lexical and syntactic features for language variety identification. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial) (blz. 190-199)