Less is Better: A cognitively inspired unsupervised model for language segmentation

Jinbiao Yang, Stefan Frank, A. van den Bosch

Onderzoeksoutput: Hoofdstuk in boek/boekdeelBijdrage aan conferentie proceedingsWetenschappelijkpeer review


Language users process utterances by segmenting them into many cognitive units, which vary in their sizes and linguistic levels. Although we can do such unitization/segmentation easily, its cognitive mechanism is still not clear. This paper proposes an unsupervised model, Less-is-Better (LiB), to simulate the human cognitive process with respect to language unitization/segmentation. LiB follows the principle of least effort and aims to build a lexicon which minimizes the number of unit tokens (alleviating the effort of analysis) and number of unit types (alleviating the effort of storage) at the same time on any given corpus. LiB’s workflow is inspired by empirical cognitive phenomena. The design makes the mechanism of LiB cognitively plausible and the computational requirement light-weight. The lexicon generated by LiB performs the best among different types of lexicons (e.g. ground-truth words) both from an information-theoretical view and a cognitive view, which suggests that the LiB lexicon may be a plausible proxy of the mental lexicon.
Originele taal-2Engels
TitelProceedings of the Workshop on the Cognitive Aspects of the Lexicon
UitgeverijAssociation for Computational Linguistics (ACL)
StatusGepubliceerd - 01 dec. 2020


Duik in de onderzoeksthema's van 'Less is Better: A cognitively inspired unsupervised model for language segmentation'. Samen vormen ze een unieke vingerafdruk.

Citeer dit