Do speech registers differ in the predictability of words?

TY - JOUR

T1 - Do speech registers differ in the predictability of words?

AU - Bentum, Martijn

AU - ten Bosch, Louis

AU - van den Bosch, A.

AU - Ernestus, Mirjam

PY - 2019/2

Y1 - 2019/2

N2 - Previous research has demonstrated that language use can vary depending on the context of situation. The present paper extends this finding by comparing word predictability differences between 14 speech registers ranging from highly informal conversations to read-aloud books. We trained 14 statistical language models to compute register-specific word predictability and trained a register classifier on the perplexity score vector of the language models. The classifier distinguishes perfectly between samples from all speech registers and this result generalizes to unseen materials. We show that differences in vocabulary and sentence length cannot explain the speech register classifier’s performance. The combined results show that speech registers differ in word predictability.

AB - Previous research has demonstrated that language use can vary depending on the context of situation. The present paper extends this finding by comparing word predictability differences between 14 speech registers ranging from highly informal conversations to read-aloud books. We trained 14 statistical language models to compute register-specific word predictability and trained a register classifier on the perplexity score vector of the language models. The classifier distinguishes perfectly between samples from all speech registers and this result generalizes to unseen materials. We show that differences in vocabulary and sentence length cannot explain the speech register classifier’s performance. The combined results show that speech registers differ in word predictability.

KW - word predictability, speech registers, statistical language modeling, text classification, register analysis

M3 - Article

SN - 1384-6655

VL - 24

JO - International Journal of Corpus Linguistics

JF - International Journal of Corpus Linguistics

IS - 1

ER -