Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora

F.B. Karsdorp; Enrique Manjavacas; Lauren Fonteyn

Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora

F.B. Karsdorp, Enrique Manjavacas, Lauren Fonteyn

Onderzoeksoutput: Hoofdstuk in boek/boekdeel › Bijdrage aan conferentie proceedings › Wetenschappelijk › peer review

Samenvatting

The question how we can reliably estimate the lexical diversity of a particular text (collection) has often been asked by linguists and literary scholars alike. This short paper introduces a way of operationalizing functional diversity measurements by means of token-based embeddings, and argues that functional diversity is not only a practically advantageous, but also a theoretically relevant addition to the Computational Humanities Research toolkit. By means of an experiment on the historical ARCHER corpus, we show that lexical diversity at the level of functional groups is less sensitive to orthographic variation, and provides insight into an important and often disregarded dimension of vocabulary diversity
in textual data.

Originele taal-2	Engels
Titel	Proceedings of the Computational Humanities Research Conference 2022
Redacteuren	Folgert Karsdorp, Alie Lassche, Kristoffer Nielbo
Uitgeverij	CEUR Workshop Proceedings
Pagina's	114-126
Volume	3290
Status	Gepubliceerd - nov. 2022

Toegang tot document

https://ceur-ws.org/Vol-3290/short_paper2780.pdfLicentie: CC BY

Citeer dit

Karsdorp, F. B., Manjavacas, E., & Fonteyn, L. (2022). Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora. In F. Karsdorp, A. Lassche, & K. Nielbo (editors), Proceedings of the Computational Humanities Research Conference 2022 (Vol. 3290, blz. 114-126). CEUR Workshop Proceedings. https://ceur-ws.org/Vol-3290/short_paper2780.pdf

@inbook{ef0b51f61e224a5fbe51c192a3b4c024,

title = "Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora",

abstract = "The question how we can reliably estimate the lexical diversity of a particular text (collection) has often been asked by linguists and literary scholars alike. This short paper introduces a way of operationalizing functional diversity measurements by means of token-based embeddings, and argues that functional diversity is not only a practically advantageous, but also a theoretically relevant addition to the Computational Humanities Research toolkit. By means of an experiment on the historical ARCHER corpus, we show that lexical diversity at the level of functional groups is less sensitive to orthographic variation, and provides insight into an important and often disregarded dimension of vocabulary diversityin textual data.",

author = "F.B. Karsdorp and Enrique Manjavacas and Lauren Fonteyn",

year = "2022",

month = nov,

language = "English",

volume = "3290",

pages = "114--126",

editor = "Folgert Karsdorp and Alie Lassche and Kristoffer Nielbo",

booktitle = "Proceedings of the Computational Humanities Research Conference 2022",

publisher = "CEUR Workshop Proceedings",

}

Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora. / Karsdorp, F.B.; Manjavacas, Enrique; Fonteyn, Lauren.
Proceedings of the Computational Humanities Research Conference 2022. redactie / Folgert Karsdorp; Alie Lassche; Kristoffer Nielbo. Vol. 3290 CEUR Workshop Proceedings, 2022. blz. 114-126.

Onderzoeksoutput: Hoofdstuk in boek/boekdeel › Bijdrage aan conferentie proceedings › Wetenschappelijk › peer review

TY - CHAP

T1 - Introducing Functional Diversity

T2 - A Novel Approach to Lexical Diversity in (Historical) Corpora

AU - Karsdorp, F.B.

AU - Manjavacas, Enrique

AU - Fonteyn, Lauren

PY - 2022/11

Y1 - 2022/11

N2 - The question how we can reliably estimate the lexical diversity of a particular text (collection) has often been asked by linguists and literary scholars alike. This short paper introduces a way of operationalizing functional diversity measurements by means of token-based embeddings, and argues that functional diversity is not only a practically advantageous, but also a theoretically relevant addition to the Computational Humanities Research toolkit. By means of an experiment on the historical ARCHER corpus, we show that lexical diversity at the level of functional groups is less sensitive to orthographic variation, and provides insight into an important and often disregarded dimension of vocabulary diversityin textual data.

AB - The question how we can reliably estimate the lexical diversity of a particular text (collection) has often been asked by linguists and literary scholars alike. This short paper introduces a way of operationalizing functional diversity measurements by means of token-based embeddings, and argues that functional diversity is not only a practically advantageous, but also a theoretically relevant addition to the Computational Humanities Research toolkit. By means of an experiment on the historical ARCHER corpus, we show that lexical diversity at the level of functional groups is less sensitive to orthographic variation, and provides insight into an important and often disregarded dimension of vocabulary diversityin textual data.

M3 - Contribution to conference proceedings

VL - 3290

SP - 114

EP - 126

BT - Proceedings of the Computational Humanities Research Conference 2022

A2 - Karsdorp, Folgert

A2 - Lassche, Alie

A2 - Nielbo, Kristoffer

PB - CEUR Workshop Proceedings

ER -

Introducing Functional Diversity: A Novel Approach to Lexical Diversity in (Historical) Corpora

Samenvatting

Toegang tot document

Vingerafdruk

Citeer dit