Beschrijving
Large language models are increasingly improving in their ability to generate convincing, human-like texts. While replicating human language has beneficial applications, it also allows for misuse, with for instance language models facilitating fraud and the spread of misinformation. For these reasons, detector models have become an active area of research. Although with some success, the accuracy of existing detector models highly depends on the combination of generation model and detector and moreover they give no direct insights into what actually distinguishes artificial from natural texts. Therefore, this thesis aims to construct a more general approach to detecting machine-generated text, and to this end identifies statistical aspects of language in terms of how artificial and natural texts differ. From experiments concerning the frequency distributions within human-written and machine-generated text, three tests were constructed to establish whether a corpus is human-written or machine-generates: the first two involve performing the Mann-Whitney test on the frequency distributions of both a corpus' top-40 words and its top-40 ranks, and the third uses a corpus' optimal Zipf parameters as its classificatoin feature. The three tests achieved accuracies of 99%, 95% and 93% respectively when classifying corpora of 10.000 tokens, therefore proving reliable methods in artificial text detection. For classification of smaller corpus sizes and different language models, future research directions concerning other statistical properties is proposed.| Periode | mrt. 2021 → jul. 2021 |
|---|---|
| Geëxamineerde | Rachel van 't Hull |
| Examen gehouden op |
|