TY - ADVS
T1 - ucto
T2 - a rule-based tokeniser
A2 - van Gompel, Maarten
A2 - van der Sloot, Ko
PY - 2020/10/8
Y1 - 2020/10/8
N2 - Ucto tokenizes text files: it separates words from punctuation, and splitssentences. This is one of the first tasks for almost any Natural LanguageProcessing application. Ucto offers several other basic preprocessing stepssuch as changing case that you can all use to make your text suited for furtherprocessing such as indexing, part-of-speech tagging, or machine translation.Ucto comes with tokenisation rules for several languages (packaged separately)and can be easily extended to suit other languages. It has been incorporatedfor tokenizing Dutch text in Frog (https://languagemachines.github.io/frog),our Dutch morpho-syntactic processor.The software is intended to be used from the command-line by researchers inNatural Language Processing or related areas, as well as software developers.
AB - Ucto tokenizes text files: it separates words from punctuation, and splitssentences. This is one of the first tasks for almost any Natural LanguageProcessing application. Ucto offers several other basic preprocessing stepssuch as changing case that you can all use to make your text suited for furtherprocessing such as indexing, part-of-speech tagging, or machine translation.Ucto comes with tokenisation rules for several languages (packaged separately)and can be easily extended to suit other languages. It has been incorporatedfor tokenizing Dutch text in Frog (https://languagemachines.github.io/frog),our Dutch morpho-syntactic processor.The software is intended to be used from the command-line by researchers inNatural Language Processing or related areas, as well as software developers.
U2 - 10.5281/zenodo.594091
DO - 10.5281/zenodo.594091
M3 - Software
ER -