The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech

Phat Do; Matt Coler; Jelske Dijkstra; Esther Klabbers

doi:10.21437/interspeech.2023-2148

The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech

Phat Do, Matt Coler, Jelske Dijkstra, Esther Klabbers

Fryske Akademy (FA)

Onderzoeksoutput: Bijdrage aan conferentie › Paper › Wetenschappelijk › peer review

Samenvatting

We compare phone labels and articulatory features as input for cross-lingual transfer learning in text-to-speech (TTS) for low-resource languages (LRLs). Experiments with FastSpeech 2 and the LRL West Frisian show that using articulatory features outperformed using phone labels in both intelligibility and naturalness. For LRLs without pronunciation dictionaries, we propose two novel approaches: a) using a massively multilingual model to convert grapheme-to-phone (G2P) in both training and synthesizing, and b) using a universal phone recognizer to create a makeshift dictionary. Results show that the G2P approach performs largely on par with using a ground-truth dictionary and the phone recognition approach, while performing generally worse, remains a viable option for LRLs less suitable for the G2P approach. Within each approach, using articulatory features as input outperforms using phone labels.

Originele taal-2	Engels
Pagina's	5461-5465
DOI's	https://doi.org/10.21437/interspeech.2023-2148
Status	Gepubliceerd - 20 aug. 2023
Evenement	Interspeech 2023 - Convention Centre, Dublin, Ierland Duur: 20 aug. 2023 → 24 aug. 2023 https://interspeech2023.org

Conferentie

Conferentie	Interspeech 2023
Land/Regio	Ierland
Stad	Dublin
Periode	20/08/2023 → 24/08/2023
Internet adres	https://interspeech2023.org

Toegang tot document

10.21437/interspeech.2023-2148Licentie: CC BY-NC-SA

Citeer dit

@conference{eadbfc01ea094a798d2654866ede6be5,

title = "The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech",

abstract = "We compare phone labels and articulatory features as input for cross-lingual transfer learning in text-to-speech (TTS) for low-resource languages (LRLs). Experiments with FastSpeech 2 and the LRL West Frisian show that using articulatory features outperformed using phone labels in both intelligibility and naturalness. For LRLs without pronunciation dictionaries, we propose two novel approaches: a) using a massively multilingual model to convert grapheme-to-phone (G2P) in both training and synthesizing, and b) using a universal phone recognizer to create a makeshift dictionary. Results show that the G2P approach performs largely on par with using a ground-truth dictionary and the phone recognition approach, while performing generally worse, remains a viable option for LRLs less suitable for the G2P approach. Within each approach, using articulatory features as input outperforms using phone labels.",

author = "Phat Do and Matt Coler and Jelske Dijkstra and Esther Klabbers",

year = "2023",

month = aug,

day = "20",

doi = "10.21437/interspeech.2023-2148",

language = "English",

pages = "5461--5465",

note = "Interspeech 2023 ; Conference date: 20-08-2023 Through 24-08-2023",

url = "https://interspeech2023.org",

}

TY - CONF

T1 - The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech

AU - Do, Phat

AU - Coler, Matt

AU - Dijkstra, Jelske

AU - Klabbers, Esther

PY - 2023/8/20

Y1 - 2023/8/20

N2 - We compare phone labels and articulatory features as input for cross-lingual transfer learning in text-to-speech (TTS) for low-resource languages (LRLs). Experiments with FastSpeech 2 and the LRL West Frisian show that using articulatory features outperformed using phone labels in both intelligibility and naturalness. For LRLs without pronunciation dictionaries, we propose two novel approaches: a) using a massively multilingual model to convert grapheme-to-phone (G2P) in both training and synthesizing, and b) using a universal phone recognizer to create a makeshift dictionary. Results show that the G2P approach performs largely on par with using a ground-truth dictionary and the phone recognition approach, while performing generally worse, remains a viable option for LRLs less suitable for the G2P approach. Within each approach, using articulatory features as input outperforms using phone labels.

AB - We compare phone labels and articulatory features as input for cross-lingual transfer learning in text-to-speech (TTS) for low-resource languages (LRLs). Experiments with FastSpeech 2 and the LRL West Frisian show that using articulatory features outperformed using phone labels in both intelligibility and naturalness. For LRLs without pronunciation dictionaries, we propose two novel approaches: a) using a massively multilingual model to convert grapheme-to-phone (G2P) in both training and synthesizing, and b) using a universal phone recognizer to create a makeshift dictionary. Results show that the G2P approach performs largely on par with using a ground-truth dictionary and the phone recognition approach, while performing generally worse, remains a viable option for LRLs less suitable for the G2P approach. Within each approach, using articulatory features as input outperforms using phone labels.

U2 - 10.21437/interspeech.2023-2148

DO - 10.21437/interspeech.2023-2148

M3 - Paper

SP - 5461

EP - 5465

T2 - Interspeech 2023

Y2 - 20 August 2023 through 24 August 2023

ER -

The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech

Samenvatting

Conferentie

Toegang tot document

Vingerafdruk

Citeer dit