Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Phat Do; Matt Coler; Jelske Dijkstra; Esther Klabbers

doi:10.21437/interspeech.2023-2158

Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Phat Do, Matt Coler, Jelske Dijkstra, Esther Klabbers

Fryske Akademy (FA)

Onderzoeksoutput: Bijdrage aan conferentie › Poster › Wetenschappelijk

Samenvatting

We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

Originele taal-2	Engels
Pagina's	5466-5470
DOI's	https://doi.org/10.21437/interspeech.2023-2158
Status	Gepubliceerd - 20 aug. 2023
Evenement	Interspeech 2023 - Convention Centre, Dublin, Ierland Duur: 20 aug. 2023 → 24 aug. 2023 https://interspeech2023.org

Conferentie

Conferentie	Interspeech 2023
Land/Regio	Ierland
Stad	Dublin
Periode	20/08/2023 → 24/08/2023
Internet adres	https://interspeech2023.org

Toegang tot document

10.21437/interspeech.2023-2158Licentie: CC BY-NC-SA

Citeer dit

@conference{4abefb3625884547b831092c38e1d989,

title = "Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages",

abstract = "We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.",

author = "Phat Do and Matt Coler and Jelske Dijkstra and Esther Klabbers",

year = "2023",

month = aug,

day = "20",

doi = "10.21437/interspeech.2023-2158",

language = "English",

pages = "5466--5470",

note = "Interspeech 2023 ; Conference date: 20-08-2023 Through 24-08-2023",

url = "https://interspeech2023.org",

}

TY - CONF

T1 - Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

AU - Do, Phat

AU - Coler, Matt

AU - Dijkstra, Jelske

AU - Klabbers, Esther

PY - 2023/8/20

Y1 - 2023/8/20

N2 - We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

AB - We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

U2 - 10.21437/interspeech.2023-2158

DO - 10.21437/interspeech.2023-2158

M3 - Poster

SP - 5466

EP - 5470

T2 - Interspeech 2023

Y2 - 20 August 2023 through 24 August 2023

ER -

Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Samenvatting

Conferentie

Toegang tot document

Vingerafdruk

Citeer dit