Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Phat Do; Matt Coler; Jelske Dijkstra; Esther Klabbers

doi:10.21437/interspeech.2023-2158

Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Phat Do, Matt Coler, Jelske Dijkstra, Esther Klabbers

Fryske Akademy (FA)

Research output: Contribution to conference › Poster › Scientific

Abstract

We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

Original language	English
Pages	5466-5470
DOIs	https://doi.org/10.21437/interspeech.2023-2158
Publication status	Published - 20 Aug 2023
Event	Interspeech 2023 - Convention Centre, Dublin, Ireland Duration: 20 Aug 2023 → 24 Aug 2023 https://interspeech2023.org

Conference

Conference	Interspeech 2023
Country/Territory	Ireland
City	Dublin
Period	20/08/2023 → 24/08/2023
Internet address	https://interspeech2023.org

Access to Document

10.21437/interspeech.2023-2158Licence: CC BY-NC-SA

Cite this

@conference{4abefb3625884547b831092c38e1d989,

title = "Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages",

abstract = "We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.",

author = "Phat Do and Matt Coler and Jelske Dijkstra and Esther Klabbers",

year = "2023",

month = aug,

day = "20",

doi = "10.21437/interspeech.2023-2158",

language = "English",

pages = "5466--5470",

note = "Interspeech 2023 ; Conference date: 20-08-2023 Through 24-08-2023",

url = "https://interspeech2023.org",

}

TY - CONF

T1 - Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

AU - Do, Phat

AU - Coler, Matt

AU - Dijkstra, Jelske

AU - Klabbers, Esther

PY - 2023/8/20

Y1 - 2023/8/20

N2 - We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

AB - We train a MOS prediction model based on wav2vec 2.0 using the open-access data sets BVCC and SOMOS. Our test with neural TTS data in the low-resource language (LRL) West Frisian shows that pre-training on BVCC before fine-tuning on SOMOS leads to the best accuracy for both fine-tuned and zero-shot prediction. Further fine-tuning experiments show that using more than 30 percent of the total data does not lead to significant improvements. In addition, fine-tuning with data from a single listener shows promising system-level accuracy, supporting the viability of one-participant pilot tests. These findings can all assist the resource-conscious development of TTS for LRLs by progressing towards better zero-shot MOS prediction and informing the design of listening tests, especially in early-stage evaluation.

U2 - 10.21437/interspeech.2023-2158

DO - 10.21437/interspeech.2023-2158

M3 - Poster

SP - 5466

EP - 5470

T2 - Interspeech 2023

Y2 - 20 August 2023 through 24 August 2023

ER -

Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages

Abstract

Conference

Access to Document

Fingerprint

Cite this