> ## Content Index
> Fetch the complete content index at: https://community.mozilladatacollective.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Fine-Tune a Speech-to-Text Model for Any Language - Including Yours
- URL: https://community.mozilladatacollective.com/fine-tune-a-speech-to-text-model-for-any-language-including-yours/
- Published: 2026-04-15T13:51:28.000Z
- Updated: 2026-04-15T14:01:29.000Z
- Description: A step-by-step developer tutorial from Kostis at Mozilla Data Collective
- Author: Mozilla Data Collective
- Tags: Guide, training data, MDC, Guides

Most speech recognition models were built with English, or a handful of well-resourced languages, in mind. If you speak Khmer, Galician, or any of the hundreds of languages underrepresented in mainstream AI, you've probably hit a wall trying to get accurate transcriptions.

The Mozilla Data Collective (MDC) is working to change that. Our mission is a tech future that is **multilingual, multicultural, and multimodal** \- built on the principle that communities should have genuine data agency and be the primary beneficiaries of sharing their data. The[ speech-to-text-finetune](https://github.com/Mozilla-Data-Collective/speech-to-text-finetune?ref=community.mozilladatacollective.com) blueprint is a practical tool for making that a reality by enabling developers to easily fine-tune Whisper-based or MMS-based STT models.

In this tutorial, we'll walk through how to fine-tune OpenAI's Whisper model on your own language using the Mozilla Data Collective platform's datasets or your own custom audio data. Everything runs locally — even on a laptop — keeping your data private.

You can also [view a video walk-through of this tutorial here](https://community.mozilladatacollective.com/dev-talks/#fine-tuning-a-whisper-model-with-mdc-datasets).

## **What You'll Build**

By the end of this tutorial, you'll have:

- A fine-tuned Whisper model optimised for your target language
- A local transcription app you can test against your own voice
- (Optionally) a model pushed to the Hugging Face Hub for sharing

As a concrete example of what's possible: a [Galician Whisper model fine-tuned with this blueprint](https://www.google.com/url?q=https://github.com/Mozilla-Data-Collective/speech-to-text-finetune?tab%3Dreadme-ov-file%23example-result-on-galician&ref=community.mozilladatacollective.com) produces dramatically cleaner transcriptions than the base `whisper-small` model, correctly handling language-specific phonology that the base model garbles.

---

## **Prerequisites**

- Python 3.10+
- `ffmpeg` installed on your system
- A Mozilla Data Collective API key (free — instructions below)
- A GPU is helpful but not required for smaller models

## **Step 1: Clone the Repo and Install Dependencies**

```Shell
git clone https://github.com/Mozilla-Data-Collective/speech-to-text-finetune
cd speech-to-text-finetune
pip install -e .
```

Then install ffmpeg:

```Shell
# Ubuntu
sudo apt install ffmpeg

# macOS
brew install ffmpeg
```

## **Step 2: Get Your MDC API Key**

The MDC platform hosts Common Voice datasets across a wide range of languages and provides a Python SDK for accessing them. To use it:

1. Create a free account at[ ](https://datacollective.mozillafoundation.org/api-reference?ref=community.mozilladatacollective.com)https://mozilladatacollective.com/
2. Generate an API key from your account dashboard
3. Copy the `.env` template and add your key:

`cp example_data/.env.example src/speech_to_text_finetune/.env`

Then open the .env file and set:

`MDC_API_KEY=your_api_key_here`

**Note:** Values in this `.env` file will override any environment variables with the same name already set on your system.

Join Mozilla Data Collective → 

## **Step 3: Choose Your Dataset**

You have two options for sourcing training data.

### **Option A: Load via the MDC Python SDK (Recommended)**

This is the easiest path. Browse datasets on the[ MDC platform](https://mozilladatacollective.com/datasets?ref=community.mozilladatacollective.com) and find a dataset for your language. Open the dataset page, click "Connect API", and copy either the "Dataset ID" or the "Dataset slug", both can be used interchangeably.

`dataset_id_or_slug="khmer-asr-cultural-dataset-4e33cd05" `

 ↑ this is your dataset\_id

**Important:** You must accept the dataset's terms and conditions on the MDC platform before downloading via the API.

### **Option B: Use a Locally Downloaded Dataset**

Download and extract the dataset zip from the MDC platform, then point your config at the local directory. This is useful if you're working offline or want to pre-process the data yourself.

### **Option C: Bring Your Own Data**

You can also record and label your own custom dataset using the built-in UI:

`python src/speech_to_text_finetune/make_custom_dataset_app.py`

The app walks you through recording audio clips and adding transcriptions. Your dataset should produce a `.csv`, `.tsv`, or `.parquet` file with `audio_path` and `transcription` columns.

## **Step 4: Configure Your Fine-tuning Run**

Create or edit a config\_whisper.yaml file. Here's a minimal example using the MDC SDK path:

```Python
model_id: openai/whisper-tiny
dataset_id: cminc35no007no707hql26lzk   # Your MDC dataset ID
language: Khmer
repo_name: whisper-tiny-km-finetuned
download_directory: /path/to/downloads  # Optional path to download the dataset
training_hp:
  push_to_hub: False
  hub_private_repo: True
  num_train_epochs: 10
  per_device_train_batch_size: 16
  learning_rate: 1e-5
```

**Choosing a model size:** Start with whisper-tiny or whisper-small for faster iteration. Move to whisper-medium or whisper-large-v3-turbo once you're happy with the pipeline and have a GPU available.

**Unsupported languages:** If your language wasn't part of Whisper's original training data, don't give up. Use[ Glottolog](https://glottolog.org/?ref=community.mozilladatacollective.com) to find the closest related language that *is* in Whisper's supported list, and set that as your language value. This gives the decoder more appropriate token priors. If no related language appears in Whisper's list, set language: None.

## **Step 5: Fine-tune**

`python src/speech_to_text_finetune/finetune_whisper.py`

The script handles dataset loading, train/test splitting, tokenization, and training. If your dataset already defines official train and test splits (as Common Voice does), those are preserved automatically and `test_size` in your config is ignored.

Training progress will be logged to the console. 

## **Step 6: Test Your Model**

Once training completes, fire up the transcription app to test your fine-tuned model against your own voice or a sample audio file:

`python demo/transcribe_app.py`

Enter the local path to your fine-tuned model (or a Hugging Face model ID if you pushed it), record a sample, and see the transcription.

To compare your fine-tuned model side-by-side against the base model:

`python demo/model_comparison_app.py`

This makes it easy to see exactly where fine-tuning improved accuracy on your language's specific phonology and vocabulary.

## **Step 7 (Optional): Speed Up Inference with Faster-Whisper**

Once you're happy with your model, you can convert it to the `faster-whisper` format for significantly faster inference — useful if you're deploying this in a production app or on lower-powered hardware. Check out the tutorial by Kathy from the MDC community:

[tutorial-whisper-fine-tuning-australian-EO2026](https://github.com/Mozilla-Data-Collective/tutorial-whisper-fine-tuning-australian-EO2026?ref=community.mozilladatacollective.com)

## **Try It Without Installing Anything**

Not ready to set up a local environment? You have some other instant options:

- **GitHub Codespaces:**[ Launch a ready-to-go environment](https://github.com/codespaces/new?hide%5Frepo%5Fselect=true&ref=main&repo=mozilla-ai/speech-to-text-finetune&skip%5Fquickstart=true&machine=standardLinux32gb) with everything pre-installed to run small scale experiments in a limited resources machine.
- **Google Colab:**[ Open the fine-tuning notebook](https://colab.research.google.com/github/mozilla-ai/speech-to-text-finetune/blob/main/demo/notebook.ipynb?ref=community.mozilladatacollective.com) which includes a full MDC flow for Khmer as a worked example (`demo/mdc_khmer.ipynb`) that can be run with free GPU access provided by Google.

## **What's Next?**

- Discover and play with new datasets [https://mozilladatacollective.com/datasets](https://mozilladatacollective.com/datasets?ref=community.mozilladatacollective.com)
- Share your own fine-tuned model and contribute back to the collection
- Join the conversation on[ Discord](https://discord.gg/YuMNeuKStr?ref=community.mozilladatacollective.com) (`#mozilla-data-collective`),[ Reddit](https://reddit.com/r/MozillaDataCollective?ref=community.mozilladatacollective.com), or[ LinkedIn](https://linkedin.com/company/mozilla-data-collective?ref=community.mozilladatacollective.com)

*The `speech-to-text-finetune` blueprint is open source under the Apache 2.0 License. It was created by Mozilla.ai and adapted by the Mozilla Data Collective community.*