> ## Content Index > Fetch the complete content index at: https://community.mozilladatacollective.com/llms.txt > Use this file to discover other available public pages before exploring further. # Updates to our Python client: faster, concurrent uploads and much more. - URL: https://community.mozilladatacollective.com/updates-to-our-python-client-faster-concurrent-uploads-and-much-more/ - Published: 2026-10-20T10:48:38.000Z - Updated: 2026-10-20T10:48:38.000Z - Author: Kostis Saitas - Zarkias **Our** [**Python SDK client**](https://github.com/Mozilla-Data-Collective/datacollective-python?ref=community.mozilladatacollective.com)**,** which lets users interact with our platform through their Python environment, just got some exciting improvements! The headline is that dataset uploads through our client are now executed concurrently by default, which **lets datasets land on the platform several times faster**. Alongside that, over the past few months we have been working continuously to support new features and make improvements to deliver a great developer experience. Our Python SDK can now download dataset samples, delete dataset submissions, list and search our dataset catalogue, extract dataset archives automatically, and report errors more accurately. In order to take advantage of the latest features, simply upgrade to the latest version with: ```Bash uv add datacollective@latest # or pip install -U datacollective ``` We recommend taking a look at our [package documentation](https://mozilla-data-collective.github.io/datacollective-python/?ref=community.mozilladatacollective.com) for the full list of features and capabilities. ## What do concurrent uploads mean to me? Before the `0.7.0` update, the SDK split the file into chunks (10 MB by default) and uploaded each part one by one sequentially. Even if you were on a really high speed connection, that stream would rarely fill up, so the SDK wouldn't utilise the full potential of your connection. With the new release, these parts are now uploaded concurrently, which can lead to significant speed ups. You don't need to change anything in your code to benefit from this. Just upgrade to the [latest](https://github.com/Mozilla-Data-Collective/datacollective-python/releases?ref=community.mozilladatacollective.com) version of our package. ### How much faster? These numbers were measured while developing the feature, on the same machine and a 300 Mbps connection for both versions: #### Before (version `0.6.4`) - Sequential: For a 100 MB dataset, the average upload speed was 5.2 MB/s finishing at 21 sec. For a 1 GB dataset, the average upload speed was 5.8 MB/s finishing at 2min 45 sec. #### After (version `0.7.0`) - 4 parallel workers: For the same 100 MB dataset, the average upload speed was 14.9 MB/s finishing at 7 seconds, leading to a **2.6x speedup**. For the 1 GB dataset, the average upload speed was 21.4 MB/s finishing at 44 seconds, leading to a **3.6x speedup.** ### Tuning it to your needs In a bit more technical terms, every upload function of the SDK (`create_submission_with_upload`, `upload_dataset_file` and `upload_sample_file`) now takes a `max_workers` argument, and it defaults to 4: ``` from datacollective import create_submission_with_upload response = create_submission_with_upload( file_path="my_dataset.tar.gz", submission=submission, max_workers=4, # number of parts uploaded at the same time ) ``` You can tune the `max_workers` argument, however, not every use case will benefit equally from a high number of workers. In summary: - **Fast, stable connection, large archive:** you can try raising the number of `max_workers=8`, as well as increasing the chunk size of the parts being uploaded `part_size=50 * 1024 * 1024` (50 MB). - **Slow or flaky connection:** keep parts small so a failure only re-uploads a little, and consider `max_workers=2`. - **Want the old behaviour:** `max_workers=1` uploads parts one at a time, exactly as before. For more details on how this feature works and how to tune it to you needs, check out [this section in our documentation](https://mozilla-data-collective.github.io/datacollective-python/upload/?ref=community.mozilladatacollective.com#tuning-upload-concurrency). ## Other interesting features - Browse and search the catalogue without an API key! `list_datasets` lists or searches our dataset catalogue with free-text queries and filters such as task, locale, license, pricing and whether a sample is available, with paginated results you can iterate over. `list_dataset_filters` returns the exact filter values currently accepted. ```Python from datacollective import list_datasets, Task page = list_datasets( "swahili speech", task=[Task.ASR, Task.TTS], sort="newest", results_per_page=20, ) print(f"{page.total} matches") for dataset in page: print(dataset.id, dataset.name, dataset.locale, dataset.sizeBytes) ``` *Results are* [*pydantic*](https://pydantic.dev/docs/validation/latest/get-started/?ref=community.mozilladatacollective.com) *models (`DatasetList` & `DatasetFilter`) so that you can have typed access to the fields* - Download a dataset's sample: `download_sample` fetches the small excerpt a publisher provides, so you can inspect the format before pulling the full archive. Samples are kept under `samples//` in your download directory. ```Python from datacollective import download_sample sample_path = download_sample("your-dataset-id") ``` - Delete a submission: `delete_submission` permanently removes a submission that has not been published yet, together with its uploaded files. Handy for cleaning up after a test upload. ```Python from datacollective import delete_submission delete_submission(submission_id="your-submission-id") ``` ## Next steps If you publish datasets, the next upload you run will already be faster. If you download datasets, check whether the one you are interested in has a sample and give `download_sample` a try before pulling the full archive. Note that samples are also available for free for compensated datasets, so you can get a glimpse before purchasing. As always, bug reports and feature requests are welcome on the [GitHub repository](https://github.com/Mozilla-Data-Collective/datacollective-python/issues?ref=community.mozilladatacollective.com), and we welcome any type of feedback you might have at our address: [support@mozilladatacollective.com](mailto:support@mozilladatacollective.com). ###