Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar; Omar  Benjelloun; Costanza Conforti; Luca Foschini; Joan Giner Miguelez; Pieter Gijsbers; Sujata Goswami; Nitisha Jain; Michalis Karamousadakis; Michael Kuchnik; Satyapriya Krishna; Sylvain Lesage; Quentin Lhoest; Pierre Marcenac; Manil Maskey; Peter Mattson; Luis Oala; Hamidah Oderinwale; Pierre Ruyssen; Tim Santos; Rajat Shinde; Elena Simperl; Arjun Suresh; Goeffry Thomas; Vyacheslav Tykhonov; Joaquin Vanschoren

Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar, Omar Benjelloun, Costanza Conforti, Luca Foschini, Joan Giner Miguelez, Pieter Gijsbers, Sujata Goswami, Nitisha Jain, Michalis Karamousadakis, Michael Kuchnik, Satyapriya Krishna, Sylvain Lesage, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Hamidah Oderinwale, Pierre Ruyssen, Tim SantosRajat Shinde, Elena Simperl, Arjun Suresh, Goeffry Thomas, Vyacheslav Tykhonov, Joaquin Vanschoren

DANS-KNAW

Research output: Contribution to conference › Poster › Scientific

Abstract

Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

Original language	English
Publication status	Published - 13 Dec 2024
Event	NeurIPS 2024: The Thirty-eighth Annual Conference on Neural Information Processing Systems - Vancouver, Canada Duration: 10 Jan 2025 → 15 Jan 2025 https://neurips.cc

Conference

Conference	NeurIPS 2024
Country/Territory	Canada
City	Vancouver
Period	10/01/2025 → 15/01/2025
Internet address	https://neurips.cc

Keywords

Machine Learning
Data management

Access to Document

https://neurips.cc/virtual/2024/poster/97627Licence: CC0

Cite this

Akthar, M., Benjelloun, O., Conforti, C., Foschini, L., Giner Miguelez, J., Gijsbers, P., Goswami, S., Jain, N., Karamousadakis, M., Kuchnik, M., Krishna, S., Lesage, S., Lhoest, Q., Marcenac, P., Maskey, M., Mattson, P., Oala, L., Oderinwale, H., Ruyssen, P., ... Vanschoren, J. (2024). Croissant: A Metadata Format for ML-Ready Datasets. Poster session presented at NeurIPS 2024, Vancouver, Canada. https://neurips.cc/virtual/2024/poster/97627

@conference{132bfe619be94107828161bedb59fddb,

title = "Croissant: A Metadata Format for ML-Ready Datasets",

abstract = "Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.",

keywords = "Machine Learning, Data management",

author = "Mubashara Akthar and Omar Benjelloun and Costanza Conforti and Luca Foschini and {Giner Miguelez}, Joan and Pieter Gijsbers and Sujata Goswami and Nitisha Jain and Michalis Karamousadakis and Michael Kuchnik and Satyapriya Krishna and Sylvain Lesage and Quentin Lhoest and Pierre Marcenac and Manil Maskey and Peter Mattson and Luis Oala and Hamidah Oderinwale and Pierre Ruyssen and Tim Santos and Rajat Shinde and Elena Simperl and Arjun Suresh and Goeffry Thomas and Vyacheslav Tykhonov and Joaquin Vanschoren",

year = "2024",

month = dec,

day = "13",

language = "English",

note = "NeurIPS 2024 : The Thirty-eighth Annual Conference on Neural Information Processing Systems ; Conference date: 10-01-2025 Through 15-01-2025",

url = "https://neurips.cc",

}

Akthar, M, Benjelloun, O, Conforti, C, Foschini, L, Giner Miguelez, J, Gijsbers, P, Goswami, S, Jain, N, Karamousadakis, M, Kuchnik, M, Krishna, S, Lesage, S, Lhoest, Q, Marcenac, P, Maskey, M, Mattson, P, Oala, L, Oderinwale, H, Ruyssen, P, Santos, T, Shinde, R, Simperl, E, Suresh, A, Thomas, G, Tykhonov, V & Vanschoren, J 2024, 'Croissant: A Metadata Format for ML-Ready Datasets', NeurIPS 2024, Vancouver, Canada, 10/01/2025 - 15/01/2025. <https://neurips.cc/virtual/2024/poster/97627>

TY - CONF

T1 - Croissant: A Metadata Format for ML-Ready Datasets

AU - Akthar, Mubashara

AU - Benjelloun, Omar

AU - Conforti, Costanza

AU - Foschini, Luca

AU - Giner Miguelez, Joan

AU - Gijsbers, Pieter

AU - Goswami, Sujata

AU - Jain, Nitisha

AU - Karamousadakis, Michalis

AU - Kuchnik, Michael

AU - Krishna, Satyapriya

AU - Lesage, Sylvain

AU - Lhoest, Quentin

AU - Marcenac, Pierre

AU - Maskey, Manil

AU - Mattson, Peter

AU - Oala, Luis

AU - Oderinwale, Hamidah

AU - Ruyssen, Pierre

AU - Santos, Tim

AU - Shinde, Rajat

AU - Simperl, Elena

AU - Suresh, Arjun

AU - Thomas, Goeffry

AU - Tykhonov, Vyacheslav

AU - Vanschoren, Joaquin

PY - 2024/12/13

Y1 - 2024/12/13

N2 - Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

AB - Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

KW - Machine Learning

KW - Data management

M3 - Poster

T2 - NeurIPS 2024

Y2 - 10 January 2025 through 15 January 2025

ER -

Croissant: A Metadata Format for ML-Ready Datasets

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this