Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar; Omar  Benjelloun; Costanza Conforti; Luca Foschini; Joan Giner Miguelez; Pieter  Gijsbers; Sujata Goswami; Nitisha Jain; Michalis Karamousadakis; Michael Kuchnik; Satyapriya Krishna; Sylvain Lesage; Quentin Lhoest; Pierre Marcenac; Manil Maskey; Peter Mattson; Luis Oala; Hamidah Oderinwale; Pierre Ruyssen; Tim Santos; Rajat Shinde; Elena Simperl; Arjun Suresh; Goeffry Thomas; Vyacheslav Tykhonov; Joaquin Vanschoren

Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar, Omar Benjelloun, Costanza Conforti, Luca Foschini, Joan Giner Miguelez, Pieter Gijsbers , Sujata Goswami, Nitisha Jain, Michalis Karamousadakis, Michael Kuchnik, Satyapriya Krishna, Sylvain Lesage, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Hamidah Oderinwale, Pierre Ruyssen, Tim SantosRajat Shinde, Elena Simperl, Arjun Suresh, Goeffry Thomas, Vyacheslav Tykhonov, Joaquin Vanschoren

DANS-KNAW

Onderzoeksoutput: Bijdrage aan conferentie › Poster › Wetenschappelijk

Samenvatting

Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

Originele taal-2	Engels
Status	Gepubliceerd - 13 dec. 2024

Toegang tot document

https://neurips.cc/virtual/2024/poster/97627Licentie: CC0

Citeer dit

Akthar, M., Benjelloun, O., Conforti, C., Foschini, L., Giner Miguelez, J., Gijsbers , P., Goswami, S., Jain, N., Karamousadakis, M., Kuchnik, M., Krishna, S., Lesage, S., Lhoest, Q., Marcenac, P., Maskey, M., Mattson, P., Oala, L., Oderinwale, H., Ruyssen, P., ... Vanschoren, J. (2024). Croissant: A Metadata Format for ML-Ready Datasets. https://neurips.cc/virtual/2024/poster/97627

@conference{132bfe619be94107828161bedb59fddb,

title = "Croissant: A Metadata Format for ML-Ready Datasets",

abstract = "Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.",

keywords = "Machine Learning, Data management",

author = "Mubashara Akthar and Omar Benjelloun and Costanza Conforti and Luca Foschini and {Giner Miguelez}, Joan and Pieter Gijsbers and Sujata Goswami and Nitisha Jain and Michalis Karamousadakis and Michael Kuchnik and Satyapriya Krishna and Sylvain Lesage and Quentin Lhoest and Pierre Marcenac and Manil Maskey and Peter Mattson and Luis Oala and Hamidah Oderinwale and Pierre Ruyssen and Tim Santos and Rajat Shinde and Elena Simperl and Arjun Suresh and Goeffry Thomas and Vyacheslav Tykhonov and Joaquin Vanschoren",

year = "2024",

month = dec,

day = "13",

language = "English",

}

Akthar, M, Benjelloun, O, Conforti, C, Foschini, L, Giner Miguelez, J, Gijsbers , P, Goswami, S, Jain, N, Karamousadakis, M, Kuchnik, M, Krishna, S, Lesage, S, Lhoest, Q, Marcenac, P, Maskey, M, Mattson, P, Oala, L, Oderinwale, H, Ruyssen, P, Santos, T, Shinde, R, Simperl, E, Suresh, A, Thomas, G, Tykhonov, V & Vanschoren, J 2024, 'Croissant: A Metadata Format for ML-Ready Datasets'. <https://neurips.cc/virtual/2024/poster/97627>

TY - CONF

T1 - Croissant: A Metadata Format for ML-Ready Datasets

AU - Akthar, Mubashara

AU - Benjelloun, Omar

AU - Conforti, Costanza

AU - Foschini, Luca

AU - Giner Miguelez, Joan

AU - Gijsbers , Pieter

AU - Goswami, Sujata

AU - Jain, Nitisha

AU - Karamousadakis, Michalis

AU - Kuchnik, Michael

AU - Krishna, Satyapriya

AU - Lesage, Sylvain

AU - Lhoest, Quentin

AU - Marcenac, Pierre

AU - Maskey, Manil

AU - Mattson, Peter

AU - Oala, Luis

AU - Oderinwale, Hamidah

AU - Ruyssen, Pierre

AU - Santos, Tim

AU - Shinde, Rajat

AU - Simperl, Elena

AU - Suresh, Arjun

AU - Thomas, Goeffry

AU - Tykhonov, Vyacheslav

AU - Vanschoren, Joaquin

PY - 2024/12/13

Y1 - 2024/12/13

N2 - Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

AB - Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, portable, and interoperable, thereby addressing significant challenges in ML data management. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, enabling easy loading into the most commonly-used ML frameworks, regardless of where the data is stored. Our initial evaluation by human raters shows that Croissant metadata is readable, understandable, complete, yet concise.

KW - Machine Learning

KW - Data management

M3 - Poster

ER -