Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar; Omar  Benjelloun; Costanza Conforti; Joan Giner Miguelez; Nitisha Jain; Michael Kuchnik; Quentin  Lhoest; Pierre Marcenac; Manil Maskey; Peter Mattson; Luis Oala; Pierre Ruyssen; Rajat Shinde; Elena  Simperl; Goeffry Thomas; Vyacheslav Tykhonov; Joaquin Vanschoren; Steffen  Vogler; Carole-Jean Wu

Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akthar, Omar Benjelloun, Costanza Conforti, Joan Giner Miguelez, Nitisha Jain, Michael Kuchnik, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Pierre Ruyssen, Rajat Shinde, Elena Simperl, Goeffry Thomas, Vyacheslav Tykhonov, Joaquin Vanschoren, Steffen Vogler, Carole-Jean Wu

DANS-KNAW

Research output: Working paper/discussion paper › Preprint

Abstract

Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

Original language	English
Publisher	Arxiv.org
Publication status	Published - 28 Mar 2024

Keywords

machine learning
artificial intelligence

Access to Document

https://arxiv.org/abs/2403.19546

Cite this

@techreport{fdacbb0addcf412eb7d16971115b78ad,

title = "Croissant: A Metadata Format for ML-Ready Datasets",

abstract = "Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.",

keywords = "machine learning, artificial intelligence",

author = "Mubashara Akthar and Omar Benjelloun and Costanza Conforti and {Giner Miguelez}, Joan and Nitisha Jain and Michael Kuchnik and Quentin Lhoest and Pierre Marcenac and Manil Maskey and Peter Mattson and Luis Oala and Pierre Ruyssen and Rajat Shinde and Elena Simperl and Goeffry Thomas and Vyacheslav Tykhonov and Joaquin Vanschoren and Steffen Vogler and Carole-Jean Wu",

note = "2403.19546v1",

year = "2024",

month = mar,

day = "28",

language = "English",

publisher = "Arxiv.org",

type = "WorkingPaper",

institution = "Arxiv.org",

}

TY - UNPB

T1 - Croissant: A Metadata Format for ML-Ready Datasets

AU - Akthar, Mubashara

AU - Benjelloun, Omar

AU - Conforti, Costanza

AU - Giner Miguelez, Joan

AU - Jain, Nitisha

AU - Kuchnik, Michael

AU - Lhoest, Quentin

AU - Marcenac, Pierre

AU - Maskey, Manil

AU - Mattson, Peter

AU - Oala, Luis

AU - Ruyssen, Pierre

AU - Shinde, Rajat

AU - Simperl, Elena

AU - Thomas, Goeffry

AU - Tykhonov, Vyacheslav

AU - Vanschoren, Joaquin

AU - Vogler, Steffen

AU - Wu, Carole-Jean

N1 - 2403.19546v1

PY - 2024/3/28

Y1 - 2024/3/28

N2 - Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

AB - Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

KW - machine learning

KW - artificial intelligence

M3 - Preprint

BT - Croissant: A Metadata Format for ML-Ready Datasets

PB - Arxiv.org

ER -

Croissant: A Metadata Format for ML-Ready Datasets

Abstract

Keywords

Access to Document

Fingerprint

Cite this