Croissant
The croissant kind stores an ML Croissant dataset defined by a metadata.json file and its referenced local files. Use this kind to load the dataset through the mlcroissant library, which is available through the digitalhub[mlcroissant] extra in the SDK installation guide.
When logging a Croissant dataitem, the metadata file must be named metadata.json; its referenced local files are uploaded with it. An explicit destination path must be an S3 partition path ending with /. See supported paths.
Create
Log
log_croissant
Create and upload a Croissant dataitem entity.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
project
|
str
|
Project name. |
required |
name
|
str
|
Entity name. |
required |
source
|
str
|
Croissant metadata JSON file path. |
required |
drop_existing
|
bool
|
Whether to drop existing entity with the same name. |
False
|
path
|
str
|
Destination path. If omitted, it is generated. |
None
|
description
|
str
|
Human-readable entity description. |
None
|
labels
|
list[str]
|
Entity labels. |
None
|
version
|
str
|
Entity version. |
None
|
**kwargs
|
dict
|
Additional dataitem specification parameters. |
{}
|
Returns:
| Type | Description |
|---|---|
DataitemCroissant
|
Created Croissant dataitem entity with uploaded files. |
Register
register_croissant
Register a Croissant dataitem entity for an existing source.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
project
|
str
|
Project name. |
required |
source
|
SourcesOrListOfSources
|
Path or URI of the existing Croissant dataitem. |
required |
name
|
str
|
Entity name. If omitted, it is inferred from |
None
|
uuid
|
str
|
Entity identifier. |
None
|
version
|
str
|
Entity version. |
None
|
description
|
str
|
Human-readable entity description. |
None
|
labels
|
list[str]
|
Entity labels. |
None
|
embedded
|
bool
|
Whether to embed the entity specification in the project specification. |
False
|
extensions
|
list[dict]
|
Entity extensions. |
None
|
**kwargs
|
dict
|
Additional dataitem specification parameters. |
{}
|
Returns:
| Type | Description |
|---|---|
DataitemCroissant
|
Registered Croissant dataitem entity. |
New
new_dataitem
Create a new dataitem entity in the backend.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
project
|
str
|
Project name. |
required |
name
|
str
|
Entity name. |
required |
kind
|
str
|
Entity kind. |
required |
uuid
|
str
|
Entity identifier. |
None
|
version
|
str
|
Entity version. |
None
|
description
|
str
|
Human-readable entity description. |
None
|
labels
|
list[str]
|
Entity labels. |
None
|
embedded
|
bool
|
Whether to embed the entity specification in the project specification. |
False
|
path
|
str
|
Entity path on the local file system or remote storage. It is also the destination path for uploads. |
None
|
extensions
|
list[dict]
|
Entity extensions. |
None
|
**kwargs
|
dict
|
Additional entity specification parameters. |
{}
|
Returns:
| Type | Description |
|---|---|
Dataitem
|
Created dataitem entity. |
Methods
The croissant kind adds dataset loading to the common Dataitem methods.