Skip to content

TVM SO

The tvm-so kind stores a compiled TVM shared-object model. Kind-specific fields, including target, optimization level, manifest, and tensor signatures, are documented with the creation functions.

The bundle contains model.so and metadata.json. Normally a compile Run creates it. For manual registration, retain the full metadata, including TVM version/commit and target: serving requires it. spec.manifest contains this metadata; it is distinct from tuning/manifest.json.

Create

Log

log_tvm_so

Create and upload a TVM SO model entity.

Parameters:

Name Type Description Default
project str

Project name.

required
source SourcesOrListOfSources

Local compiled bundle containing both model.so and metadata.json.

required
name str

Entity name. If omitted, it is inferred from source.

None
drop_existing bool

Whether to drop an existing entity with the same name.

False
path str

Destination path. If omitted, it is generated.

None
description str

Human-readable entity description.

None
labels list[str]

Entity labels.

None
version str

Entity version.

None
framework str

Model framework.

None
algorithm str

Model algorithm.

None
parameters dict

Model parameters.

None
entry str

Compiled model entry function.

None
inputs list[dict]

Input tensor signatures. Each item can define name (str), dtype (str), shape (list[int]), scale (list[float]), zero_point (list[int]), and quantized_dimension (int).

None
outputs list[dict]

Output tensor signatures, with the same fields as inputs.

None
target str

TVM hardware target used to compile the shared object.

None
opt_level int

TVM optimization level, from 0 to 3.

None
manifest dict

Full parsed metadata.json produced by the compile Run, including TVM version/commit and the effective target. Not tuning/manifest.json.

None
**kwargs dict

Additional model specification parameters.

{}

Returns:

Type Description
ModelTvmSo

Created TVM SO model entity with uploaded files.

Examples:

Log a compiled TVM shared-object model:

import json
from pathlib import Path
import digitalhub as dh

metadata = json.loads(Path("compiled-model/metadata.json").read_text())
model = dh.log_tvm_so(
    project="my-project",
    name="resnet18-tvm",
    source="./compiled-model/",
    target=metadata["target"],
    opt_level=metadata["opt_level"],
    entry=metadata["entry"],
    inputs=metadata["inputs"],
    outputs=metadata["outputs"],
    manifest=metadata,
)

Register

register_tvm_so

Register a TVM SO model entity for an existing source.

Parameters:

Name Type Description Default
project str

Project name.

required
source SourcesOrListOfSources

Directory or URI of the existing compiled bundle, including metadata.json.

required
name str

Entity name. If omitted, it is inferred from source.

None
uuid str

Entity identifier.

None
version str

Entity version.

None
description str

Human-readable entity description.

None
labels list[str]

Entity labels.

None
embedded bool

Whether to embed the entity specification in the project specification.

False
extensions list[dict]

Entity extensions.

None
framework str

Model framework.

None
algorithm str

Model algorithm.

None
parameters dict

Model parameters.

None
entry str

Compiled model entry function.

None
inputs list[dict]

Input tensor signatures. Each item can define name (str), dtype (str), shape (list[int]), scale (list[float]), zero_point (list[int]), and quantized_dimension (int).

None
outputs list[dict]

Output tensor signatures, with the same fields as inputs.

None
target str

TVM hardware target used to compile the shared object.

None
opt_level int

TVM optimization level, from 0 to 3.

None
manifest dict

Full parsed metadata.json produced by the compile Run, including TVM version/commit and the effective target. Not tuning/manifest.json.

None
**kwargs dict

Additional model specification parameters.

{}

Returns:

Type Description
ModelTvmSo

Registered TVM SO model entity.

Examples:

Register an uploaded bundle using a local copy of its metadata:

import json
from pathlib import Path
import digitalhub as dh

metadata = json.loads(Path("compiled-model/metadata.json").read_text())
model = dh.register_tvm_so(
    project="my-project",
    name="registered-resnet18-tvm",
    source="s3://my-bucket/models/resnet18-so/",
    target=metadata["target"],
    opt_level=metadata["opt_level"],
    entry=metadata["entry"],
    inputs=metadata["inputs"],
    outputs=metadata["outputs"],
    manifest=metadata,
)

New

Create a Model reference to an already uploaded bundle; this call does not compile or upload files.

new_model

Create a new model entity in the backend.

Parameters:

Name Type Description Default
project str

Project name.

required
name str

Entity name.

required
kind str

Entity kind.

required
uuid str

Entity identifier.

None
version str

Entity version.

None
description str

Human-readable entity description.

None
labels list[str]

Entity labels.

None
embedded bool

Whether to embed the entity specification in the project specification.

False
path str

Entity path on the local file system or remote storage. It is also the destination path for uploads.

None
extensions list[dict]

Entity extensions.

None
**kwargs dict

Additional entity specification parameters.

{}

Returns:

Type Description
Model

Created model entity.

import json
from pathlib import Path
import digitalhub as dh

metadata = json.loads(Path("compiled-model/metadata.json").read_text())

model = dh.new_model(
    project="my-project",
    name="resnet18-tvm",
    kind="tvm-so",
    path="s3://my-bucket/models/resnet18-so/",
    target=metadata["target"],
    opt_level=metadata["opt_level"],
    entry=metadata["entry"],
    inputs=metadata["inputs"],
    outputs=metadata["outputs"],
    manifest=metadata,
)

Retrieve and manage

Use the shared Model management reference to retrieve, update, or delete a TVM SO Model.

See the TVM runtime, including its compile and serve workflows.