Serve a TVM Model
Call function.run(action="serve", ...) with the service settings. The Run uses
function.so_model unless you set model_path explicitly.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| action | str | ✓ | Action to execute. Must be serve. |
| volumes | list[dict] | List of volumes. | |
| resources | dict | Resource values with optional cpu, mem, gpu, and disk keys. |
|
| envs | list[dict] | Environment variables. | |
| secrets | list[str] | List of secret names. | |
| profile | str | Scheduling/resource profile; does not select the model target. | |
| model_path | str | Optional store:// reference to a tvm-so Model; otherwise uses function.so_model. |
|
| served_name | str | Model name in /v2/models/<name>; defaults to the Function name. |
|
| image | str | Go image by default; set the Rust image to use Rust for this Run. | |
| replicas | int | Number of pods, zero or greater; default 1. |
|
| workers | int | Parallel model copies per pod, at least one; default 1. |
|
| service_type | str | ClusterIP (default), NodePort or LoadBalancer. |
|
| service_name | str | Extra Kubernetes service name, <function>-<service_name>; distinct from served_name. |
Serving options belong to the Run. Reusable Tasks contain only common infrastructure settings, for example:
The selected Model is pinned in this Run. Compiling another target does not change
a running service; use a versioned model_path to select an earlier build.
profile must select a node compatible with the Model's architecture and CPU.
These parameters configure the service; inference tensors are sent directly to its
REST or gRPC endpoint. Request validation is implemented by the Go/Rust serving
image and does not require new SDK parameters. Go 0.26.0-r3 and Rust 0.26.0-r2
include these checks. Set image to the desired version if the platform default
still uses an older runtime. See the
input validation guide.