Skip to content

Run a vLLM Pooling Function

Call function.run(action="serve", ...) to deploy the vLLM pooling model. Configure the service with the following parameters.

Parameters

Name Type Required Description
action str ✓ Action to execute. Must be serve
volumes list[dict] List of volumes.
resources dict Resource limits and requests.
envs list[dict] Environment variables.
secrets list[str] List of secret names.
profile str Profile template.
replicas int Number of replicas.
service_type str Service type.
service_name str Service name.
url str URL of the vLLM service.
args list[str] Arguments for the vLLM server.
enable_telemetry bool Enable or disable telemetry.
use_cpu_image bool Use a CPU image for serving.
storage_space str Storage space for model artifacts.

Example

run = function.run(
    action="serve",
    args=["--pooling", "last"]
)