Skip to content

Use a vLLM Text Run

The service is executed remotely. Call run.refresh() and inspect run.status to check its progress. When ready, the status includes the service details. Use list_endpoints() to inspect the model's available endpoints, then call invoke() to send a request.

run.refresh()
run.status

endpoints = run.list_endpoints()
response = run.invoke(json={"inputs": "Hello"})
invoke

Invoke served model. The method defaults to "POST" if data or json is provided in kwargs, otherwise it defaults to "GET". The function returns a requests.Response object.

Parameters:

Name Type Description Default
method str

Method of the request (e.g., "GET", "POST").

'POST'
url str

URL to invoke. If specified, it must start with the service URL (http:// or https:// prefixes are required and stripped before comparison).

None
**kwargs dict

Keyword arguments to pass to the request.

{}

Returns:

Type Description
Response

Response from the request.

list_endpoints

List all available endpoints for the served model.

Returns:

Type Description
list[str]

List of available endpoints.