Skip to content

vLLM Pooling Serve

Use the serve action to deploy pooling models with vLLM on Kubernetes. Pooling models support tasks such as embedding and reranking.

  • Create


    Configure a Function with the vLLM model source URL.

  • Run


    Configure and deploy the vLLM service.

  • Use


    Invoke the model and list its available endpoints.