Skip to content

OpenInference Serve

Use the serve action to deploy an OpenInference function as an inference service.

What the action does

The inference endpoint uses the model name and tensor schemas declared in the function specification. Requests and responses follow the Open Inference Protocol.

Build requirements first

If the Function has requirements, run the OpenInference Build action before deploying it so the dependencies are installed in its image.

  • Create


    Define the inference handler, model name and tensor schemas.

  • Run


    Deploy the service and configure its replicas.

  • Use


    Invoke the endpoint with a tensor-oriented request payload.