Skip to content

OpenInference runtime

OpenInference functions are Python handlers for inference services. Their function specification declares a model name and the tensor schemas accepted by the endpoint. See the Python handler definition for general handler setup and the OpenInference handler definition for protocol-specific behavior.

Prerequisites

Actions

  • Serve

    Deploy an OpenInference function as an inference service.

  • Build

    Build a container image for an OpenInference function.

Examples

  • Examples

    Explore examples for OpenInference builds and inference services.