OpenInference runtime
OpenInference functions are Python handlers for inference services. Their function specification declares a model name and the tensor schemas accepted by the endpoint. See the Python handler definition for general handler setup and the OpenInference handler definition for protocol-specific behavior.
Prerequisites
- Python >= 3.10, < 3.15.
- Install
digitalhub-runtime-pythonusing the installation instructions.
Actions
Examples
-
Explore examples for OpenInference builds and inference services.