ModelServe runtime
ModelServe deploys machine-learning models as services on Kubernetes using serving kinds for scikit-learn, MLflow, KubeAI and vLLM.
Prerequisites
- Python >= 3.10, < 3.15.
- Install
digitalhub-runtime-modelserveusing the installation instructions.
Actions
-
Serve a scikit-learn model saved in pickle format.
-
Build a serving image for an MLflow model.
-
Deploy an MLflow model as a service.
-
Deploy a text model through KubeAI.
-
Deploy a speech model through KubeAI.
-
Deploy a vLLM text-generation model.
-
Deploy a vLLM speech model.
-
Deploy a vLLM model with pooling support.
Examples
-
Explore complete examples for supported model-serving kinds.