Use it when
- You want an opinionated pipeline orchestration toolbox that is focused on ML-specific workloads on Kubernetes.
- You want a tool that is cloud provider agnostic.
- You want a framework that integrates all components to cover each phase of the ML lifecycle.
- You want to run Jupyter Notebooks on GPU instances with shared data backends.
- You want to autoscale compute resources to your workload needs.
- You want to deploy ML models to production.
Watch out
- Extensive configuration options require significant expertise and experimentation to get the optimal configuration.
- Reliability issues may arise from component dependencies and their version incompatibilities. Updating one component might break other parts due to incompatibilities.
- Kubeflow expects that containers are in cloud container registries.
Example stacks
Airflow + MLflow stack