Use it when
- You want to create reproducible ML pipelines for production.
- You want an open-source framework that combines pipeline orchestrator, artifact, and metadata store for production pipelines.
- You want a cloud-agnostic framework that could be expanded with other tools.
- You want to migrate workflows from on-prem to cloud, keeping the pipelines and steps intact.
- You want an orchestrator that is lightweight.
- The scalability of the created pipelines is limited to which backend tools have been used.
- Currently does not support DAG or Steps based declaration of workflows.
Airflow + MLflow stack
pip install zenml