Kestra logo


Pipeline orchestration

Kestra is a highly scalable data pipeline and workflow orchestration platform.

Use it when

  • You want a platform flexible workflow orchestrator deployed either on-prem, Kubernetes cluster, or as a Docker container.
  • You want to define pipelines declaratively as YAML code.
  • You want a data orchestrator that supports ETL and ELT pipelines.
  • You want support for parallel and branched tasks in a pipeline.
  • You want event-based pipeline scheduling triggered by API, a timed schedule, detection, and events.
  • You want to track pipeline metrics.
  • You want to use Terraform to manage cloud resources.
  • You want a developer-friendly UI.

Watch out

  • Production workflows require a Kubernetes cluster.

Example stacks

Airflow + MLflow stack