Dagster logo


Pipeline orchestration

Dagster is an orchestration platform for developing, producing, and observing data assets.

Use it when

  • You want to create workflows as DAGs (Direct Acyclic Graphs).
  • You need flexibility (manual, scheduled, modify individual jobs according to schedule) when running DAGs.
  • You want to define inputs and output for individual jobs more explicitly to streamline data movement between tasks.
  • You want an easy way to create dynamic workflows.
  • You want to generate pipelines as Python code.
  • You want to automate ML workflows.
  • You want to separately abstract compute and storage.

Example stacks

Airflow + MLflow stack


pip install dagster dagit