Argo Workflows logo

Argo Workflows

Pipeline orchestration

Argo Workflows is an open-source container-native workflow engine for orchestrating parallel jobs on Kubernetes.

Use it when

  • You want to visualize pipelines running in production.
  • You want to generate pipelines as YAML code.
  • You want to deploy machine learning models.
  • You want to adopt containers and Kubernetes for developing and deploying distributed applications.
  • You want to create workflows as DAGs (Direct Acyclic Graphs).
  • You want to run each task on a separate Kubernetes pod.
  • You want a workflow orchestrator that natively adds services like, secrets, roles, and volumes to Kubernetes.
  • You want to configure infrastructure as YAML code.
  • You want resilience to container crashes.
  • You want support for both time and event-based workflow execution.
  • You want autoscaling features.
  • You want a workflow orchestrator that easily installs to existing Kubernetes clusters.

Watch out

  • Maintaining the YAML files for large projects is difficult.
  • Requires expertise in Kubernetes to work safely in production workloads.
  • The maintenance of an enterprise setup is complex.

Example stacks

Airflow + MLflow stack


kubectl create ns argo
kubectl apply -n argo -f