Nextflow logo


Pipeline orchestration

Nextflow enables scalable and reproducible scientific workflows using software containers.

Use it when

  • You want a workflow management system specializing in managing computational data analysis for fields such as Bioinformatics, Imaging, Physics, and Chemistry.
  • You need a framework that supports domain-specific languages (DSL).
  • You need workflows that are highly portable and are computing platform agnostic.
  • You need workflows that are reproducible in different computing platforms.
  • You want to support containerization.
  • You want to use Kubernetes.
  • You want to execute tasks in parallel efficiently.
  • You want continuous checkpoints to allow workflows to resume from the last executed step.

Watch out

  • Nextflow is geared towards highly specific scientific computational workflows. It can be adapted for more general ML workflows.
  • Bash may complicate using Nextflow for general ML workflows.

Example stacks

Airflow + MLflow stack


curl -s | bash
chmod +x nextflow
sudo install nextflow /usr/local/bin/