(Minor) co-author on the paper here, just wanted to second your experience in the field.
The big win I've found with Nextflow is that once you've written a workflow, you have a lot of flexibility in the execution environment: Have all the tools already installed on your workstation or large compute instance? Use the local executor to saturate the box with concurrently running jobs. Don't have or want all those tools installed? Use the local executor with Docker images. Have access to a traditional compute cluster (e.g. LSF, SGE, Torque, etc.)? Use the cluster executor with Docker images.
That sounds fantastic. I no longer work in bioinformatics but regularly keep in touch with some of my old colleagues. Definitely going to speak in person about this with them.
The big win I've found with Nextflow is that once you've written a workflow, you have a lot of flexibility in the execution environment: Have all the tools already installed on your workstation or large compute instance? Use the local executor to saturate the box with concurrently running jobs. Don't have or want all those tools installed? Use the local executor with Docker images. Have access to a traditional compute cluster (e.g. LSF, SGE, Torque, etc.)? Use the cluster executor with Docker images.
A couple other resources worth checking out:
Toil workflow engine https://github.com/BD2KGenomics/toil
Common Workflow Language (CWL) specification https://github.com/common-workflow-language/common-workflow-...