Developing Portable Pipelines

Workflow Description Language (WDL) is a portable, open standard for specifying data processing workflows. WDL makes it straightforward to define complex analysis tasks, chain them together in workflows, and parallelize their execution. Common Workflow Language (CWL) is a portable, open standard for describing how to run command line tools and how to connect them in order to run workflows. CWL makes it straightforward to scale complex data analysis workflows from a single workstation to larger cloud or high performance computing environments.

The recorded webinars below show how to use WDL and Docker to develop locally, then migrate your pipelines to the DNAnexus platform. The material includes

• An overview of the DNAnexus platform, and an introduction to building workflows on the platform

• An introduction to Docker, Docker best practices and ways to use Docker images on the platform

• An introduction to writing workflows in WDL and running them on the platform. Note that the dxWDL tool used in the webinar to compile WDL workflows for execution on the platform has been replaced by the dxCompiler tool.