dx-toolkit
, if you want to automate workflow creation or lock down your workflow. In this tutorial we will show you how to do that step by step from your local workstation.stages
field of our JSON file holds a list of executables for the workflow. We'll add two stages to our workflow: the first one will run the app BWA-MEM FASTQ Read Mapper and the second one will run Freebayes Variant Caller. We'll also specify a name and an output folder where our results will be saved. Our dxworkflow.json
will look as follows:sorted_bams
andsorted_bam
, in the dxworkflow.json
file we are using. The sorted_bams
input field for the Freebayes app is bound to the sorted_bam
field of the BWA step.stages
list should have an id
, which is a free-form string unique in a given workflow, and anexecutable
field, which holds either the ID/name of an app, or an ID of an applet that we want to run in that stage.input
field for a stage if we want to bind the input of that stage with an output/input of a different stage. For example, the file array input sorted_bams
of our second stage, call_variants
, will receive values from the output field sorted_bam
of the first stage, align_reads
:dx describe
command in the command line.input
section of a stage to set default values for a field. We select the file hs37d5.bwa-index.tar.gz
(file-B6ZY4942J35xX095VZyQBk0v
), which is publicly available in the reference project "Apps Data: AWS US (East)" (project-BQpp3Y804Y0xbyG4GJPQ01xv
) on the DNAnexus platform, to be the default reference file for the alignment step, align_reads
.dxworkflow.json
file in the newly created directory. inputs
and outputs
to the workflow during creation, with links to inputs and outputs of specific stages. When the workflow is run, the user will be able to pass values only to the fields defined in inputs
, and all the parameters that are not visible in this workflow-level I/O interface will be unchangeable/non-overridable.align_reads
, reads_fastqgzs
. When locking workflows we always define those inputs that are not locked, by listing them in the workflow-level inputs
field. All the other inputs will be automatically locked and users will not be able to override their values when running this workflow.inputs
field, which may look like this:reads
.workflowInputField
, as in the example below. If a file is supplied to reads
when the workflow is run, it will be directed to reads_fastqgzs
of the stage align_reads
.genomeindex_targz
and genome_fastagz
are not put into the workflow-level input field, indicating that these fields are locked. Since the user cannot input values for locked fields, we have to set the value for these fields using each individual stage'sinput
field as above (not the workflow-level inputs
), and the workflow will only be run with the values file-B6ZY4942J35xX095VZyQBk0v
and file-B6ZY7VG2J35Vfvpkj8y0KZ01
respectively.dxworkflow.json
. In the example, our stages have the following required inputs:align_reads
stage has the inputs reads_fastqgzs
and genomeindex_targz
call_variants
stage has the inputs sorted_bams
and genome_fastagz
reads_fastqgzs
is created as a workflow-level input in inputs
(so it is not locked, and the user will set the input) while the remaining inputs are locked. The values used in the locked input fields must be set by the creator of the workflow, and for our example, they have been set according to the code snippet seen above.inputs
specification is null or not specified at all, the workflow can accept inputs provided directly to the workflow stages by the user.outputs
:outputSource
allows us to configure which stage-level outputs will be the outputs of the workflow. This, together with inputs
, is especially useful when we want to set a workflow as an executable within another workflow.dxworkflow.json
workflow description will look as follows:dxworkflow.json
):reads
input field:-ialign_reads.reads_fastqgzs=my_input_file.fastq.gz
, is not possible for locked workflows.dx get "BWA MEM + Freebayes Exome Workflow"
, add inputs
to the downloaded dxworkflow.json
, set workflowInputField
references from stages to these inputs
as explained above, and run dx build
again.