Global workflows are immutable workflows that can be versioned and published to make them available to other users. They reside in a global namespace, shared with apps.

They are implemented as wrappers of regular, project-based workflow objects so any workflow can be converted to a global workflow with a unique name and version.

Conceptually, and on the API level, global workflows are similar to apps. Both are executables that:

• are global objects that are not stored in any particuar project but can be accessed by users from the DNAnexus tools library

• have a unique name in the global namespace (this namespace includes both apps and global workflows)

• can have multiple versions. Together with the name, versions allow users to uniquely identify the executable and access its provenance through the immutable version history

• can be accessed by users or orgs listed in the authorized users list and the developers list

• can be updated only by users or orgs listed in the developers list

• can be labelled with a category (e.g. Structural Variation, RNA-seq, etc.) and searched by that category.

A resource container is associated with each version of a global workflow, a separate one is used for each of the enabled regions. The resource container contains the underlying (wrapped) workflow and all the dependencies necessary to run the global workflow.

When a global workflow is launched, an analysis object is created, like for any workflow on the DNAnexus platform.

Name and version

The name, together with a version, uniquely identifies a global workflow. Once someone claims a name (for an app or another global workflow), it cannot be changed even if the global workflow has been deleted.

When a developer creates a global workflow with a name and version, the DNAnexus platform will generate a unique ID of the form "globalworkflow-xxxx". The workflow can be uniquely referred to by that ID or by the alias "globalworkflow-<name>/<version>>", e.g. "globalworkflow-micromap/0.0.1".

One, and only one, version can be labelled with an additional "default" alias. It is assigned to the first version automatically and can be shifted to any other version. It allows the workflow to be referred to by its name only, e.g. "globalworkflow-micromap", then the version associated with the "default" flag is invoked.

Before a particular version of the workflow is published, it is not visible to other users on the platform except to the developers of the same global workflow.

Publishing

Once the developer calls the /globalworkflow-xxxx/publish route, the version of the workflow is made available to the users that are specified as the workflow's authorized users. Before publishing only the developers can access it.

The title, summary, and documentation can still be updated after the workflow is published but all the other metadata and functionality will remain immutable. It is not possible to "unpublish" a global workflow version.

Developers

Similar to app developers, global workflow developers is a set of users or orgs that can update any version of the workflow, create new versions of the workflow with a particular name, and release (publish) workflow versions to authorized users. They can also update the list of developers and authorized users of the workflow. Developers have access to the workflow versions that are unpublished (under development).

When a user first creates a global workflow with a particular name and version, that user is initially the sole developer.

One difference between apps and workflows is that any developer of a global workflow can update, publish, or delete it, not only the creator of the version.

As in the case of apps, the list of developers can be updated at any time and the list is applied to all (past and future) versions of the workflow.

Authorized users

A list of users or organizations that can view and run the published versions of the global workflow. If the workflow is unpublished, only developers can access it.

In order to run and view the workflow, the users must also be the authorized users of all the apps of the workflow and the apps must be published.

As in the case of apps, developers can add and remove the members of the list at any time and the user list is applied to all (past and future) versions of the workflow.

Categories

As with apps, developers can add categories to a global workflows. See App Categories section for details on app categories that also apply to global workflows.

Updating workflow versions

Unlike apps, global workflows cannot be updated in place (i.e. without changing the version and hash ID) even when they are unpublished. A version string must be changed when the developer wants to make changes to the workflow. The only exception is the metdata fields unrelated to the function of the workflow, such as title, summary, and documentation. They can be changed at any time for both published and unpublished versions of the workflow.

Deleting workflow versions

Global workflows can be marked as deleted using the /globalworkflow-xxxx[/yyyy]/delete route. After a global workflow version is deleted, it can still be described (the response of which will indicate that it has been deleted), but it can no longer be run. In order to help maintain provenance, even if all the versions of the workflow are deleted the name of the workflow remains claimed and cannot be re-used.

Workflow resource container

Just like apps, each global workflow has access to its resource container, which is used to store objects needed by the workflow. One resources container is maintained in each region in which the workflow may be run. These containers are created at global workflow creation time and are persistent as long as the workflow object exists.

When running an analysis in a certain region, the resource container of the corresponding workflow in that region is read-only to the jobs of the analysis, and it is not accessible to the user executing the workflow.

The resource containers belong to the developer. For billing purposes, data in the container is the responsibility of the user or organization specified as billTo for the global workflow.

Multi-region global workflows

A global workflow can be enabled in multiple regions. In each of the enabled regions there exists a container storing an underlying workflow that represents a region-specific executable of the global workflow. The set of enabled regions may be specified by the developer at the time the version is created, and the set is fixed thereafter for this version.

When a global workflow is executed, the region of the project context decides which region the workflow will be run in.

If the workflow contains any apps, all the apps must be enabled in the regions in which the workflow will be executed.

Global Workflow API Method Specifications

API method: /globalworkflow/new

Specification

Creates a global workflow.

A read-only container for the global workflow's resources is created and initialized as specified by the resources option. In addition, the workflow object defined under regionalOptions and the applets (together with their assets) that are defined in the workflow stages are copied into it. Developers of the global workflow have VIEW access to the resources container. After the workflow is published, the authorized users will also have VIEW access to this container.

If the workflow is being created for the first time, then the billTo will default to the billTo of the requesting user (this is the same default billing account for new projects created by the user) unless otherwise specified. This same billing account must be used for all subsequent versions created under the same global workflow name.

The fields name, version, regionalOptions (pointing to least one workflow ID in one region) are required. The set of enabled regions of a particular version will be the keys of the regionalOptions mapping. The set of enabled regions is fixed once the version is created.

Inputs

• regionalOptions mapping Configurations for all regions (e.g., "aws:us-east-1", "azure:westus") in which this global workflow will be enabled

  • key Name of a region in which this workflow may be run

  • value mapping Configuration of this workflow in the corresponding region, with the following key-value pairs:

    • workflow string ID of the underlying workflow of this global workflow in the corresponding region. It is a regular workflow, stored in any project as a data-object. The I/O specifications of all specified underlying workflow must be equal

• name string Name for the global workflow, e.g. "micromap". Must match /^[a-zA-Z0-9._-]+$/ and cannot start with the prefix "app-" or "globalworkflow-". See Name and version.

• version string A nonempty freeform string representing the global workflow’s version; must be unique for the given global name (string provided must match /^[a-zA-Z0-9._+-]+$/). See Name and version.

• title string (optional) Title for the global workflow, e.g. "Micro Map". If no title (or an empty string) is found, then name is used as the title

• summary string (optional) A one-line description for the global workflow

• description string (optional) A longer description of the workflow, formatted in Markdown, to be displayed on the workflow's detail page)

• developerNotes string (optional) More advanced documentation for the global workflow

• billTo string (optional, default is billTo of the requesting user issuing the API call) ID of the entity to which all resource/storage costs associated with this global workflow will be billed; must be the ID of the requesting user or an org of which the requesting user is a member with allowBillableActivities permission. If this is a subsequent version of an existing global workflow, then billTo must be the same as billTo of the existing global workflow

• nonce string (optional) Unique identifier (idempotency token) for this request. Ensures that even if multiple requests fail and are retried, only a single global workflow is created. For more information, see Nonces.

Outputs

• id string ID of this workflow (of the form "globalworkflow-<hash id>").

• resourcesByRegion mapping Mapping of the resources containers created for this workflow.

  • key Region name (e.g., "aws:us-east-1", "azure:westus").

  • value string ID of the resources container of this global workflow in the corresponding region.

Errors

• InvalidInput

  • If regionalOptions is specified, then it must be non-empty.

  • If workflow is specified for one region in regionalOptions, then it must be specified for all regions.

  • The workflow denoted by regionalOptions[r].workflow (where r is a region name) must reside in the region denoted by r

  • version is already in use by an existing version of this global workflows, or is a tag for a published version of this workflow.

  • A nonce was reused in a request but some of the other inputs had changed signifying a new and different request

  • A nonce may not exceed 128 bytes

• PermissionDenied

  • VIEW access is required to the underlying workflow and to all required objects.

  • If the name provided is in use, then the user must also be listed as a developer for the name

  • A full scope token or one with developer access is required

  • billTo must be either the ID of the requesting user or the ID of an org of which the requesting user is a member with allowBillableActivities permission

  • The billTo of the invoking user is an org and the user is no longer a member of that org

  • The region of the workflow that this global workflow was created from must be one of the permittedRegions of this global workflows's billTo

• SpendingLimitExceeded

  • billTo has reached its spending limit

API method: /globalworkflow-xxxx[/yyyy]/update

Specification

Updates the global workflow version with any fields that are specified below. Any developer of the global workflow can update it and unpublished as well as published versions can be updated.

Inputs

• title string (optional) The global workflow’s title, e.g. "Micro Map"

• summary string (optional) The global workflow’s summary

• description string (optional) The global workflow’s description

• developerNotes string (optional) Advanced documentation for the global workflow

Outputs

• id string ID of the global workflow (of the form "globalworkflow-<hash id>")

Errors

• InvalidInput

  • Invalid type values for any of the input fields

• ResourceNotFound

  • The specified global workflow does not exist

• PermissionDenied

  • User must be the developer global workflow

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/addTags

Specification

Adds the specified tags to this version of the global workflow. If any of the tags are already present, no action is taken for those tags. If any of the tags are already in use by different versions of this workflow, those tags are removed from the other versions and added to this version. This maintains the invariant that any tag only applies to a single version of a global workflow.

Example: you have two published versions of a global workflow, globalworkflow-foo/1.0.0 and globalworkflow-foo/2.0.0, and version 1.0.0 is the current default. You can make version 2.0.0 the default by calling /globalworkflow-foo/2.0.0/addTags with input {"tags":["default"]}

Inputs

• tags array of strings List of tags to add to the specified global workflow

  version

Outputs

• id string ID of the workflow (of the form "globalworkflow-<hash id>")

Errors

• ResourceNotFound (the specified global workflow does not exist)

• InvalidInput (input is not a hash, tags is not present or is not an array of nonempty strings, a tag is equal to an existing version, whether published or not)

• PermissionDenied

  • User must be a developer of the global workflow

  • A full scope token or one with developer access is required

• InvalidState (the global workflow object has already been deleted)

API method: /globalworkflow-xxxx[/yyyy]/removeTags

Specification

Remove the requested aliases of "globalworkflow-<name>/<tag>"

Inputs

• tags array of strings List of tags to remove from all versions of the global workflow

Outputs

• name string Global workflow name modified (e.g. "bwa")

Errors

• ResourceNotFound (the specified global workflow name does not exist)

• InvalidInput (input is not a hash, tags is not present or is not an array of nonempty strings, a tag is equal to an existing version, or the tag includes the special tag "default")

• PermissionDenied

  • User must be a developer of the global workflow

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/publish

Specification

Releases the version of the global workflow so that it is now discoverable and runnable by authorized users on the platform and can be accessed by object ID, "globalworkflow-/", and "globalworkflow-/" if the global workflow object also has tags. Any developer of the global workflow can publish the version.

Inputs

• makeDefault boolean (optional, default false) Whether this global workflow should take on the "default" tag, removing it from whichever workflow version was previously marked as "default"

Outputs

• id string ID of the workflow (of the form "globalworkflow-<hash id>")

Errors

• PermissionDenied

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/describe

Specification

Describe the global workflow.

Inputs

• fields mapping (optional) Restrict the output of this method to have only the provided keys in this field

  • key Output field; see the "Outputs" section below for valid values here

  • value boolean or mapping A boolean value indicates whether to include the output field.

Outputs

• id string ID of the global workflow (of the form "globalworkflow-<hash id>")

• class string The value "globalworkflow"

• billTo string ID of the user/organization that owns this global workflow's name (and is billed for the storage incurred by the workflow versions)

• name string Name of the workflow

• title string Freeform name

• version string Version of the workflow

• aliases array of strings List that includes the version and all tags of this workflow by which it can be addressed using "globalworkflow-/"

• summary string A one-line description of the workflow

• description string A long description of the workflow, formatted in Markdown, to be displayed on the workflow's detail page

• categories array of strings List of categories the workflow belongs to

• dxapi string The version of the API used

• createdBy string ID of the user who created the workflow

• created timestamp When the workflow was created

• modified timestamp When this workflow was last modified

• deleted boolean Whether the workflow has been deleted

• isDeveloperFor boolean Whether the requesting user has developer permissions for the workflow AND the token provided has full scope or has developer access

• authorizedUsers array of strings List of user and org IDs representing who can describe and run this workflow; if the string "PUBLIC" is present, then all users can access this workflow

• regionalOptions mapping Mapping that contains the regional configurations for this workflow

  • key Region name (e.g., "aws:us-east-1", "azure:westus"). Note that the existence of a region here indicates that this workflow may be run in this region, and that absence of a region indicates otherwise.

  • value mapping Mapping of the configuration of this workflow in the corresponding region, with the following key-value pairs:

    • workflow string ID of the underlying workflow of this global workflow in the corresponding region.

    • resources string ID of the resources container of this global workflow in the corresponding region. It contains the underlying workflow and any other dependencies (files, applets).

If the workflow is published:

• published timestamp Time at which this workflow was published

Errors

• InvalidInput (input is not a hash, or fields if present, is not a hash or has a non-boolean key)

• ResourceNotFound (the specified workflow does not exist)

• PermissionDenied

  • The requesting user must be represented in the access list or must be one of the workflow's developers; if the latter, a full scope token or one with developer access is required

  • If the workflow is unpublished, the requesting user must be one of the workflow's developers, and a full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/run

Specification

Run the specified global workflow.

Inputs

• name name (optional, default is the global workflow's title if set and otherwise its name) Name for the resulting analysis

• input mapping Input that the global workflow is launched with

  • key Input field name (as it appears in the input specification of any of the underlying workflows)

  • value Input field value

• project string (required when called by a user; cannot be provided when called by a job) ID of the project in which the global workflow will be run, i.e. the project context; any output objects will be placed into this project. If this method is called by a job (and therefore this field is not present), the project context of the new job is set to that of the job making the request, but the output objects will be placed in the temporary workspace of the job. See The Project Context and Temporary Workspace for more details

• folder string (optional) The folder into which objects output by the analysis will be placed. If the folder does not exist when the job(s) complete, it (and any parent folders necessary) will be created. See the Customizing Output Folders for more details on how it interacts with stages' output folders

• stageFolders mapping (optional) Override any stored options for the workflow stages' folder fields. See the Customizing Output Folders section for more details

  • key Stage ID or "*" to indicate that the value should be applied to all stages not otherwise mentioned

  • value null or string Value to replace the stored default

• details array or mapping (optional, default { }) Any conformant JSON (i.e. a JSON object or array, per RFC4627), which will be stored with the created job

• nonce string (optional) Unique identifier for this request. Ensures that even if multiple requests fail and are retried, only a single analysis is created. For more information, see Nonces.

• tags array of strings (optional) Tags to associate with the resulting analysis

• properties mapping (optional) Properties to associate with the resulting analysis

  • key Property name

  • value string Property value

• details mapping or array (optional, default { }) JSON object or array that is to be associated with the analysis

Outputs

• id string ID of the created analysis object (i.e. a string in the form “analysis-xxxx”).

• stages array of strings List of job IDs that will be created for each stage, as ordered in the workflow

Errors

• ResourceNotFound (the specified workflow object, any referenced workflows, apps or applets, default input files, or project context does not exist)

• PermissionDenied

  • The requesting user must be represented in the access list or must be one of the global workflow's developers; if the latter, a full scope token or one with developer access is required

  • If the workflow is unpublished, the requesting user must be one of the global workflow's developers, and either (1) a full scope token or one with developer access is required or (2) the token belongs to a job that is already running the same global workflow

  • CONTRIBUTE access to the project context is required unless called by a job

  • The requesting user must be represented in the access list of all apps or must be the apps' developer; if the latter, a full scope token or one with developer access is required

  • When specifying allowSSH or debug options, the user must have developer access to all apps in the workflow, or the apps must have the openSource field set to true

• InvalidInput

  • The workflow spec is not complete

  • The workflow must be enabled in the region of the project context

  • All data object inputs must be in the same region as the project context

  • All inputs that are job-based object references must refer to a job that was run in the same region as the project context.

  • A nonce was reused in a request but some of the other inputs had changed signifying a new and different request

  • A nonce may not exceed 128 bytes

• InvalidState

  • Some input to the workflow is not in the "closed" state

  • The project context's region is not among the enabled regions of the workflow

API method: /globalworkflow-xxxx[/yyyy]/delete

Specification

Mark a global workflow version as deleted. A deleted version can no longer be run, modified, nor published. This state is reflected in the output of describe. The associated resources container is made inaccessible.

Inputs

• None

Outputs

• id string ID of the workflow (of the form "globalworkflow-<hash id>")

Errors

• PermissionDenied

  • Requesting user must be the developer of the workflow

  • A full scope token or one with developer access is required

• InvalidState

  • If the workflow has multiple versions, all other versions must be deleted before a workflow with the "default" can be deleted

API method: /globalworkflow-xxxx[/yyyy]/addDevelopers

Specification

Adds developers (users and/or orgs) to the global workflow.

If the requesting user is not a developer of the global workflow, but he or she is the billTo of the workflow or the billTo of the workflow is an org and the user is an ADMIN of that org, then the user may invoke this method to grant themselves, and only themselves, developer access to the workflow.

Inputs

• developers array of strings List of user and/or org IDs that will be added as developers

Outputs

• name string The global workflow name (e.g. "bwa")

Errors

• ResourceNotFound

  • Specified global workflow does not exist

  • A specified user does not exist

• InvalidInput

  • If the requesting user is not a developer of the workflow and is or administers the billTo of the workflow, then developers may only contain the ID of the requesting user

• PermissionDenied

  • The requesting user must be a developer of the workflow, or he or she must be or administer the billTo of the workflow

  • If the requesting user is a developer of the workflow, then the user must have either a full-scope token or a token with developer access

  • If the requesting user is not a developer of the workflow but is or administers the billTo of the workflow, then the user must have a full scope token

API method: /globalworkflow-xxxx[/yyyy]/listDevelopers

Specification

List the developers of the global workflow.

Inputs

• None

Outputs

• developers array of strings List of user and/or org IDs that

  are developers

Errors

• ResourceNotFound (the specified workflow does not exist)

API method: /globalworkflow-xxxx[/yyyy]/removeDevelopers

Specification

Remove a list of developers from the global workflow. Attempting to remove a user or org who is not listed as a developer will not throw an error. If the billing account of the global workflow is a user ID, then the developer access of that user may not be revoked.

Inputs

• developers array of strings list of user and/or org IDs that will be removed as developers

Outputs

• name string the global workflow name (e.g. "bwa")

Errors

• ResourceNotFound

  • The specified workflow does not exist

• InvalidInput

  • The billing account of the workflow is a user ID and developers contains that user ID

  • PermissionDenied

  • User issuing the API call must be a developer of the workflow

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/addAuthorizedUsers

Specification

Add users and/or org IDs to the access list. The access list applies to all global workflow versions.

Inputs

• authorizedUsers array of strings List of user and/or org IDs to add to the access list

Outputs

• name string The global workflow name (e.g. "bwa")

Errors

• ResourceNotFound (a specified global workflow or user/org does not exist)

• InvalidInput (input is not a hash, authorizedUsers is missing or is not an array of nonempty strings)

• PermissionDenied

  • User must be a developer of the global workflow

  • A full scope token or one with developer access is required

  • "PUBLIC" cannot be added as an authorized user. If you are interested in making an app or global workflow available to all users of the DNAnexus Platform, please contact us at support@dnanexus.com.

API method: /globalworkflow-xxxx[/yyyy]/listAuthorizedUsers

Specification

List the users and/or orgs IDs on the access list for the workflow. The string "PUBLIC" represents the set of all users on DNAnexus Platform. The access list applies to all global workflow versions.

Inputs

• None

Outputs

• authorizedUsers array of strings List of user and/or org IDs that can describe and run the workflow; if "PUBLIC" is present, this means all users can access the workflow.

Errors

• ResourceNotFound (the specified global workflow does not exist)

• PermissionDenied

  • The requesting user must be represented in the access list or must be one of the workflow's developers; if the latter, a full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/removeAuthorizedUsers

Specification

Remove a list of user and/or org IDs from the access list for the global workflow. Removing an entity that is not currently listed in the access list does not throw an error.

Inputs

• authorizedUsers array of strings List of user and/or org IDs to remove from the access list

Outputs

• name string The global workflow name (e.g. "bwa")

Errors

• ResourceNotFound (a specified global workflow does not exist)

• InvalidInput (input is not a hash, authorizedUsers is missing or is not an array of nonempty strings)

• PermissionDenied

  • User must be a developer of the global workflow

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/addCategories

Specification

Add categories for the global workflow.

Inputs

• categories array of strings List of categories to add the

  workflow to

Outputs

• name string The global workflow name (e.g. "bwa")

Errors

• ResourceNotFound (a specified global workflow does not exist)

• InvalidInput (input is not a hash, categories is missing or is not an array of nonempty strings, a listed category is not allowed: "new", "recommended", or "popular")

• PermissionDenied

  • User must be a developer of the global workflow

  • A full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/listCategories

Specification

List the categories that a global workflow is a member of. All authorized users can list the categories of a published workflow. Categories apply to all the versions of the global workflow.

Inputs

• None

Outputs

• categories array of strings List of categories the global workflow belongs to

Errors

• ResourceNotFound (the specified global workflow does not exist)

• PermissionDenied

  • The requesting user must be represented in the access list or must be one of the workflow's developers; if the latter, a full scope token or one with developer access is required

  • If the workflow is unpublished, the requesting user must be one of the workflow's developers, and a full scope token or one with developer access is required

API method: /globalworkflow-xxxx[/yyyy]/removeCategories

Specification

Remove the categories from a global workflow.

Inputs

• categories array of strings List of categories (strings) to remove from the global workflow

Outputs

• name string The global workflow name (e.g. "bwa")

Errors

• ResourceNotFound (a specified global workflow does not exist)

• InvalidInput (input is not a hash, categories is missing or is not an array of nonempty strings)

• PermissionDenied

  • User must be a developer of the workflow

  • A full scope token or one with developer access is required