# Versioning and Publishing Global Workflows

## Workflows and Global Workflows

[Workflows](https://documentation.dnanexus.com/developer/workflows) created on the DNAnexus Platform with an ID in the form of `workflow-xxxx` are data objects stored in a project.

Because they are stored in a project, you can share the workflow with other users by adding them to the project. These local workflows are great for fast iterations for development and testing. You can always delete the workflow and create a new one in its place.

A global workflow is an executable that can be versioned and published to other users. It is implemented as a wrapper around an existing project-based workflow. For users and organizations collaborating on multiple private or public projects, these local workflows may be less suitable for long-term maintenance and collaboration.

For example, an organization administrator or workflow developer may want to restrict execution of a particular workflow to specific users and organizations, or they may want to allow execution of this workflow across cloud regions and providers. [Global Workflows](https://documentation.dnanexus.com/developer/api/running-analyses/global-workflows) can be used for those use cases and more.

## Global Workflow Use Cases

Global workflows enable version management for tracking different iterations of a workflow's source code, providing an explicit provenance history of changes. Each version is immutable, allowing users to revert to previous states while ensuring data integrity and preventing unintended modifications to existing workflow versions.

They make collaborative development across projects and organizations easier by allowing workflows to be shared with authorized users who have execution privileges but cannot modify the core workflow definition. Global workflows also support deployment and execution across multiple geographic regions and cloud providers, and enable discovery of shared workflows within the DNAnexus Tools Library.

## Step 1: Build a Local Workflow

The easiest way to build a workflow is by using the web interface. In your project's **Manage** tab, click **Add** > **New Workflow**. For details, see [building-and-running-workflows](https://documentation.dnanexus.com/developer/workflows/building-and-running-workflows "mention").

You can also use the [DNAnexus SDK](https://documentation.dnanexus.com/downloads), `dx-toolkit` .

## Step 2: Make the Workflow Global

### Download the Original Workflow

For a local workflow you'd like to convert to a global workflow, use the [`dx get`](https://documentation.dnanexus.com/user/helpstrings-of-sdk-command-line-utilities#get) command to download a JSON representation (`dxworkflow.json`) of the local workflow:

```shell
dx get "Exome Analysis Demo:Exome Analysis Workflow"
```

This creates a local directory "Exome Analysis Workflow" with the `dxworkflow.json` file in it.

### Build a Global Workflow

Adjust the downloaded `dxworkflow.json` before you create a new global workflow.

* Global workflows require the `name` and `version` fields.
  * The `name` has to be unique in the global namespace shared by apps and global workflows.
  * The `name` can use lowercase letters, numbers, `-`, `.`, and `_`, and must not contain spaces.
* You can add fields such as `title` and `summary` to make the workflow more user-friendly.
* You can leave the `stages` and any other fields unchanged.

{% code title="dxworkflow\.json" %}

```json
{
  "name": "my_global_workflow",
  "title": "Exome Analysis Workflow",
  "version":"1.0.0",
  "stages": [...]
}
```

{% endcode %}

{% hint style="success" %}
It's a good practice to include documentation in the `Readme.md` file in the same [directory](https://documentation.dnanexus.com/developer/workflow-build-process#workflow-directory-structure) as the workflow.
{% endhint %}

If the workflow uses globally available apps or applets from non-[PHI projects](https://documentation.dnanexus.com/getting-started/key-concepts/projects#phi-data-protection), you can use `dx build` to create your global workflow. Workflows using resources from PHI-enabled projects cannot be built as global workflows due to the project's protective restrictions.

```shell
# Create your global workflow
dx build --globalworkflow "Exome Analysis Workflow"
```

Because `Exome Analysis Workflow` is the name of the directory storing the `dxworkflow.json`, it can be changed freely.

### Describe and Run the Workflow

The `dx build` command returns the unique ID of the created version, such as `globalworkflow-xxxx` .

You can use the ID to refer to the global workflow.

```shell
dx describe globalworkflow-xxxx
```

Running a global workflow is the same as running any other workflow.

```shell
dx run my_global_workflow \
  -i0.reads_fastqgzs="Exome Analysis Demo:/Input/SRR504516_1.fastq.gz"
```

## Step 3: Add Authorized Users or Orgs

You can specify a list of users with whom you want to share the workflow by using [`dx add users`](https://documentation.dnanexus.com/user/helpstrings-of-sdk-command-line-utilities#add-users).

The users can find and access the global workflow only *after* the workflow is published. The list of users can be updated by developers before or after it is published, and it applies to all existing and future versions of the workflows.

```shell
# Share the workflow with a user AND with an organization
dx add users my_global_workflow user-bob org-partnerorg

# View who's on the workflow's access list
dx list users my_global_workflow

# Remove a user from the access list
dx remove users my_global_workflow org-partnerorg
```

The authorized user permissions do not propagate to apps automatically. If the workflow contains any apps, the users need access to those apps to run the workflow. To add users to an app, you can use the same `dx add users` command.

## Step 4: Publish a Version to Users

Once you've tested the workflow version, you can release it to the authorized users using `dx publish` .

```shell
dx publish my_global_workflow/0.0.1
```

Publishing a workflow version has the following effects:

* The authorized users can discover the version via `dx find globalworkflow` .
* The authorized users can describe and run the version.
* The authorized users can download the workflow's `dxworkflow.json` source code with `dx get` (as well as the dependencies, such as applets), though they cannot build new versions with the same global workflow name.

Developers can add and remove users and other developers at any time. Adding users to the workflow gives the users access to all the published versions but it has no effect on the unpublished versions. Unpublished versions are only accessible to the people listed as [developers](#add-developers) of the workflow.

### The "Default" Alias

Executing [`dx publish`](https://documentation.dnanexus.com/user/helpstrings-of-sdk-command-line-utilities#publish) command makes the published version a default one by adding a "default" alias to this version (and removing it from whichever workflow version was previously marked as "default").

The "default" alias indicates that this version is invoked whenever the workflow name is used without specifying a version. For example, users can run the workflow:

```shell
dx run my_global_workflow   # equivalent to "dx run my_global_workflow/0.0.1"
```

The first created version of a global workflow gets "default" flag automatically.

## Next Steps

### Add Developers

You can enable multiple users to update the current version or build new versions of the global workflow.

```shell
# Add a user to the workflow's developer list
dx add developers my_global_workflow user-eve
```

Either specific users or whole orgs can be developers of a global workflow.

```shell
# See who's on the workflow's developer list
dx list developers my_global_workflow

# Remove a user from the developer list
dx remove developers my_global_workflow user-bob
```

### Create a New Version

You can create a new version of the workflow by updating the `dxworkflow.json` and using `dx build --globalworkflow`. To create the new version, you need to explicitly update the `version` string, for example, to "0.0.2".

If you lose the source `dxworkflow.json`, you can download it at any time using `dx get` .

```shell
# Download dxworkflow.json for a specific version of a global workflow
dx get globalworkflow-my_global_workflow/0.0.2
```

Any developer of the workflow can download its source code and build new versions. Authorized users that have access to published versions can only download or run it.

### Delete

You can mark a global workflow version as *deleted*, which makes the version non-runnable. It is still possible to describe the workflow for provenance purposes.

```shell
dx api globalworkflow-xxxx delete
```

{% hint style="danger" %}
**Use with caution**

* Deleting a global executable can break users' reproducibility requirements.
* Deleting all workflow versions does not release the workflow name. It is impossible to reuse the name for a different global workflow by another user.
  {% endhint %}

## Discovering Global Workflows

### Searching Available Versions

You can use the command [`dx find globalworkflows`](https://documentation.dnanexus.com/user/helpstrings-of-sdk-command-line-utilities#find-globalworkflows) to browse the global workflows available to you.

```shell
# List the current default versions of published global workflows
dx find globalworkflows

# List the complete version history of published global workflows
dx find globalworkflows --all

# List the global workflows that are not published
dx find globalworkflows --unpublished --all
```

### Searching by Category

You can add the `--category` parameter to restrict the search to a specific category.

Common categories are available as tab completions. For example:

```shell
dx find globalworkflows --category Variation\ Calling
```

To view all available categories you can search by:

```shell
dx find globalworkflows --category-help
```

## Summary

The following table summarizes the main steps and stages required to create and publish the workflow.

| Object name                 | ID prefix        | Namespace | Access                       | Definition and purpose                                                                                                                                                                               |
| --------------------------- | ---------------- | --------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Workflow                    | `workflow`       | Project   | based on project permissions | A file-like workflow object stored in a project. Used for private, light-weight development                                                                                                          |
| Global workflow unpublished | `globalworkflow` | Global    | developers                   | A development version of a workflow that is not visible to users but is to developers. Used for development and testing in multiple regions, release management, preparation for publishing to users |
| Global workflow published   | `globalworkflow` | Global    | users, developers            | A version of a workflow that is available to authorized users                                                                                                                                        |
| Global workflow deleted     | `globalworkflow` | Global    | users, developers            | A version (previously published or unpublished) that is not runnable but can be described                                                                                                            |