Comment on page
Learn about the Cloud Workstation app, which lets you explore and manipulate data on the DNAnexus Platform, as you would on a local Linux machine.
Cloud Workstation sets up a virtual workstation that lets you access and work with data stored on the DNAnexus Platform, just as you would on a local Linux machine, but without having to download files to your computer.
You can configure this workstation to use any of the wide range of powerful instance types available on the Platform. You can easily save files or analysis results from your workstation session, to the project within which you launched it. You can also create a snapshot of the session, if you want to pick up work at a later date.
Note that to run the Cloud Workstation app from within a project, you must have either CONTRIBUTE or ADMINISTER access to that project.
dx runto launch Cloud Workstation, taking care to add the
$ dx run app-cloud_workstation --ssh
Once the app launches, you'll see a list of optional parameters:
Select an optional parameter to set by its # (^D or <ENTER> to finish):
 Maximum Session Length (suffixes allowed: s, m, h, d, w, M, y) (max_session_length) [default="1h"]
 Files (fids)
 Snapshot (snapshot)
Once you've either set or skipped the optional parameters, you'll be connected to the worker running Cloud Workstation. You'll see the following in your terminal:
Calling app-cloud_workstation with output destination
Job ID: job-xxxx
Waiting for job-xxxx to start......
Resolving job hostname and SSH host key...........................
Checking connectivity to ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com...OK
Connecting to ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
Welcome to DNAnexus!
This is the DNAnexus Execution Environment, running job-xxxx.
Job: Cloud Workstation
Instance type: mem1_ssd1_v2_x8
Project: emiai (project-xxxx)
Running since: Mon Nov 1 17:40:51 UTC 2021
Running for: 0:00:38
The public address of this instance is ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com.
You are running byobu, a terminal session manager.
If you get disconnected from this instance, you can log in again; your work will be saved as long as the job is running.
For more information on byobu, press F1.
The job is running in terminal 1. To switch to it, use the F4 key (fn+F4 on Macs; press F4 again to switch back to this terminal).
Use sudo to run administrative commands.
From this window, you can:
- Use the DNAnexus API with dx
- Monitor processes on the worker with htop
- Install packages with apt-get install or pip3 install
- Use this instance as a general-purpose Linux workstation
OS version: Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-1056-aws x86_64)
To access a file within your virtual workstation, you must download it from a Platform project to the workstation. If you want to save a file from your workstation session, you must upload it to the workstation's parent project.
In order to do either, you must prepare your workstation by running the following two commands:
$ unset DX_WORKSPACE_ID
$ dx cd $DX_PROJECT_CONTEXT_ID:
The first command unsets an environment variable that's set when the workstation is launched. This allows you to navigate from within the workstation to any of the projects to which you have access. The second command invokes
dx cdto change your workstation's working directory, to that of the parent project.
Your virtual workstation is now ready to use.
This includes files stored in the workstation's parent project.
You can download any file from any project to which you have access.
To download a file named
my-file.txtfrom the parent project, use the command:
$ dx download my-file.txt
$ dx download project-BQbJpBj0bvygyQxgQ1800Jkk:/SRR100022/SRR100022_1.filt.fastq.gz
When downloading a file or files from a project other than the parent project, you might want to avoid having to enter the project ID. To do this, start by getting a list of projects to which you have access. Then choose a project:
$ dx select --level=VIEW
Available projects (VIEW or higher):
0) Working Project (CONTRIBUTE)
1) Research Project (VIEW)
2) Production Project (VIEW)
Pick a numbered choice or "m" for more options : 1
Setting current project to: Research Project
Now you can download a file to your virtual workstation, by entering only the filename:
$ dx download my-file-1.txt
Your virtual workstation has network access, so you can download and use any tool you need during your session, just as you would on a Linux workstation.
$ dx upload --path "$DX_PROJECT_CONTEXT_ID:" <FILE>
Use the following commands to perform a test upload:
$ dx ls
$ echo "This is a test file" > file_from_workstation.txt
$ dx upload --path "$DX_PROJECT_CONTEXT_ID:" test_file_from_workstation.txt
$ dx ls
You should see the contents of your project change, with the new file
test_file_from_workstation.txtappearing in the file list, between the first and second invocations of
By default, your virtual workstation will automatically shut down once the maximum session length is reached. If you want to shut it down earlier, use the
dx terminatecommand, taking care to use
$DX_JOB_IDto include the workstation's job ID:
$ dx terminate $DX_JOB_ID
You can also shut down the workstation by finding the job in the UI's Monitor page, and clicking the Terminate button at the right end of the row displaying info on the job.
The contents of your virtual workstation will be destroyed on termination. Before the end of your session, be sure to upload any files you want to save.
By default, your virtual workstation will launch on a
mem1_ssd1_v2_x8instance type, which has 8 cores, 16 GB of memory, and 180 GB of storage. To run the app on a different instance type, use the
dx run, as in:
$ dx run --instance-type mem1_ssd1_v2_x36 --ssh app-cloud_workstation
The Cloud Workstation app is set up to use Ubuntu 20.04.
When connecting to the execution environment, you are using the job's credentials to interact with the DNAnexus API. The job has a limited subset of your Platform user permissions. By default, a job running the Cloud Workstation app has VIEW permissions to all projects to which you have VIEW or greater permissions.
The Cloud Workstation app provides the minimum functionality necessary to support an interactive workstation.
To make your own version with enhanced functionality, you can create a custom applet, based on the Cloud Workstation app. To get the original source code for the the Cloud Workstation app, run
dx get app-cloud_workstation. See Introduction to Building Apps to learn how to build a custom applet that incorporates an existing executable.
Possible customizations include:
- Specifying different inputs
- Prepackaging external utilities for use within the worker
- Changing the instance type of the worker
- Changing access permissions