This page is a reference for most useful operations and features in the DNAnexus JupyterLab environment:

• ​Download files from the project to the local execution environment​

• ​Upload data from the session to the project​

• ​Download and upload data to your local machine​

• ​Use the terminal​

• ​Install custom packages in the session environment​

Download files from the project to the local execution environment

bash

You can download input data from a project using dx download in a notebook cell:

%%bash
dx download input_data/reads.fastq

The %%bash keyword converts the whole cell to a magic cell which allows us to run bash code in that cell without exiting the Python kernel. See more examples of magic commands in the IPython documentation. The ! prefix to achieves the same result:

! dx download input_data/reads.fastq

Alternatively, the dx command can be executed from the terminal.

Python

To download data with Python in the notebook, you can use the download_dxfile function:

import dxpy
dxpy.download_dxfile(dxid='file-xxxx',
                     filename='unique_name.txt')

Check dxpy helper functions for details on how to download files and folders.

Upload data from the session to the project

bash

Any files from the execution environment can be uploaded to the project using dx upload:

%%bash
dx upload Readme.ipynb

Python

To upload data using Python in the notebook, you can use the upload_local_file function:

import dxpy
dxpy.upload_local_file('variants.vcf')

Check dxpy helper functions for details on how to upload files and folders.

Download and upload data to your local machine

By selecting a notebook or any other file on your computer and dragging it into the DNAnexus project file browser, you can upload the files directly to the project. To download a file, right-click on it and click Download (to local computer).

You may upload and download data to the local execution environment in a similar way, i.e. by dragging and dropping files to the execution file browser or by right-clicking on the files there and clicking Download.

Use the terminal

It is useful to have a terminal provided by JupyterLab at hand, which uses bash shell by default and lets you execute shell scripts or interact with the platform via dx toolkit. For example, the command:

$ dx pwd
MyProject:/

will confirm what the current project context is.

Running pwd will show you that the working directory of the execution environment is /opt/notebooks. The JupyterLab server is launched from this directory, which is also the default location of the output files generated in the notebooks.

To open a terminal window, go to File > New > Terminal or open it from the Launcher (using the "Terminal" box at the bottom). To open a Launcher, select File > New Launcher.

Install custom packages in the session environment

You can install pip, conda, apt-get, and other packages in the execution environment from the notebook:

%%bash
pip install torch
pip install torchvision
conda install -c conda-forge opencv

By creating a snapshot, you can start subsequent sessions with these packages pre-installed by providing the snapshot as input.