dx download

Learn how to use the dx download command to download data and share files among collaborators.

Downloading a Single File

The dx download command can download a file using either its file ID or its name. Files are downloaded into the current local directory and inherit their names from the platform unless specified otherwise.

Downloading Using a File ID

Using dx download to download a file using its ID is particularly useful in situations where there may be different files with the same name in the same folder. To get a file's ID via the web UI, select the file and click "Info". The "ID" field has a corresponding file ID of the format file-xxxx.

$ dx download file-xxxx
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example.bam

You can remove the progress bar using the flag --no-progress, so that no feedback is provided after executing the command dx download. This is useful for scripting.

Downloading Using a Filename

Using dx download to download a file using its name differs slightly based on whether the filename is unique within its folder. To find your current working directory within DNAnexus, use the command dx pwd to list the files in a directory, use the command dx ls.

Unique Filename

If the filename is unique within its folder, downloading a file using its name looks similar to downloading a file using its ID.

$ dx download "different.bam"
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) different.bam

Non-Unique Filename

If a filename is not unique within its folder, on executing dx download you are prompted to select which file you intend on downloading.

If you use the -a/--all flag, all files are downloaded. However, your local operating system may not allow multiple files of the same name to be in the same folder.

$ dx download "same.bam"
The given path "same.bam" resolves to the following data objects:
0) closed  2017-01-01 09:00:00 1.51 MB   same.bam (file-xxxx)
1) closed  2017-01-02 09:00:00 1.64 MB   same.bam (file-yyyy)

Pick a numbered choice: 0
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) same.bam

Downloading Using a File Path

If the file is contained within a folder, dx download can also use file paths as input. For example, if file file_name is contained within the folder my_folder/ in the project project-xxxx, the command changes accordingly. Projects can be identified by either their name or ID. If multiple projects are named identically, you are prompted to select one.

$ dx download "project-xxxx:/my_folder/example.bam"
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example.bam

$ dx download "Project_Name:/my_folder/example.bam"
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example.bam

Downloading Multiple Files

If you only want to download a few files at once, you can use multiple filenames or file IDs as inputs to dx download.

$ dx download "example_1.bam" "example_2.bam" "project-xxxx:file-xxxx"
The given path "example_1.bam" resolves to the following data objects:
0) closed  2017-01-01 09:00:00 1.51 MB   example_1.bam (file-xxxx)
1) closed  2017-01-02 09:00:00 1.64 MB   example_1.bam (file-yyyy)

Pick a numbered choice: 0
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example_1.bam
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example_2.bam
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) example_3.bam

Downloading a Folder

If you want to download multiple files at once using dx download, moving desired files to a single folder can be the simplest solution. The -r/--recursive flag is required to download folders.

By default, if multiple files of the same name exist in a folder, only one of the files is downloaded. Add the -a/--all flag to download all files, even if their names match. Your local operating system may not allow multiple files of the same name to be in the same folder.

$ dx ls "example_folder/"
example_1.bam
example_2.bam
example_3.bam
$ dx download "example_folder" -r
$ ls "example_folder/"
example_1.bam  example_2.bam    example_3.bam

Generating Download URLs

Download URLs are useful for sharing files with collaborators who may or may not have accounts on DNAnexus. URLs may be pasted directly into a browser window for automatic download or used as input for the bash command wget.

Generating a Single URL

The dx make_download_url command is used to generate download URLs.

The command can be used on both file names and file IDs. The path to the file should include the project context, such as project-xxxx:file-yyyy, or project-xxxx:/path/to/file.txt.

$ dx make_download_url "project-xxxx:example.bam"
https://dl.dnanex.us/F/D/xxxx/example.bam
$ dx make_download_url project-xxxx:file-yyyy
https://dl.dnanex.us/F/D/xxxx/example.bam

As shown above, if the file name used as input is not unique within its folder, you are prompted to select a file.

$ dx make_download_url "project-xxxx:example.bam"
The given path "example.bam" resolves to the following data objects:
0) closed  2017-01-01 09:00:00 1.51 MB   example.bam (file-xxxx)
1) closed  2017-01-02 09:00:00 1.64 MB   example.bam (file-yyyy)

Pick a numbered choice: 0
https://dl.dnanex.us/F/D/xxxx/example.bam

Generating Multiple URLs

You must write a script to generate download URLs for multiple files.

URL Expiration Dates

By default, URLs expire after one day. However, you can change the duration so that the URL lasts for a longer or shorter period of time.

$ dx make_download_url "project-xxxx:test.bam" --duration 1h
https://dl.dnanex.us/F/D/xxxx/test.bam

After the specified duration of time, downloading the file using the URL no longer works. This is useful for restricting access to files.

Advanced Options

Resuming Downloads

If a file has only been partially downloaded, the -f/--overwrite flag resumes the download. However, if a file has been fully downloaded, the -f/--overwrite flag overwrites the downloaded file.

Redirecting Downloaded Files

By default, files assume the same name they had on the platform when downloaded. However, you can override this with the -o/--output flag. This is useful if your script assumes a specific file name.

$ dx download "example.bam" -o "not_example.bam"
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) not_example.bam

Redirecting to a Different Folder

You may also redirect the outputs to a different folder using the -o/--output flag. If you are redirecting the outputs to a different folder, dx download does not create the folders. They must already exist before downloading the files.

$ dx download "example.bam" -o "newdir/not_example.bam"
[                                                            ] Downloaded 0 byte
[===========================================================>] Downloaded 1,582,
[===========================================================>] Completed 1,582,904 of 1,582,904 bytes (100%) newdir/not_example.bam

Printing to stdout

The command dx download file-xxxx -o - redirects the contents of the file file-xxxx to stdout. This has the same functionality as the command dx cat file-xxxx.

$ dx download "sample.txt" -o -
This is a sample text file.

Additional information

Executing the dx download --help command shows all the flags available to use with dx download. The message printed by this command is same as the one displayed in the brief description of dx download. More information about dx make_download_url can be found on the same page and by running the command dx make_download_url --help.