Ctrlk
APIDownloadsIndex of dx CommandsLegal
For the complete documentation index, see llms.txt. This page is also available as Markdown.
ChatGPT

dx upload

You can upload files to the DNAnexus Platform using the command dx upload. You can also upload data using the DNAnexus Upload Agent, a fast and convenient command-line client. For uploading multiple or large files (>50 MB), we recommend that you use Upload Agent, which uploads up to 1000 files concurrently and resumes uploads after network interruption.

Uploading Files

Uploading One File

You can use the dx upload command followed by a file path to upload one local file.

$ dx upload /Users/alice/README.txt
[===========================================================>] Uploaded 597 of 597 bytes (100%) /Users/alice/README.txt
ID                  file-xxxx
Class               file
Project             project-xxxx
Folder              /
Name                README.txt
State               closing
Visibility          visible
Types               -
Properties          -
Tags                -
Outgoing links      -
Created             Sun Jan  1 12:00:01 2017
Created by          alice
Last modified       Sun Jan  1 12:00:02 2017
archivalState       "live"
Media type

You can use the --brief flag with the dx upload command to print only a DNAnexus ID rather than the full command output. This flag is useful if you want to save the file ID of the uploaded file.

If you are trying to upload a file hosted at a publicly accessible URL rather than locally, you can use the URL Fetcher app.

Uploading Multiple Files

You can also use the dx upload command followed by multiple file paths to upload multiple local files.

You can use the wildcard * to upload multiple files.

The following command uploads all the files inside the /Users/alice/ directory. For this example, the /Users/alice/ directory contains only files and not sub-directories.

Uploading Directories

You can specify the ‑r/‑‑recursive parameter with the dx upload command to recursively upload one or more directories or folders and maintain their respective structures.

You can use the dx ls command to list the uploaded folders and files.

If the local uploaded directory ends with the / character, only the contents of the directory are uploaded, not the directory itself.

Uploading Files With Metadata

You can specify object metadata using the dx upload command.

If multiple files are uploaded, tags and properties are applied to all the files.

Properties

You can use the --property KEY=VALUE parameter to add metadata to the file being uploaded. The parameter may be repeated as necessary, for example, --property key1=val1 --property key2=val2, to link multiple metadata fields as key-value pairs.

Tags

You can also add tags to uploaded files with the dx upload command using the --tag TAG parameter. The parameter may be repeated as necessary, for example, --tag tag1 --tag tag2, to link multiple tags to a file.

Advanced Options

Redirecting Uploaded Files

You can use the --path/‑‑destination parameter to specify the DNAnexus destination path. If the path is not specified, dx upload defaults to the current project and folder. You can determine your current project and folder using the command dx pwd.

Uploading from stdin

You can upload data directly from standard input. This is useful when streaming the upload while the file is generated.

Buffers

You can specify the --buffer-size parameter with the dx upload command to set the write buffer size in bytes. For large files from standard input, set --buffer-size manually because the dx command cannot determine the file size in advance.

Hiding the Progress Bar

You can specify the --no-progress flag with the dx upload command to hide the progress bar.

Upload Integrity

dx upload splits files into parts and sends each part with an MD5 checksum. The platform verifies this checksum when recording the part and only marks the part as complete when the value matches. A file object cannot be closed until all its parts are in the complete state. You can submit a job with a reference to an open file as an app input, but the job cannot start, and the file cannot be copied between projects, until the file is closed.

For uploading large files or many files concurrently, Upload Agent provides the same integrity guarantees with higher throughput and resume support.

Last updated

Was this helpful?