Download Agent

The DNAnexus Download Agent (dxda) is a command-line tool for downloading large numbers of files from the DNAnexus Platform to a local environment. The Download Agent is designed for downloads that involve many files, large individual files, or transfers that may run for an extended period.

Installation

Download releases are listed in the Releases section of the Download Agent GitHub repository. Docker images for experimental builds are available in the Download Agent Docker repository.

Usage

Download Agent accepts a BZIP2-compressed JSON manifest file, typically named manifest.json.bz2, that describes which files to download. Manifest entries identify files by project, folder, name, and part metadata. For instructions on generating a manifest and for full usage details including available flags, see the Quick Start in the Download Agent repository.

Transfer Integrity

Download Agent verifies the integrity of each file part as it is received. A checksum is computed in real time during transfer and compared against the value recorded by the platform. If a mismatch is detected, Download Agent retries the affected part automatically.

After a download completes, you can re-validate the downloaded files using the inspect subcommand. The inspect subcommand reads a previously used manifest and checks each downloaded part against its stored checksum. It supports multiple checksum algorithms, including MD5, CRC32, CRC32C, SHA1, SHA256, and CRC64NVME.

CLI Reference

Start with the top-level help output to see the available Download Agent subcommands. The command-specific references below focus on the outputs that are most useful in practice when you are downloading files, checking transfer results, or reviewing the supported flags.

Command Overview

Usage: dx-download-agent <flags> <subcommand> <subcommand args>

Subcommands:
	commands         list all command names
	download         Download files in a manifest
	flags            describe all known top-level flags
	help             describe subcommands and their syntax
	inspect          Inspect files downloaded in a manifest + additional 'health' checks
	progress         show current download progress
	version          get the version

Download Command

Use the download command to start a transfer from a manifest file. This synopsis shows the manifest argument and the primary command-specific options.

dx-download-agent download [-num_threads=N] <manifest.json.bz2>
  -gc_info
    	report statistics for golang garbage collection
  -max_threads int
    	An alias for num_threads
  -num_threads int
    	Number of threads to use when downloading files. By default (or if zero), this number is chosen according to machine memory and CPU constraints.
  -verbose
    	verbose logging

Download Flags

The flags download output is a reference for the options that control download concurrency and logging.

  -gc_info
    	report statistics for golang garbage collection
  -max_threads int
    	An alias for num_threads
  -num_threads int
    	Number of threads to use when downloading files. By default (or if zero), this number is chosen according to machine memory and CPU constraints.
  -verbose
    	verbose logging

Progress Command

Use the progress command to check the status of an active or previous manifest download.

dx-download-agent progress <manifest.json.bz2>