App Metadata
The file
dxapp.json is a DNAnexus application metadata file. Its presence in a directory tells DNAnexus tools that this directory contains DNAnexus app source code. The file's format is defined by convention and operationally by DNAnexus tools that use it (usually invoked through dx build and dx build --create-app).The format of the file closely resembles that of the corresponding calls to
/applet/new and /app/new, except for some differences as noted below. Fields found in dxapp.json but not recognized by the build tools are passed through to the API methods.The next section is a detailed example showing most of the commonly used fields.
The following lists the contents of an example
dxapp.json to be provided in a directory for use with the dx build command. For fields that are marked for apps only, it is harmless to provide them when building applets; they will simply only take effect when building apps.Note that comments as shown below are not valid in the JSON format but are provided here for easy reference.
{
"name": "mapper", # Simple name; cannot be changed across versions of an app
"title": "Simple Read Mapper", # Human-readable name (shown on the App's page and used as the default
# job name)
"summary": "Maps reads to a genome", # One-liner description of what the app(let) does
"description": "This app...", # (optional) Longer description of what the app(let) does; if not
# provided, dx build will inline the Readme.md file if available
"developerNotes": "...", # (optional) Detailed notes covering extra details; if not provided,
# dx build will inline the Readme.developer.md file if available
"details": { # (optional) This can contain arbitrary metadata, but there are some
"contactEmail": "[email protected]" # key/value pairs that are recognized as conventions by the website
}, # when rendering your app, e.g. for displaying contact information
"dxapi": "1.0.0", # The version of the API that your app uses
"version": "0.0.1", # Apps only: version of the app that is being built
"categories": ["Read Mapping"], # (optional) Apps only: list of categories describing the app (see
# below for a list of recognized categories)
"inputSpec": [ # Input specification (see API documentation)
# An array:int input "1,2,3" would be launched with the input [1, 2, 3]
# An array:strin "1,2,3" would be launched with the input ["1", "2", "3"]
{
"name": "reads",
"class": "file",
"patterns": ["*.fastq", "*.fastq.gz"],
"help": "Input FASTQ files ... ",
"optional": false
},
{
"name": "genomeindex_targz",
"label": "BWA reference genome index",
"help": "A file ...",
"class": "file",
"patterns": ["*.bwa-index.tar.gz"],
"suggestions": [
{
"name": "DNAnexus Reference Genomes",
"project": "project-BQpp3Y804Y0xbyG4GJPQ01xv",
"path": "/"
}
],
"optional": false
},
{
"name": "reads_type",
"class": "string",
"default": "paired-end",
"choices": ["single-end", "paired-end"],
"group": "Advanced Options"
}
],
"outputSpec": [ # Output specification (see API documentation)
{
"name": "mappings",
"class": "file",
"patterns": ["*.bam"]
}
],
"runSpec": { # Run specification (for more details, see the API documentation)
"interpreter": "python3", # Either "bash" or "python3" to indicate the interpreter for which
# the script in "file" was written
"file": "src/mapper.py", # Path to the script that will be run when this app(let) is run
"headJobOnDemand": true # (optional) Whether headJobOnDemand is enabled by default for this applet or app..
# Can be overridden at runtime.
"restartableEntryPoints": "all", # All entry points may be restarted if a restartable error occurs
"assetDepends": [
{
"id": "record-Bq320600kXGxY7PyB43GZ69j"
# Asset bundle record ID
},
{
"name": "qiime_asset", # (optional) Asset bundle record name
"version": "0.0.1", # (optional) Asset bundle record version number
"project": "project-Bq1pqV00kXGgzp45j4jGyYY7",
# (optional) ID of the project containing the asset bundle record
"folder": "/records" # (optional) Folder in the project containing the asset bundle record.
} # By default the folder is "/"
],
"systemRequirements": { # (optional, deprecated, moved to "regionalOptions") Request different
# instance types for different entry points
"main": { # "main" here is the name of the entry point that is called when an
"instanceType": "mem1_ssd1_x4" # app(let) is run
},
"process": { # "process" is the name of an entry point (used when spawning a new job
# via /job/new that runs the function called "process")
"instanceType": "mem3_ssd2_fpga1_x8",
"fpgaDriver": "edico-1.4.5" # fpga driver to install on fpga instances
},
"*": { # "*" indicates that the request applies to all other entry points
"instanceType": "mem3_ssd1_x4",# (when calling /job/new any function value other than "main" and "process")
"clusterSpec": { # (optional) request a cluster of instances
"type": "dxspark", # "dxspark", "apachespark","generic"
"version": "3.2.3", # "2.4.4" or "3.2.3" for spark, not present for generic
"initialInstanceCount": 3, # Size of the cluster (min 1) including the master
"ports": "1000, 1100-1200", # (optional) Ports/port-ranges to open for communication between nodes in the cluster
"bootstrapScript": "./bootstrap.sh" # (optional) Script to run on all nodes before running app code on master node
}
} #
},
"executionPolicy": { # (optional) Options governing job restart policy
"restartOn": {
"*": 3 # Restart automatically up to 3 times for system errors
}
},
"timeoutPolicy": { # (optional) User-specified timeout policy for a job running this executable.
"main": { # "main" here is the name of the entry point that is called when this executable
"hours": 12 # is run. The "main" entry point will timeout after 12 hours.
},
"*": { # "*" indicates that all other entry points except those explicitly specified in
"hours": 4, # `timeoutPolicy` (i.e. "main") will timeout after 4 hours 30 minutes.
"minutes": 30
}
},
"distribution": "Ubuntu", # The flavor of Linux that your app(let) will run on. Supported value: "Ubuntu"
"release": "16.04", # The release of the Linux distribution your app(let) will run on.
# Supported values: "20.04", "16.04".
"version": "1", # Default: "0". A triplet of (distribution, release, version)
# uniquely identifies an application execution environment. Supported versions:
# ("Ubuntu","20.04","0")
# ("Ubuntu","16.04","0")
# ("Ubuntu","16.04","1") (includes full python3 support)
"execDepends": [ # (optional) A list of packages that your app(let) requires to be
# installed before it can be run
{
"name": "samtools" # "samtools" is an APT package name (other package managers also
} # supported; see documentation on the run specification for details)
]
},
"resources": "project-BBF4Jp80vVky55Vvgkb0028v",
# (optional, deprecated, moved to the "regionalOptions" field) Apps only: request
# some resource(s) to be made (privately) accessible to the app when it is run;
# can be a project ID or a list of data object IDs. It can be used only for apps
# enabled in a single region and when "regionalOptions" field is not specified.
"ignoreReuse": true, # (optional) Whether job reuse is disabled by default for this applet or app.
# Can be overriden at runtime.
"openSource": true, # (optional) Apps only: Whether the contents of this app version's
# resources container should be viewable by all members of the app's
# developers and authorized users
"httpsApp": { # (optional) externally expose jobs to the internet via an HTTPS proxy
# at https://job-xxxx.dnanexus.cloud/
"ports": [443, 8080, 8081], # Array of ports to allow access. Allowed values are 443, 8080, 8081
"shared_access": "VIEW" # Restrict access to users with project access VIEW, CONTRIBUTE, ADMINISTER, or NONE to limit to the launching user
},
"access": { # (optional) Request that the app(let) be given additional access
# permissions
"project": "CONTRIBUTE", # (optional) Give the app(let) CONTRIBUTE access to the project in
# which it is run. By default, an app will receive NO access, or if
# you build this as an applet, it will receive VIEW access.
"allProjects": "VIEW", # (optional) Give the app(let) VIEW access to all projects that the
# user running the app can access. Use this carefully.
"network": [ # (optional) Give the app(let) network access to all domains; you can
"*" # also list specific domains such as "github.com".
],
"developer": true # (optional) Give the app(let) permissions to act as a developer on
# behalf of the user (e.g. it will be able to create new apps and
# access unpublished apps)
},
"developers": [ # Apps only: these users will be allowed to publish new versions of
"user-you" # this app and perform other administrative functions.
],
"authorizedUsers": [ # Apps only: any published version of this app will be runnable by the
"user-alice", # developers of the app, and by user-alice and user-bob. (Access to
"user-bob" # an applet is determined by access to the project containing the
], # applet.)
# In the case of apps: set this field to allow your app to be run in more
# than one region. Provide as keys all the regions in which you wish the
# app to be enabled. At a minimum, supply an empty object mapping {}
# as the corresponding value. Within each value, you may also supply
# any of the keys "systemRequirements", "bundledDepends",
# "assetDepends", or "resources". The corresponding values will be treated
# as if you had supplied them in the run specification above, but only
# for the associated region.
#
# In the case of applets: request specific "systemRequirements"
# in the region in which the applet will be built.
#
# You can use this to request different instance types that might be
# available in different clouds, or to supply copies of app resources
"regionalOptions": { # that are colocated with the specific region where the app will be
"aws:us-east-1": { # run.
"systemRequirements": {"*": {"instanceType": "mem2_ssd1_x2"}},
"bundledDepends": [{"name": "bundle.tar.gz", "id": {"$dnanexus_link": "file-xxxx"}}],
"assetDepends": [{"id": "record-yyyy"}],
"resources": "project-zzzz"
},
"azure:westus": {
"systemRequirements": {"*": {"instanceType": "azure:mem2_ssd1_x2"}},
"bundledDepends": [{"name": "bundle.tar.gz", "id": {"$dnanexus_link": "file-zzzz"}}],
"assetDepends": [{"id": "record-aaaa"}],
"resources": ["file-aaaa", "file-bbbb"]
}
}
}
Other options for the
/applet/new and /app/new calls, such as specifying in which project or folder to create an applet, are populated via command-line flags of dx build.The
name string The unique name of the applet / app. The name can have lower case letters, numbers, -, and _ but cannot have spaces.If the applet is published into an app, then its ID will be of the form
app-[name]/[version]. An app's name cannot be changed across versions.Example:
{
...
"name": "bwa_mem_fastq_read_mapper",
...
}
title string (optional) A human-readable name of the app(let). If published as an app, this name will be shown on the Apps page and as the default job name.Example:
{
...
"title": "BWA-MEM FASTQ Read Mapper",
...
}
summary string (optional) A one-liner description of the app(let). If published as an app, this summary will be displayed under the app name on the Apps page.Example:
{
...
"summary": "Maps FASTQ reads to a reference genome with the BWA-MEM algorithm.",
...
}
description string (optional) Longer description of what the app(let) does. If not provided, dx build will inline the Readme.md file, if available.Example:
{
...
"description": "This app maps FASTQ reads to a reference genome ... inputs ...",
...
}
developerNotes string (optional) Detailed notes covering extra details. If not provided, dx build will inline the Readme.developer.md file, if available.Example:
{
...
"developerNotes": "...",
...
}
details string (optional) This can contain arbitrary metadata, but there are some key/value pairs that are recognized as conventions by the website when rendering your app, e.g. for displaying contact information. Recognized metadata fields include the following:- Author contact information
- Upstream executable version number (i.e. bwa-mem 0.7.12)
- Upstream author name
- URL for app executable
- Upstream licenses
- Change Log
- Citations for papers
Example:
{
...
"details": {
"contactEmail": "[email protected]",
"citations": [ "doi:10.1093/bioinformatics/btp352"],
"upstreamVersion": "0.9.12",
"upstreamAuthor": "Alice Bob",
"upstreamUrl": "http://foo.bar.edu/",
"upstreamLicenses": ["GLPv3"],
"whatsNew": "## Changelog\n* Optionally markdown formated"
}
...
}
dxapi string (optional) The version of the API that your app uses.Example:
{
...
"dxapi": "1.0.0",
}
version string Apps only: The version of the app to be built. This version number must be unique from all other versions of the app (published or not).Example:
{
...
"version": "0.0.2",
}
You can specify any categories you wish for your app in the "categories" field of the dxapp.json. However, the following categories are specially recognized by DNAnexus, and authorized users of your app can use them to browse for your app in the App Store if you specify any of them:
- Annotation
- Assembly
- Debugging
- Export
- Import
- Mappings Manipulation
- Read Manipulation
- Read Mapping
- Reports
- RNA-Seq
- Statistics
- Structural Variation
- Variation Calling
Example:
{
...
"categories": [
"Read Mapping"
],
}
JSON array containing the specifications for each input to the app. These fields are required for apps and highly recommended when developing applets. The full documentation for specifying an input or output specification can be found here.
If you do choose to exclude an input (output) specification for an applet (i.e. one or more of the keys
inputSpec and outputSpec is missing), this means that the system will perform no checking on the input (output) provided to (returned by) the applet, respectively. Note, however, that applets that do not have both an input and output specification cannot be added as a stage in a workflow.The
runSpec field specifies your app's entry and the interpreter used to evaluate it:{ ...,
"runSpec": {
"interpreter": "python3",
"file": "src/code.py"
},
...
}
In the above example, the "runSpec.file" parameter provides the path to a Python file that contains your application logic.
In this case, we've provided Python code (
code.py) that uses the DNAnexus Python 3 execution template (indicated in the interpreter field). This template takes care of parsing your app's input from the DNAnexus execution environment, handling possible errors, and saving your app's output. See the example applications for examples of the entry point Python function signatures.For more details about execution templates and running code in the execution environment, see the Execution Environment Reference.
You can designate a specific version number, such as
3.1.41-rc.1, to identify each build of your app.We recommend following the Semantic Versioning conventions for numbering the versions of your app. Semantic Versioning also specifies how you should change the version number for various kinds of updates to your app (that is, bug-fix only, backwards compatible, or backwards incompatible). Using the Semantic Versioning guidelines will help users and other developers to understand when it is safe to move between different versions of your app.
In order to both make reproducibility guarantees feasible and protect user privacy, apps are generally heavily restricted in what data they can read and write. Apps may request relaxation of these restrictions.
See also the App Permissions Guide for more information about the permissions available and their implications.
You can specify who can describe and run your app by providing a list of users. The default access list is the empty list
[], which indicates that no one except the developers of the app can find and run your app, even if it has been published. You may provide a list of users and/or organizations here to grant them access the app.Note: the access list is shared for all past and future versions of the app but can be modified at any time. If you need to create another version of your app which has a different access list, you will need to publish it separately under a different name.
You can also edit the list of authorized users after you've built and published an app by using the
dx add users and dx remove users commands. However, if you (or another developer) rebuilds the app with authorizedUsers in the dxapp.json file, the access list will be overwritten to match the one found in the file.You can specify what regions the app can be run in, and specify advanced options that control the app's behavior in multiple regions. The
regionalOptions field should be a mapping with keys corresponding to each region in which the app should be runnable. (A region is given by a string like "aws:us-east-1"; see Regions for more information about regions.) If you don't specify this field, the app will be enabled in only one region: the region of your project context when the app is built.The values associated with each key are themselves mappings. In the most basic usage, you can specify an empty mapping for each region. This is generally sufficient to allow apps that don't use certain advanced features to be run in multiple regions.
For apps that make use of bundled dependencies, additional resources, or specify that they should run on specific instance types, each value of
regionalOptions may contain the following keys and values:-
systemRequirementsmapping Request specific instance types and (optional) define a clusterSpec for some of the entry points of the app when it is run in the corresponding region. bundledDependslist bundled dependencies for the corresponding region. For the syntax of this field, see the discussion ofbundledDependsin the run specification.assetDependslist asset dependencies for the corresponding region. See theassetDependsfield above for the syntax of this field.resourcesstring or array of strings Either a string containing the ID of a project which will be snapshotted into a resources container, or an array of data object IDs, which will all be cloned into the root folder of the resources container. All specified objects must reside in the specified region and will be available to the app when it is run in that region.
If you specify a value for a key, say "systemRequirements", you must do so in all regions, and you may not specify the same key in the run specification.
Last modified 17d ago