DNAnexus Documentation
APIDownloadsIndex of dx CommandsLegal
  • Overview
  • Getting Started
    • DNAnexus Essentials
    • Key Concepts
      • Projects
      • Organizations
      • Apps and Workflows
    • User Interface Quickstart
    • Command Line Quickstart
    • Developer Quickstart
    • Developer Tutorials
      • Bash
        • Bash Helpers
        • Distributed by Chr (sh)
        • Distributed by Region (sh)
        • SAMtools count
        • TensorBoard Example Web App
        • Git Dependency
        • Mkfifo and dx cat
        • Parallel by Region (sh)
        • Parallel xargs by Chr
        • Precompiled Binary
        • R Shiny Example Web App
      • Python
        • Dash Example Web App
        • Distributed by Region (py)
        • Parallel by Chr (py)
        • Parallel by Region (py)
        • Pysam
      • Web App(let) Tutorials
        • Dash Example Web App
        • TensorBoard Example Web App
      • Concurrent Computing Tutorials
        • Distributed
          • Distributed by Region (sh)
          • Distributed by Chr (sh)
          • Distributed by Region (py)
        • Parallel
          • Parallel by Chr (py)
          • Parallel by Region (py)
          • Parallel by Region (sh)
          • Parallel xargs by Chr
  • User
    • Login and Logout
    • Projects
      • Project Navigation
      • Path Resolution
    • Running Apps and Workflows
      • Running Apps and Applets
      • Running Workflows
      • Running Nextflow Pipelines
      • Running Batch Jobs
      • Monitoring Executions
      • Job Notifications
      • Job Lifecycle
      • Executions and Time Limits
      • Executions and Cost and Spending Limits
      • Smart Reuse (Job Reuse)
      • Apps and Workflows Glossary
      • Tools List
    • Cohort Browser
      • Chart Types
        • Row Chart
        • Histogram
        • Box Plot
        • List View
        • Grouped Box Plot
        • Stacked Row Chart
        • Scatter Plot
        • Kaplan-Meier Survival Curve
      • Locus Details Page
    • Using DXJupyterLab
      • DXJupyterLab Quickstart
      • Running DXJupyterLab
        • FreeSurfer in DXJupyterLab
      • Spark Cluster-Enabled DXJupyterLab
        • Exploring and Querying Datasets
      • Stata in DXJupyterLab
      • Running Older Versions of DXJupyterLab
      • DXJupyterLab Reference
    • Using Spark
      • Apollo Apps
      • Connect to Thrift
      • Example Applications
        • CSV Loader
        • SQL Runner
        • VCF Loader
      • VCF Preprocessing
    • Environment Variables
    • Objects
      • Describing Data Objects
      • Searching Data Objects
      • Visualizing Data
      • Filtering Objects and Jobs
      • Archiving Files
      • Relational Database Clusters
      • Symlinks
      • Uploading and Downloading Files
        • Small File Sets
          • dx upload
          • dx download
        • Batch
          • Upload Agent
          • Download Agent
    • Platform IDs
    • Organization Member Guide
    • Index of dx commands
  • Developer
    • Developing Portable Pipelines
      • dxCompiler
    • Cloud Workstation
    • Apps
      • Introduction to Building Apps
      • App Build Process
      • Advanced Applet Tutorial
      • Bash Apps
      • Python Apps
      • Spark Apps
        • Table Exporter
        • DX Spark Submit Utility
      • HTTPS Apps
        • Isolated Browsing for HTTPS Apps
      • Transitioning from Applets to Apps
      • Third Party and Community Apps
        • Community App Guidelines
        • Third Party App Style Guide
        • Third Party App Publishing Checklist
      • App Metadata
      • App Permissions
      • App Execution Environment
        • Connecting to Jobs
      • Dependency Management
        • Asset Build Process
        • Docker Images
        • Python package installation in Ubuntu 24.04 AEE
      • Job Identity Tokens for Access to Clouds and Third-Party Services
      • Enabling Web Application Users to Log In with DNAnexus Credentials
      • Types of Errors
    • Workflows
      • Importing Workflows
      • Introduction to Building Workflows
      • Building and Running Workflows
      • Workflow Build Process
      • Versioning and Publishing Global Workflows
      • Workflow Metadata
    • Ingesting Data
      • Molecular Expression Assay Loader
        • Common Errors
        • Example Usage
        • Example Input
      • Data Model Loader
        • Data Ingestion Key Steps
        • Ingestion Data Types
        • Data Files Used by the Data Model Loader
        • Troubleshooting
      • Dataset Extender
        • Using Dataset Extender
    • Dataset Management
      • Rebase Cohorts and Dashboards
      • Assay Dataset Merger
      • Clinical Dataset Merger
    • Apollo Datasets
      • Dataset Versions
      • Cohorts
    • Creating Custom Viewers
    • Client Libraries
      • Support for Python 3
    • Walkthroughs
      • Creating a Mixed Phenotypic Assay Dataset
      • Guide for Ingesting a Simple Four Table Dataset
    • DNAnexus API
      • Entity IDs
      • Protocols
      • Authentication
      • Regions
      • Nonces
      • Users
      • Organizations
      • OIDC Clients
      • Data Containers
        • Folders and Deletion
        • Cloning
        • Project API Methods
        • Project Permissions and Sharing
      • Data Object Lifecycle
        • Types
        • Object Details
        • Visibility
      • Data Object Metadata
        • Name
        • Properties
        • Tags
      • Data Object Classes
        • Records
        • Files
        • Databases
        • Drives
        • DBClusters
      • Running Analyses
        • I/O and Run Specifications
        • Instance Types
        • Job Input and Output
        • Applets and Entry Points
        • Apps
        • Workflows and Analyses
        • Global Workflows
        • Containers for Execution
      • Search
      • System Methods
      • Directory of API Methods
      • DNAnexus Service Limits
  • Administrator
    • Billing
    • Org Management
    • Single Sign-On
    • Audit Trail
    • Integrating with External Services
    • Portal Setup
    • GxP
      • Controlled Tool Access (allowed executables)
  • Science Corner
    • Scientific Guides
      • Somatic Small Variant and CNV Discovery Workflow Walkthrough
      • SAIGE GWAS Walkthrough
      • LocusZoom DNAnexus App
      • Human Reference Genomes
    • Using Hail to Analyze Genomic Data
    • Open-Source Tools by DNAnexus Scientists
    • Using IGV Locally with DNAnexus
  • Downloads
  • FAQs
    • EOL Documentation
      • Python 3 Support and Python 2 End of Life (EOL)
    • Automating Analysis Workflow
    • Backups of Customer Data
    • Developing Apps and Applets
    • Importing Data
    • Platform Uptime
    • Legal and Compliance
    • Sharing and Collaboration
    • Product Version Numbering
  • Release Notes
  • Technical Support
  • Legal
Powered by GitBook

Copyright 2025 DNAnexus

On this page
  • Org Admin Guide
  • Update Org Policies
  • Edit Org Membership
  • Update Membership Access
  • Org Projects
  • Grant Yourself Access to Org Projects
  • List All Org Projects
  • Org Billing and Finance
  • Accessing Org Billing Information
  • Setting up or Updating Billing Information for an Org
  • Monitoring Account Spending and Usage
  • Per-Project Usage Report
  • Root Execution Stats Report
  • Learn More About An Org
  • List Org Members
  • List Org Apps
  • Grant Yourself Access to Org Objects
  • Give Yourself Access to any Org Project
  • Grant Yourself Access to any Org App
  • Creating an Org
  • Provisioning New DNAnexus Accounts
  • Enabling PHI Data Protection for an Org
  • Data Protection Notice
  • Forwarding Job Logs To Customer's Splunk
  • Enabling Forwarding for an Org
  • Setting Up Forwarding
  • Disabling Forwarding
  • How It Works
  • Learn More

Was this helpful?

Export as PDF
  1. Administrator

Org Management

On the DNAnexus Platform, several users can form an organization, which is an entity with a common pool of funds that all users can draw from. All projects billed to that organization (as well as all jobs that are run in those projects) will incur charges using the same pool of funds, regardless of user.

The All Orgs tab, which you can select from Org Admin option on the top left menu, shows you a list of all orgs to which you have admin access. On this page, you can quickly see all of your orgs, the number of projects per org, and the funds available to the org. You can also see additional settings for each org by enabling optional columns (icon on the right side of the column headers).

Org Admin Guide

These operations are restricted to only org admins - that is, org members with "Admin" membership level.

If you are an admin of an org, you will be able to access the org admin tools from the main Platform menu. From here, you can quickly navigate to the list of orgs you administer (via "All Orgs"), or to a specific org (e.g. Demo Organization in this example).

In the "Basic Information" section of the org settings tab, you will be able to see an overview of the org.

  • View and edit the organization name. This is how the org will be referred to in the UI and in email notifications

  • View the organization ID, the unique ID used to reference a particular org on the CLI (e.g. org-demo_org)

  • View the number of org members, org projects and org apps

  • View the list of organization admins

Update Org Policies

You can use the dx update org command to update your org policies.

To update the membership list visibility policy so all org members can view the list of org members, use the following command, replacing org-demo with your org ID.

$ dx update org org-demo --member-list-visibility MEMBER
Updated org-demo

To update the restrict project transfer policy so that only org admins can change the billing account of an org project, use the following command, replacing org-demo with your org ID.

$ dx update org org-demo --project-transfer-ability ADMIN
Updated org-demo

Via the UI

You can also update org policies for your org in the "Policies and Administration" section of the org settings tab. Here, you can both change the membership list visibility and restrict project transfer policies for the org and contact DNAnexus support to enable PHI data policies for org projects.

Edit Org Membership

Within an org page, the Members tab allows you to view all the members of the org, invite new members, remove existing members, and update existing members' permission levels.

To add existing DNAnexus user to your org, you can use the "+ Invite New Member" button from the org's Members tab. This will open a dialog where you can enter the user's username (e.g. smithj) or user-ID (e.g. user-smithj). Then you can configure the user's access level in the org.

  • Organization Membership: Whether they are a member or admin of the org.

  • Billable Activities Access: Whether the user should be able to create projects and apps billed to the org.

  • Shared Project Access: The maximum permission the user will have to projects that were shared with the org. For example, if this is set to "Contributor" but someone shares a project with the org at an Administrator level, this user will still only have Contributor access, even if other members of this org receive the full Administrator permissions.

  • Shared Apps Access: Whether the user should have access to apps shared with the org.

If you add a member to the org with billable activities access set to billing allowed, they will have the ability to create new projects billed to the org. However, adding the member will not change their default billing account. If the user wishes to use the org as their default billing account, they will have to set their own default billing account.

Additionally, if the member has any pre-existing projects that are not billed to the org, the user will need to transfer the project to the org if they wish to have the project billed to the org.

The user will receive an email notification informing them that they have been added to the organization.

Via the CLI

The dx add member command allows org admins to add existing DNAnexus users to a pre-existing org. The command will add a member to the org but will not set their default billing account; the member will have to set their own default billing account. Additionally, if the member has any pre-existing projects that are not billed to the org, the user will need to transfer the project to the org if they wish to have the project billed to the org.

Below are some examples of adding users to an org.

To add an existing user to an org with:

  • Membership level: MEMBER

  • Billable activities access: Allowed

  • Shared apps access: Allowed

  • Shared projects access: CONTRIBUTE

Use the following command, replacing org-demo with your org ID and xxxx in user-xxxx with the new member's username.

$ dx add member org-demo user-xxxx --level MEMBER --allow-billable-activities --project-access CONTRIBUTE
Invited user-xxxx to org-demo

To add an existing user to an org with:

  • Membership level: MEMBER

  • Billable activities access: Not allowed

  • Shared apps access: Not allowed

  • Shared projects access: VIEW

Use the following command, replacing org-demo with your org ID and xxxx in user-xxxx with the new member's username.

$ dx add member org-demo user-xxxx --level MEMBER --project-access VIEW --no-app-access
Invited user-xxxx to org-demo

To add another org admin to the org, use the following command, replacing org-demo with your org ID and xxxx in user-xxxx with the new member's username.

$ dx add member org-demo user-xxxx --level ADMIN
Invited user-xxxx to org-demo

For more information about adding members to an org, use the dx add members command with the -h flag.

Update Membership Access

From the org Members tab, you can edit the permissions for one or multiple members of the org. The option to "Edit Access" appears when you have one or more org members selected on the Members tab.

If you select only one member, you can specify their new access via the dialog.

You can also edit multiple members at the same time by selecting multiple members from the members list and clicking "Edit Access".

When you're editing multiple members, you have the option of changing only one access while leaving the rest as is. In the example below, we are editing the member access for User Alice and Jennifer Smith. In this case, we updating their access so both have the ability to create new projects billed to the org, Billing Allowed. We will keep all the rest of their permissions the same.

Via the CLI

The dx update member command allows org admins to update a member's access within an org.

To grant an existing org member billable activities access, use the following command, replacing org-demo with your org ID and xxxx in user-xxxx with the member's username.

$ dx update member org-demo user-xxxx --level MEMBER --allow-billable-activities true
Updated membership of user-xxxx in org-demo

To grant an existing org member shared projects access, use the following command, replacing org-demo with your org ID and xxxx in user-xxxx with the member's username.

$ dx update member org-demo user-xxxx --level MEMBER --app-access true
Updated membership of user-xxxx in org-demo

To update an existing org member's shared projects access, use the following command, replacing org-demo with your org ID, xxxx in user-xxxx with the member's username, and ACCESS with one of ADMINISTER, CONTRIBUTE, UPLOAD, VIEW, or NONE.

$ dx update member org-demo user-xxxx --level MEMBER --project-access ADMINISTER
Updated membership of user-xxxx in org-demo

To make an existing org member an org admin, use the following command, replacing org-demo with your org ID and xxxx in username with the member's username.

$ dx update member org-demo user-xxxx --level ADMIN
Updated membership of user-xxxx in org-demo

For more information about updating members' access and level in an org, use the dx update members command with the -h flag.

Remove a Member from an Org

Org admins can remove a member from an org using the dx remove member command. This command will remove the user from the org, revoking the user's access to projects and apps shared with the org. This command will also, by default and if applicable, remove the user's access to apps and projects billed to the org.

From the org Members tab, you can remove one or more members from the org. The option to "Remove" appears when you have one or more org members selected on the Members tab.

This will open the following dialog where you can confirm the action to remove members from the org. This action will revoke the user's access to all projects and apps billed to or shared with the org.

Via the CLI

To remove a member from an org, use the following command, replacing org-demo with your org ID and xxxx user-xxxx with the member's username. You will be prompted to confirm the removal of the user from the org.

$ dx remove member org-demo user-xxxx

WARNING: About to remove user-xxxx from org-demo; project
permissions will be removed and app permissions will be removed
Please confirm [y/n]: y
Removed user-xxxx from org-demo
Removed user-xxxx from the following projects:
  None
Removed user-xxxx from the following apps:
  None

For more information about removing members from an org, use the dx remove members command with the -h flag.

Org Projects

The org's Projects tab allows you to see the list of all projects billed to the org. This list will include all projects in which you have View and above permissions as well as projects that are billed to the org in which you do not have permissions (Not a Member).

You will be able to view all project metadata (e.g. the members list, data usage, creation date), as well as some other optional columns (e.g. project creator). To enable the optional columns, select the column from the dropdown menu to the right of the column names.

Grant Yourself Access to Org Projects

In addition to viewing the list of projects, org admins can give themselves access to any project billed to the org. If you select a project in which you are not a member, you will still be able to navigate into the project's settings page. From the project settings page, you will see a button allowing you to grant yourself Administer permissions to the project.

You will also be able to grant yourself Administer permissions if you are currently a member of a project billed to your org but you only have View, Contribute, or Upload permissions.

List All Org Projects

To list all org projects, including projects to which you do not have access, use the dx find org projects command, replacing org-demo with your org ID.

# List all org projects
$ dx find org projects org-demo
project-xxxx : my_new_project (ADMINISTER)
project-yyyy : example_project (CONTRIBUTE)

# Return only the project IDs for all projects billed to the
org
$ dx find org projects org-demo --brief
project-xxxx
project-yyyy

# Return JSON-formatted string of org projects and project metadata
$ dx find org projects org-demo --json
[
    {
        "level": "ADMINISTER",
        "describe": {
            "id": "project-xxxx",
            ...

You can also use JQ (a [command-line JSON parser](https://stedolan.github.io/jq/))
to format a command-delimited list of project-ID, project name, your access
level , project creator, and number of bytes stored in the project.

$ dx find org projects org-demo --json | jq -r '.[] .describe | .id + "," + .name + "," + .level + "," + .createdBy.user + "," + (.dataUsage | tostring)'
project-xxxx,my_new_project,ADMINISTER,user-xxxx,0.059
project-yyyy,example_project,CONTRIBUTE,user-yyyy,0.048

Org Billing and Finance

Accessing Org Billing Information

  1. Click on your user icon in the top right corner of your screen.

  2. Select Billing from the dropdown menu.

  3. Find the tile with the name of the billing account for which you'd like to see detailed info. Click on the downward-facing arrow at the bottom right of the tile.

  4. In the Billing Details section, you'll see either a prompt to set up billing, or information on billing responsibility for the org.

  5. In the Account Summary section, you'll see detail on the billing account's spending limit and total charges incurred to date, including those for storage, compute, and egress.

The values shown in the Account Summary are lifetime values for the billing account. They do not represent outstanding charges for usage since the last contract or account settlement date.

The Remaining value, as noted above, represents the total additional charges that can be incurred, before the spending limit is reached.

The Storage and DB Cluster Charge figures are updated monthly. Note that this update is typically done by the end of the first week of each month, with the updated figure representing storage charges incurred during the previous month.

Setting up or Updating Billing Information for an Org

To set up billing for an org, or change billing information for that org:

  1. Open the Billing screen by following the instructions in “Access org billing information” above.

  2. For the org for you wish to update billing information, follow the instructions in the Billing Details section to set up or update your billing information.

Request a Spending Limit Change for an Org

Monitoring Account Spending and Usage

Configuration of these features, and report delivery, is handled by DNAnexus Support.

Per-Project Usage Report

Root Execution Stats Report

A monthly org-level root execution report that lists every root execution associated with a cost. The report is generated as a CSV and is delivered to a user-specified destination project.

Learn More About An Org

List Org Members

From the org Members tab, you can quickly see the names and access levels for all org members.

To list all the members of an org using the CLI, use the dx find org members command, replacing org-demo with your org ID.

# List all org members & admin
$ dx find org members org-demo
user-xxxx : Jane Doe  (ADMIN)
user-yyyy : John Doe  (MEMBER)

# List only org admin
$ dx find org members org-demo --level ADMIN
user-xxxx : Jane Doe  (ADMIN)

# Return only user IDs of org members & admin
$ dx find org members org-demo --brief
user-xxxx
user-yyyy

List Org Apps

To list all the apps billed to your app, use the dx find org apps command, replacing org-demo with your org ID.

# Return list of all org apps

$ dx find org apps org-demo
  Example App (example_app), v0.0.1

Grant Yourself Access to Org Objects

Give Yourself Access to any Org Project

Org admins can give themselves access to any project billed to the org by using the dx invite command. To invite yourself to a project with ADMINISTER access, use the following command, replacing project-xxxx with the project ID of the org project you want access to and user-xxxx with your username. The --no-email flag will suppress the email that would normally be sent notifying you that you have been given access to a project.

$ dx invite user-xxxx project-xxxx --no-email
Invited user-xxxx to project-xxxx (accepted)

For more information about inviting yourself to a project, use the dx invite command with the -h flag.

Grant Yourself Access to any Org App

Org admins can grant themselves developer access to any app billed to the org. App developers can publish new app versions, deprecate old app versions, and manage the group of developers and authorized users of the app.

$ dx add developers app-xxxx user-xxxx

Creating an Org

Provisioning New DNAnexus Accounts

Admins of orgs with the proper license have the ability to provision new DNAnexus accounts on behalf of the org. The user will then receive an email with instructions to activate their account and set their password.

If this feature has already been turned on for an org you administer, you can create new accounts using the dx new user command.

To create a new user and add them to your org as a member with billable activities access and set their default billing account to the org, use the following command. You will need to provide the following information:

  • EMAIL: the user's email -- there cannot be multiple accounts associated with the same email

  • USERNAME: the user's username -- must be a unique username

  • FIRST: the user's first name

  • LAST: the user's last name

$ dx new user --username USERNAME --email EMAIL --first FIRST --last LAST --org org-demo --level MEMBER --set-bill-to
Created new user account (user-xxxx)

For more information about provisioning new user accounts, run the dx new user command with the -h flag.

Org admins have the ability to create new DNAnexus accounts on behalf of the org. The user will then receive an email with instructions to activate their account and set their password.

Limited Feature Note

If this feature has already been turned on for an org you administer, you will see an option to "Create New User" when you invite a new member.

The dialog will expand to allow you to specify a username (e.g. alice or smithj), the new user's name, and email address. This dialog will automatically create a new user account for the given email address and add them as a member in the org.

Please note that if you create a new user and set their Billable Activities Access to "Billing Allowed", we recommend that you set the org as the user's default billing account. This option is available as a checkbox under the Billable Activities Access dropdown.

Enabling PHI Data Protection for an Org

Protected Health Information, or PHI, refers to identifiable health information that can be linked to a specific person. Once enabled for a project, DNAnexus PHI Data Protection safeguards the confidentiality of data stored in that project, in compliance with HIPAA.

Data Protection Notice

If the Data Protection Notice feature is enabled for a project, all users accessing the project for the first time will be required to review and confirm their acceptance of a requirement not to egress data from the project. This feature can be enabled or disabled for a project by the project administrator. For orgs licensed to use this feature, it can be enabled or disabled for a project within the Access Policies section of the project's Settings screen.

Forwarding Job Logs To Customer's Splunk

To aid in better understanding and diagnosing any issues with executions, you can have job logs (i.e. the output of dx watch) forwarded to your Splunk instance.

Enabling Forwarding for an Org

Setting Up Forwarding

Once this feature is enabled for an org, an org admin with a full-scope DNAnexus token can proceed with setup, following these instructions:

  1. To set up forwarding, use a dx update command in the following format, substituting the URL of your HEC, your HEC authentication token, and the name of the org: dx update org --job-logs-forwarding-json '{"url":"https://http-inputs-examplecompany.splunkcloud.com/services/collector/event","token":"abc11d22-de34-567f-g8h9-i012jk34lm34"}' org-exampleorg Note that a message is logged to Splunk on successful update of Splunk credentials with the following text: user-xxxx, an admin of org-yyyy is enabling DNAnexus job logs forwarding with these parameters{"url": "https://http-inputs-examplecompany.splunkcloud.com/services/collector/event", "tokenSignature": "<sha256OfSplunkToken>"}

  2. Optionally, verify your org's job logs forwarding configuration, using dx describe as follows: dx describe org-exampleorg --json --verbose |jq .jobLogsForwarding This will return details of the configuration, in the following format: {"jobLogsForwarding": { "url":"https://http-inputs-examplecompany.splunkcloud.com/services/collector/event", "tokenSignature": "<sha256OfSplunkToken>", "updated": "timestamp-in-milliseconds-from-epoch", "updatedBy": "user-xxxx" }}

Disabling Forwarding

To disable job logs forwarding for an org, use:

dx update org --job-logs-forwarding-json '{}' org-exampleorg

In addition to disabling forwarding for the org, this will record the user who has disabled forwarding, and the timestamp at which this was done, as part of the jobLogsForwarding field for the org. You can see the updated configuration details by entering a command in the following format:

dx describe org-exampleorg --json --verbose |jq .jobLogsForwarding

This will return details of the configuration, which will appear in the following format:

{"jobLogsForwarding": { "updated": "timestamp-in-milliseconds-from-epoch", "updatedBy": "user-xxxx" }}

If forwarding has already been disabled for the org, the field jobLogsForwarding will not be updated, with the API returning the code HTTP 200.

How It Works

Once the feature is set up for an org, job logs for all jobs billed to the org will be forwarded to the Splunk instance via the specified Splunk HEC.

Job logs forwarding takes place outside of the application execution environment. Org admin is responsible for the Spunk instance's ability to accept the volume of job logs generated by jobs in the org.

For cluster jobs, only job logs from the driver node will be sent to Splunk.

Forwarded Job Log Format

Forwarded job log lines will include the folowing Splunk event fields to enable efficient searching and filtering in Splunk:

  • jobId The id of the job generating the log message

  • try The try of the job generating the log message

  • project The project id in which the job is executing

  • billTo The billTo of the job generating the log message

  • launchedBy The user id of the job generating the log message

  • rootExecution The job or analysis id of the root execution of which the job is a part

  • executableName The name of the DNAnexus app or applet being executed by the job

  • entryPoint The entrypoint in the app or applet being executed by the job

  • level The log level of the jog log line such as EMERG, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG, STDERR, STDOUT or METRICS

Job log lines are truncated to 8015 utf-8-encoded bytes before being forwarded to Splunk.

Job Log Limit

Errors

Data Egress Charges

Security Considerations

The customer bears responsibility for managing Splunk users' access to, and protecting sensitive/confidential information contained in, job logs and metadata that have been forwarded to the customer’s Splunk instance as part of this feature. This includes responsibility for compliance with applicable data security and privacy regulations, such as the General Data Protection Regulation (GDPR).

Customer also owns all liability for maintaining the security and privacy of job logs from the point of egress from the DNAnexus Platform to storage within the Customer’s system.

DNAnexus is not responsible for the accuracy of the provided HEC URL.

Resource Usage

Customers who have this feature enabled agree to allocate a small amount of worker resources to send job logs to Splunk.

The job logs are forwarded from each worker to the customer's Splunk endpoint via an API request every 30 seconds, or after every 100 job log lines, whichever comes first. Unsuccessful API requests are retried three (3) times. Up to 10MiB of job logs are buffered in the worker's memory for delivery to the Splunk endpoint. If the Splunk endpoint can not keep up with the rate of job log production, some job log lines may be dropped and counted in the job's jobLogsForwardingStatus.linesDropped field. The additional overhead of managing the 10MiB buffer varies from 5 to 15MiB depending on the size of individual job log lines. Job log text is sent to Splunk as JSON strings, along with the metadata associated with each job log line.

Learn More

The video tutorials provide walkthroughs of key org admin tasks:

Last updated 6 months ago

Was this helpful?

For general information about org policies, please see the page.

To see detailed billing info for an org of which you’re a member, and within which you have :

You'll see a list of billing accounts for orgs of which you're a member, and within which you have . Information for each account is shown in a tile. This information includes the account's name and ID. For accounts with and a in place, a Remaining figure is also shown, representing the additional charges that can be incurred, before the spending limit is reached.

As an org admin, you can request a spending limit increase for the org by contacting DNAnexus Support at . Please include your user ID and attach a PO.

Licenses are required to use the Per-Project Usage Report and Root Execution Stats Report features. for more information.

This is a report issued monthly for an individual org. This report contains information on incurred compute, storage, egress, and archival costs for each project within that org, as well as any . The report is generated as a CSV file and is delivered to a destination project selected by the org admin.

If you would like to set up an org for several DNAnexus users to join, please contact us at .

A license is required to enable org admins to provision accounts for other users. for more information.

A license and a signed Business Associate Agreement are required to enable and use PHI Data Protection. for more information.

For more on PHI data protection, consult the section.

A license is required to use this feature. for more information.

A license is required to use this feature. For more information, .

Job logs forwarding is an org-level feature. To have this feature enabled for an org, .

Set up the via the Splunk web interface.

The Platform will make reasonable attempts to upload job logs to Splunk in the face of communication errors, while .

Detailed job metrics are forwarded as a "Detailed job metrics" job log lines that have detailed metrics values attached as event fields named after column headings in .

For orgs with this feature enabled, job logs are forwarded to Splunk without being subject to the imit that applies to job logs displayed by dx watch.

If an error is encountered during job logs forwarding, forwarding will be retried three times. The job's description, accessible by running , will contain the count of job log lines that could not be forwarded after three (3) retries.

Job logs forwarding incurs .

For more information, .

Organizations
support@dnanexus.com
Contact DNAnexus Sales
DNAnexus support
Contact DNAnexus Sales
Contact DNAnexus Sales
PHI Data Protection
Contact DNAnexus Sales
contact DNAnexus Sales
contact DNAnexus Sales
Splunk HTTP Event Collector (HEC)
dx watch --metrics csv
4 MiB l
data egress charges
contact DNAnexus Sales
Managing members
Removing members
Creating new accounts
limiting its use of memory and CPU on the worker
charges incurred for using public IPv4 address for workers used for compute
billing set up
spending limit
dx describe for the job
billable activities access
billable activities access