Job Notifications

Learn how to set job notification thresholds on the DNAnexus Platform.

A license is required to use the functionality described on this page. Contact DNAnexus Sales for more information.

Being notified of when a job may be stuck can help users to troubleshoot problems in a timely manner. On DNAnexus, users can set timeouts to limit the amount of time their jobs can run, or set a threshold on how long a job can take to run before the user is notified. The notification threshold can be specified in the executable at compile time via dx or dxCompiler.

When the threshold has been reached for a job tree, the user who launched the executable and the org admin will receive an email notification.

Setting Thresholds From the Command Line

For a root execution, the turnaround time is the time between its creation time and the time it reaches the terminal state (or the current time if it is not in a terminal state). The terminal states of an execution are done, terminated and failed. The job tree turnaround time threshold can be set from the dxapp.json app metadata file using the treeTurnaroundTimeThreshold supported field, where the threshold time is set in seconds. Note that when a user runs an executable that has a threshold, only the resulting root execution will have that threshold applied to it. See here for more details on the treeTurnAroundTimeThreshold API.

Example of including the treeTurnaroundTimeThreshold field in dxapp.json:

{
  "treeTurnaroundTimeThreshold": {threshold},
  ...
}

In the command-line interface (CLI), the dx build and dx build --app commands can accept the treeTurnaroundTimeThreshold field from dxapp.json, and the resulting app will be built with the job tree turnaround time threshold from the JSON file.

To check the treeTurnaroundTimeThreshold value of an executable, users can use dx describe {app, applet, workflow or global workflow id} --json command.

Using the dx describe {execution_id} --json command will display the selectedTreeTurnaroundTimeThreshold, selectedTreeTurnaroundTimeThresholdFrom, and treeTurnaroundTime values of root executions.

WDL Workflows

For WDL workflows and tasks, dxCompiler allows users to specify tree turnaround time using the extras JSON file. dxCompiler uses the value of the treeTurnaroundTimeThreshold field from the perWorkflowDxAttributes and defaultWorkflowDxAttributes sections in extras and passes on this threshold to the new workflow generated from this file. To set a job tree turnaround time threshold for an applet using dxCompiler, add the treeTurnaroundTimeThreshold field to the perTaskDxAttributes and defaultTaskDxAttributes sections in the extras JSON file.

Example of including the treeTurnAroundTimeThreshold field in perWorkflowDxAttributes:

{
  "perWorkflowDxAttributes": {
    {workflow_name}: {
      "treeTurnaroundTimeThreshold": {threshold},
      ...
    },
    ...
  }
}