Release Notes
Get details on new features, changes, and bug fixes for each Platform and toolkit release.
- Refactoring of the asynchronous archive services
- Refactoring of the job re-use and support workflow (e.g., WDL)
- Apollo: Refactoring of the geno filters for the command line interface
- Security patching for Docker and VMs
- Refactoring of job restart, asyncrhonous archival and UI backend changes
- Apollo: Spark server upgrades, Python upgrades
- Security patching on Docker containers and lambdas
- Fixes for job preservation functionality
- Security patching for Docker container, lambdas and Ubuntu instances
- Fixes to the Titan file uploader and cosmetic changes to the UI in the input components
- Apollo: depracation of the Association Browser
- Monthly security patching for Ubuntu instances, patching of vulnerabilities for Docker containers and AWS lambdas
- Migration of platform service instances from Ubuntu 18.04 to 22.04 or 20.04 when not possible to fully migrate
- UI fix to SSO users who are confronted with separate MFA prompt
- Apollo: fix issues in dxfuse
- Granular wait time for spot instances for workflows (subscriber licensees only)
- Docker container security patching
- Refactoring of job refuse to support workflows (e.g., WDL)
- Refactoring of known issues in Titan including s3deleter, spot granlarity issues, expansion of the metadata capacity in the describe API call, UI fixes to SSO users
- Refactor of known issues in the initial login of the thrift cluster, more granularity in log reporting
- Patching of Docker container vulnerabilities
- Titan: Fixes to the UI and s3deleter functionality
- Apollo: fixes to enforcing region boundaries and API proxy issues
- Workflow Job Reuse will be deprecated completely, replaced with functionality elsewhere
- Change GiB to GB e in the job disk comparison
- Added new functionality to Titan, specifically:
- DNA_API_PROVIDER_PERMISSION_RESTRICTIONS - Project/org restrictions should be enforced based on the provider constraint options choice
- DNA_API_PROVIDER_DATA_CLONING_RESTRICTIONS - User is not able to clone data from restricted to unrestricted project or from restricted to restricted project with different studyID
- DNA_API_PROVIDER_JOB_RESTRICTIONS - Restriction on building tools into the project from the user's computer or analyses
- FastQC app (v2.2.3 and earlier) is deprecated
- NextFlow - GA to a limited set of users
- Added new functionality to Apollo, specifically:
- TIP_DATA_ACCESS_PROJECT_RESTRICTIONS - The system can limit users access to data based on project settings and license
- TIP_DATA_DISPENSAL - User is able to create project containing data specified by studyID
- New Titan Functionality
- DNA_API_INTERNET_JOB_RESTRICTIONS - Data providers are able restrict the internet access for jobs running in Trusted Research Environment (TRE) projects in order to prevent egress of sensitive data from the TRE
- DNA_API_EXTERNAL_UPLOAD_RESTRICTIONS - Project admin is able to restrict users ability to upload data into trusted research environment (TRE) projects outside of the jobs running in the TRE
- DNA_GUI_EXTERNAL_UPLOAD_ACCESS_RESTRICTED - User is not able to upload files into the External Upload restricted project
- DNA_GUI_EXTERNAL_UPLOAD_ALLOWED_FILE_UPLOAD - User is able to upload files into the External Upload allowed project
- DNA_GUI_EXTERNAL_UPLOAD_PROJECT_SETTING - User can change External Upload Access setting
- DNA_GUI_JUPYTERLAB_EXTERNAL_UPLOAD_RESTRICTED - External upload options are not available in JupyterLab job running over the External Upload restricted project
- DNA_API_OFH_PROJECT_CREATE - User is able to create new project with data dispense or without data dispense option
- DNA_GUI_OFH_PROJECT_CREATE - User is able to create new project with data dispense or without data dispense option
- DNA_GUI_OFH_CREATE_ACCOUNT - User is able to create new account via OFH Landing page
- DNA_GUI_OFH_USER_ASSOCIATION - User is able to make association of DNAnexus and AMS account
- DNA_GUI_OFH_USER_2FA - User is prompted to set 2FA and it is not possible to remove it
- DNA_GUI_OFH_USER_BILLING_INFO - User has XX GBP billing trial balance and default region is Azure OFH-TRE (London) after successful association
- G5 GPU workers available in specific cloud
- Apollo bug fixes: Fix for license and region checks
- New Apollo Functionality
- TIP_JOB_INTERNET_RESTRICTION - The system is able to restrict internet access in worker jobs
- TIP_EXTERNAL_UPLOAD_RESTRICTIONS - Upload restricted projects have limited access
- Titan bug fixes: Updates for the API server and Azure Cloud Manager. Fix of granularity for spot instances, fix of race condition for hard links updates; archival state fixes
- Docker Container vulnerability patching
- ​DNAnexus is on AWS' Marketplace, so if you have AWS-issued credits, you can consume them via this mechanism!
- Vulnerability patching of worker AMIs and Docker containers
- Fixes to the external uploader restrictions, code cleanup
- Apollo: Fixes to the Visualisation Server and API server. Cleanup of error messages
- Vulnerability remediation of Docker vulnerabilities
- Completion of the annual SSL cert rotation
- Apollo: fixes to the metastore, fixes to the clipboard functionality in the visualization module and other bugfixes
- Complete migration to new SSL certificates throughout the infrastructure
- Bugfixes for billing, upload restrictions and SSO logins will no longer be redirected to the Project Page
- Apollo: fixes to encryption of upload URLs in the API proxy and other minor fixes
- Security patching of the Titan User interface
- Upgrading upload restrictions at the project level
- Patching Docker containers to address new vulnerabilities
- Apollo: updating SSL certs and refacting app interactions
- Docker security patching and hardening
- User Interface security patching
- Apollo: bugfixes, library updates, security patching
- Update OpenSSL v3 to 3.0.7 patch level
- Patching Docker containers and CIS level 1 hardening
- Fixes of known Titan issues
- Apollo: bugfixes to known issues.
- Code optimization, bugfixes in the User Interface
- Vulnerability patching for docker containers
- Apollo: fix API proxy issues, regex fixes for routes and other fixes.
- Security patching of operating system and Docker containers
- Additional Nextflow functionality added to Titan and Apollo
​
- Bugfixes to Titan and Apollo
- Docker vulnerability patching
- Fix for 2FA prompt for existing SSO accounts
- Fix for Jupyter Notebook calls
- Fix to block archiving of object linked to applets
- Vulnerability patching of Docker containers
- Apollo: upgrades to the internal cache and related internal upgrades
- Filecloser, asynchronous archival function, and API proxy bug fixes
- Internal upgrades of the metadata database
- Apollo: fixes to the cohort browser and fixes to internal API routes
- End-to-end FIPS 140-2 validated ciphers within the FedRAMP boundary
- Bugfixes to the authentication server
- Apollo: bugfixes to known issues
- Hardening of routes behind the User Interface
- Vulnerability patches to the Docker containers
- Apollo: Additional granularity to the logging and billing. Cleanup of error messages.
- Added Monitoring and Restriction Controls regarding download and egress, ability to preview, IP addresses of users, etc.
- Bug fixes on streams containing gzipped files
- Apollo: Fixes to the 30K cohort browser encoding
- Complete migration to OpenSSL v3
- New functionality to restrict IP domains for certain API calls
- Updates to the NVIDIA drivers for GPU users
- Increase the partition sizes for LXC
- Security patching of Docker Containers
- Apollo: Restrictions to downloads from Apollo
- Security patching for Docker Container vulnerabilities
- Usage report fixes
- Hardening of routes in the User Interface
- Increases in timeout period for dxproxy
- Apollo: enhanced CLI based querying, fixes to HDFS filesystem
- Upgrade the metadata database to Mongo 4.4
- Upgrade node.js to v14
- fixes to Archiving in AWS regions
- Apollo: no release
- LXC filesystem limit increased
- Monthly security patching
- Upgrade for SSO users who switch accounts
- Apollo: CLI based data querying
- Security patching of Docker containers
- User Interface self-service billing functionality
- Improved clinical data ingestion experience
- Apollo: Refactoring of VizServer
- Fix of the "create" functionality of DBCluster (AWS deprecated previous engine on 25-Jun-2022)
- Refactoring of user interface issues
- Patching security vulnerabilities of EC2 and Docker containers
- Refactoring issues of the cloud manager on Azure
- Apollo: Enhance validations and experience around ingesting clinical data
- Internal refactoring of reported issues
- End-to-End FIPS 140-2 validated encryption throughout Titan
- Vulnerability patching of Docker containers
- Apollo: refactoring user interface issues
- Vulnerability patching for images and Docker Containers
- Titan new functionality: DNA_API_ORG_REPORT_HISTORICAL_CHARGES
- Apollo new functionality:
- TIP_VIZ_COHORT_COMBINE
- TIP_APPS_COHORT_COMBINE_REBASE
- DNA_GUI_COHORT_COMBINE
- Apollo: fixes on Azure deployment; upgrades to the Spark engine.
- Updates to the internal CloudManager and fixes to microservices
- Vulnerability patching of images and Docker containers
- UI fixes
- Refactoring of multiple analyses workflows, more info on the spending limits and updates to the cookie notice
- Security patching of the underlying operating systems and worker AMIs
- Apollo: no release
- FIPS 140-2 encryption implemented for uploading/downloading from the storage plane
- Apollo: no release
- Minor fixes and normal vulnerability patching of the User Interfaces and supporting libraries
- Apollo: minor fixes, including the ability to query without a primary key
- Security updates for the infrastructure
- Backend fixes to support upgrading the metadata database software version upgrades
- Apollo: Enhancements of existing functionalities to ease the creation of mixing multiple cohorts
- Restrictions for SSH functionality
- Refactoring of known issues related to container cleanup, caching, etc.
- Apollo: Ability to export up to 200 columns in a table; UI fixes
- Tuning to optimize workflow runs
- Updates to the documentation pages of the applications
- UI fixes to show app versions in descending order, other minor UI fixes
- Apollo: refactoring of known issues in the user interface
- Order pending jobs by rank
- Node.js updates
- Tuning of the thrift server throughput; facilitate the combination of cohorts in the cohort browser.
- Fixes to known issues, such as a fix to the scope of unarchiving to better control the scope, updated the internationalization library (fixes for text fields in the UI).
- Apollo: Refactoring of issues with field mapping and support for ordered fields.
- Refactoring the User Interface to address issues, including moving the workflow input to the Project workflow pages.
- Fixes to the unarchiving functionality, tighter control of scope of the unarchiving
- Apollo: Refactoring of issues arising from the visualization APIs
- Refactoring known issues in the dbcluster, internal changes to FIPS 140-2 validated ciphers, Cloud Manager, fixes to the app updater when archive resources are involved, etc.
- Updates to community portals - caching is now disabled
- Apollo: Refactoring of the issue involving model input.
- Vulnerability patching of operating systems
- Fixes to code compatibility to rush.js
- Support ticket generation will be tagged with the Platform page of origin
- Apollo: Enhanced support for bulk RNAseq data in a dataset
- Apollo: Security and version number upgrades of servers
- Apollo: Removal of autoloading a chart, so the user will be prompted to generate the chart
- Implemented FIPS 140-2 validated endpoints for archival functions
- Bugfixes, such as issues reported in unarchiving function and email org_invite function
- Updating reCaptcha functionality
- Apollo: Added functionality to allow a user to incorporate Molecular Expression data into a dataset
- Apollo: Fixes to issues in Thrift and VizServer.
- Update to Node.js v12 for the infrastructure
- Backend updates to rush.js version in some of the lambdas, revising metadata sharding approach, updates to s3_deleter and minor bugfixes to the User Interface
- Apollo: Address known bugs, including limiting the number of tiles in the dashboard to a maximum of 15
- Bug fixes on job restarts, rush.js updates and increased granularity of data in the audit trail
- Apollo: in addition to bugfixes, the following functionality was added: TIP_APPS_CLINICAL_DATASET_MERGER - Users will be able to merge phenotypic datasets and TIP_APPS_ASSAY_DATASET_MERGER - Users will be able to merge an assay dataset and a phenotypic dataset
- Update of security patches of identified vulnerabilities
- Spend limit enforcements
- Enhancements for job filters
- Updates to the email filters
- Apollo: fixes to address solution-exception messages, fixes of the update API calls, and removal of blank pages in the User Interface
- Service Provider Single Sign On (SSO)
- User Interface fixes
- CloudManager enhancements (internal)
- Apollo: updates and fixes of reported issues
- Fixes to Cloud Manager, visibility into the org license for the org-admins
- Remove all Ubuntu 16.04 instances from production. Ubuntu 16.04 is available at the app level
- UI fixes to the monitor pages; improve state transition with regard to Jupyter Notebooks
- Apollo: refactoring known issues, such as nametags, etc.
- User Interface fixes to cancel transfers
- Fixes for the initiation of nested workers and the API call for apprun worker limits
- Monthly Security patches
- Apollo: fixes for zero length files, security patches to packages and libraries
- Button changes on the Web User Interface
- Throttling renabled, refactoring of known issues
​
- Complete migration of the UI to React 17 with UI changes to the Data manager, removal of the Global Search button
- Refactoring of known issues, such as fixes to watermarks, etc.
- Additional metrics for spinup and interval time for instances
- User Interface infrastructure updates to React 17
- Infrastructure consolidation for load balancing
- Security patching
- Apollo: refactoring of known issues
- Org Admin can view each org member's userID and email address
- Refactoring minor bugs, including fixing timoues and fixes to allow additional flexibility for UKB dispensals
- Apollo: Refactoring known issues
- UI fix to https workers
- Refactoring of many reported UI issues
- Refactoring of unarchive issues
- Org Manager can access the org user's email addresses
- Refactoring of the UKB rate card
- Self-service billing for UKB RAP and PayGo customers only
- Apollo: no release
- Monthly Security Patching
- Deprecation of Global Search
- Apollo: no release
- Refactoring of CloudManager
- Additional heartbeat monitoring of the infrastructure
- Refactoring of user interface issues
- Apollo: Refactoring of known lissues and updating the version of tRNA to feed Apollo
- Changes to timeout on instance re-use
- Updates to the billing daemons
- Updates on data withdraw and refresh process
- Moving infrastructure components to node.js v12
- Apollo: Update of Spark to support v3.1.2 and fixes to database pointers
- Apollo available in AWS-Frankfurt
- Additional logging for cluster jobs
- Added additional time for reserve instances
- Efficiency enhancements for listing projects
- User Interface fixes to reported bugs
- Chef upgrade on the infrastructure
- Apollo: Deprecation of the legacy chort browser, efficiency enhancements of Spark and other fixes.
- Updates to Job Manager and Cloud Manager
- Refactoring of issues on the User Interface
- Apollo: refactoring of filter parameters, UI and Spark 3.x upgrades
- Audit logging updates
- Refactoring of the User Interface issues
- Sendgrid changes for event-based email notifications
- Apollo: refactoring reported issues
- London region archiving
- Refactoring of emall alerts
- Code efficiencies pertaining to audit trail generation
- Efficiencies added to the backend code to allow more efficient list of projects.
- Apollo: refactoring known issues, efficiency enhancements and deprecation of the legacy Cohort browser
- Cloud Manager caching implemented in regions that use autoscaling
- Refactoring of internal database to increase efficiency
- Opening AWS region in London, will be visible on the user interface and rate cards in UK currency
- Internal Splunk updated to v8.1.5
- OpenJDK v11 updated on Ubuntu 18.04 AMIs
- Apollo: No Release
- Switch to the Pannexin User Interface only
- Data master fixes in the User Interface
- Query optimizations
- Internal migration of the org-specific audit logs to a more efficient process
- Apollo: Refactoring of error messages, UI fixes to gene filter parsing an semiphore fixes for API calls
- Reset job fix
- jQuery update
- Master Daata UI fixes
- lambda functions update
- increase Cloud Manager Resilience
- Apollo: Refactoring known inssues, including updates for code efficiencies and documentation
- Restart job fix
- Internal update of jQuery to v3.6.0
- Monthly vulnerability patching implemented in production
- Apollo: Refactoring of known issues, including some minor refactor of UI for efficiencies
- Additions to ProjectDescribe API to include ranking field
- Visualization tab of the User Interface will be migrated to the newer version (Pannexin)
- Job Manager efficiences
- Apollo: Added TIP_VIZ_COHORT_ANDOR, DNA_GUI_COHORT_ANDOR_JOIN_FILTER, and DNA_GUI_COHORT_ANDOR_FILTER_GROUP functionality
- Archive API updates
- Platform refactoring: Fixes to the User Interface and error handling when project files do not exist
- UUbuntu 12.04 and 14.04 instances will be deprecated. Workers using Ubuntu from 16.04 to 20.04 will continue to be supported.
- Apollo: Refactoring of R functionality and fixes to license checks
- Updates to the billing page
- Fix to the GetDetails call, handling of incorrect API calls and code efficiencies
- Updating API servers to Ubuntu 18.04, security patching of infrastructure
- Fixes to getFileStatusURL, verify all filter values in an array, fixes in the data picker, fixes in the histogram functionality
- Upgrade of the dependencies of the Auth Server
- Titan bugfixes
- Org Page UI changes
- Apollo: UI fixes
- Refactoring for handling regions, fixes on cost limits, fixed prioritization and long-term fix for watermarking
- Egress Reporting
- Apollo: refactoring issues
- Apollo: TABBY release - moving sections to multiple tabs
- Fixes on Rush.js project dependencies
- Correction of defects in the JobManager and JobEvents
- DBCluster updates to the latest TypeScript
- React.js library upgrade, which may have slightly changes to the visual appearance as well as possibly behavior changes
- Corrections to UserDescribe functionality
- Fixes FileStatus calls, changes for efficiency of filesystem calls, tuning cache sizes
- Minor AMI changes for logging
- Revert of egress accounting change and some watermark functionality
- Additional information from file describe on watermarked files
- Functionality to allow org-admins to share projects across orgs
- dxcompile support for bash variables
- Ubuntu 14.04 depracated
- Apollo: Fixing the total count number when multiple alleles are involved, performance enhancement on the underlying database access methods
- Job re-use with watermarking
- Cloud Manager updates
- Increase granularity in egress billing
- Apollo: fixes of known issues, such as retry logic on read conditions and efficiency changes in the User Interface
- Updates to the DBcluster monitoring process
- Re-deployment of dl-proxy modules to support ungraded infrastructure
- Application context ID now included in the profiled projects, workers handle watermarked APIs, etc.
- Implementing monthly vulnerability patching
- Apollo: Fixes of known issues within Apollo, including data filter conditions and fixes to the Locus filter
- Refactoring of watermarking code to address issues
- Refactoring of code performing bulk calls to reduce the volume of information returned
- Updated the user experience (UI) with a new root component error page. Formerly, an error would show a blank screen when the worker crashed.
- Implement Apollo on Azure West-US
- DBCluster services were updated to be more efficient.
- Okta updates on dependencies
- Fixes to the autoscaling of the Cloudmanager infrastructure
- Fixes to the timestamps on expired/destroyed containers to integrate correctly in the metadata database
- Expanded IP range on workers
- PHI flag implemented on Azure
- Implemented the 2FA authentication in the pannexin code (formerly only on the membrane UI code)
- Apollo: Updates to the metadata connect functionality
- Apollo: Fixes with the concurrency issues with API calls
- Apollo: Efficiency updates to reduce the number of API calls
- Fix to allow UKB RAP users to download from multiple resources
- Continued rollout of v6 to v12 upgrades to node.js in the APIproxy and CloudManager infrastructure
- Updates to allow the cloud manager to handle tagged AMIs
- Efficiency changes to the APIserver from the Monitor tab. UI fixes to the cohort browser where screen disappears and table crashes
- All AMIs on AWS will be Ubuntu 18.04. AMI updates on Azure will come later. Older AMIs will continue to be supported
- Apollo: Updates to the Vizserver to more efficiently use the API server and efficiencies implemented on the monitor badges.
- Apollo: Resolved access issues to non-DNAnexus S3 buckets.
- Apollo: Added DXDATA_CREATE_COHORT functionality to the product.
- Fix to userDescribe functionality
- Fix to DBClusters
- Okta SCIM update
- Vulnerability patching
- Refactoring of SSO token exchange
- Apollo: Resolving issues accessing data in public S3 buckets, fixes for the data filters, and increased the efficiency of displaying tables in the User Interface
- Refactoring user describe functionality to provide greater visbility to the org-admin
- Refactoring of minor User interface bugs
- Apollo: Minor fixes to the filters
- Refactoring of the watermarking and UKB withdraws
- Refactoring of User Interfaces, including multiple wildcard support
- Apollo: refactoring of the Azure filesystem support and entity relationship filters
- Refactoring of functional issues related to: update job pricing, updating autoscaling of cloud manager, fixes to the event emitter components, updates to analysis describe to get increased mapping, increase org-admin ability to assign roles within their role, and efficiency fixes to the watermarking API calls.
- Performance refactoring such as deletes on autothrottled instances, fixes of updates to update describe for field mapping to the User Interfaces as well as removing deprecated instances out of the code base.
- User interface currency awareness
- User interface capability to upload directly from S3 and Blob storage
- Apollo: refactoring of issues arising from Azure filesystemRelease for 09-Mar-2021
- Supporting GPU and high memory instances on Azure
- Apollo: Now available on Azure (US-West)
- Refactoring of Titan to fix hanging and uassigned cluster jobs, etc.
- Updates to the dependencies of the Okta-SCIM servers
- Bugfixes to projectDescribe
- Expanding the ability to CLI ability to lower a job's priority to the UI
- Apollo: fixes to configuration tables on Azure; when a configuration table is incomplete, it is no longer flagges as an error.
- Failed job restart fixes
- Implemented the autoscaling Cloud Manager in the AWS-London region for UKB RAWP
- Increased the granularity of reporting of Worker egress and access to heartbeat frequency
- Restricted ability to change addBilltoRegion
- File closer efficiency for files >1TB
- New worker types in Azure
- UI refractoring of bugs
- Fix issues with orchestration scripting for running cluster jobs, bare SQL supported
- Cloud Manager on Azure updated for cluster jobs
- Fixes on UI components to limit length of org-names, etc.
- Apollo: Cosmetic fixes to Scatterplot and rerfactoring filters in the query builder. Added "is-not" filter condition
- Minor changes to org-design descriptor
- Additional logging info for terminated spot jobs on AWS
- Fixes to org-describe API to identify project calls for UKB RAP
- Node.js updated to v12 for archiving
- Apollo: Fixes to close database command and readfile cache. Added efficiencies for field selection UI activity
- User Interface fixes
- Monthly security vulnerability patches, including the sudo vulnerability in Ubuntu
- Updates to node.js v12 for Execserver
- Apollo: fixes to vizserver and UI of cohort browser
- Additional enhancements to gather information from job fails to be captured in the audit log.
- Graphic redesign of the download proxy (allow folders and multiple files (up to 100) to be downloaded)
- Moving primary Authentication Server to Ubuntu 18.04
- Apollo: R-based Apollo Dataset SDK
- Apollo: Completion of filters
- Apollo: User interface efficiencies
- Backend fixes to DBCluster
- UI fixes to project transfer flow
- UI vulnerability fix for PRSSI
- Monthly updates to address infrastructure vulnerabilities
- Added one more access priv to HTTPS-worker (fix)
- Limiting Customer Cloud Account to specific API calls, etc.
- Backend changes to cloning files from the archive
- UI fixes to datamanager
- Apollo: refresh of apps on Apollo to address bugs
- UI refactoring to address 3.0 re-route issue and a fix to allow the batch input selection
- Change to address email notifications when there are failed jobs. When massive (e.g. >thousand of more) fail from a workflow, email was sent for every job failure. This behavior will change to a limited set of emails focusing on the parent jobs only.
- Health checks on the internal Cloud Manager
- Transition of internal servers to node.js v12
- UI changes in for UK BioBank users
- Apollo: adding existential filter
- Updating Sendgrid emails (job status emails are processed via sendgrid)
- Updating SSL credentials for 2021
- UI fixes to allow the user to switch between cohort browsers
- UI fix for the processing of long and complicated workflows/pipelines
- Fix of job manager to accomodate detached jobs from appropriately licensed customers
- Add access to Jupyter Notebook Previewer via the UI
- Apollo: refactoring of the rsid filter
- Apollo: Allow for version switch to support Apache Spark V3.0.1
- Fix for CanSeePrices to check the currency type (USD or GBP)
- Fix of issues in the download proxy
- Fix of the withdraw service for UK BioBank
- Efficiency and performance enhancements for job re-use functionality
- Update logic for price currency (USD and GBP)
- Adding functionality: DNA_API_RDS_CREATE_DBCLUSTER, DNA_API_RDS_TERMINATE_DBCLUSTER, DNA_API_RDS_USE_DBCLUSTER for Titan
- Jupyter Notebook file viewer
- Apollo: Minor refactoring of the filters