Running RStudio Shiny Server & Apps

These are descriptions and examples that demonstrate how to build and run RStudio Shiny applications on DNAnexus.

Shiny – an RStudio package – is becoming more and more popular among R language developers for ability to rapidly create web apps from R scripts. The following documents three different methods of using Shiny apps on the DNAnexus platform.

Prerequisites

To be able to successfully build apps described below, you will need to have the following already set up:

DNAnexus account
DNAnexus Platform SDK: The DNAnexus SDK (aka. dx-toolkit) allows you to develop software (primarily in Python) that interacts with the DNAnexus platform. It includes the command-line interface (CLI) client, tools for building and debugging apps, etc. You can download it here.
Docker: Docker allows you to package your code/software and data into containers that can then be run as a virtual Linux machine. Largest container repositories with lots of different container images are Docker Hub and Quay. Many bioinformatics tools have already been containerized. Their Dockerfiles and/or container images can be found at Biocontainers (website link, Github link) and Dockstore.

Source code

Full source code of the apps described in the following chapters is deposited in the Github repository in this folder:

dnanexus-example-applets/Tutorials/webapp/r-shiny-server at master · dnanexus/dnanexus-example-appletsGitHub

Method 1: Pull Everything On-the-fly

This is the simplest method and is useful if you want to create the app with the least effort. The following example will demonstrate how to create a new DNAnexus web app based on the K-means R Shiny app example, deploy it to DNAnexus, and interact with it via a web browser. It will pull everything it needs from the internet on-the-fly: an example Shiny app's R source code and Shiny Server's docker image.

DNAnexus apps can be created using a special DX app wizard that comes with the SDK. To do so, follow the steps below:

Logging in and running `dx-app-wizard`

Log into the platform and select your project using the CLI client.

dx login
# Enter your username, password
# Select the project where you want to deploy your app.

Now, to create a new app called "flying_kmeans", start the app wizard by executing the command:

dx-app-wizard

# The wizard will ask you questions to finalize the app's configuration.
# Here are some example answers:

App Name: flying_kmeans
Title []: Flying K-Means
Summary []: K-means Shiny app on DNAnexus
Version [0.0.1]:    # click <ENTER>
1st input name (<ENTER> to finish):    # click <ENTER>
1st output name (<ENTER> to finish):    # click <ENTER>
Timeout policy [48h]:    # click <ENTER>
Programming language: bash
Will this app need access to the Internet? [y/N]: y
Will this app need access to the parent project? [y/N]: y
Choose an instance type for your app [mem1_ssd1_v2_x4]:  # click <ENTER>

If you are familiar with this wizard, you can change some of the other parameters. For example, if your app requires more resources than the default server instance (mem1_ssd1_v2_x4) provides, you may elect a larger instance.

The wizard will create a folder flying_kmeans on your computer with this hierarchy:

flying_kmeans
├── dxapp.json
├── src
│   └── flying_kmeans.sh
├── Readme.developer.md
├── Readme.md
├── resources
└── test

Create Web App

To convert this app into a web app, modify the dxapp.json in the app's root folder. This is a JSON-formatted file containing the app's configuration as key-value pairs. (Docs are here.) In the file, find the place where it says:

    "version": "0.0.1",
    "inputSpec": [],

And modify it to:

    "version": "0.0.1",
    "httpsApp": {
          "ports": [443],
          "shared_access": "VIEW"
      },
    "inputSpec": [],

Adding the new httpsApp key and its values enable communication via port 443 (HTTPS). The shared_access is a string and its values can be: VIEW, CONTRIBUTE, or NONE. To access your running app, a user must have at least VIEW permission to your project. If the web app must only be accessed by whoever launched it then set the value of this key to NONE.

Keys in the httpsApp describe how and who can access the web app. The "ports": [443] means that we've enabled communication via port 443 (HTTPS; one can add more ports). The setting "shared_access": "VIEW" means that to access this web app while it is running, another user must have at least VIEW permission to your project. If the web app must only be accessed by whoever launched it, change this value to NONE. Read more about web applications here.

Create App Code

To create the app's code, use the code of the K-means example app from Github, get a Docker image of the R Shiny Server, and connect them together. A Docker image created by Rocker Project is used here. Open file src/flying_kmeans.sh. This is the Bash script that contains the function, where the app starts from - the main() function. The script is as follows:

#!/bin/bash
# flying_kmeans 0.0.1
set -eux
main() {
 # get K-means app code
 mkdir kmeans_app
url=https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example
 wget -P kmeans_app/ $url/DESCRIPTION $url/server.R $url/ui.R
 # pull and run Shiny Server docker image
 # attach our K-means app's folder as a volume
 docker run --rm -p 443:3838 -v $PWD/kmeans_app:/srv/shiny-server/ rocker/shiny
}

Copy this code, paste into your file, and save it.

Build and Deploy the App

Now build and deploy your app. In your terminal, navigate to the folder above flying_kmeans and execute command:

dx build -f flying_kmeans

Once properly built and deployed to the DNAnexus platform, the applet ID will be listed in the CLI, like this:

{"id": "applet-Fkxz86j0ffy5jfXgJXGq9732"}

The new app will appear in your DNAnexus project:

Start the App

To start the app, click on it. In the app's window, click on Run button.

The app will start launching that can be seen in the Platform's Monitor tab:

In a few minutes, the app's status will change to "Running". After a moment, a web URL will appear in the last column of the table called "Worker URL".

This means that the app is ready to accept connections. Click on the app's link. A new web browser tab will open and display the interface of your "Flying K-Means" app:

If you got "502 Bad Gateway" browser error when you loaded the page, please wait about a minute or so and reload the page. This happens when the web server part is ready and the URL is available but your Shiny app or some internal process is not yet ready to listen to connections yet.

Notes

With this method, you can quickly build apps from existing code. The caveats are that (1) your Shiny app's code must be accessible on the internet to be pulled; (2) every time you run the app, it will pull the 1.43 GB rocker/shiny docker image anew, which takes platform's bandwidth and your time, and (3) the app does not have any input files that are in your DNAnexus project.

Method 2. Multi-purpose Shiny Server

Multi-purpose Shiny servers can be created instead if users do not want to build and deploy multiple Shiny apps, or to keep dynamically pulling an app's code and Rocker/Shiny image.

Create a New Applet Using the Wizard

Just like in the Method 1, invoke the dx-app-wizard and provide the following parameters:

App Name: dx_shiny
Title []: Shiny Server
Summary []: Shiny Server on DNAnexus
Version [0.0.1]:    # click <ENTER>

Unlike in the previous method, configure the app to have an input:

1st input name (<ENTER> to finish):  app_gz
Label (optional human-readable name) []: Gzip archive of my Shiny app
Choose a class (<TAB> twice for choices): file
This is an optional parameter [y/n]: n

Your new applet will be created in the dx_shiny folder.

Convert it to a web app

As before, to convert it to a web app, open its dxapp.json and add:

"httpsApp": { "ports": [443], "shared_access": "VIEW" },

Enable Input File Validation

To validate that the input files have .gz extension, change the value of the "patterns" list from "*" to "*.gz" under the "inputSpec" key so that it looks like this:

"inputSpec": [
    {
      "name": "app_gz",
      "label": "Gzip archive of my Shiny app",
      "class": "file",
      "optional": false,
      "patterns": [
        "*.gz"
      ],
      "help": ""
    }
  ],

Save the dxapp.json file.

Create Main App Script

Now modify the src/dxshiny.sh script. Here is the content:

#!/bin/bash
# dxshiny 0.0.1

set -eux

main() {
    # Download the input: your Shiny app's source code archive from the project
    echo "Value of app_gz: '$app_gz'"
    dx download "$app_gz" -o app.gz
    
    # Unpack the app code. Here, `--strip 1` allows to skip the top folder
    # of the archive and unpack the code directly underneath ./app/
    # This allows you to go to the job's URL and launch your Shiny app directly.
    # Otherwise, the code would be one folder level deeper and one would have to
    # click on the app folder's link in the browser.
    mkdir app
    tar -zxvf app.gz -C app --strip 1
    
    # Set up the Shiny Server
    # Assuming that the archive of the docker image of the Shiny Server is
    # in your project at /rshiny/rshiny.docker.gz
    dx download $DX_PROJECT_CONTEXT_ID:/rshiny/rshiny.docker.gz
    # ($DX_PROJECT_CONTEXT_ID contains your project ID)
    # Now load the image from the archive, create docker image
    docker load -i rshiny.docker.gz
    # Run the docker image. Attach your app's folder as a volume.
    docker run --rm -p 443:3838 -v $PWD/app:/srv/shiny-server/ rocker/shiny
}

Essentially, this downloads the Shiny app's source code archive -- a single input that this app will require. This also downloads the archive of the Shiny Server's Docker image, loading and running it. Importantly, this allows the attachment of the Shiny app's code as a volume to the Shiny Server's Docker container.

Build the Applet

Navigate to the parent of the dx_shiny folder and run:

dx build -f dx_shiny

The new applet dx_shiny should appear in the current directory of your DNAnexus project.

Archive and Upload the File Archives

The app requires two file archives: (1) the K-means Shiny App, and (2) Shine Server's docker image.

Create and Upload the K-means App's Archive

Run the following command in the terminal:

mkdir kmeans_app 
url=https://raw.githubusercontent.com/rstudio/shiny-examples/master/050-kmeans-example
wget -P kmeans_app/ $url/DESCRIPTION $url/server.R $url/ui.R
tar -zcvf kmeans_app.gz kmeans_app

This will download all three files of the app to your local kmeans_app folder and archive it. The new archive will be called kmeans_app.gz.

Upload the File to the Platform

dx upload kmeans_app.gz

Pull the Docker image from Docker Hub:

docker pull rocker/shiny

Save it to an archive file:

docker save -o rshiny.docker.gz rocker/shiny

Now upload it to the platform:

dx upload rshiny.docker.gz

Let's assume your current directory in the DNAnexus platform is /shiny. After these operations, you should have three objects in the folder: archive files kmeans_app.gz, rshiny.docker.gz, and the dx_shiny applet.

Run

To test if everything worked, run the applet. You can run it from the UI by clicking on it, providing the kmeans_app.gz archive as an input.

It should look like this:

Click on the Run as Analysis… button.

Alternatively, start your app with this command:

dx run dx_shiny -y -i app_gz=kmeans_app.gz

As before in Method 1, the job will soon start and provide the URL to access it. Open it to see the familiar interface of the "K-means Shiny" app.

Notes

This method allows you to create a DNAnexus applet for Shiny Server. It can take an archive of a Shiny app code and run it. You can run different Shiny apps on this server, which makes it a versatile tool. There are caveats, however, the most obvious and important ones being that this applet does not take any input files from its DNAnexus project, neither can it store results back to the project. These can be addressed by adding additional input and output slots during the creation of the app, like for the app_gz input.

Method 3: Mount Your Project as a Folder

Many Shiny apps out there will require one or multiple input files. They may also need to save some results as files. Shuttling files back and forth from your project could be done by adding inputs and outputs to your app.

dxFUSE (currently in Beta mode) allows you to mount one or more DNAnexus projects to your web app as a folder. The Shiny app can then open project files as if they were local ones, and would save outputs the same way, and be agnostic that these files are physically stored elsewhere.

Create a New Applet

1. Create New Applet

Invoke the dx-app-wizard and provide the the same answers for the app mentioned in Method 2, except for these sections, which will be the following:

App Name: fused_shiny
Title []: dxFUSE Shiny Server
Summary []: Shiny Server with mounted DNAnexus project

2. Modify the Main App Script

Now modify the fused_shiny.sh; add code to download the dxfuse tool, set it up, mount your DNAnexus project as a folder, and then mount this folder to Shiny Server's Docker image. The code is a modification of the script from dx_shiny app. Not all the code is repeated, but the snippet of interest is as follows:

# Mount the parent project using dxFUSE
wget https://github.com/dnanexus/dxfuse/releases/download/v0.21/dxfuse-linux
...
FUSE_MOUNT=$HOME/projects
mkdir -p $FUSE_MOUNT
sudo -E ./dxfuse-linux -uid $(id -u) -gid $(id -g) -verbose 2 $FUSE_MOUNT $DX_PROJECT_CONTEXT_ID
...
...
docker run --rm -p 443:3838 -v $PWD/app:/srv/shiny-server/ -v $PROJ_PATH:/srv/project/ rocker/shiny

The above command downloads the dxfuse tool and sets it up. It also dxfuse-mounts the DNAnexus project as a folder, and then mounted this folder to the Shiny Server's Docker image.

There are many different Shiny apps and lots of different use cases, so the methods described above should be viewed as a jumping off point for your exploration.

PreviousR Shiny Example Web App NextTensorBoard Example Web App

Last updated 7 months ago