Load Testing Shiny Applications

Load testing helps developers and administrators estimate how many users their application can support. If an application requires tuning, load testing and load test result analysis can be used to identify performance bottlenecks and to guide changes to infrastructure, configuration, or code.

It’s a common misconception that “Shiny doesn’t scale.” In actuality, properly-architected Shiny applications can be scaled horizontally, a fact which Sean Lopp was recently able to demonstrate at rstudio::conf 2018. We used shinycannon to simulate 10,000 concurrent users interacting with an application deployed to AWS. You can see a recording of Sean’s talk and the load test demonstration here: Scaling Shiny

To get started with shinyloadtest, read through the quick start guide below.

Analysis Example

Installation

To perform a load test you’ll need two pieces of software: shinyloadtest and shinycannon.

shinyloadtest is an R package used to generate recordings and analyze results. You should install it on your development machine.

shinycannon is the command-line replay tool. You can install it on your development machine for testing, but for best results we recommend installing it on a server, and preferably not the one the application under test is also on.

You should substitute https://shinyapp.example.com/ for the URL of the Shiny app you’d like to record. If your application requires authentication you should consult the [authentication article][authentication]. You should also be aware that [certain Shiny features][limitations] are not compatible with shinyloadtest.

Running the function will open a browser displaying the app. Once open, interact with the application as a normal user would and then close the browser. After closing the browser window or tab, a file will be created that contains a recording of the session. This recording, written to recording.log by default, will serve as the basis for the load test.

Step 2: Run the Load Test

With the recording in hand, we’re ready to run the load test. The actual test is conducted outside of R using the shinycannon command-line tool. You can run it using your system’s terminal or console program, or you can run it from the RStudio terminal tab.

In addition to these two required arguments, shinycannon accepts a number of optional arguments that can be specified with flags. Of these, the most interesting are:

–workers: The number of concurrent users to simulate. shinycannon uses threads to represent each user. It defaults to 1.

–loaded-duration-minutes: The duration of the load test, in minutes. This does not include “warmup time”, which is the time shinycannon spends gradually increasing the number of workers, or “cooldown time”, which is the time spent decreasing the number of workers. It defaults to 0, meaning that after all workers have “warmed up”, they will immediately begin to “cool down”. Generally, you’ll want to set this to a duration greater than 0. shinycannon will re-run the recording as necessary to fill the time.

–output-dir: Name of the directory to create and store timing information in.

As an example, to run a load test simulating 5 concurrent users for at least 2 minutes, outputting to the directory run1:

During the test, shinycannon reports the progress and number of simulated users. The result of the test is an output directory (run1 in the example above) which includes timing information for each session.

Step 3: Analyze the Results

The output directory created in step 2 (run1) is the focus of step 3. The shinyloadtest R package provides functions for analyzing the output files from the test.