I am creating a fairly large shiny app which allows users to create multiple time series plots using the dygraphs package. The users can dynamically select the variables to be plotted and several other display settings. This results in frequent replotting of the dygraphs. For this topic I have simplified the app to selecting the variables to be plotted. The issue is that there appears to be a memory leak using the dygraph package in combination wisth shiny. I have also found that the memory build-up is not in R itself.
First attempt
The code below is my first attempt where I use the uiOutput/renderUI/tagList combination to create the plot element for the output. If you use the Web Inspector to monitor memory usage you can see that it never decreases when (de)selecting variables. Eventually the app crashes (in my application I actually use data by the minute so after around 10 selections this occurs on my machine).
library(shiny)
library(dygraphs)
library(xts)
library(pryr)
used_mem <- function() {paste0(round(mem_used()/1e6, 3)," mb")}
# Data -----
time <- seq.POSIXt(from = ISOdatetime(2018, 9, 1, 0, 0, 0, "Europe/Brussels"),
to = ISOdatetime(2018, 12, 31, 23, 59, 59, "Europe/Brussels"),
by = "10 min")
dat <- xts(x = data.frame(a = rnorm(length(time)),
b = rexp(length(time)),
c = runif(length(time))),
order.by = time,
tzone = "Europe/Brussels")
# UI -----
ui <- fluidPage(
selectInput("vars", "Select variables", choices = names(dat), multiple = TRUE),
uiOutput("plot"),
textOutput("text")
)
# Server -----
server <- function(input, output) {
counter_plot <- counter_text <- 1L
output$plot <- renderUI({
validate(need(!is.null(input$vars), "Please select at least one variable"))
plot_list <- vector(mode = "list", length = length(input$vars))
for (i in seq_along(input$vars)) {
plot_list[[i]] <- dygraph(data = dat[, which(names(dat) == input$vars[i])],
main = paste("Iteration", counter_plot), xlab = "",
ylab = input$vars[i], width = "100%", height = "200px")
}
counter_plot <<- counter_plot + 1L
do.call(tagList, plot_list)
})
output$text <- renderText({
validate(need(!is.null(input$vars), ""))
counter_text <<- counter_text + 1L
paste0("Iteration ", counter_text - 1L, ": ", used_mem())
})
}
# Run -----
shinyApp(ui, server)
Second attempt
I figured this might have something to do with the fact the elements of the tagList are not assigned any IDs. I found an example here which does assign user specified IDs to each plot. I modified it a bit to use dygraphs in the example below. However, running this results in the same steady build-up of memory usage.
library(shiny)
library(dygraphs)
library(xts)
library(pryr)
max_plots <- 3
used_mem <- function() {paste0(round(mem_used()/1e6, 3)," mb")}
# Data -----
time <- seq.POSIXt(from = ISOdatetime(2018, 9, 1, 0, 0, 0, "Europe/Brussels"),
to = ISOdatetime(2018, 12, 31, 23, 59, 59, "Europe/Brussels"),
by = "10 min")
n <- length(time)
dat <- xts(x = data.frame(a = rnorm(n),
b = rexp(n),
c = runif(n)),
order.by = time,
tzone = "Europe/Brussels")
# UI -----
ui <- pageWithSidebar(
headerPanel("Dynamic number of plots"),
sidebarPanel(
sliderInput("n", "Number of plots", value = 1L, min = 1L, max = max_plots, step = 1L)
),
mainPanel(
uiOutput("plots")
)
)
# Server -----
server <- function(input, output) {
counter <- 1L
output$plots <- renderUI({
plot_output_list <- lapply(1:input$n, function(i) {
plotname <- paste("plot", i, sep = "")
dygraphOutput(plotname)
})
counter <<- counter + 1
print(paste0("Iteration ", counter - 1L, " ", used_mem()))
do.call(tagList, plot_output_list)
})
for (i in 1:max_plots) {
local({
my_i <- i
plotname <- paste("plot", my_i, sep = "")
output[[plotname]] <- renderDygraph({
dygraph(data = dat[, my_i], main = names(dat)[my_i], xlab = "", ylab = "",
width = "100%", height = "200px")
})
})
}
}
# Run -----
shinyApp(ui, server)
Third attempt
Then I figured I might be able to manually reset the memory. I tried using the removeUI/insertUI combination below, which again does not resolve the memory issue. So I guess this implies that the memory build-up occurs in the output list. I know output elements can be removed (see). This might be a possible workaround but I haven't been able to figure out how to recreate the plot output after removing it.
library(shiny)
library(dygraphs)
library(xts)
library(pryr)
used_mem <- function() {paste0(round(mem_used()/1e6, 3)," mb")}
# Data -----
time <- seq.POSIXt(from = ISOdatetime(2018, 9, 1, 0, 0, 0, "Europe/Brussels"),
to = ISOdatetime(2018, 12, 31, 23, 59, 59, "Europe/Brussels"),
by = "10 min")
dat <- xts(x = data.frame(a = rnorm(length(time)),
b = rexp(length(time)),
c = runif(length(time))),
order.by = time,
tzone = "Europe/Brussels")
# UI -----
ui <- fluidPage(
selectInput("vars", "Select variables", choices = names(dat), multiple = TRUE),
uiOutput("plot"),
textOutput("text")
)
# Server -----
server <- function(input, output, session) {
counter_plot <- counter_text <- 1L
output$plot <- renderUI({
validate(need(!is.null(input$vars), "Please select at least one variable"))
removeUI(selector = "#plot")
insertUI(selector = "#text", where = "beforeBegin", ui = uiOutput("plot"))
plot_list <- vector(mode = "list", length = length(input$vars))
for (i in seq_along(input$vars)) {
plot_list[[i]] <- dygraph(data = dat[, which(names(dat) == input$vars[i])],
main = paste("Iteration", counter_plot), xlab = "",
ylab = input$vars[i], width = "100%", height = "200px")
}
counter_plot <<- counter_plot + 1L
do.call(tagList, plot_list)
})
output$text <- renderText({
validate(need(!is.null(input$vars), ""))
counter_text <<- counter_text + 1L
paste0("Iteration ", counter_text - 1L, ": ", used_mem())
})
}
# Run -----
shinyApp(ui, server)
Workaround
The code below is a workaround I have found but I am not really satisfied with it. I suppose it would be workable if the number of simultaneous plots is limited (in my application there are around 800 possible variables that can be plotted). But I would prefer the (in my opinion) simplicity of the uiOutput/renderUI/tagList combination.
library(shiny)
library(dygraphs)
library(xts)
library(pryr)
used_mem <- function() {paste0(round(mem_used()/1e6, 3)," mb")}
# Data -----
time <- seq.POSIXt(from = ISOdatetime(2018, 9, 1, 0, 0, 0, "Europe/Brussels"),
to = ISOdatetime(2018, 12, 31, 23, 59, 59, "Europe/Brussels"),
by = "10 min")
n <- length(time)
dat <- xts(x = data.frame(a = rnorm(n),
b = rexp(n),
c = runif(n)),
order.by = time,
tzone = "Europe/Brussels")
# UI -----
ui <- fluidPage(
selectInput("vars", "Select variables", choices = names(dat), multiple = TRUE),
dygraphOutput("plot1", width = "100%", height = "200px"),
conditionalPanel("output.nvars > 1",
dygraphOutput("plot2", width = "100%", height = "200px")),
conditionalPanel("output.nvars > 2",
dygraphOutput("plot3", width = "100%", height = "200px")),
textOutput("text")
)
# Server -----
server <- function(input, output, server) {
counter_plot <- counter_text <- 1L
output$nvars <- reactive(return(length(input$vars)))
outputOptions(output, "nvars", suspendWhenHidden = FALSE)
output$plot1 <- renderDygraph({
validate(need(length(input$vars) > 0L, "Please select at least one variable"))
counter_plot <<- counter_plot + 1L
dygraph(dat[, input$vars[1L]], main = paste("Iteration", counter_plot - 1L),
xlab = "", ylab = input$vars[1L])
})
output$plot2 <- renderDygraph({
validate(need(length(input$vars) > 1L, ""))
dygraph(dat[, input$vars[2L]], main = paste("Iteration", counter_plot - 1L),
xlab = "", ylab = input$vars[2L])
})
output$plot3 <- renderDygraph({
validate(need(length(input$vars) > 2L, ""))
dygraph(dat[, input$vars[3L]], main = paste("Iteration", counter_plot - 1L),
xlab = "", ylab = input$vars[3L])
})
output$text <- renderText({
validate(need(!is.null(input$vars), ""))
counter_text <<- counter_text + 1L
paste0("Iteration ", counter_text - 1L, ": ", used_mem())
})
}
# Run -----
shinyApp(ui, server)
Questions
- Is this an actual bug/memory leak that should be reported or am I missing something? And if it is a bug, is it a shiny issue or purely a dygraph issue? I suspect there might be a connection to this issue which I can reproduce on my machine.
- Are there alternative solutions that are more elegant and simple than the workaround I have found? Any suggestions would be more than welcome!
devtools::session_info()
-
Session info -----------------------------------------------------------------------------------------------------------------
setting value
version R version 3.5.2 (2018-12-20)
os Windows >= 8 x64
system x86_64, mingw32
ui RStudio
language (EN)
collate English_Belgium.1252
ctype English_Belgium.1252
tz Europe/Paris
date 2019-01-16 -
Packages ---------------------------------------------------------------------------------------------------------------------
package * version date lib source
assertthat 0.2.0 2017-04-11 [1] CRAN (R 3.5.1)
backports 1.1.3 2018-12-14 [1] CRAN (R 3.5.2)
callr 3.1.1 2018-12-21 [1] CRAN (R 3.5.2)
cli 1.0.1 2018-09-25 [1] CRAN (R 3.5.2)
codetools 0.2-15 2016-10-05 [2] CRAN (R 3.5.2)
crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.1)
desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.1)
devtools 2.0.1 2018-10-26 [1] CRAN (R 3.5.2)
digest 0.6.18 2018-10-10 [1] CRAN (R 3.5.2)
dygraphs * 1.1.1.6 2018-07-11 [1] CRAN (R 3.5.2)
fortunes 1.5-4 2016-12-29 [1] CRAN (R 3.5.0)
fs 1.2.6 2018-08-23 [1] CRAN (R 3.5.1)
glue 1.3.0 2018-07-17 [1] CRAN (R 3.5.1)
htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.1)
htmlwidgets 1.3 2018-09-30 [1] CRAN (R 3.5.2)
httpuv 1.4.5.1 2018-12-18 [1] CRAN (R 3.5.2)
jsonlite 1.6 2018-12-07 [1] CRAN (R 3.5.2)
later 0.7.5 2018-09-18 [1] CRAN (R 3.5.2)
lattice 0.20-38 2018-11-04 [2] CRAN (R 3.5.2)
magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.1)
memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.1)
mime 0.6 2018-10-05 [1] CRAN (R 3.5.2)
pkgbuild 1.0.2 2018-10-16 [1] CRAN (R 3.5.2)
pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.2)
prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.1)
processx 3.2.1 2018-12-05 [1] CRAN (R 3.5.2)
promises 1.0.1 2018-04-13 [1] CRAN (R 3.5.1)
pryr * 0.1.4 2018-02-18 [1] CRAN (R 3.5.2)
ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.2)
R6 2.3.0 2018-10-04 [1] CRAN (R 3.5.2)
Rcpp 1.0.0 2018-11-07 [1] CRAN (R 3.5.2)
remotes 2.0.2 2018-10-30 [1] CRAN (R 3.5.2)
rlang 0.3.0.1 2018-10-25 [1] CRAN (R 3.5.2)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.1)
rsconnect 0.8.12 2018-12-05 [1] CRAN (R 3.5.2)
rstudioapi 0.8 2018-10-02 [1] CRAN (R 3.5.2)
sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.2)
shiny * 1.2.0 2018-11-02 [1] CRAN (R 3.5.2)
stringi 1.2.4 2018-07-20 [1] CRAN (R 3.5.1)
stringr 1.3.1 2018-05-10 [1] CRAN (R 3.5.1)
testthat 2.0.1 2018-10-13 [1] CRAN (R 3.5.2)
usethis 1.4.0 2018-08-14 [1] CRAN (R 3.5.2)
withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.1)
xtable 1.8-3 2018-08-29 [1] CRAN (R 3.5.1)
xts * 0.11-2 2018-11-05 [1] CRAN (R 3.5.2)
yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.1)
zoo * 1.8-4 2018-09-19 [1] CRAN (R 3.5.2)
[1] C:/Users/ndep/Documents/R/Packages
[2] C:/Program Files/R/R-3.5.2/library