I get the following error message when deploying my shiny App: Container event from container-7154229: oom (out of memory).
The app works when run locally. When using ListBundleFiles() the app size is listed as 26478691 which is significantly smaller than some other apps running on my account that work just fine. I have a standard account which should be able to host apps up to 8gb in size. Does anyone know what might be causing this error, or how to troubleshoot it?
The out of memory doesn't refer to disk space but to RAM, your files might be small but you might be performing memory intensive tasks with your code.
Can you provide more details? Like the complete console output you get and your app's code?
I can share my UI and server if that helps maybe the code is redundant in a few areas and the issue can be solved by trying to reduce the amount of work the functions are doing?
UI
ui <- shiny::navbarPage(
title = paste0("Data Kubb"),
theme = shinythemes::shinytheme(theme = "flatly"),
shiny::tabPanel(
title = "14064 US Hand and Body Care",
shiny::fluidRow(
shiny::column(
width = 4,
shiny::wellPanel(
shiny::h2("Data Slicing"),
shiny::h4("Customise data slice below"),
# Create a UI drop-down menu...
shiny::selectizeInput(
inputId = "dimension",
label = paste0("Data Dimension"),
choices = c("Purchase", "Trips"),
selected = "Purchase",
),
shiny::selectizeInput(
inputId = "data_lens",
label = paste0("Data Lens (Rows)"),
choices = NULL,
selected = character(0),
options = list(
placeholder = "Choose One...",
onInitialize = I('function() { this.setValue(""); }')
)
),
shiny::selectizeInput(
inputId = "columns",
label = paste0("Breakdown data lens (columns)"),
choices = NULL,
selected = character(0),
options = list(
placeholder = "Choose One...",
onInitialize = I('function() { this.setValue(""); }')
)
),
shiny::selectizeInput(
inputId = "cut",
label = paste0("Target Group (Table Filter)"),
choices = NULL,
selected = "No Filter",
),
shiny::br(),
shiny::div(
class = "float-right",
shiny::actionButton(
inputId = "flip_data_lens",
label = "Flip Data Lens",
icon = icon("fas fa-sync"),
width = "100%"
)
),
shiny::br(),
shiny::div(
class = "float-right",
shiny::actionButton(
inputId = "clear_selections",
label = "Clear selections",
icon = icon("hand-sparkles"),
width = "100%"
)
)
)
),
shiny::column(
width = 8,
shiny::column(
width = 12,
shiny::h4(strong(textOutput(
"main_output_title"))),
shiny::h6(textOutput(
"main_output_subtitle")),
DT::DTOutput(
outputId = "tbl"
),
shiny::br(),
shiny::textOutput("base_text")
)
)
)
)
)
Server function
server <- function(input, output, session) {
#Update variable select choices based on the dimension selected (Purchase or Trips)
observeEvent(input$dimension, {
x <- input$dimension
if (x == "Trips") {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = c("Age group"),
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = "No filter",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "cut",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$`Filter Displayed`)
)
)
),
selected = "No Filter", #Please change to character(0) and fix functions to match
server = TRUE
)
}
else {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = "Age group",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = "No filter", #Please change to character(0) and fix functions to match
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "cut",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$`Filter Displayed`)
)
)
),
selected = "No Filter", #Please change to character(0) and fix functions to match
options = list(
placeholder = "Optional...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
}
})
#Flip data lens option
observeEvent(input$flip_data_lens, {
x <- input$dimension
col <- input$columns
row <- input$data_lens
if (x == "Trips") {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = col,
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = row,
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
}
else {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = col,
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = row,
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
}})
observeEvent(input$clear_selections, {
x <- input$dimension
col <- input$columns
row <- input$data_lens
if (x == "Trips") {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = "Age group",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$data_lens)
)
),
c("No filter")
),
selected = "No filter",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "cut",
choices = c(sort
(na.omit
(unique
(data_cuts$Trips$`Filter Displayed`)
)
)
),
selected = "No Filter", #Please change to character(0) and fix functions to match
server = TRUE
)
}
else {
shiny::updateSelectizeInput(
session = session,
inputId = "data_lens",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = "Age group",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "columns",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$data_lens)
)
),
c("No filter")
),
selected = "No filter",
options = list(
placeholder = "Choose One...",
render = I('function() { this.setValue(""); }')
),
server = TRUE
)
shiny::updateSelectizeInput(
session = session,
inputId = "cut",
choices = c(sort
(na.omit
(unique
(data_cuts$Purchase$`Filter Displayed`)
)
)
),
selected = "No Filter", #Please change to character(0) and fix functions to match
server = TRUE
)
}})
#Create reactive values for data, vars, rowvar, and colvar
data <- reactive({
get_data(df, input$dimension, input$cut)
})
vars <- reactive({
get_vars(input$dimension, input$data_lens, input$columns)
})
row_var <- reactive({
get_row_var(input$dimension, input$data_lens)
})
col_var <- reactive({
get_col_var(input$dimension, input$columns)
})
#Reactive value for table output
tab <- reactive({
data <- data()
row_var <- row_var()
col_var <- col_var()
if(length(row_var) >1 & identical(col_var, character(0))) { #Use get_count here if None is selected for the column input
tab.prop.count <- get_multi_none_count(data, row_var, input$data_lens)# Problem with naming this table as well as missing overall count potentially only needed in the base
tab.prop <- tab.prop.count[[1]]
tab.count <- tab.prop.count[[2]]
create_dt(tab.prop)
#DONE: July 7
}
else if (identical(col_var, character(0))) {
tab.prop.count <- get_count(data, row_var, input$data_lens) # Problem with naming this table as well as missing overall count potentially only needed in the base
tab.prop <- tab.prop.count[[1]]
tab.count <- tab.prop.count[[2]]
create_dt(tab.prop)
#DONE: July 7
}
else if (length(col_var) > 1) { # if the column variable is a multiselect then output multiselect table
tab.prop.count <- get_multi_table_col(data, row_var, col_var, input$data_lens)
tab.prop <- tab.prop.count[[1]]
tab.count <- tab.prop.count[[2]]
sig.table <- multi_col_sig_table(tab.prop, tab.count)
tab.full <- cbind(tab.prop, sig.table)
create_dt(tab.prop, tab.full, sig.table)
#DONE: July 7
}
else if (length(row_var)>1){# if the row variable is a multiselect then output multiselect table
tab.prop.count <- get_multi_table_row(data, row_var, col_var, input$data_lens)
tab.prop <- tab.prop.count[[1]]
tab.count <- tab.prop.count[[4]]
tab.n <- tab.prop.count[[2]]
sig.table <- multi_row_sig_table(tab.prop, tab.count, tab.n)
tab.full <- cbind(tab.prop, sig.table)
create_dt(tab.prop, tab.full, sig.table)
#DONE:July 7
}
else {
tab.prop.count <- get_table(data, row_var, col_var, input$data_lens)
tab.prop <- tab.prop.count[[1]]
tab.count <- tab.prop.count[[2]]
sig.table <- sig_table(tab.prop, tab.count)
tab.full <- cbind(tab.prop, sig.table)
create_dt(tab.prop, tab.full, sig.table)
#DONE: July 7
}})
#Get base text string
base <- reactive({
data <- data()
row_var <- row_var()
col_var <- col_var()
if (length(row_var) >1 & identical(col_var, character(0))) { #Multi-none
tab <- get_multi_none_count(data, row_var, input$data_lens)
footer <- paste0("Base = ", tab[[3]]) # Add "Total Purchases?"
footer
#Done: July 7
}
else if(identical(col_var, character(0))) {
tab <- get_count(data, row_var, input$data_lens)
footer <- get_footer(tab)
#Done: July 7
}
else if (length(col_var)>1) {
mydata <- get_multi_table_col(data, row_var, col_var, input$data_lens)
footer <- multi_col_footer(mydata)
footer
#Done: July 7
}
else if (length(row_var)>1){
mydata <- get_multi_table_row(data, row_var, col_var, input$data_lens)
footer <- multi_row_footer(mydata)
footer
#DONE: July 7
}
else {
mydata <- get_table(data, row_var, col_var, input$data_lens)
footer <- get_footer(mydata)
footer
#Done: July 7
}})
output$tbl <- renderDT({tab()})
output$base_text <- renderText({base()})
#Aesthetic Titles - Pulled from orignal data cube scripts
output$main_output_title <- renderText({
req(input$dimension, input$columns, input$data_lens)
if(input$columns == "No filter"){
columns_text <- ""
}
else {
columns_text <- paste0(" - Overall vs by ", input$columns)
}
if (input$cut == "no_filter"){
filter_text <- ""
}
else {
filter_text <- paste0(" - filtered for ", input$cut)
}
paste0(input$dimension,
" by ",
input$data_lens,
columns_text,
filter_text)
})
#Pulled from original datacube scripts
output$main_output_subtitle <- renderText({
req(input$dimension, input$columns, input$data_lens)
if(input$columns == "No filter") {
return("Share of total")
} else if(input$columns != "No filter"){
return(paste0("Share of total vs by ", input$columns))
}
})
}
Your app seems incomplete; where does data_cuts come from ?
Regardless, its probably overly-optimistic that a forum-mite will do you a substantial favour of a full review of your entire app; I recommend that you look into the practice of 'profiling' your app. That should shed light on the memory consumption issues if there are any. start here : Shiny - Profiling your Shiny app (rstudio.com)
Yes apologies I'm not able to share the whole app. Thank you for directing me to profiling I will see if that can help!
Same issue. In my global.R, I have about 2 GB of data.RDS files loading. On the paid Shiny Apps server, I can increase the RAM, and it does work. However, I need to host this on a client's free tier shinyapps.iso server, and we receive this 2 Gb of data as .RDS files, which may increase in size in the future.
How to handle this situation with a free shinyapps.iso server?
Any suggestions would be helpful.
You would need to change to an "on-disk" approach to handle your data, arrow and duckdb are popular options but I'm not sure if they are available at Shinyapp.io.
Another option would be to host shiny-server yourself on a free cloud computing service, if you can work your way around managing arm64 systems, Oracle Cloud offers a 4 cores 24GB RAM arm64 virtual machine for free. On the same line of thought you could set up a DBMS on a virtual machine and push the processing there using SQL.
1 Like