I am running out of memory due to multiple caret predictions

I have a Shiny app that uses 5 xgbTree models to make predictions. The app runs locally just fine but when I try to run it on shinyapps.io it runs out of memory. I am using Instance Size 3X-Large (8 GB). The part that causes the app to run out of memory is making all 5 predict() function calls all together.

This is what runs out of memory on shinyapps.io. (I read in all 5 of my models outside of the server() function and these predict functions are inside the server() function)

  # Predictions output:
  preds <- eventReactive(input$gobutton, {

    user_data <- df()

    pred_sed <- predict(xgbTree_CEAP1and2_sed, newdata = user_data)
    pred_SurfN <- predict(xgbTree_CEAP1and2_SurfN, newdata = user_data)
    pred_SubN <- predict(xgbTree_CEAP1and2_SubN, newdata = user_data)
    pred_P <- predict(xgbTree_CEAP1and2_P, newdata = user_data)
    pred_Psol <- predict(xgbTree_CEAP1and2_Psol, newdata = user_data)

  })
  

Here is my temporary fix but it is SLOW. (I read in one model at a time, make the prediction, then remove the model. This all occurs in the server() function)

  # Predictions output:
  preds <- eventReactive(input$gobutton, {

    user_data <- df()

    # Sediment
    xgbTree_CEAP1and2_sed <- readRDS("xgbTree_CEAP1and2_sed_trimmed.rds")
    pred_sed <- predict(xgbTree_CEAP1and2_sed, newdata = user_data)
    remove(xgbTree_CEAP1and2_sed)
    
    # Surface Nitrogen
    xgbTree_CEAP1and2_SurfN <- readRDS("xgbTree_CEAP1and2_SurfN_trimmed.rds")
    pred_SurfN <- predict(xgbTree_CEAP1and2_SurfN, newdata = user_data)
    remove(xgbTree_CEAP1and2_SurfN)
    
    # Subsurface Nitrogen
    xgbTree_CEAP1and2_SubN <- readRDS("xgbTree_CEAP1and2_SubN_trimmed.rds")
    pred_SubN <- predict(xgbTree_CEAP1and2_SubN, newdata = user_data)
    remove(xgbTree_CEAP1and2_SubN)
    
    # Total Phosphorus
    xgbTree_CEAP1and2_P <- readRDS("xgbTree_CEAP1and2_P_trimmed.rds")
    pred_P <- predict(xgbTree_CEAP1and2_P, newdata = user_data)
    remove(xgbTree_CEAP1and2_P)
    
    # Soluble Phosphorus
    xgbTree_CEAP1and2_Psol <- readRDS("xgbTree_CEAP1and2_Psol_trimmed.rds")
    pred_Psol <- predict(xgbTree_CEAP1and2_Psol, newdata = user_data)
    remove(xgbTree_CEAP1and2_Psol)

  })
  

I also already tried trimming away any parts of the models not needed for prediction:

# function to remove unnecessary parts of the caret model
trim_model_fun <- function(model) {
  model$trainingData <- NULL
  model$control <- NULL
  model$modelInfo <- NULL
  
  return(model)
}

Any advice for how to read in all 5 models outside of the server() function and be able to use predict() to make all 5 predictions inside the server() function? Thanks