Crashing RStudio Cloud with spatial data...

[AmeliaMN]

I'm teaching a class using RStudio Cloud, and it's been going well so far. But today, we were doing some stuff with spatial data (a similar set of code is available here: https://www.amelia.mn/sds236/MakingMaps.html) and when we got to the leaflet piece, everyone's R sessions crashed, losing everything in their Environment and their History

I think everyone was working in my STAT360 workspace (Posit Cloud) and we got a variety of error messages. One interesting/weird feature is that after the crash, the sf library could no longer be loaded into most of their new sessions. Here's an error message:

> library(sf)
Error: package or namespace load failed for ‘sf’:
.onLoad failed in loadNamespace() for 'sf', details:
 call: get(genname, envir = envir)
 error: object 'group_map' not found

Sorry if I've crashed Cloud somehow!

[technocrat]

group_map is a dplyr function, which I think has been recently updated, as has sf. Also, sf has to be able to link to GEOS 3.6.1, GDAL 2.1.3, PROJ 4.9.3. I can't log into the cloud account, of course, so it's hard to tell what's loaded. Could you post

sessionInfo()

to take a look at the set-up?

[AmeliaMN]

What makes me think I did it is that the code was working for me earlier when it was just me, but once I let my students loose we started having problems. Here's the session info:

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lubridate_1.7.4

loaded via a namespace (and not attached):
[1] compiler_3.5.2 magrittr_1.5   tools_3.5.2    Rcpp_1.0.0    
[5] stringi_1.2.4  knitr_1.21     stringr_1.3.1  xfun_0.4      

[AmeliaMN]

Ah, that was after the crash. Here is is after loading dplyr, rgdal, leaflet and trying to load sf but failing.

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

Matrix products: default
BLAS: /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] leaflet_2.0.2   rgdal_1.4-3     sp_1.3-1        dplyr_0.7.8    
[5] lubridate_1.7.4

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0       later_0.8.0      pillar_1.3.1     compiler_3.5.2  
 [5] bindr_0.1.1      class_7.3-14     tools_3.5.2      digest_0.6.18   
 [9] tibble_2.0.1     lattice_0.20-38  pkgconfig_2.0.2  rlang_0.3.1     
[13] shiny_1.3.0      DBI_1.0.0        rstudioapi_0.9.0 crosstalk_1.0.0 
[17] xfun_0.4         bindrcpp_0.2.2   e1071_1.7-1      stringr_1.3.1   
[21] knitr_1.21       htmlwidgets_1.3  classInt_0.3-1   grid_3.5.2      
[25] tidyselect_0.2.5 glue_1.3.0       R6_2.3.0         purrr_0.2.5     
[29] magrittr_1.5     promises_1.0.1   htmltools_0.3.6  units_0.6-2     
[33] assertthat_0.2.0 xtable_1.8-3     mime_0.6         httpuv_1.5.1    
[37] stringi_1.2.4    crayon_1.3.4    

[technocrat]

I can't reproduce the problem, of course, without the code, but I've been running sf without difficulty. Here's what I have after loading the same packages:

sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rgdal_1.4-3   sp_1.3-1      leaflet_2.0.2 sf_0.7-3      dplyr_0.8.0.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1       rstudioapi_0.10  magrittr_1.5     units_0.6-2     
 [5] tidyselect_0.2.5 lattice_0.20-38  xtable_1.8-3     R6_2.4.0        
 [9] rlang_0.3.4      tools_3.5.3      grid_3.5.3       packrat_0.5.0   
[13] e1071_1.7-1      DBI_1.0.0        htmltools_0.3.6  crosstalk_1.0.0 
[17] class_7.3-15     assertthat_0.2.1 digest_0.6.18    tibble_2.1.1    
[21] crayon_1.3.4     shiny_1.3.0      later_0.8.0      purrr_0.3.2     
[25] htmlwidgets_1.3  promises_1.0.1   mime_0.6         glue_1.3.1      
[29] compiler_3.5.3   pillar_1.3.1     classInt_0.3-1   httpuv_1.5.1    
[33] pkgconfig_2.0.2 
> 

What would worry me most is Ubuntu, R, dplyr, Rcpp, rstudioapi, compiler ... just say everything that's not current. I'm not a cloud user, so I don't know whether you, as a user, can do all of that.

[AmeliaMN]

I suspect it has to do with the size of the SpatialPolygonsDataFrame we were creating, which is pretty large. What I don't understand is why crashing R then makes sf not work. I'll try to create a more reproducible issue!

[mfherman]

Have you tried upgrading to the current CRAN or dev version of dplyr (0.8)? I remember having some weirdness like you’re describing after upgrading sf while having an older version of dplyr installed. As @technocrat mentioned, the reference to dplyr::group_map() in the error message makes me think it’s related to this issue.

[seans]

Hi Amelia,

Sorry you encountered issues using cloud. I think the R session crashes are memory related. I looked up a couple recent projects in your space and it does look like a number of them hit the 1GB memory limit earlier in the day. This would result in a session crash similar to what you describe.

Sean

[technocrat]

I woke up thinking the same thing. With a hard limit like that, it makes sense that it would shut down the largest packages first, which might be sf.`

[AmeliaMN]

Thanks! I'll try to come up with smaller activities for my students in the future

[technocrat]

You should also talk to the RStudio folk about an academic license with more generous limits; I know they have academic rates on their commercial products, so maybe ...